What's happening with the last child process here????
in [Perl]
|
Prev: Math::GMP make test fails.
Next: System Return Value?
From: trxrse on 10 Jan 2007 04:34 Hi, everyone can you tell me please what's happening here? This piece of code is supposed to launch 3 children processes and to keep track of each one when they finish. THE PROBLEM: SOMETIMES (50% OF THE TIME) THE LAST PROCESS NEVER RETURNS THE USR1 MESSAGE TO THE PARENT. For example: ../fork3 Started Wed Jan 10 09:23:36 2007 Forked by Wed Jan 10 09:23:36 2007 I am finished... Process 1 Look:3 $nfound:0 Look:2 $nfound:0 Look:1 $nfound:1 Wed Jan 10 09:23:37 2007 - Completed 1 I am finished... Process 2 Look:3 $nfound:0 Look:2 $nfound:0 Look:1 $nfound:0 I am finished... Process 3 Look:3 $nfound:0 Look:2 $nfound:2 Wed Jan 10 09:23:39 2007 - Completed 2 Look:1 $nfound:0 >>>>>>>>>>>>>>>>>>>>>>>> And here it gets stuck forever!!!!!!!!!!!!!!!!! And now the code: #! /bin/perl # Forking out a series of processes which may take # a while to run. System waits for any signal to be # received back so that it can get the faster ones # finalised quickly! Once a signal is received back # it finds the appropriate pipe and reads from it @waitlist = (3,2,1); $SIG{USR1} = "doneit"; $start = localtime(); print "Started $start\n"; $parent = $$; # Use a "typeglob" to store, in effect, a list of # file handles. foreach $item (@waitlist) { pipe *{$item},FH; unless ($pid = fork()) { # Child process sleep $item; print FH "Completed $item\n"; kill "USR1",$parent; print "I am finished... Process ",$item,"\n"; exit(); } # Parent process - loop to start others $kids++; } $mid = localtime(); print "Forked by $mid\n"; # Parent - wait for returned data # To add - uses select to check channels! while ($kids >0) { while (! $gotone) { sleep 1; } $gotone = 0; foreach $item (@waitlist) { $rin = $win = ""; vec($rin, fileno(*{$item}), 1) = 1; $ein = $rin ; $nfound=select($rin,$win,$ein,0); print "Look:",$item," \$nfound:",$nfound,"\n"; if ($nfound) { sysread($item,$response,40); $now = localtime(); print "$now - $response"; close $item; $kids--; } } } $end = localtime(); print "Completed $end\n"; sub doneit { $gotone = 1; }
From: anno4000 on 10 Jan 2007 07:00 trxrse <Remus.SEPP(a)t-mobile.co.uk> wrote in comp.lang.perl.misc: > Hi, everyone > > can you tell me please what's happening here? > > This piece of code is supposed to launch 3 children processes and to > keep track of each one when they finish. > THE PROBLEM: SOMETIMES (50% OF THE TIME) THE LAST PROCESS NEVER RETURNS > THE USR1 MESSAGE TO THE PARENT. No need to shout. [output sample snipped] > And now the code: > > #! /bin/perl You are missing "use strict" and "use warnings" here. That makes your code hard to debug. > # Forking out a series of processes which may take > # a while to run. System waits for any signal to be > # received back so that it can get the faster ones > # finalised quickly! Once a signal is received back > # it finds the appropriate pipe and reads from it > > @waitlist = (3,2,1); > $SIG{USR1} = "doneit"; > $start = localtime(); > print "Started $start\n"; > $parent = $$; > > # Use a "typeglob" to store, in effect, a list of > # file handles. > > foreach $item (@waitlist) { > pipe *{$item},FH; The last statement only works without strictures. You are creating filehandles with non-standard names "3", "2" and "1". Is that what you intend? I haven't followed your code any further. Please make your code strict-compliant. If the use of numeric filehandles is somehow essential to the program revoke strict 'refs' locally, but I think you should be able to work with standard filehandles. Repost your code with strict and warnings in place. You'll have better chances of someone finding the error or the explanation. Anno -- $anagram = 'Knuth heals rare project'; # by Abigail push @{ $pos{ $_}}, $pos ++ for split //, lc $anagram; print "print +(split //, '$anagram')[ $_]\n" for join ', ', map shift @$_, @pos{ split //, lc "Just another Perl hacker"};
From: xhoster on 10 Jan 2007 12:52 "trxrse" <Remus.SEPP(a)t-mobile.co.uk> wrote: > Hi, everyone > > can you tell me please what's happening here? > > This piece of code is supposed to launch 3 children processes and to > keep track of each one when they finish. > THE PROBLEM: SOMETIMES (50% OF THE TIME) THE LAST PROCESS NEVER RETURNS > THE USR1 MESSAGE TO THE PARENT. USR1 isn't a message. > $SIG{USR1} = "doneit"; Do you need to reinstall signals after they've been fired on your system? > foreach $item (@waitlist) { > pipe *{$item},FH; > unless ($pid = fork()) { > # Child process > sleep $item; > print FH "Completed $item\n"; > kill "USR1",$parent; > print "I am finished... Process ",$item,"\n"; > exit(); > } Why mix signals and select? When the child is done writing on it's handle, then it is done. No need for signals at all. Xho -- -------------------- http://NewsReader.Com/ -------------------- Usenet Newsgroup Service $9.95/Month 30GB
From: Uri Guttman on 10 Jan 2007 13:49 >>>>> "t" == trxrse <Remus.SEPP(a)t-mobile.co.uk> writes: t> can you tell me please what's happening here? not really. it is one of the most convoluted and bizarre forking examples i have seen. why are you doing it the hard way? t> THE PROBLEM: SOMETIMES (50% OF THE TIME) THE LAST PROCESS NEVER RETURNS t> THE USR1 MESSAGE TO THE PARENT. signals can be merged into one actual call to your signal handler. this is an OS dependent thing and perl can't fix it. when you get a signal, you have to make sure it was only one or possible more than one that were merged. the way to do this with subprocesses is to call wait/waitpid in nonblcoking mode and loop until no more processes are reaped. use the sigchild signal which is better as it will be delivered after the child exits unlike yours which could be delivered before it really exited. t> @waitlist = (3,2,1); as anno said, this is a very bad way to track the processes. save the pids and use those to track what is alive and dead. t> $SIG{USR1} = "doneit"; you can use a code ref there since your callback is so small. $SIG{USR1} = sub { $doneit = 1 } ; but as i said use sigchild. and use strict and warnings also as anno said. t> # Use a "typeglob" to store, in effect, a list of t> # file handles. that is not a good way to do this. use lexical file handles and make one for each call to pipe. use Symbol::gensym or IPC::Open2 but not typeglobs which are symbolic refs. t> foreach $item (@waitlist) { t> pipe *{$item},FH; the FH will be the overwritten in the parent for all the children which is not smart. one issue with pipes to children is detecting closed pipes with select as you seem to be doing below. you have to close the other side in the parent or child since those will be dupped and stay open. t> unless ($pid = fork()) { t> # Child process t> sleep $item; reusing the item number for sleep time is very nutso. i can't say anything more as i am just shocked by that code. t> print FH "Completed $item\n"; t> kill "USR1",$parent; that signal could be delivered before this process exits so you won't be able to immediately detect it. but my guess is that the signals were merged and you don't use wait/waitpid to really check for childred to reap. actually you aren't even reaping the children (only wait/waitpid does that) so you create zombie children which will get reaped by the init process when your parent exits. as i said, this is all done totally the wrong way. t> $mid = localtime(); t> print "Forked by $mid\n"; forking will happen much faster than the granulaity of localtime (which is normally 1 second). t> # Parent - wait for returned data t> # To add - uses select to check channels! t> while ($kids >0) { t> while (! $gotone) { t> sleep 1; t> } t> $gotone = 0; t> foreach $item (@waitlist) { t> $rin = $win = ""; t> vec($rin, fileno(*{$item}), 1) = 1; t> $ein = $rin ; t> $nfound=select($rin,$win,$ein,0); coding select directly is not for the faint of heart. use IO::Select or event.pm. event.pm can also handle signals which makes it very good for process management. enough for now. my brane hertz. uri -- Uri Guttman ------ uri(a)stemsystems.com -------- http://www.stemsystems.com --Perl Consulting, Stem Development, Systems Architecture, Design and Coding- Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
From: trxrse on 10 Jan 2007 14:11
Thanks very much for your replies. I appologize, I am not a professional perl developer... even if I'd like to... :( - probably I need some years for this. Anyway, I appreciate your answers. To be honest, I am trying to write a very simple "scheduler" that gets an amount of work as input (eg process a file having 1 million lines), splits this work in chunks and launches a limited number of parallel processes, each one doing some small chunk of work. When a child process is finished OK, the "scheduler" has to launch another child that does the next chunk. Finally, I want to embed some elegant behavior in the master/parent - aka re-launch the chunk of work for the subprocess that collapsed, etc... Can you give me a hint, please? Of course, I am not asking you people to solve my problem, but I would like to find your professional views. Thanks again R PS I will rewrite the code as Anno said earlier. |