Creating a 'load simulator' by calling Perl Programs - or Forking?
in [Perl]
|
Prev: Spelling suggestions for common words - ispell, etc.
Next: Creating a 'load simulator' by calling Perl Programs - or Forking?
From: pgodfrin on 3 Apr 2008 16:02 Greetings, I would like to create a 'driver' (simulation) program that calls existing perl programs, waits for them to complete and provides some feedback. Ideally I would like to have the driver echo to the screen that the simulation is still running and perhaps provide some information. That may be too ambitious, but that is the basic idea. I have a handful of perl programs that do things - read a database (kudos to the DBI modules) and output a report, from a few lines to a few thousand. Question #1: What is the proper way to execute the perl programs? I am familiar with exec() and system() and have done this: system("sallmr1 -n50000 >/dev/null"); system("srngmr1 -sr -es -n10000 >/dev/null"); system("srngmr2 -sr -es -n10000 >/dev/null"); Which executes these three programs sequentially - or by adding '&' at the end I can execute them at the same time. Of course the 'calling' program cannot wait for the background tasks because of the shell return. Which leads me down the path of fork(). If I were to code the following: for(1..2) { if($pid=fork) { # parent } elsif (defined $pid) { # child exec("sallmr1 -n50000 >/dev/null"); } # end child (elsif) } # end for loop do { $x=wait; print "$x died\n";} until $x==-1; Then I would get a pair of runs, and wait until each child dies. Question #2: Is there a way to echo to the screen that the wait() is happening and provide some indication of progress? To clarify this question, I had originally tried: do {$x=wait; print ".";} until $x==-1; Thinking I would get a period printed during the loop. (silly me - wait only comes back when a child process dies...). Of course if this is much to complicated a way to so a simulated load, then so be it - just a plain old fork and wait works fine... And I could always remove the /dev/null and actually see the called programs output, which will tell me that it's still running... But I was curious if there is a more elegant or at least interesting way of doing this? thanks, phil
From: Joost Diepenmaat on 3 Apr 2008 16:30 pgodfrin <pgodfrin(a)gmail.com> writes: > Greetings, > > I would like to create a 'driver' (simulation) program that calls > existing perl programs, waits for them to complete and provides some > feedback. Ideally I would like to have the driver echo to the screen > that the simulation is still running and perhaps provide some > information. That may be too ambitious, but that is the basic idea. It may take a bit of getting used to, but POE::Wheel::Run can do this very nicely, including monitoring of the fork'd procesesses' STDOUT and STDERR, catching signals etc. And knowing POE will be useful in many event-based processes. See: http://search.cpan.org/~rcaputo/POE-1.0000/lib/POE/Wheel/Run.pm and the section "Process Management" in the POE cookbook: http://poe.perl.org/?POE_Cookbook -- Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
From: pgodfrin on 3 Apr 2008 16:41 On Apr 3, 3:30 pm, Joost Diepenmaat <jo...(a)zeekat.nl> wrote: > pgodfrin <pgodf...(a)gmail.com> writes: > > Greetings, > > > I would like to create a 'driver' (simulation) program that calls > > existing perl programs, waits for them to complete and provides some > > feedback. Ideally I would like to have the driver echo to the screen > > that the simulation is still running and perhaps provide some > > information. That may be too ambitious, but that is the basic idea. > > It may take a bit of getting used to, but POE::Wheel::Run can do this > very nicely, including monitoring of the fork'd procesesses' STDOUT and > STDERR, catching signals etc. And knowing POE will be useful in many > event-based processes. > > See:http://search.cpan.org/~rcaputo/POE-1.0000/lib/POE/Wheel/Run.pm > > and the section "Process Management" in the POE cookbook:http://poe.perl.org/?POE_Cookbook > > -- > Joost Diepenmaat | blog:http://joost.zeekat.nl/| work:http://zeekat.nl/ I was looking at IPC::Run as well. What do you think of it? pg
From: pgodfrin on 3 Apr 2008 16:57 On Apr 3, 3:49 pm, Joost Diepenmaat <jo...(a)zeekat.nl> wrote: > pgodfrin <pgodf...(a)gmail.com> writes: > > I was looking at IPC::Run as well. What do you think of it? > > Haven't used it all. And to be honest, from skimming the docs I can't > really see if it will even do what you want it to do if you have more > than one child process running, which I can guarantee you POE will. > > But as I implied, POE has a bit of a learning curve, and it requires you > to (re)structure your code in a event handling way. YMMV, probably :-) > > -- > Joost Diepenmaat | blog:http://joost.zeekat.nl/| work:http://zeekat.nl/ Yeah - I agree. As I've mentioned in the past, I'm just a lowly DBA (out of work btw) who's trying to not only learn Perl real good, but use it simply to drive my databases so I can do DBA stuff - you know like look at CPU util, I/O, funky SQL, etc. So - I'm certain the POE will do the trick, on your say-so, but I'm not sure if it is worth the learning curve at this juncture. But I have bookmarked it and I will at least peruse it to see what it's all about. You never know... smiles, pg
From: xhoster on 3 Apr 2008 17:22
pgodfrin <pgodfrin(a)gmail.com> wrote: .... > Question #1: What is the proper way to execute the perl programs? You gave two examples, sequentially and in parallel. Which one is appropriate depends on what kind of load you want to simulate, which in turn depends on why are you are doing the simulation. > Question #2: Is there a way to echo to the screen that the wait() is > happening and provide some indication of progress? > > To clarify this question, I had originally tried: > do {$x=wait; print ".";} until $x==-1; > > Thinking I would get a period printed during the loop. (silly me - > wait only comes back when a child process dies...). So you do get a period printed, once per child-exit. This is what I would want to do, anyway, but I seem to be in a philosophical minority in this regard (I hate progress indicators that merely indicate that the system clock is ticking, and nothing about the actual progress of the actual program.) I'm not sure what you originally thought it might do instead. Periods to be printed as fast as Perl possible can print them? > Of course if this is much to complicated a way to so a simulated load, > then so be it - just a plain old fork and wait works fine... And I > could always remove the /dev/null and actually see the called programs > output, which will tell me that it's still running... But I was > curious if there is a more elegant or at least interesting way of > doing this? If the waiting program is still running, and is still waiting, then the program it is waiting on must still be running. What other possibilities are there? If you really want the "time has not come to a halt" thing, then something like this: perl -le '$|=1; fork or do {sleep 4*$_; exit} foreach (1..3); \ $SIG{ALRM}=sub{print "."; alarm 1}; alarm 1; \ do {$x=wait; print "$x";} until $x==-1; \ alarm 0' .. .. .. 20287 .. .. .. .. 20288 .. .. .. .. 20289 -1 Xho -- -------------------- http://NewsReader.Com/ -------------------- The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact. |