Prev: FAQ 5.1 How do I flush/unbuffer an output filehandle? Why must I do this?
Next: what cpu core is running the script?
From: Peter Valdemar Mørch on 29 Jun 2010 03:06
On Jun 28, 5:07 pm, Willem <wil...(a)turtle.stack.nl> wrote:
> Isn't there some easier method, where you don't have to screw around with
> output maps at all ?
>
> If the following API would work, that would be the easiest, IMO:
>
>   my @result = async_map { do_something($_) } @array;
>
> Where async_map takes care of all the details of creating the threads,
> gathering all the output, et cetera.  Or does that already exist ?

It doesn't pre-exist to my knowledge. Not with such a simple syntax.

Ben wrote (in another branch of this tread,
http://groups.google.com/group/comp.lang.perl.misc/msg/0dbec9f2d0e37750
):
> It's now more analogous to map than foreach, but I don't see that as a
> problem.

Given these two inputs (thank you!), I propose an addition to $pl-
>foreach and $pl->while:

my @result = $pl->map(sub { do_something($_) }) @array;

And that will be $pl->map(sub {}) instead of async_map {} so the
object holds the number of processes to use. Alternatively, async_map
would have to be passed the number of processes to use. Which is also
a possibility. Or the number of processes is shared among all
Parallel::Loop async_map calls (which I like less).

> (The simple implementation is only a few lines of code, but it could
> then be easily extended to use a limited number of threads, or keep
> a thread pool handy, or something like that.)

The problem with a thread pool is that then we need to keep all
variables synchronized between them. And I'm focusing on forking - not
threads - here.

But yeah, it isn't that difficult to write. Already, there is more pod
than code! :-) There have just been so many instances already where I
find myself thinking: "This loop could and should be parallelized. But
(I'm too lazy|the schedule is too tight|who cares) right now."

Peter
From: Ted Zlatanov on 29 Jun 2010 10:47
On Fri, 25 Jun 2010 23:34:14 +0100 Ben Morrow <ben(a)morrow.me.uk> wrote:

BM> Personally I find

BM> my %output :shared;

BM> for my $i (@input) {
BM> async {
BM> $output{$i} = do_some_hefty_calculation($i);
BM> }
BM> }

BM> somewhat clearer, but that's just a matter of taste. (With 5.10
BM> presumably a 'my $_' would make $_ work too.)

I personally don't like "inline tagged" code blocks as much as passing
them off to a library subroutine. Inline tagging IMO creates spaghetti
code and is harder to refactor. But I can see the appeal :)

On Tue, 29 Jun 2010 00:06:41 -0700 (PDT) Peter Valdemar M�rch <4ux6as402(a)sneakemail.com> wrote:

PVM> The problem with a thread pool is that then we need to keep all
PVM> variables synchronized between them. And I'm focusing on forking -
PVM> not threads - here.

Please don't try to make your module do everything for everyone. It's
OK to say "it won't support XYZ." Do a few things well rather than many
things badly.

Ted
First  |  Prev  | 
Pages: 1 2 3 4
Prev: FAQ 5.1 How do I flush/unbuffer an output filehandle? Why must I do this?
Next: what cpu core is running the script?