Speed of reading some MB of data using qx(...)
in [Perl]
|
From: Wolfram Humann on 22 Jul 2010 09:02 I have a program that processes PDF files by converting them to Postscript, read the ps and do something with it. I use pdftops (from xpdf) for the pdf->ps conversion and retrieve the result like this: $ps_text = qx( pdftops $infile - ); On win32 using strawberry perl (tried 5.10 and 5.12) this takes much more time than I expected so I did a test and first converted the PDF to Postscript, then read the Postscript (about 12 MB) like this (cat on win32 provided by cygwin): perl -E" $t = qx(cat psfile.ps); say length $t " This takes about 16 seconds on win32 but only <1 seconds on Linux. I was afraid that this might be a 'binmode' problem so I also tried this: perl -E" open $in,'cat psfile.ps |'; binmode $in; local $/; $t=<$in>; say length $t " But the effect is the same: fast on linux, slow on win32. Besides bashing win32 :-) and ideas for reason and (possibly) cure? Wolfram
From: Uri Guttman on 22 Jul 2010 12:39 >>>>> "WH" == Wolfram Humann <w.c.humann(a)arcor.de> writes: WH> perl -E" $t = qx(cat psfile.ps); say length $t " WH> This takes about 16 seconds on win32 but only <1 seconds on Linux. I WH> was afraid that this might be a 'binmode' problem so I also tried WH> this: WH> perl -E" open $in,'cat psfile.ps |'; binmode $in; local $/; $t=<$in>; WH> say length $t " WH> But the effect is the same: fast on linux, slow on win32. Besides WH> bashing win32 :-) and ideas for reason and (possibly) cure? you also bashed File::Slurp in a bug report you sent me. obviously it is a winblows issue. one possibility is that winblows does poor process context switching and piping between processes causes lots of that. there may be other reasons but i stay away from redmond. uri -- Uri Guttman ------ uri(a)stemsystems.com -------- http://www.sysarch.com -- ----- Perl Code Review , Architecture, Development, Training, Support ------ --------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
From: Ilya Zakharevich on 22 Jul 2010 17:14 On 2010-07-22, Uri Guttman <uri(a)StemSystems.com> wrote: > WH> But the effect is the same: fast on linux, slow on win32. Besides > WH> bashing win32 :-) and ideas for reason and (possibly) cure? > > you also bashed File::Slurp in a bug report you sent me. obviously it is > a winblows issue. one possibility is that winblows does poor process > context switching and piping between processes causes lots of > that. there may be other reasons but i stay away from redmond. Disable your network adapters, and switch off virus checking. Recheck speed. Hope this helps, Ilya
From: Wolfram Humann on 23 Jul 2010 04:53 On Jul 22, 6:39 pm, "Uri Guttman" <u...(a)StemSystems.com> wrote: > >>>>> "WH" == Wolfram Humann <w.c.hum...(a)arcor.de> writes: > > WH> perl -E" $t = qx(cat psfile.ps); say length $t " > > WH> This takes about 16 seconds on win32 but only <1 seconds on Linux.. I > WH> was afraid that this might be a 'binmode' problem so I also tried > WH> this: > > WH> perl -E" open $in,'cat psfile.ps |'; binmode $in; local $/; $t=<$in>; > WH> say length $t " > > WH> But the effect is the same: fast on linux, slow on win32. Besides > WH> bashing win32 :-) and ideas for reason and (possibly) cure? > > you also bashed File::Slurp in a bug report you sent me. obviously it is > a winblows issue. one possibility is that winblows does poor process > context switching and piping between processes causes lots of > that. there may be other reasons but i stay away from redmond. > > uri Uri, it was certainly not my intention to bash File::Slurp in my bug report -- and I when I re-read what I wrote I don't think I did. If I did bash, I am sorry for it (and would be grateful for explanation from your side). I'm not a native speaker; maybe things came out different from what I intended. Besides that, I think the two issues (my bug report and my post here) are unrelated. It looks like File::Slurp needs to do substitutions on win32. I was not aware of that as I do not seem to need them when I read line-by-line on win32. Those substitutions naturally suffer from the use of $& elsewhere in the program. I think it would be o.k. to mention this briefly in the docs for File ::Slurp. This is a special case and you might disagree that it should be in the docs. In that case, just reject and close my bug report. Regards, Wolfram
From: Uri Guttman on 23 Jul 2010 05:00 >>>>> "WH" == Wolfram Humann <w.c.humann(a)arcor.de> writes: WH> On Jul 22, 6:39�pm, "Uri Guttman" <u...(a)StemSystems.com> wrote: >> you also bashed File::Slurp in a bug report you sent me. obviously it is >> a winblows issue. one possibility is that winblows does poor process >> context switching and piping between processes causes lots of >> that. there may be other reasons but i stay away from redmond. WH> it was certainly not my intention to bash File::Slurp in my bug report WH> -- and I when I re-read what I wrote I don't think I did. If I did WH> bash, I am sorry for it (and would be grateful for explanation from WH> your side). I'm not a native speaker; maybe things came out different WH> from what I intended. WH> Besides that, I think the two issues (my bug report and my post here) WH> are unrelated. It looks like File::Slurp needs to do substitutions on WH> win32. I was not aware of that as I do not seem to need them when I WH> read line-by-line on win32. Those substitutions naturally suffer from WH> the use of $& elsewhere in the program. I think it would be o.k. to WH> mention this briefly in the docs for File ::Slurp. This is a special WH> case and you might disagree that it should be in the docs. In that WH> case, just reject and close my bug report. i don't think i should worry about someone else using $& in their code. most modules use s/// with grabbing and will suffer for the same reason. this is a silly idea to warn about something i have no control over. but you showed with your slurp code test that it was slower on winblows without my module. so telling me to warn about using $& is wrong as it isn't the reason for the slowdown. winblows doesn't do forks/pipe well since it doesn't like process context switching. they push threads which suck for other reasons. on most unix flavors process and thread context switching are about equally fast. uri -- Uri Guttman ------ uri(a)stemsystems.com -------- http://www.sysarch.com -- ----- Perl Code Review , Architecture, Development, Training, Support ------ --------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
|
Next
|
Last
Pages: 1 2 3 4 5 6 7 Prev: Extracting table in html page Next: Use of uninitialised value, how to avoid? |