Prev: NEWBIE: how exclude files in ftp
Next: NEWBIE: how exclude files in ftp
From: Chad on 17 Jun 2010 13:15
On Jun 14, 9:20 am, John Kelly <j...(a)isp2dial.com> wrote:
> On Mon, 14 Jun 2010 08:34:47 -0700 (PDT), srikanth
>
>
>
>
>
> <srikanth0...(a)gmail.com> wrote:
> >Hi All,
> >I need to browse some URLs in browser. Here I am having 600+ URLs to
> >browse. I have written a shell script to do this but the script is not
> >working properly. Here is my script.
>
> >#!/bin/bash
> >if [ -z "$1" ]
> >then
> > printf "Provide input text file to process the URLs\n"
> > exit 0
> >fi
> >{
> >   for i in `cat $1`
> >   do
> >   echo "`xdg-open $i`"
> >  done
> >}
> >exit 0
>
> a for loop is OK with a small number of elements, but when the data set
> is large, a while read loop is better.
>
> while read; do
>     xdg-open "$REPLY"
> done <$1
>
>

Why is a while read loop better for large a data set?


Chad
From: John Kelly on 17 Jun 2010 13:29
On Thu, 17 Jun 2010 10:15:47 -0700 (PDT), Chad <cdalten(a)gmail.com>
wrote:

>> a for loop is OK with a small number of elements, but when the data set
>> is large, a while read loop is better.
>>
>> while read; do
>> � � xdg-open "$REPLY"
>> done <$1

>Why is a while read loop better for large a data set?

Because the for loop reads the whole file into memory at once. Like
shooting yourself in the foot with Perl, heh.

The while read loop reads the whole file too, but only stores one line
at a time into memory. In this example, the application can only work
with one line at a time, so there is no value cramming the whole file
into memory all at once.

The while read loop is a more common sense way to process a large data
set. Don't be a memory hog. Be considerate of other system users, even
if they're just programs and not people.

If you're working with a smaller list of items, say 40 or less, then a
for loop makes good sense.


--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php

From: Seebs on 17 Jun 2010 15:42
On 2010-06-17, Chad <cdalten(a)gmail.com> wrote:
> On Jun 14, 9:20�am, John Kelly <j...(a)isp2dial.com> wrote:
>> a for loop is OK with a small number of elements, but when the data set
>> is large, a while read loop is better.

> Why is a while read loop better for large a data set?

Imagine that the data set is, say, 2GB.

The for loop has to parse an entire 2GB string into 2GB of smaller
strings (plus another gigabyte or so of overhead for data structures),
and have all of that in memory.

The while loop has to keep an 8KB buffer or so full of stuff and read single
lines at a time.

It's a LOT more efficient, in that case.

(All numbers approximate to give you an idea.)

Both are potentially vulnerable to weirdness in cases where there's newlines
or spaces or whatever, but the while loop is a bit more robust usually.

-s
--
Copyright 2010, all wrongs reversed. Peter Seebach / usenet-nospam(a)seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
First  |  Prev  | 
Pages: 1 2 3 4 5 6 7
Prev: NEWBIE: how exclude files in ftp
Next: NEWBIE: how exclude files in ftp