Prev: atexit/signal for non-interactive jobs
Next: type error raise
From: Stefan Behnel on 26 May 2010 16:26
Shashank Singh, 26.05.2010 21:48:
> What is the most efficient way of applying a function to all the elements of
> an iterable while discarding the
> result (i.e. operations are done only for side-effects).
>
> For example if I want to save all elements in a list of items (and am not
> interested in what save() returns), the
> simplest way is:
>
> itemlist = [i1, i2, i3....]
> for item in itemlist: item.save()
>
> It might be squeezing too much but is it possible to do it more efficiently
> by pushing the looping inside the C code
> and achieve some gains as is done by using map in place of a for loop when
> the return values need to be saved?

If all items have the exact same type, you can get away with an unbound method:

class MyType(object):
...
def safe(self):
...

itemlist = [ MyType() for i in range(20) ]

# this is what you want:

from itertools import imap
for _ in imap(MyType.save, itemlist): pass


But some people may consider this too ugly to actually use in real code.

Stefan

From: Peter Otten on 26 May 2010 17:09
Stefan Behnel wrote:

> Shashank Singh, 26.05.2010 21:48:
>> What is the most efficient way of applying a function to all the elements
>> of an iterable while discarding the
>> result (i.e. operations are done only for side-effects).
>>
>> For example if I want to save all elements in a list of items (and am not
>> interested in what save() returns), the
>> simplest way is:
>>
>> itemlist = [i1, i2, i3....]
>> for item in itemlist: item.save()
>>
>> It might be squeezing too much but is it possible to do it more
>> efficiently by pushing the looping inside the C code
>> and achieve some gains as is done by using map in place of a for loop
>> when the return values need to be saved?
>
> If all items have the exact same type, you can get away with an unbound
> method:
>
> class MyType(object):
> ...
> def safe(self):
> ...
>
> itemlist = [ MyType() for i in range(20) ]
>
> # this is what you want:
>
> from itertools import imap
> for _ in imap(MyType.save, itemlist): pass
>
>
> But some people may consider this too ugly to actually use in real code.

Which doesn't mean it can't get worse:

>>> from sys import maxint
>>> from operator import attrgetter
>>> from itertools import imap, islice
>>> class I:
.... def __init__(self, i):
.... self.i = i
.... def save(self):
.... print "saving", self.i
....
>>> items = [I(i) for i in range(5)]
>>> next(islice(imap(apply, imap(attrgetter("save"), items)), maxint,
maxint), None)
saving 0
saving 1
saving 2
saving 3
saving 4


See also the following thread:

http://mail.python.org/pipermail/python-list/2010-January/1233307.html

Most real-world applications should spend significantly more time inside the
save() methods than in the enclosing loop, so I'd expect the effect of this
kind of optimization on the speed of your program to be negligable.

Peter
From: Stefan Behnel on 26 May 2010 17:26
Shashank Singh, 26.05.2010 23:16:
> I probably didn't state the problem properly. I was assuming the
> availability of a static method that could be passed on to map based
> solution (or imap for that matter).
>
> The question was, if one wants to apply a function on each member of list
> and discard the return value, is it possible to do it more efficiently than
> having a for loop in python and applying the function of each of the
> members?
>
> Take this run:
>
> from itertools import imap
> from timeit import Timer
>
>
> def save(x): 2 * x
>
> from itertools import imap
> from timeit import Timer
>
>
> def save(x): 2 * x
>
> def f1(): map(save, range(1000)) #simple map
>
> def f2():
> for _ in imap(save, range(1000)): pass #imap
>
> def f3():
> for x in range(1000):save(x) #simple iteration
>
> t1 = Timer("f1()", "from __main__ import f1")
> print "f1", t1.timeit(number=1000)
>
> t2 = Timer("f2()", "from __main__ import f2")
> print "f2", t2.timeit(number=1000)
>
> t3 = Timer("f3()", "from __main__ import f3")
> print "f3", t3.timeit(number=1000)
>
> The output for one run was:
>
> f1 0.393015034034
> f2 0.476230638252
> f3 0.376324923978
>
> => simple for loop performs better than map/imap.

Interesting. So you've found the fastest solution already. If a simple for
loop is both the most readable and the fastest way to do it, it sounds to
me like you have solved your problem yourself.


> Another problem with
> map/imap is that the memory cost is dependent on the length of the list
> (because of the intermediate mapped list stored) which is not the case for
> simple for loop.

.... neither is it the case for the imap() solution - but that still proves
to be slower in your timings.

BTW, assuming that you are using Python 2, you'd better use xrange() in
benchmarks. The list creation of range() can easily spoil your timings.

Stefan

 | 
Pages: 1
Prev: atexit/signal for non-interactive jobs
Next: type error raise