Prev: Mark built-in module as deprecated
Next: Colour TIFF support (PIL or otherwise)
From: Robert Kern on 13 Jun 2010 16:20
On 2010-06-13 14:17 , Stefan Behnel wrote:
> Stephen Hansen, 13.06.2010 21:05:
>> On 6/13/10 11:41 AM, Stefan Behnel wrote:
>>> Take a look at a) NumPy and b) Cython. You can also use Cython with the
>>> array module, but NumPy is a much more common way to deal with "number
>>> crunching routines", especially multi-dimentional arrays.
>>
>> Does Cython support Py3k yet? The OP seemed to be concerned about using
>> numpy due to it not yet being ready for 3+.
>
> I didn't read that from the post, but you may be right.
>
> Cython supports Py 2.3-3.1+, but NumPy still does not support Py3k (they
> are actually considering to rewrite parts of it in Cython to simplify
> the migration ;).

The latest SVN of numpy works in Python 3.1 already. There are still some things
to be fixed, but it's mostly done.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

From: Martin on 13 Jun 2010 16:36
On Jun 13, 6:15 pm, Thomas Jollans <tho...(a)jollans.com> wrote:
> Hi,
>
> I'm writing some buffer-centric number-crunching routines in C for
> Python code that uses array.array objects for storing/manipulating data.
> I would like to:
>
> 1. allocate a buffer of a certain size
> 2. fill it
> 3. return it as an array.
>
> I can't see any obvious way to do this with the array module, but I was
> hoping somebody here might be able to help. My best shot would be to:
>
> 1. create a bytearray with PyByteArray_FromStringAndSize(NULL, byte_len)
> 2. fill its buffer
> 3. initialize an array from the bytearray.
>
> The issue I have with this approach is that array will copy the data to
> its own buffer. I'd much rather create an array of a certain size, get a
> write buffer, and fill it directly -- is that possible?
>
> I expect that numpy allows this, but I don't really want to depend on
> numpy, especially as they haven't released a py3k version yet.
>
> -- Thomas

You want Numpy...

e.g.

import numpy as np
array = np.zeros(100, dtype=np.uint8)

then either something like this to fill it

for i in xrange(len(100)):
array[i] = 2

or

array = np.zeros(0)
for i in xrange(len(100)):
array = np.append(array, 2)


Mart
From: geremy condra on 13 Jun 2010 18:20
On Sun, Jun 13, 2010 at 1:20 PM, Robert Kern <robert.kern(a)gmail.com> wrote:
> On 2010-06-13 14:17 , Stefan Behnel wrote:
>>
>> Stephen Hansen, 13.06.2010 21:05:
>>>
>>> On 6/13/10 11:41 AM, Stefan Behnel wrote:
>>>>
>>>> Take a look at a) NumPy and b) Cython. You can also use Cython with the
>>>> array module, but NumPy is a much more common way to deal with "number
>>>> crunching routines", especially multi-dimentional arrays.
>>>
>>> Does Cython support Py3k yet? The OP seemed to be concerned about using
>>> numpy due to it not yet being ready for 3+.
>>
>> I didn't read that from the post, but you may be right.
>>
>> Cython supports Py 2.3-3.1+, but NumPy still does not support Py3k (they
>> are actually considering to rewrite parts of it in Cython to simplify
>> the migration ;).
>
> The latest SVN of numpy works in Python 3.1 already. There are still some
> things to be fixed, but it's mostly done.

This is excellent news, thanks for pointing this out.

Geremy Condra
From: Hrvoje Niksic on 14 Jun 2010 07:18
Thomas Jollans <thomas(a)jollans.com> writes:

> 1. allocate a buffer of a certain size
> 2. fill it
> 3. return it as an array.

The fastest and more robust approach (I'm aware of) is to use the
array.array('typecode', [0]) * size idiom to efficiently preallocate the
array, and then to get hold of the pointer pointing into array data
using the buffer interface.

Please send a message to capi-sig(a)python.org, a SIG specializing in the
Python/C API, if you need more help with implementing this.
From: Thomas Jollans on 14 Jun 2010 08:28
On 06/14/2010 01:18 PM, Hrvoje Niksic wrote:
> Thomas Jollans <thomas(a)jollans.com> writes:
>
>> 1. allocate a buffer of a certain size
>> 2. fill it
>> 3. return it as an array.
>
> The fastest and more robust approach (I'm aware of) is to use the
> array.array('typecode', [0]) * size idiom to efficiently preallocate the
> array, and then to get hold of the pointer pointing into array data
> using the buffer interface.

Ah, create a single-element array, and multiply that. That's not a bad
approach, the overhead is probably equivalent to what I have now:
currently, I create an uninitialized(!) bytes of the correct size, fill
it myself, and initialize an array from that. Both approaches have the
overhead of creating one extra Python object (bytes/single-element
array) and either copying one element over and over, or memcpy'ing the
whole buffer.

>
> Please send a message to capi-sig(a)python.org, a SIG specializing in the
> Python/C API, if you need more help with implementing this.

I'll probably subscribe to that list, thanks for the hint.

-- Thomas
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: Mark built-in module as deprecated
Next: Colour TIFF support (PIL or otherwise)