From: Kevin Ar18 on

I'm not sure my previous message went through (I wasn't subscribe), so I'm gonna try again.

The multiprocessing module has 4 methods for sharing data between processes:
Queues
Pipes
Shared Memory Map
Server Process

Which of these use shared memory?

I understand that the 3rd (Shared Memory Map) does, but what about Queues?

Thanks,
Kevin

_________________________________________________________________
Hotmail is redefining busy with tools for the New Busy. Get more from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_2
From: MRAB on
Kevin Ar18 wrote:
> I'm not sure my previous message went through (I wasn't subscribe), so I'm gonna try again.
>
> The multiprocessing module has 4 methods for sharing data between processes:
> Queues
> Pipes
> Shared Memory Map
> Server Process
>
> Which of these use shared memory?
>
> I understand that the 3rd (Shared Memory Map) does, but what about Queues?
>
The documentation says:

"""class multiprocessing.Queue([maxsize])
Returns a process shared queue implemented using a pipe and a few
locks/semaphores. When a process first puts an item on the queue a
feeder thread is started which transfers objects from a buffer into the
pipe."""
From: John Nagle on
On 7/27/2010 12:30 PM, MRAB wrote:
> Kevin Ar18 wrote:
>> I'm not sure my previous message went through (I wasn't subscribe), so
>> I'm gonna try again.
>>
>> The multiprocessing module has 4 methods for sharing data between
>> processes:
>> Queues
>> Pipes
>> Shared Memory Map
>> Server Process
>>
>> Which of these use shared memory?
>>
>> I understand that the 3rd (Shared Memory Map) does, but what about
>> Queues?
>>
> The documentation says:
>
> """class multiprocessing.Queue([maxsize])
> Returns a process shared queue implemented using a pipe and a few
> locks/semaphores. When a process first puts an item on the queue a
> feeder thread is started which transfers objects from a buffer into the
> pipe."""

Python's "shared memory" between processes is un-Pythonic. You
can't share Python object, only C objects. The locking mechanisms
are very limited, and slow; locking actually takes place via
messages over pipes. There's no dynamic allocation in the shared area.

Realistically, if you're using Python, you're not that concerned
about compute speed. So don't bother with shared memory, which is
a performance optimization.

John Nagle