Prev: Castrated traceback in sys.exc_info()
Next: url2lib (windows 7) does not notice when network reconnects (getaddrinfoproblem)
From: Tim Arnold on 17 Mar 2010 09:30 Hi, I'm checking to see if multiprocessing works on freebsd for any version of python. My server is about to get upgraded from 6.3 to 8.0 and I'd sure like to be able to use multiprocessing. I think the minimal test would be: --------------------- import multiprocessing q = multiprocessing.Queue() --------------------- with 6.3, I get File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line 212, in Queue from multiprocessing.queues import Queue File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22, in <module> from multiprocessing.synchronize import Lock, BoundedSemaphore, Semaphore, Condition File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. thanks for any info, --Tim Arnold
From: Philip Semanchuk on 17 Mar 2010 11:26 On Mar 17, 2010, at 9:30 AM, Tim Arnold wrote: > Hi, > I'm checking to see if multiprocessing works on freebsd for any > version of python. My server is about to get upgraded from 6.3 to 8.0 > and I'd sure like to be able to use multiprocessing. > > I think the minimal test would be: > --------------------- > import multiprocessing > q = multiprocessing.Queue() > --------------------- > > with 6.3, I get > > File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line > 212, in Queue > from multiprocessing.queues import Queue > File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22, > in <module> > from multiprocessing.synchronize import Lock, BoundedSemaphore, > Semaphore, Condition > File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line > 33, in <module> > " function, see issue 3770.") > ImportError: This platform lacks a functioning sem_open > implementation, therefore, the required synchronization primitives > needed will not function, see issue 3770. Hi Tim, Under FreeBSD 8/Python 2.6.2 I get the same result, unfortunately. That's a pity because sem_open works under FreeBSD >= 7.2 as we discussed before. Issue 3770 is closed with the note, "we've removed hard-coded platform variables for a better autoconf approach." I'm using the Python built from FreeBSD's ports, and the note makes me think that it's possible that if I built my own Python from the Python.org tarball rather than ports the problem would go away due to autoconf magic. I don't have the time to offer to do this for you, unfortunately. But why not install FreeBSD 8 under VirtualBox or somesuch and give it a go yourself? A couple of quirks I noted related to FreeBSD & POSIX IPC that you might find useful -- - The sem and mqueuefs kernel modules must be loaded, otherwise you'll get a message like this when you try to create a semaphore or message queue: Bad system call: 12 (core dumped) Under 8.0 they're loaded by default, I think. - C apps that want to use message queues must link to the realtime libs (pass -lrt to the linker). This tripped me up for a while. Linking to the realtime libs is required for all POSIX IPC calls under Linux; FreeBSD does not require it for semaphores or shared mem, only message queues. Hope this helps Philip
From: Tim Arnold on 17 Mar 2010 13:57 On Mar 17, 11:26 am, Philip Semanchuk <phi...(a)semanchuk.com> wrote: > On Mar 17, 2010, at 9:30 AM, Tim Arnold wrote: > > > > > > > Hi, > > I'm checking to see if multiprocessing works on freebsd for any > > version of python. My server is about to get upgraded from 6.3 to 8.0 > > and I'd sure like to be able to use multiprocessing. > > > I think the minimal test would be: > > --------------------- > > import multiprocessing > > q = multiprocessing.Queue() > > --------------------- > > > with 6.3, I get > > > File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line > > 212, in Queue > > from multiprocessing.queues import Queue > > File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22, > > in <module> > > from multiprocessing.synchronize import Lock, BoundedSemaphore, > > Semaphore, Condition > > File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line > > 33, in <module> > > " function, see issue 3770.") > > ImportError: This platform lacks a functioning sem_open > > implementation, therefore, the required synchronization primitives > > needed will not function, see issue 3770. > > Hi Tim, > Under FreeBSD 8/Python 2.6.2 I get the same result, unfortunately. > That's a pity because sem_open works under FreeBSD >= 7.2 as we > discussed before. > > Issue 3770 is closed with the note, "we've removed hard-coded platform > variables for a better autoconf approach." I'm using the Python built > from FreeBSD's ports, and the note makes me think that it's possible > that if I built my own Python from the Python.org tarball rather than > ports the problem would go away due to autoconf magic. I don't have > the time to offer to do this for you, unfortunately. But why not > install FreeBSD 8 under VirtualBox or somesuch and give it a go > yourself? > > A couple of quirks I noted related to FreeBSD & POSIX IPC that you > might find useful -- > - The sem and mqueuefs kernel modules must be loaded, otherwise you'll > get a message like this when you try to create a semaphore or message > queue: > Bad system call: 12 (core dumped) > > Under 8.0 they're loaded by default, I think. > > - C apps that want to use message queues must link to the realtime > libs (pass -lrt to the linker). This tripped me up for a while. > Linking to the realtime libs is required for all POSIX IPC calls under > Linux; FreeBSD does not require it for semaphores or shared mem, only > message queues. > > Hope this helps > Philip Hi Philip, Thanks for that information (esp the linker info). Once the machine is upgraded, I'll try building python from the tarball. I'll post back here with the results. here's hoping! thanks, --Tim
From: Martin P. Hellwig on 17 Mar 2010 15:25 On 03/17/10 13:30, Tim Arnold wrote: > Hi, > I'm checking to see if multiprocessing works on freebsd for any > version of python. My server is about to get upgraded from 6.3 to 8.0 > and I'd sure like to be able to use multiprocessing. > > I think the minimal test would be: > --------------------- > import multiprocessing > q = multiprocessing.Queue() > --------------------- > > with 6.3, I get > > File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line > 212, in Queue > from multiprocessing.queues import Queue > File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22, > in<module> > from multiprocessing.synchronize import Lock, BoundedSemaphore, > Semaphore, Condition > File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line > 33, in<module> > " function, see issue 3770.") > ImportError: This platform lacks a functioning sem_open > implementation, therefore, the required synchronization primitives > needed will not function, see issue 3770. > Build mine from ports, with the following options (notice SEM & PTH): [martin(a)aspire8930 /usr/home/martin]$ cat /var/db/ports/python26/options # This file is auto-generated by 'make config'. # No user-servicable parts inside! # Options for python26-2.6.4 _OPTIONS_READ=python26-2.6.4 WITH_THREADS=true WITHOUT_HUGE_STACK_SIZE=true WITH_SEM=true WITH_PTH=true WITH_UCS4=true WITH_PYMALLOC=true WITH_IPV6=true WITHOUT_FPECTL=true [martin(a)aspire8930 /usr/home/martin]$ uname -a FreeBSD aspire8930 8.0-STABLE FreeBSD 8.0-STABLE #3: Wed Feb 3 17:01:18 GMT 2010 martin(a)aspire8930:/usr/obj/usr/src/sys/ASPIRE8930 amd64 [martin(a)aspire8930 /usr/home/martin]$ python Python 2.6.4 (r264:75706, Mar 17 2010, 18:44:24) [GCC 4.2.1 20070719 [FreeBSD]] on freebsd8 Type "help", "copyright", "credits" or "license" for more information. >>> import multiprocessing as mp >>> queue = mp.Queue() >>> hth -- mph
From: Tim Arnold on 18 Mar 2010 12:38
"Martin P. Hellwig" <martin.hellwig(a)dcuktec.org> wrote in message news:hnrabj$c4v$1(a)news.eternal-september.org... > On 03/17/10 13:30, Tim Arnold wrote: >> Hi, >> I'm checking to see if multiprocessing works on freebsd for any >> version of python. My server is about to get upgraded from 6.3 to 8.0 >> and I'd sure like to be able to use multiprocessing. >> >> I think the minimal test would be: >> --------------------- >> import multiprocessing >> q = multiprocessing.Queue() >> --------------------- >> >> with 6.3, I get >> >> File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line >> 212, in Queue >> from multiprocessing.queues import Queue >> File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22, >> in<module> >> from multiprocessing.synchronize import Lock, BoundedSemaphore, >> Semaphore, Condition >> File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line >> 33, in<module> >> " function, see issue 3770.") >> ImportError: This platform lacks a functioning sem_open >> implementation, therefore, the required synchronization primitives >> needed will not function, see issue 3770. >> > > Build mine from ports, with the following options (notice SEM & PTH): > [martin(a)aspire8930 /usr/home/martin]$ cat /var/db/ports/python26/options > # This file is auto-generated by 'make config'. > # No user-servicable parts inside! > # Options for python26-2.6.4 > _OPTIONS_READ=python26-2.6.4 > WITH_THREADS=true > WITHOUT_HUGE_STACK_SIZE=true > WITH_SEM=true > WITH_PTH=true > WITH_UCS4=true > WITH_PYMALLOC=true > WITH_IPV6=true > WITHOUT_FPECTL=true > > [martin(a)aspire8930 /usr/home/martin]$ uname -a > FreeBSD aspire8930 8.0-STABLE FreeBSD 8.0-STABLE #3: Wed Feb 3 17:01:18 > GMT 2010 martin(a)aspire8930:/usr/obj/usr/src/sys/ASPIRE8930 amd64 > [martin(a)aspire8930 /usr/home/martin]$ python > Python 2.6.4 (r264:75706, Mar 17 2010, 18:44:24) > [GCC 4.2.1 20070719 [FreeBSD]] on freebsd8 > Type "help", "copyright", "credits" or "license" for more information. > >>> import multiprocessing as mp > >>> queue = mp.Queue() > >>> > > hth > -- > mph Hi Martin, thanks very much for posting that. All I can say is YAY! I'm really looking forward to my machine's upgrade now! thanks, --Tim |