From: Cameron Simpson on
[ Please don't top post. Post below so that things read like a
conversation. (And trim excess quoted junk.) It doesn't take long and
makes things a lot easier for your readers. ]

On 25Jul2010 04:41, Navkirat Singh <navkirats(a)gmail.com> wrote:
| On 25-Jul-2010, at 4:37 AM, Navkirat Singh wrote:
| > I have been meddling around with forking and multiprocessing. Now
| > both of them spawn new processes from parent (atleast from what I have
| > understood). I have been able to reproduce a zombie state in a fork with:
| >
| > import os,time
| > print('before fork',os.getpid())
| > pid = os.fork()
| > if pid:
| > print('child: ',pid)
| > time.sleep(120)
| >
| > Now doing a ps ax | grep <pid> I can find a zombie child process,
| > but I am not being able to reproduce the same with multiprocessing. Am
| > I missing something? Or multiprocessing does not have the zombie problem?
|
| OK I wanted zombie processes and have been able to regenerate them with multiprocessing. Now lets see how I can handle them.

A zombie process is a fairly common thing and nothing to panic about
usually.

When you have a normal, running process it has an entry in the
system process table (that "ps" shows) that points at it and shows its
state.

When a process exits, its exit status (exit code, whether it exited
normally or from a signal, and if so which signal, some stats) _remain_
in the process table as a record of the exit status. This is shown as a
"zombie" process in "ps" - a process which has exited and whose exit
status is hanging around waiting to be collected.

After the process' parent has issued a wait() call to collect the status
the process slot if released and the :zombie" is gone.

In your fork() example above, if that's the whole program, the child
exits immediately and the parent has not waited for it.

If the parent exits without waiting for all its children, the init
process (process 1) inherits them, and it collects the statuses.

Having a few zombies lying around isn't a disaster provided your program
does get around to waiting for them at some point (or if your main
program is short lived, so they get reaped by init after it exits).
It's only a process table slot after all - little memory is needed for
it.

A program that spawns a lot of children and never waits for them _is_
bad in the long run. Process ids are often 2-byte integers on many
systems for it's quite feasible to fill the table, and depending on the
kernel's process table implementation a big process table might have other
performance side effects.

Cheers,
--
Cameron Simpson <cs(a)zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I'm behind a corporate Exchange Server which seems to have
changed recently to converting everything it sees to HTML.
How embarrassing. - Dan Espen <despen(a)telcordia.com>
From: Navkirat Singh on

On 25-Jul-2010, at 5:25 AM, Cameron Simpson wrote:

> [ Please don't top post. Post below so that things read like a
> conversation. (And trim excess quoted junk.) It doesn't take long and
> makes things a lot easier for your readers. ]
>
> On 25Jul2010 04:41, Navkirat Singh <navkirats(a)gmail.com> wrote:
> | On 25-Jul-2010, at 4:37 AM, Navkirat Singh wrote:
> | > I have been meddling around with forking and multiprocessing. Now
> | > both of them spawn new processes from parent (atleast from what I have
> | > understood). I have been able to reproduce a zombie state in a fork with:
> | >
> | > import os,time
> | > print('before fork',os.getpid())
> | > pid = os.fork()
> | > if pid:
> | > print('child: ',pid)
> | > time.sleep(120)
> | >
> | > Now doing a ps ax | grep <pid> I can find a zombie child process,
> | > but I am not being able to reproduce the same with multiprocessing. Am
> | > I missing something? Or multiprocessing does not have the zombie problem?
> |
> | OK I wanted zombie processes and have been able to regenerate them with multiprocessing. Now lets see how I can handle them.
>
> A zombie process is a fairly common thing and nothing to panic about
> usually.
>
> When you have a normal, running process it has an entry in the
> system process table (that "ps" shows) that points at it and shows its
> state.
>
> When a process exits, its exit status (exit code, whether it exited
> normally or from a signal, and if so which signal, some stats) _remain_
> in the process table as a record of the exit status. This is shown as a
> "zombie" process in "ps" - a process which has exited and whose exit
> status is hanging around waiting to be collected.
>
> After the process' parent has issued a wait() call to collect the status
> the process slot if released and the :zombie" is gone.
>
> In your fork() example above, if that's the whole program, the child
> exits immediately and the parent has not waited for it.
>
> If the parent exits without waiting for all its children, the init
> process (process 1) inherits them, and it collects the statuses.
>
> Having a few zombies lying around isn't a disaster provided your program
> does get around to waiting for them at some point (or if your main
> program is short lived, so they get reaped by init after it exits).
> It's only a process table slot after all - little memory is needed for
> it.
>
> A program that spawns a lot of children and never waits for them _is_
> bad in the long run. Process ids are often 2-byte integers on many
> systems for it's quite feasible to fill the table, and depending on the
> kernel's process table implementation a big process table might have other
> performance side effects.
>
> Cheers,
> --
> Cameron Simpson <cs(a)zip.com.au> DoD#743
> http://www.cskk.ezoshosting.com/cs/
>
> I'm behind a corporate Exchange Server which seems to have
> changed recently to converting everything it sees to HTML.
> How embarrassing. - Dan Espen <despen(a)telcordia.com>

Thanks a lot for all the information. It cleared a lot of my doubts.

Nav
From: Lawrence D'Oliveiro on
In message <mailman.1127.1280014712.1673.python-list(a)python.org>, Chris
Rebert wrote:

> "Paging Dr. Frankenstein. Dr. Frankenstein to the lab. Paging Dr.
> Frankenstein."
>
> Most people try to /avoid/ making zombies.

Is there some connection between Frankenstein and zombies?
From: Chris Rebert on
On Mon, Jul 26, 2010 at 3:30 AM, Lawrence D'Oliveiro
<ldo(a)geek-central.gen.new_zealand> wrote:
> In message <mailman.1127.1280014712.1673.python-list(a)python.org>, Chris
> Rebert wrote:
>
>> "Paging Dr. Frankenstein. Dr. Frankenstein to the lab. Paging Dr.
>> Frankenstein."
>>
>> Most people try to /avoid/ making zombies.
>
> Is there some connection between Frankenstein and zombies?

His monster is zombie-ish; it's made of reanimated corpse parts. It's
also mentioned in Wikipedia's "Zombie" article as influencing future
works about the undead.

Cheers,
Chris
--
Not a perfect comedic reference, I admit.
From: Michele Simionato on
On Jul 25, 1:11 am, Navkirat Singh <navkir...(a)gmail.com> wrote:
> OK I wanted zombie processes and have been able to regenerate them with multiprocessing. Now lets see how I can handle them.

The multiprocessing docs say:

"""
Joining zombie processes

On Unix when a process finishes but has not been joined it becomes a
zombie. There should never be very many because each time a new
process starts (or active_children() is called) all completed
processes which have not yet been joined will be joined. Also calling
a finished process’s Process.is_alive() will join the process. Even so
it is probably good practice to explicitly join all the processes that
you start.
"""