Prev: Pickling an extension type subclasses problems
Next: timer for a function
From: mk on 5 Feb 2010 10:25
Hello everyone,

I have a problem with a threaded program: it frequently hangs on sys.exit.

The problem is that my program uses threads which in turn use paramiko
library, which itself is threaded.

I try to gracefully close the threads (below), but it doesn't always
work, if paramiko calls happen to be at stage of negotiating ssh
connection or smth similar.

The only workable solution I have is a program sending itself SIGKILL,
which makes it terminated by OS (I think so).

Is there any way to brutally close the threads? I know that normally
that should not be done, but shutdown when you don't care about writing
out to disk is the only situation where it doesn't apply.

def ctrlchandler(signal, frame):
print
print ENDC + "Terminating on Ctrl-C, closing threads for:",
while queue:
for ip, th in queue:
print ip,
try:
lock.acquire()
th.abort = True
lock.release()
except RuntimeError:
pass
queue.remove((ip,th))
print
pid = os.getpid()
print "Finished closing threads."
# suicide - it's the only way of preventing frequent hangup on sys.exit
os.kill(pid, SIGTERM)
os.kill(pid, SIGKILL)
sys.exit(0)

From: Aahz on 11 Feb 2010 11:43
In article <mailman.1961.1265383809.28905.python-list(a)python.org>,
mk <mrkafk(a)gmail.com> wrote:
>
>I have a problem with a threaded program: it frequently hangs on sys.exit.
>
>The problem is that my program uses threads which in turn use paramiko
>library, which itself is threaded.
>
>I try to gracefully close the threads (below), but it doesn't always
>work, if paramiko calls happen to be at stage of negotiating ssh
>connection or smth similar.
>
>The only workable solution I have is a program sending itself SIGKILL,
>which makes it terminated by OS (I think so).

You can also use os._exit().
--
Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
From: ssteinerX on 11 Feb 2010 13:57

On Feb 11, 2010, at 11:43 AM, Aahz wrote:

> In article <mailman.1961.1265383809.28905.python-list(a)python.org>,
> mk <mrkafk(a)gmail.com> wrote:
>>
>> I have a problem with a threaded program: it frequently hangs on sys.exit.
>>
>> The problem is that my program uses threads which in turn use paramiko
>> library, which itself is threaded.
>>
>> I try to gracefully close the threads (below), but it doesn't always
>> work, if paramiko calls happen to be at stage of negotiating ssh
>> connection or smth similar.
>>
>> The only workable solution I have is a program sending itself SIGKILL,
>> which makes it terminated by OS (I think so).
>
> You can also use os._exit().

This just came up on the Twisted IRC channel last night and came to the same conclusion. Python's going to wait for threads to terminate and if they're not going to, neither is Python. os._exit() came up as the 'last resort' way out of the app.

S

From: mk on 11 Feb 2010 15:17
Stephen Hansen wrote:
> I use threads all the time (well, for certain types of workloads) and
> have never seen this.
>
> Are your threads daemon threads? The only time I've seen sys.exit() not
> close out my program is when I'm launching non-daemon threads on accident.

The snag is that my program is using calls to another threaded library
(paramiko) which does lots of stuff with sockets and communication
(SSH). Paramiko in turn calls Crypto which has compiled C extensions. I
really don't know what's going on in there, just guessing that this
might trigger this behavior.

> Now, I do get some slightly random errors on close due to threads doing
> stuff or returning from doing stuff while the shutdown procedure is
> going on, but I don't really care, cuz, well, I'm shutting everything
> down and nothing worth knowing is going on in the rest of the program.

Same as me, except I get lots of exceptions in threads if I shut down
with sys.exit. SIGTERM somehow gets around this problem.

I'll try os._exit.

Regards,
mk

From: mk on 11 Feb 2010 15:18
Aahz wrote:

> You can also use os._exit().

Thanks!


 |  Next  |  Last
Pages: 1 2
Prev: Pickling an extension type subclasses problems
Next: timer for a function