fork + exec; what are the possible resource leaks?
in [Unix Programming]
|
From: Rainer Weikusat on 20 Apr 2010 07:59 David Given <dg(a)cowlark.com> writes: > On 19/04/10 20:39, Rainer Weikusat wrote: > [...] >> Yes. The actual solution is still (and will remain forever): DO NOT DO >> THIS. This cannot be that complicated, can it? > > So, basically, you're saying, don't use fopen()? I am hard-pressed to try a reply in simplified kindergarten language here, but I am not entirely certain if you didn't perhaps honestly misinterprete my text completely. See, if you desire to change the default policy, it is necessary that you have sufficient control of your environment to actually change the policy without 'something' you don't control causing the yet unchanged default policy to be applied. The environment we are "discussing" here has been carefully defined as to offer no opportunity for changing the default policy, consequently, the default policy cannot be changed. How do you proprose to deal with a multithreaded library which remaps the text segment of your application with r/w permission and overwrites your code with one million copies of "Oft glaubt der Mensch, wenn er nur Worte hoert, es muesse sich dabei doch auch was denken lassen" (Goethe, Faust I)? [...] > We all know that the default for close-on-exec is wrong; That's your opinion on this topic. Apparently, the opinion of the people who designed the interface was different. My opinion on this topic, being a completely pragmatic person with no desire to evangelize in favor of the one true someone's way is that "whatever the default happens to be, it will be inconvenient in some situations".
From: Geoff Clare on 20 Apr 2010 08:47 David Given wrote: > My understanding of the problem is that *any* call to fopen(), in a > multithreaded application, may result in a leaked file descriptor > (because even if the caller to fopen() remembers to call > fcntl(fileno(fp), F_SETFD, FD_CLOEXEC) afterwards, there's still a > window whereby another thread calling fork() might propagate the file > descriptor). > > Is this correct? > > If so, is there any actual solution? Because fopen() is not going to go > away. Here are three possible solutions: 1. Instead of using fopen(), use open() with O_CLOEXEC and then fdopen() to get a stream from the file descriptor. Obviously this is only portable to systems that provide O_CLOEXEC, but O_CLOEXEC has been mandated by POSIX since 2008. 2. Fork before creating any threads. Then later on tell the child what to do (via IPC) instead of forking at that point. (The child can safely fork another child to do the work if necessary.) Obviously this requires cooperation from the other threads that would have called fork(), so it may not be possible when using 3rd party libraries. 3. Use a mutex to ensure fork() cannot be called concurrently with fopen(). Again this requires cooperation. -- Geoff Clare <netnews(a)gclare.org.uk>
From: William Ahern on 20 Apr 2010 14:48 Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups(a)ntlworld.com> wrote: > [-- text/html, encoding 7bit, charset: ISO-8859-1, 67 lines --] No doubt Windows has myriad techniques and devices for handling thread issues. In fact, some of them clearly aren't resolvable. Windows Vista seems to terminate a process which has a thread which tries to access a critical section left inconsistent by ExitProcess terminating another thread! If these are arguments for how things should be done in Unix, they strongly suggest circumscribing the unstructured use of threads in Unix, and heaping scorn on libraries and their developers that do stupid things. Unix has a long tradition of implementing solutions through wetware, rather than software or hardware. This is why there is so much opprobium on this and similar Unix groups, which Windows developers chafe at. This thread is a case in point. It's also not unrelated to the fact that access to source code, even for proprietary platforms, has always been much easier than on Windows. It's a reasonable proposition to fix bugs rather than work around them. This tact isn't acceptable in the "corporate" world. But consider that Unix libraries aren't as often commercial products--libc a notable exception--unlike in Windows, and we're of course speaking of libraries. Windows provids the canonical XML parser, but in Unix you use libxml2 or something similar. Similarly, system-level "solutions" are more difficult in Unix, because portability across implementations is more highly valued, and unless a solution is clearly a winner it will never be widely adopted by vendors or users. This arguably results in evolutionarily stronger solutions, but in the short-term leaves more gaps to be handled ad hoc--burdensome but IMO acceptable as long nothing else precludes you from solving the problem. This is also related to theories of software composition. Windows developers often balk at the inefficiency of poll()+read() versus Windows' completion ports. But poll()+read() are far more composable than completion ports. They're moderately less efficient and yet almost infinitely more composable--relative to being forced into a threaded, reentrant model. Unix culture has always been more conservative and protective of unrestricted composition of primitives than Windows culture. (The uglier Unix interfaces are ugly by this measure in particular--and it's why most people in this thread aren't much bothered by the descriptor "leaking" issue; it's not ugly by this measure). That's not necessarily a judgment one way or another, just the way it is.
From: David Given on 19 Apr 2010 15:08 On 19/04/10 03:31, Joshua Maurice wrote: [...] > It's kind of difficult to do > what you suggest in the real world for general purpose libraries, both > users and writers. My understanding of the problem is that *any* call to fopen(), in a multithreaded application, may result in a leaked file descriptor (because even if the caller to fopen() remembers to call fcntl(fileno(fp), F_SETFD, FD_CLOEXEC) afterwards, there's still a window whereby another thread calling fork() might propagate the file descriptor). Is this correct? If so, is there any actual solution? Because fopen() is not going to go away. -- ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ───── │ │ "In the beginning was the word. │ And the word was: Content-type: text/plain" --- Unknown sage
From: Rainer Weikusat on 19 Apr 2010 15:39
David Given <dg(a)cowlark.com> writes: > On 19/04/10 03:31, Joshua Maurice wrote: > [...] >> It's kind of difficult to do >> what you suggest in the real world for general purpose libraries, both >> users and writers. > > My understanding of the problem is that *any* call to fopen(), in a > multithreaded application, may result in a leaked file descriptor > (because even if the caller to fopen() remembers to call > fcntl(fileno(fp), F_SETFD, FD_CLOEXEC) afterwards, there's still a > window whereby another thread calling fork() might propagate the file > descriptor). > > Is this correct? > > If so, is there any actual solution? Yes. The actual solution is still (and will remain forever): DO NOT DO THIS. This cannot be that complicated, can it? Since executing fork will create a new process running the same program with much of the environment of the old process inherited, insofar parts of the environment which would otherwise be inherited must not be transferred across a fork (possibly followd by an exec), the only way to do so is to either (temporarily) modify the original environment before fork or adjust the inherited environment after fork. There is no other way. The much more interesting question (as compared to 'is there any actual solution') would be 'Is there any actual problem', that is, a real-world situation which does not include the generous assumption that 'unknown and buggy code which exists only in binary form is going to be executed as part of the forking process'. Because, as soon as you assume that "the software doesn't work", you'll obviously get just that (software doesn't work) by definition (and discussing this is as obviously useless). BTW, given that the DRAMs usually used in PCs are known to be inherently unreliable, is their any way to program a computer at all? |