Prev: Kill process tree, again
Next: ANN: Seed7 Release 2010-04-18
From: Rainer Weikusat on 20 Apr 2010 15:29
David Given <dg(a)cowlark.com> writes:
> On 20/04/10 01:15, Ersek, Laszlo wrote:
>> On Mon, 19 Apr 2010, David Given wrote:
> [...]
>>> So, basically, you're saying, don't use fopen()? In any multithreaded
>>> apps? And in any *library* which might be used by a multithreaded app?
>>
>> s/fopen/fork, I guess.
>
> I did mean fopen(): if I have a multithreaded application where one
> thread, *any* thread, creates a file descriptor, and another thread,
> *any* thread, calls exec(), then there's a potential problem.

If you write a single line of code, there is a potential problem
because it could be the wrong line of code. But since this is
completely hypothetical, discussing the possibility is useless. One
should rather check the line instead insofar there is any reason to
assume that a problem actually exists. And despite you are probably
not willing to believe this, there is no difference between my
contrived example and your contrived example: As soon as something is
done at all, it could have been done wrongly according to some
definition of 'wrong' and hence, every action is 'a potential problem'
(and discussing 'potential problems' at this level of detail is an
exercise in generating unproductive noise).

I asked you for a real problem two postings ago. Where is it?

[...]

> Ah, but it's not me that's calling socket() or pipe() --- it's a
> third-party library that's not under my control.

Ok. Fine. What do you propose as solution to the problem that the
third-party library could just overwrite all of your code with trash
data? Maybe even because of a programming error in said library or a
few bits ins some DIMM which have chosen to change their values at an
inconvenient time?

Answer: You ignore the possibility, at least until 'problems' actually
manifest themselves.

Regarding your problems wrt 'it is impossible to program for
UNIX(*)', hire a programmer and stay away from this stuff. It is
beyond you. *plonk*
From: Rainer Weikusat on 20 Apr 2010 15:43
Rainer Weikusat <rweikusat(a)mssgmbh.com> writes:

[...]

> Ok. Fine. What do you propose as solution to the problem that the
> third-party library could just overwrite all of your code with trash
> data? Maybe even because of a programming error in said library or a
> few bits ins some DIMM which have chosen to change their values at an
> inconvenient time?

As an addition to that: One of the nastiest problems I had to deal
with so far was "a third-party libray I couldn't control" (Opsec SDK)
which apparently corrupted the malloc-heap every once in a while,
leading (except causing random crashes) to the phenomenon that every
free-call in my code (or in library code) could possibly end in an
endless loop beause of trying to find the end of a linked list of
structures with the next-pointer of one structure pointing to the
structure itself. To add insult to injury, this was for a 24x7
firewall monitoring application which was supposed to 'just work'
without any human intervention and whose malfunctioning would
immediately cause phone calls from angry customers. It was possible to
deal with this because it was an actual problem with specific
properties. It is not possible to deal with hypothetical problems with
the nice property of being immediately redefined in a suitable way
once someone suggests a possible solution for the vaguely defined
issue (which is vaguely defined exactly in order to enable 'problem
redefinition' whenever the threat of a solution arises).
From: Ersek, Laszlo on 20 Apr 2010 18:01
On Tue, 20 Apr 2010, Rainer Weikusat wrote:

> One of the nastiest problems I had to deal with so far was "a
> third-party libray I couldn't control" (Opsec SDK) which apparently
> corrupted the malloc-heap every once in a while, leading (except causing
> random crashes) to the phenomenon that every free-call in my code (or in
> library code) could possibly end in an endless loop beause of trying to
> find the end of a linked list of structures with the next-pointer of one
> structure pointing to the structure itself. To add insult to injury,
> this was for a 24x7 firewall monitoring application which was supposed
> to 'just work' without any human intervention and whose malfunctioning
> would immediately cause phone calls from angry customers. It was
> possible to deal with this because it was an actual problem with
> specific properties.

Can you / would you / are you permitted to tell us: how?

Thanks,
lacos
From: Joshua Maurice on 20 Apr 2010 18:57
On Apr 20, 11:48 am, William Ahern <will...(a)wilbur.25thandClement.com>
wrote:
> Jonathan de Boyne Pollard <J.deBoynePollard-newsgro...(a)ntlworld.com> wrote:
>
> > [-- text/html, encoding 7bit, charset: ISO-8859-1, 67 lines --]
>
> No doubt Windows has myriad techniques and devices for handling thread
> issues. In fact, some of them clearly aren't resolvable. Windows Vista seems
> to terminate a process which has a thread which tries to access a critical
> section left inconsistent by ExitProcess terminating another thread!
>
> If these are arguments for how things should be done in Unix, they strongly
> suggest circumscribing the unstructured use of threads in Unix, and heaping
> scorn on libraries and their developers that do stupid things.
>
> Unix has a long tradition of implementing solutions through wetware, rather
> than software or hardware. This is why there is so much opprobium on this
> and similar Unix groups, which Windows developers chafe at. This thread is a
> case in point.
>
> It's also not unrelated to the fact that access to source code, even for
> proprietary platforms, has always been much easier than on Windows. It's a
> reasonable proposition to fix bugs rather than work around them. This tact
> isn't acceptable in the "corporate" world. But consider that Unix libraries
> aren't as often commercial products--libc a notable exception--unlike in
> Windows, and we're of course speaking of libraries. Windows provids the
> canonical XML parser, but in Unix you use libxml2 or something similar.
>
> Similarly, system-level "solutions" are more difficult in Unix, because
> portability across implementations is more highly valued, and unless a
> solution is clearly a winner it will never be widely adopted by vendors or
> users. This arguably results in evolutionarily stronger solutions, but in
> the short-term leaves more gaps to be handled ad hoc--burdensome but IMO
> acceptable as long nothing else precludes you from solving the problem.
>
> This is also related to theories of software composition. Windows developers
> often balk at the inefficiency of poll()+read() versus Windows' completion
> ports. But poll()+read() are far more composable than completion ports.
> They're moderately less efficient and yet almost infinitely more
> composable--relative to being forced into a threaded, reentrant model. Unix
> culture has always been more conservative and protective of unrestricted
> composition of primitives than Windows culture. (The uglier Unix interfaces
> are ugly by this measure in particular--and it's why most people in this
> thread aren't much bothered by the descriptor "leaking" issue; it's not ugly
> by this measure). That's not necessarily a judgment one way or another, just
> the way it is.

I don't know if I have a windows-centric or whatever software value
system.

My own view of processes and threads is the standard one: threads of a
single process share a common memory space, file handles etc., and
different processes have different memory spaces, file handles, etc.
One uses threads when one needs this common memory space, and one uses
processes when one wants some degree of fault tolerance and
decoupling.

I see no a priori reason that creating a new process and opening a
file should require cooperation between threads to avoid resource
leaks. If these threads were instead processes, it would be simple;
the operations are independent, so they can be done concurrently.
However, because they are threads, and because fork + exec is defined
the way it is, these operations are in fact not independent. I think
this is silly. Nearly all cases of fork + exec do not want to inherit
all file handles. They want to inherit 3 handles and none of the rest.
The point of separate processes is to have separate resource pools,
not to share resource pools. There is no POSIX portable fast way to do
this, and thus I claim POSIX is broken in this regard if it cannot
handle the common case.

(It seems to be a recurring problem with POSIX, that to write correct
applications, all pieces of code must cooperate when it shouldn't be
required. POSIX should make independent those tasks which logically
are independent and can easily be made independent. This coupling of
code is to no one's benefit.)

I am not here to troll. I made it clear from the beginning that I know
this is how POSIX is defined, and I merely asked if there was a way to
wrap fork + exec calls to give a sane interface to callers, a way to
close all unrelated resources between fork and exec. I was asking for
a practical solution to this reasonable use. Instead, I had trolls
come in and give not useful replies like "I shouldn't hit myself with
a hammer". Luckily, not everyone is so indoctrinated and asinine, and
I was able to get some useful answers and replies. I thank those
people.
From: Joshua Maurice on 20 Apr 2010 18:57
On Apr 20, 11:48 am, William Ahern <will...(a)wilbur.25thandClement.com>
wrote:
> Jonathan de Boyne Pollard <J.deBoynePollard-newsgro...(a)ntlworld.com> wrote:
>
> > [-- text/html, encoding 7bit, charset: ISO-8859-1, 67 lines --]
>
> No doubt Windows has myriad techniques and devices for handling thread
> issues. In fact, some of them clearly aren't resolvable. Windows Vista seems
> to terminate a process which has a thread which tries to access a critical
> section left inconsistent by ExitProcess terminating another thread!
>
> If these are arguments for how things should be done in Unix, they strongly
> suggest circumscribing the unstructured use of threads in Unix, and heaping
> scorn on libraries and their developers that do stupid things.
>
> Unix has a long tradition of implementing solutions through wetware, rather
> than software or hardware. This is why there is so much opprobium on this
> and similar Unix groups, which Windows developers chafe at. This thread is a
> case in point.
>
> It's also not unrelated to the fact that access to source code, even for
> proprietary platforms, has always been much easier than on Windows. It's a
> reasonable proposition to fix bugs rather than work around them. This tact
> isn't acceptable in the "corporate" world. But consider that Unix libraries
> aren't as often commercial products--libc a notable exception--unlike in
> Windows, and we're of course speaking of libraries. Windows provids the
> canonical XML parser, but in Unix you use libxml2 or something similar.
>
> Similarly, system-level "solutions" are more difficult in Unix, because
> portability across implementations is more highly valued, and unless a
> solution is clearly a winner it will never be widely adopted by vendors or
> users. This arguably results in evolutionarily stronger solutions, but in
> the short-term leaves more gaps to be handled ad hoc--burdensome but IMO
> acceptable as long nothing else precludes you from solving the problem.
>
> This is also related to theories of software composition. Windows developers
> often balk at the inefficiency of poll()+read() versus Windows' completion
> ports. But poll()+read() are far more composable than completion ports.
> They're moderately less efficient and yet almost infinitely more
> composable--relative to being forced into a threaded, reentrant model. Unix
> culture has always been more conservative and protective of unrestricted
> composition of primitives than Windows culture. (The uglier Unix interfaces
> are ugly by this measure in particular--and it's why most people in this
> thread aren't much bothered by the descriptor "leaking" issue; it's not ugly
> by this measure). That's not necessarily a judgment one way or another, just
> the way it is.

I don't know if I have a windows-centric or whatever software value
system.

My own view of processes and threads is the standard one: threads of a
single process share a common memory space, file handles etc., and
different processes have different memory spaces, file handles, etc.
One uses threads when one needs this common memory space, and one uses
processes when one wants some degree of fault tolerance and
decoupling.

I see no a priori reason that creating a new process and opening a
file should require cooperation between threads to avoid resource
leaks. If these threads were instead processes, it would be simple;
the operations are independent, so they can be done concurrently.
However, because they are threads, and because fork + exec is defined
the way it is, these operations are in fact not independent. I think
this is silly. Nearly all cases of fork + exec do not want to inherit
all file handles. They want to inherit 3 handles and none of the rest.
The point of separate processes is to have separate resource pools,
not to share resource pools. There is no POSIX portable fast way to do
this, and thus I claim POSIX is broken in this regard if it cannot
handle the common case.

(It seems to be a recurring problem with POSIX, that to write correct
applications, all pieces of code must cooperate when it shouldn't be
required. POSIX should make independent those tasks which logically
are independent and can easily be made independent. This coupling of
code is to no one's benefit.)

I am not here to troll. I made it clear from the beginning that I know
this is how POSIX is defined, and I merely asked if there was a way to
wrap fork + exec calls to give a sane interface to callers, a way to
close all unrelated resources between fork and exec. I was asking for
a practical solution to this reasonable use. Instead, I had trolls
come in and give not useful replies like "I shouldn't hit myself with
a hammer". Luckily, not everyone is so indoctrinated and asinine, and
I was able to get some useful answers and replies. I thank those
people.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6 7 8 9 10
Prev: Kill process tree, again
Next: ANN: Seed7 Release 2010-04-18