Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?
From: Peter Olcott on 9 Apr 2010 10:38

"Casper H.S. Dik" <Casper.Dik(a)Sun.COM> wrote in message
news:4bbef62d$0$22917$e4fe514c(a)news.xs4all.nl...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>
>>I just checked the third party provider provides UPS. Most
>>experts seem to say not to worry about the disk drive's
>>onboard cache if UPS is available. It looks like the only
>>alternative that can be counted on for this would be to
>>disable write caching for the drive.
>
>
> I disagree; in holland we have reliable powergrid. My
> experience
> is that UPS fail more often than the powergrid. I.e.,
> even if
> you have a UPS, you may still lose the data in the write
> cache.
>
> This needs to be fixed in software; ZFS is able to handle
> this properly with the write caches enabled.
>

Yet ZFS can not be run on Linux because of licensing issues.

> (Note that most PC drivers are designed for deployment on
> Windows with the write cache enabled; it is not clear that
> disabling the write crache will work for all models.
>
> Casper
> --
> Expressed in this posting are my opinions. They are in no
> way related
> to opinions held by my employer, Sun Microsystems.
> Statements on Sun products included here are not gospel
> and may
> be fiction rather than truth.


From: Peter Olcott on 9 Apr 2010 10:53

"Jasen Betts" <jasen(a)xnet.co.nz> wrote in message
news:hpncoj$uuu$3(a)reversiblemaps.ath.cx...
> On 2010-04-07, Peter Olcott <NoSpam(a)OCR4Screen.com> wrote:
>>
>> A 3.5 minute long low priority process could be already
>> executing when a 50 ms high priority job arrives. The 3.5
>> minute long low priority must give up what it is doing
>> (sleep) so that the 50 ms high priority process has
>> exclusive use of the CPU. If the 50 ms job does not have
>> exclusive use of the CPU it may become A 500 ms job due
>> to
>> the lack of cache spatial locality of reference. I am
>> trying
>> to impose a 100 ms real-time limit on the high priority
>> jobs.
>
> write your 210000ms job to check for waiting 50ms jobs
> every 25ms or so.
>
> one way to do this waould be to take and immediately
> relinquish a
> mutex every N loops. in the low priority task
> but in the high priority task take the mutex and keep it.
>

This is exactly what I originally proposed, but, David
Schwartz seemed to think it was a horribly stupid idea.

His idea may be good enough at probably at least a slight
performance penalty to some of the high priority jobs.
Simply give the high priority jobs much more process
priority than the low priority jobs. For example each of the
three low priority jobs would get 10% of the CPU, and the
high priority job would get 70%. Or 7,7,7 and 79.

> --- news://freenews.netfront.net/ - complaints:
> news(a)netfront.net ---


From: Jasen Betts on 9 Apr 2010 10:50
On 2010-04-07, David Schwartz <davids(a)webmaster.com> wrote:
> On Apr 7, 1:55 pm, sc...(a)slp53.sl.home (Scott Lurndal) wrote:
>> David Schwartz <dav...(a)webmaster.com> writes:
>> >On Apr 7, 4:43=A0am, Jasen Betts <ja...(a)xnet.co.nz> wrote:
>
>> >> select() prefers the lowest numbered file descriptor it's asked to
>> >> test/watch so that should be easy to arrange,
>
>> >Huh?!
>
>> on the call, but not subsequent to the wait.
>
>
> I still have no idea what you're talking about.
>
> DS

I was wrong above, I totally misremembered how select() works.


--- news://freenews.netfront.net/ - complaints: news(a)netfront.net ---
From: David Schwartz on 9 Apr 2010 17:39
On Apr 8, 2:44 pm, "Ersek, Laszlo" <la...(a)caesar.elte.hu> wrote:

> Now on to my own (botched) ideas:
>
> The parent may rely on SIGCHLD, which is generated for the process. The
> signal might be delivered to a "shadow thread".

The same problem would occur with a multi-threaded process. If the
library created a new thread in a multi-threaded process, it might get
misdirected SIGCHLDs. It could handle this by blocking SIGCHLD.


> > The differences in signal handling would also be invisible to a
> > process designed to be single-threaded.

> See alarm(), followed by a connect(). The SIGALRM is generated for the
> process, which might be delivered to a "shadow thread".

The same problem would occur with a multi-threaded process. Any
library that creates a thread has to deal with this, likely by
blocking SIGALRM in the thread.

> Another example:
>
> static volatile sig_atomic_t caught_int,
>      caught_term;
>
> static void
> sigs_handle(int sig)
> {
>    *(SIGINT == sig ? &caught_int : &caught_term) = 1;
>
> }
>
> static void
> sigs_inst(void)
> {
>    struct sigaction sa;
>    int ret;
>
>    sa.sa_handler = &sigs_handle;
>
>    ret = sigemptyset(&sa.sa_mask);
>    assert(0 == ret);
>
>    sa.sa_flags = 0; /* note, SA_RESTART is absent */
>
>    ret = sigaction(SIGINT, &sa, 0);
>    assert(0 == ret);
>
>    ret = sigaction(SIGTERM, &sa, 0);
>    assert(0 == ret);
>
> }
>
> (No comments on style now, please; we're talking about correct
> single-threaded processes, not *nice* ones.)

> Suppose the program is blocked in a read() call and expects it to return
> with -1/EINTR (suppose the two signals mentioned above are not blocked at
> all and EINTR is handled everywhere), after which the program does some
> cleanup. If another process generates SIGTERM for this process (that is,
> asynchronously), and the signal is delivered to a "shadow thread", that
> thread will execute the handler, and the "main thread" won't return from
> read() with -1/EINTR.

Again, the same problem would occur whether the original program is
single-threaded or multi-threaded. So this isn't a reason a library
can't create a thread if called by a single-threaded program.

> Suppose now that yet another process sends a SIGINT similarly, which is
> this time delivered to the "correct thread". If the main thread, woken up
> from read(), checks "caught_term" *first*, the behavior is undefined,
> because that object was modified by a different thread with (possibly or
> probably) no intervening call to any routine that would synchronize memory
> state (ie. the listed pthread_*() functions). "volatile" is absolutely no
> help here.

That I don't follow. What other thread modified the object? If you
mean a shadow thread, shadow threads could only modify objects that
were accessed by the calling thread into the library, which obviously
*would* use the proper synchronization functions.

Again, the same problem would occur if the original process was multi-
threaded.

> "Shadow threads" might want to call pthread_sigmask() as soon as they are
> created to block all signals. There seems to be a race, though. Of course,
> the C library may make use of a kernel facility that takes a signal mask
> as a thread attribute before spawning the thread. I guess a *C* library
> can do whatever it wants, since everything goes through it in a portable
> program.

Exactly.

> For the likely case of all my examples being epic failures, please note
> that the inability to provide any counterexample for conjecture C doesn't
> render conjecture C theorem T, that is, it doesn't prove it.

The argument that something is not possible requires as evidence more
than a list of easy to solve problems that thing creates.

DS
From: Ersek, Laszlo on 9 Apr 2010 19:29
On Fri, 9 Apr 2010, David Schwartz wrote:

> On Apr 8, 2:44�pm, "Ersek, Laszlo" <la...(a)caesar.elte.hu> wrote:
>
>> Now on to my own (botched) ideas:
>>
>> The parent may rely on SIGCHLD, which is generated for the process. The
>> signal might be delivered to a "shadow thread".
>
> The same problem would occur with a multi-threaded process. If the
> library created a new thread in a multi-threaded process, it might get
> misdirected SIGCHLDs. It could handle this by blocking SIGCHLD.

Perhaps I'm losing the original context here, but if the original process
is multi-threaded, it most likely already calls pthread_sigmask() to
constrain delivery of SIGCHLD (or SIGALRM in the other example) to a
specific thread. The external library (not the C library) creates its
(first) own threads when the program calls one of the functions the
library provides. If this function call happens from an original program
thread that has SIGCHLD (SIGALRM) already blocked, the whole thread-tree
rooted at said function call will inherit this signal mask.

Of course there is no assurance that no such shadow thread creating
library function will be called from an original program thread with a
"leaky" signal mask. But a multi-threaded program is at least aware of the
issue, and it knows to call pthread_sigmask() instead of sigprocmask().


>> Another example:
>>
>> static volatile sig_atomic_t caught_int,
>> � � �caught_term;
>>
>> static void
>> sigs_handle(int sig)
>> {
>> � �*(SIGINT == sig ? &caught_int : &caught_term) = 1;
>>
>> }
>>
>> static void
>> sigs_inst(void)
>> {
>> � �struct sigaction sa;
>> � �int ret;
>>
>> � �sa.sa_handler = &sigs_handle;
>>
>> � �ret = sigemptyset(&sa.sa_mask);
>> � �assert(0 == ret);
>>
>> � �sa.sa_flags = 0; /* note, SA_RESTART is absent */
>>
>> � �ret = sigaction(SIGINT, &sa, 0);
>> � �assert(0 == ret);
>>
>> � �ret = sigaction(SIGTERM, &sa, 0);
>> � �assert(0 == ret);
>>
>> }
>>

[snip]

>> Suppose the program is blocked in a read() call and expects it to
>> return with -1/EINTR (suppose the two signals mentioned above are not
>> blocked at all and EINTR is handled everywhere), after which the
>> program does some cleanup. If another process generates SIGTERM for
>> this process (that is, asynchronously), and the signal is delivered to
>> a "shadow thread", that thread will execute the handler, and the "main
>> thread" won't return from read() with -1/EINTR.
>
> Again, the same problem would occur whether the original program is
> single-threaded or multi-threaded. So this isn't a reason a library
> can't create a thread if called by a single-threaded program.

If the original program is multi-threaded, it probably won't go with "the
two signals mentioned above are not blocked at all". I assume it would
already block both signals in all threads except the one expecting
-1/EINTR in read(). Its designers would likely be more aware of the issue,
and they would try to call all external library initialization functions
with the signal mask already set up.

.... Yes, this is mostly speculation; but in a single threaded process, the
programmer is originally not *forced* to think about this.


>> Suppose now that yet another process sends a SIGINT similarly, which is
>> this time delivered to the "correct thread". If the main thread, woken
>> up from read(), checks "caught_term" *first*, the behavior is
>> undefined, because that object was modified by a different thread with
>> (possibly or probably) no intervening call to any routine that would
>> synchronize memory state (ie. the listed pthread_*() functions).
>> "volatile" is absolutely no help here.
>
> That I don't follow. What other thread modified the object? If you mean
> a shadow thread, shadow threads could only modify objects that were
> accessed by the calling thread into the library, which obviously *would*
> use the proper synchronization functions.

This second paragraph of mine follows my previous paragraph
chronologically. I imagined the following steps:

1. The original program is single-threaded. It doesn't block either of
SIGINT or SIGTERM anywhere. It handles -1/EINTR everywhere. It also
installs a signal handler for both signals. (In the example that is a
common function, but it doesn't really matter.) The signal handler (which
otherwise can be reentered within the same thread too, ie. the processing
of SIGINT can be interrupted by SIGTERM and vice versa) sets different
volatile sig_atomic_t objects (with static storage duration and internal
linkage and file scope), choosing one depending on the signal delivered.

2. The library creates shadow threads, when the application calls a
library function. They all inherit the signal mask of the spawner, that
is, asynchronously generated SIGINT and SIGTERM signals can be delivered
to any of them.

3. Signal actions are process-wide. That is, whichever thread is chosen
for delivery, it will enter the signal handler and modify the
corresponding static variable the library writer knew nothing about.

4.In my example, there are two signals sent, with convenient time distance
between them. The main thread is blocking in a read(), while shadow
threads are doing whatever. A different process sends a SIGTERM. It is
delivered to a shadow thread. Suppose it was writing a huge block, so it
is interrupted after writing nonzero bytes. It enters the signal handler
installed by the original program and sets "caught_term" to 1. Then it
returns with a small value from write(). The library writer knew about
"short writes", and the loop he wrapped around write() simply handles this
condition.

This modification of "caught_term" by shadow thread S1 makes any access to
"caught_term" by any other thread, without an intervening call to a memory
synchronization routine
<http://www.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_11>,
undefined behavior.

5. Now the second signal, a SIGINT is sent by yet another process, and
this actually happens to wake up the main thread from read(). The
developer of the main program decided to check "caught_term" first, to see
which signal(s) were processed. Since there was no protection and no
memory synchronization between the previous and the current access to
"caught_term", by two different threads, the behavior is undefined. The
main program may see the old value of "caught_term", or something
completely unexpected. If the delivery of SIGINT coincides with SIGTERM
sufficiently tightly, then the main thread may wake up first, return from
the handler, and be checking "caught_term" while S1 is modifying it.
"volatile sig_atomic_t" doesn't protect against this, at least not
portably. If it did, then modern standards would explicitly define it as a
type with atomic access in multi-threaded programs.


>> For the likely case of all my examples being epic failures, please note
>> that the inability to provide any counterexample for conjecture C
>> doesn't render conjecture C theorem T, that is, it doesn't prove it.
>
> The argument that something is not possible requires as evidence more
> than a list of easy to solve problems that thing creates.

Yes, but the question is, which standpoint needs to shoulder the burden of
proof. As I understand it, the basic rule has been laid down by the SUS
(or POSIX); I only tried to bring up some borked examples for the
usefulness and validity of the rule. Those examples don't try and don't
have to prove the rule. If you defeated my examples, well, that was well
expected on my part :), but your standpoint would need to defeat the
reasoning of the people who came up with the rule.

On Thu, 8 Apr 2010, David Schwartz wrote:

> On Apr 8, 9:28�am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:

>> [...] Some UNIX(*) interface have different semantics for
>> single-threaded and multi-threaded processes, the most prominent one
>> being fork.
>
> So what? Can you cite a single example where these semantics require a
> difference that would be visible to an application that was designed to
> be single-threaded?

I was probably unable to cite a single example, but that means nothing. I
trust the people who created the rule can name classes of such cases, and
I think they actually did in the rationale I linked to previously.

http://www.opengroup.org/onlinepubs/9699919799/functions/pthread_atfork.html#tag_16_402_08

On Thu, 8 Apr 2010, Ersek, Laszlo wrote:

> ----v----
> As explained, there is no suitable solution for functionality which
> requires non-atomic operations to be protected through mutexes and
> locks. This is why the POSIX.1 standard since the 1996 release requires
> that the child process after fork() in a multi-threaded process only
> calls async-signal-safe interfaces.
> ----^----

If a program unknowingly becomes multi-threaded from single-threaded, it
may suddenly violate this rule, after the child process returns from
fork().

Do you think this topic should be brought up in austin-group-l?


Thank you for answering. I hope I'm not too dense for this discussion.
lacos
First  |  Prev  |  Next  |  Last
Pages: 27 28 29 30 31 32 33 34 35 36 37 38
Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?