Prev: [PATCH] x86: Fix vsyscall on gcc 4.5 with -Os
Next: [PATCH 10/12] scsi: megaraid_sas - Add input parameter for max_sectors
From: Roland McGrath on 18 Jun 2010 17:10
I think you're right. I can't see what would prevent that race.

So for_each_process and do_each_thread are safe only under
read_lock(&tasklist_lock) and while_each_thread is only safe under
either that or siglock. (Also a few places using next_thread in
similar loops outside those macros.)

Perhaps we could move those del's from __unhash_process to
__put_task_struct (or just delayed_put_task_struct?) and then
they wouldn't need to be rculist.h calls after all. But we
would a need careful audit to figure out the assumptions about
being on the list meaning not reaped yet.

I think de_thread() in exec-by-nonleader is the only case where this
can happen, right? So then perhaps we could make it call release_task
only via call_rcu?


Thanks,
Roland
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Frederic Weisbecker on 19 Jun 2010 01:40
On Fri, Jun 18, 2010 at 10:00:54PM -0700, Mandeep Baines wrote:
> On Fri, Jun 18, 2010 at 12:34 PM, Oleg Nesterov <oleg(a)redhat.com> wrote:
> > (add cc's)
> >
> > Hmm. Once I sent this patch, I suddenly realized with horror that
> > while_each_thread() is NOT safe under rcu_read_lock(). Both
> > do_each_thread/while_each_thread or do/while_each_thread() can
> > race with exec().
> >
> > Yes, it is safe to do next_thread() or next_task(). But:
> >
> > � � � �#define while_each_thread(g, t) \
> > � � � � � � � �while ((t = next_thread(t)) != g)
> >
> > suppose that t is not the group leader, and it does de_thread() and then
> > release_task(g). After that next_thread(t) returns t, not g, and the loop
> > will never stop.
> >
> > I _really_ hope I missed something, will recheck tomorrow with the fresh
> > head. Still I'd like to share my concerns...
> >
>
> Yep. You're right. Not sure what I was thinking. This is only case
> where do_each_thread
> is protected by an rcu_read_lock. All others, correctly use read_lock.



cgroup does too.
taskstats also uses rcu with while_each_threads, and may be some
others.

It's not your fault, theses iterators are supposed to be rcu safe,
we are just encountering a bad race :)



> > If I am right, probably we can fix this, something like
> >
> > � � � �#define while_each_thread(g, t) \
> > � � � � � � � �while ((t = next_thread(t)) != g && pid_alive(g))
> >
>
> This seems like a reasonable approach. Maybe call it:
>
> while_each_thread_maybe_rcu() :)



Hmm, no while_each_thread must really be rcu_safe.



>
> This makes hung_task a little less useful for failure fencing since
> this (and rcu_lock_break)
> increases the potential for never examining all threads but its still
> a nice lightweight diagnostic
> for finding bugs.



In fact may be we could just drop the rcu break, people who really
care about latencies can use the preemptable version.

I know the worry is more about delaying too much the grace period if
we walk a too long task list, but I don't think it's really a problem.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Mandeep Baines on 19 Jun 2010 11:50
On Fri, Jun 18, 2010 at 10:35 PM, Frederic Weisbecker
<fweisbec(a)gmail.com> wrote:
> On Fri, Jun 18, 2010 at 10:00:54PM -0700, Mandeep Baines wrote:
>> On Fri, Jun 18, 2010 at 12:34 PM, Oleg Nesterov <oleg(a)redhat.com> wrote:
>> > (add cc's)
>> >
>> > Hmm. Once I sent this patch, I suddenly realized with horror that
>> > while_each_thread() is NOT safe under rcu_read_lock(). Both
>> > do_each_thread/while_each_thread or do/while_each_thread() can
>> > race with exec().
>> >
>> > Yes, it is safe to do next_thread() or next_task(). But:
>> >
>> > � � � �#define while_each_thread(g, t) \
>> > � � � � � � � �while ((t = next_thread(t)) != g)
>> >
>> > suppose that t is not the group leader, and it does de_thread() and then
>> > release_task(g). After that next_thread(t) returns t, not g, and the loop
>> > will never stop.
>> >
>> > I _really_ hope I missed something, will recheck tomorrow with the fresh
>> > head. Still I'd like to share my concerns...
>> >
>>
>> Yep. You're right. Not sure what I was thinking. This is only case
>> where do_each_thread
>> is protected by an rcu_read_lock. All others, correctly use read_lock.
>
>
>
> cgroup does too.
> taskstats also uses rcu with while_each_threads, and may be some
> others.
>
> It's not your fault, theses iterators are supposed to be rcu safe,
> we are just encountering a bad race :)
>

Thanks:) Feel less dumb now. My quick grep only turned up hung_task:

$ find . -name \*.c | xargs fgrep -B 10 do_each_thread | grep rcu
../kernel/hung_task.c- rcu_read_lock();

>
>
>> > If I am right, probably we can fix this, something like
>> >
>> > � � � �#define while_each_thread(g, t) \
>> > � � � � � � � �while ((t = next_thread(t)) != g && pid_alive(g))
>> >
>>
>> This seems like a reasonable approach. Maybe call it:
>>
>> while_each_thread_maybe_rcu() :)
>
>
>
> Hmm, no while_each_thread must really be rcu_safe.
>

I didn't realize there were other cases which need while_each_thread to
be rcu-safe. For hung_task, its OK to break out on a release_task(g).
We'll just check the threads we missed on the next iteration.

>
>
>>
>> This makes hung_task a little less useful for failure fencing since
>> this (and rcu_lock_break)
>> increases the potential for never examining all threads but its still
>> a nice lightweight diagnostic
>> for finding bugs.
>
>
>
> In fact may be we could just drop the rcu break, people who really
> care about latencies can use the preemptable version.
>

For large systems, you'd pin a CPU for a very long time checking for
hung_tasks. You'd cause a lot of memory pressure by delaying the
grace period for such a long time. You'd also cause softlockups with
the huge burst of call_rcus being processed by rcu_process_callbacks.

> I know the worry is more about delaying too much the grace period if
> we walk a too long task list, but I don't think it's really a problem.
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Oleg Nesterov on 19 Jun 2010 15:30
On 06/18, Mandeep Baines wrote:
>
> On Fri, Jun 18, 2010 at 12:34 PM, Oleg Nesterov <oleg(a)redhat.com> wrote:
> >
> >
> > I _really_ hope I missed something, will recheck tomorrow with the fresh
> > head. Still I'd like to share my concerns...
>
> Yep. You're right. Not sure what I was thinking. This is only case
> where do_each_thread
> is protected by an rcu_read_lock. All others, correctly use read_lock.

No, no. while_each_thread() is supposed to be rcu-safe, we should fix it.
It has many rcu_read_lock() users.

The patch which fixes rcu_lock_break() is orthogonal to this problem.

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Oleg Nesterov on 21 Jun 2010 13:50
On 06/21, Oleg Nesterov wrote:
>
> So, I am thinking about the first attempt
>
> #define while_each_thread(g, t) \
> while ((t = next_thread(t)) != g && pid_alive(g))
>
> again. But this means while_each_thread() can miss more threads
> than it currently can under the same conditions. Correct, but
> not good.

Not good, but correct ;) Probably it makes sense to fix the problem
anyway, then think about the more optimal fix.

static inline struct task_struct *
next_thread_careful(const struct task_struct *g, const struct task_struct *t)
{
t = next_thread(t);
/*
* this pairs with the implicit barrier between detach_pid()
* and list_del_rcu(g->thread_group) in __unhash_process(g).
*/
smp_rmb();
if (likely(pid_alive(g)))
return t;
else
return g;
}

#define while_each_thread(g, t) \
while ((t = next_thread_careful(t)) != g)

I think this should work. detach_pid() does unlock + lock at least
once and thus we have the barrier (this worth a comment or we
can add the explicit wmb() in __unhash_process).

Paul, Roland, do you see any problems from the correctness pov,
or a better fix for now?

Perhaps it also makes sense to keep the old variant renamed to
while_each_thread_locked(), I dunno.

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
 |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: [PATCH] x86: Fix vsyscall on gcc 4.5 with -Os
Next: [PATCH 10/12] scsi: megaraid_sas - Add input parameter for max_sectors