From: Anton Blanchard on

I'm seeing spikes of up to 0.5ms in khungtaskd on a large machine. To reduce
this source of jitter I tried setting hung_task_check_count to 0:

# echo 0 > /proc/sys/kernel/hung_task_check_count

which didn't have the intended response. Change to a post increment of
max_count, so a value of 0 means check 0 tasks.

Signed-off-by: Anton Blanchard <anton(a)samba.org>
---

Index: linux.trees.git/kernel/hung_task.c
===================================================================
--- linux.trees.git.orig/kernel/hung_task.c 2009-11-27 13:11:46.000000000 +1100
+++ linux.trees.git/kernel/hung_task.c 2009-11-27 13:11:57.000000000 +1100
@@ -144,7 +144,7 @@ static void check_hung_uninterruptible_t

rcu_read_lock();
do_each_thread(g, t) {
- if (!--max_count)
+ if (!max_count--)
goto unlock;
if (!--batch_count) {
batch_count = HUNG_TASK_BATCHING;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Frederic Weisbecker on
On Fri, Nov 27, 2009 at 01:28:20PM +1100, Anton Blanchard wrote:
>
> I'm seeing spikes of up to 0.5ms in khungtaskd on a large machine. To reduce
> this source of jitter I tried setting hung_task_check_count to 0:
>
> # echo 0 > /proc/sys/kernel/hung_task_check_count
>
> which didn't have the intended response. Change to a post increment of
> max_count, so a value of 0 means check 0 tasks.
>
> Signed-off-by: Anton Blanchard <anton(a)samba.org>


Acked-by: Frederic Weisbecker <fweisbec(a)gmail.com>


> ---
>
> Index: linux.trees.git/kernel/hung_task.c
> ===================================================================
> --- linux.trees.git.orig/kernel/hung_task.c 2009-11-27 13:11:46.000000000 +1100
> +++ linux.trees.git/kernel/hung_task.c 2009-11-27 13:11:57.000000000 +1100
> @@ -144,7 +144,7 @@ static void check_hung_uninterruptible_t
>
> rcu_read_lock();
> do_each_thread(g, t) {
> - if (!--max_count)
> + if (!max_count--)
> goto unlock;
> if (!--batch_count) {
> batch_count = HUNG_TASK_BATCHING;

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Américo Wang on
On Fri, Nov 27, 2009 at 10:28 AM, Anton Blanchard <anton(a)samba.org> wrote:
>
> I'm seeing spikes of up to 0.5ms in khungtaskd on a large machine. To reduce
> this source of jitter I tried setting hung_task_check_count to 0:
>
> # echo 0 > /proc/sys/kernel/hung_task_check_count
>
> which didn't have the intended response. Change to a post increment of
> max_count, so a value of 0 means check 0 tasks.
>
> Signed-off-by: Anton Blanchard <anton(a)samba.org>


Ack.

I would also suggest to make 'max_count' as unsigned long,
since sysctl_hung_task_check_count is.

Thanks.

> ---
>
> Index: linux.trees.git/kernel/hung_task.c
> ===================================================================
> --- linux.trees.git.orig/kernel/hung_task.c     2009-11-27 13:11:46.000000000 +1100
> +++ linux.trees.git/kernel/hung_task.c  2009-11-27 13:11:57.000000000 +1100
> @@ -144,7 +144,7 @@ static void check_hung_uninterruptible_t
>
>        rcu_read_lock();
>        do_each_thread(g, t) {
> -               if (!--max_count)
> +               if (!max_count--)
>                        goto unlock;
>                if (!--batch_count) {
>                        batch_count = HUNG_TASK_BATCHING;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Frederic Weisbecker on
On Fri, Nov 27, 2009 at 10:46:53AM +0800, Am�rico Wang wrote:
> On Fri, Nov 27, 2009 at 10:28 AM, Anton Blanchard <anton(a)samba.org> wrote:
> >
> > I'm seeing spikes of up to 0.5ms in khungtaskd on a large machine. To reduce
> > this source of jitter I tried setting hung_task_check_count to 0:
> >
> > # echo 0 > /proc/sys/kernel/hung_task_check_count
> >
> > which didn't have the intended response. Change to a post increment of
> > max_count, so a value of 0 means check 0 tasks.
> >
> > Signed-off-by: Anton Blanchard <anton(a)samba.org>
>
>
> Ack.
>
> I would also suggest to make 'max_count' as unsigned long,
> since sysctl_hung_task_check_count is.
>
> Thanks.


Also, the batch_count thing should be dropped I think.
This is a hardcoded, not overridable pause after 1024
threads checks to avoid latencies caused by rcu_read_lock.
But now we have PREEMPT_RCU so people can enable it if
they care about latency. We should remove it as it adds
unnecessary complexity.

I'm preparing a patch for that, on top of Anton patch.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Américo Wang on
On Fri, Nov 27, 2009 at 10:55 AM, Frederic Weisbecker
<fweisbec(a)gmail.com> wrote:
> On Fri, Nov 27, 2009 at 10:46:53AM +0800, Américo Wang wrote:
>> On Fri, Nov 27, 2009 at 10:28 AM, Anton Blanchard <anton(a)samba.org> wrote:
>> >
>> > I'm seeing spikes of up to 0.5ms in khungtaskd on a large machine. To reduce
>> > this source of jitter I tried setting hung_task_check_count to 0:
>> >
>> > # echo 0 > /proc/sys/kernel/hung_task_check_count
>> >
>> > which didn't have the intended response. Change to a post increment of
>> > max_count, so a value of 0 means check 0 tasks.
>> >
>> > Signed-off-by: Anton Blanchard <anton(a)samba.org>
>>
>>
>> Ack.
>>
>> I would also suggest to make 'max_count' as unsigned long,
>> since sysctl_hung_task_check_count is.
>>
>> Thanks.
>
>
> Also, the batch_count thing should be dropped I think.
> This is a hardcoded, not overridable pause after 1024
> threads checks to avoid latencies caused by rcu_read_lock.
> But now we have PREEMPT_RCU so people can enable it if
> they care about latency. We should remove it as it adds
> unnecessary complexity.

This sounds OK for me.

>
> I'm preparing a patch for that, on top of Anton patch.
>

Great!

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/