A few questions and issues with dynticks, NOHZ and powertop
in [Kernel]
|
Prev: [PATCH 2/4] panic: Add taint flag TAINT_FIRMWARE_WORKAROUND ('I')
Next: [PATCH] raw: fsync method is now required
From: Dominik Brodowski on 4 Apr 2010 12:50 On Sun, Apr 04, 2010 at 11:17:37AM -0400, Alan Stern wrote: > On Sun, 4 Apr 2010, Dominik Brodowski wrote: > > > Booting a SMP-capable kernel with "nosmp", or manually offlining one CPU > > (or -- though I haven't tested it -- booting a SMP-capable kernel on a > > system with merely one CPU) means that in up to about half of the calls to > > tick_nohz_stop_sched_tick() are aborted due to rcu_needs_cpu(). This is > > quite strange to me: AFAIK, RCU is an excellent tool for SMP, but not really > > needed for UP? > > I can't answer the real question here, not knowing enough about the RCU > implementation. However, your impression is wrong: RCU very definitely > _is_ useful and needed on UP systems. It coordinates among processes > (and interrupt handlers) as well as among processors. Okay, but still: can't this be sped up by much on UP (especially if CONFIG_RCU_FAST_NO_HZ is set), so that we can go to sleep right away? > > 3) USB: built-in UHCI and a built-in 0a5c:2101 Broadcom Corp. A-Link > > BlueUsbA2 Bluetooth module; built-in EHCI and a built-in 0ac8:c302 Z-Star > > Microelectronics Corp. Vega USB 2.0 Camera. > > > > usbcore.autosuspend is enabled (= 2), of course. > > > > Recent USB suspend statistics > > Active Device name > > 100.0% USB device 7-1 : BCM92045NMD (Broadcom Corp) > > 100.0% USB device 1-2 : Vega USB 2.0 Camera. (Vimicro Corp.) > > 100.0% USB device usb7 : UHCI Host Controller (Linux 2.6.34-rc3 uhci_hcd) > > 100.0% USB device usb1 : EHCI Host Controller (Linux 2.6.34-rc3 ehci_hcd) > > > > Booting into /bin/bash on a SMP kernel booted with "nosmp" leads to ~ 10 > > wakeups per second; disabling the cursor helps halfway (~ 5 wakeups); and > > manually unbinding the USB host drivers from the USB host devices finally > > lead to ~ 1.1 wakeups per second. What's keeping USB from suspending these > > unused devices here? > > Either the drivers don't support autosuspend or the devices aren't > enabled for autosuspend. By default, autosuspend is disabled for > (almost) all non-hub devices. You or your distribution must enable > it manually by doing: > > echo auto >/sys/bus/usb/devices/.../power/level > > If the driver doesn't support autosuspend then enabling it won't be > enough; you'll also have to unbind the driver from the device. The > easiest way to do this is to unconfigure the device: > > echo 0 >/sys/bus/usb/devices/.../bConfigurationValue Thanks! This way, it works, even without manually unbinding the host drivers. Best, Dominik -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Paul E. McKenney on 4 Apr 2010 16:50 On Sun, Apr 04, 2010 at 06:39:24PM +0200, Dominik Brodowski wrote: > On Sun, Apr 04, 2010 at 11:17:37AM -0400, Alan Stern wrote: > > On Sun, 4 Apr 2010, Dominik Brodowski wrote: > > > > > Booting a SMP-capable kernel with "nosmp", or manually offlining one CPU > > > (or -- though I haven't tested it -- booting a SMP-capable kernel on a > > > system with merely one CPU) means that in up to about half of the calls to > > > tick_nohz_stop_sched_tick() are aborted due to rcu_needs_cpu(). This is > > > quite strange to me: AFAIK, RCU is an excellent tool for SMP, but not really > > > needed for UP? > > > > I can't answer the real question here, not knowing enough about the RCU > > implementation. However, your impression is wrong: RCU very definitely > > _is_ useful and needed on UP systems. It coordinates among processes > > (and interrupt handlers) as well as among processors. > > Okay, but still: can't this be sped up by much on UP (especially if > CONFIG_RCU_FAST_NO_HZ is set), so that we can go to sleep right away? One situation that will prevent CONFIG_RCU_FAST_NO_HZ from putting the machine to sleep right away is if there is an RCU callback posted that spawns another RCU callback, and so on. CONFIG_RCU_FAST_NO_HZ will handle one callback that spawns another, but it gives up if the second callback spawns a third. Might this be what is happening to you? If so, would you be willing to patch your kernel? RCU_NEEDS_CPU_FLUSHES is currently set to 5, and might be set to (say) 8. This is defined in kernel/rcutree_plugin.h, near line 990. Another thing to try would be running with TINY_RCU, at least if it is OK that RCU be non-preemptible. Thanx, Paul > > > 3) USB: built-in UHCI and a built-in 0a5c:2101 Broadcom Corp. A-Link > > > BlueUsbA2 Bluetooth module; built-in EHCI and a built-in 0ac8:c302 Z-Star > > > Microelectronics Corp. Vega USB 2.0 Camera. > > > > > > usbcore.autosuspend is enabled (= 2), of course. > > > > > > Recent USB suspend statistics > > > Active Device name > > > 100.0% USB device 7-1 : BCM92045NMD (Broadcom Corp) > > > 100.0% USB device 1-2 : Vega USB 2.0 Camera. (Vimicro Corp.) > > > 100.0% USB device usb7 : UHCI Host Controller (Linux 2.6.34-rc3 uhci_hcd) > > > 100.0% USB device usb1 : EHCI Host Controller (Linux 2.6.34-rc3 ehci_hcd) > > > > > > Booting into /bin/bash on a SMP kernel booted with "nosmp" leads to ~ 10 > > > wakeups per second; disabling the cursor helps halfway (~ 5 wakeups); and > > > manually unbinding the USB host drivers from the USB host devices finally > > > lead to ~ 1.1 wakeups per second. What's keeping USB from suspending these > > > unused devices here? > > > > Either the drivers don't support autosuspend or the devices aren't > > enabled for autosuspend. By default, autosuspend is disabled for > > (almost) all non-hub devices. You or your distribution must enable > > it manually by doing: > > > > echo auto >/sys/bus/usb/devices/.../power/level > > > > If the driver doesn't support autosuspend then enabling it won't be > > enough; you'll also have to unbind the driver from the device. The > > easiest way to do this is to unconfigure the device: > > > > echo 0 >/sys/bus/usb/devices/.../bConfigurationValue > > Thanks! This way, it works, even without manually unbinding the host > drivers. > > Best, > Dominik > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo(a)vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Paul E. McKenney on 4 Apr 2010 19:40 On Sun, Apr 04, 2010 at 01:47:25PM -0700, Paul E. McKenney wrote: > On Sun, Apr 04, 2010 at 06:39:24PM +0200, Dominik Brodowski wrote: > > On Sun, Apr 04, 2010 at 11:17:37AM -0400, Alan Stern wrote: > > > On Sun, 4 Apr 2010, Dominik Brodowski wrote: > > > > > > > Booting a SMP-capable kernel with "nosmp", or manually offlining one CPU > > > > (or -- though I haven't tested it -- booting a SMP-capable kernel on a > > > > system with merely one CPU) means that in up to about half of the calls to > > > > tick_nohz_stop_sched_tick() are aborted due to rcu_needs_cpu(). This is > > > > quite strange to me: AFAIK, RCU is an excellent tool for SMP, but not really > > > > needed for UP? > > > > > > I can't answer the real question here, not knowing enough about the RCU > > > implementation. However, your impression is wrong: RCU very definitely > > > _is_ useful and needed on UP systems. It coordinates among processes > > > (and interrupt handlers) as well as among processors. > > > > Okay, but still: can't this be sped up by much on UP (especially if > > CONFIG_RCU_FAST_NO_HZ is set), so that we can go to sleep right away? > > One situation that will prevent CONFIG_RCU_FAST_NO_HZ from putting the > machine to sleep right away is if there is an RCU callback posted that > spawns another RCU callback, and so on. CONFIG_RCU_FAST_NO_HZ will handle > one callback that spawns another, but it gives up if the second callback > spawns a third. > > Might this be what is happening to you? > > If so, would you be willing to patch your kernel? RCU_NEEDS_CPU_FLUSHES > is currently set to 5, and might be set to (say) 8. This is defined > in kernel/rcutree_plugin.h, near line 990. > > Another thing to try would be running with TINY_RCU, at least if it is > OK that RCU be non-preemptible. And you did mention offlining some CPUs above. The folloiwng patch (from Lai Jiangshan) is needed to handle this case. Thanx, Paul ------------------------------------------------------------------------ commit 6a2ae79877827355b747c0b91133a963b74ed396 Author: Lai Jiangshan <laijs(a)cn.fujitsu.com> Date: Tue Mar 30 18:40:36 2010 +0800 rcu: ignore offline CPUs in last non-dyntick-idle CPU check Offline CPUs are not in nohz_cpu_mask, but can be ignored when checking for the last non-dyntick-idle CPU. This patch therefore only checks online CPUs for not being dyntick idle, allowing fast entry into full-system dyntick-idle state even when there are some offline CPUs. Signed-off-by: Lai Jiangshan <laijs(a)cn.fujitsu.com> Signed-off-by: Paul E. McKenney <paulmck(a)linux.vnet.ibm.com> diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 79b53bd..687c4e9 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -1016,7 +1016,7 @@ int rcu_needs_cpu(int cpu) /* Don't bother unless we are the last non-dyntick-idle CPU. */ for_each_cpu_not(thatcpu, nohz_cpu_mask) - if (thatcpu != cpu) { + if (cpu_online(thatcpu) && thatcpu != cpu) { per_cpu(rcu_dyntick_drain, cpu) = 0; per_cpu(rcu_dyntick_holdoff, cpu) = jiffies - 1; return rcu_needs_cpu_quick_check(cpu); -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Arjan van de Ven on 4 Apr 2010 23:50 > And you did mention offlining some CPUs above. The folloiwng patch > (from Lai Jiangshan) is needed to handle this case. btw on x86... don't offline CPUs if you want to save power.. it doesn't. (at least not during idle.. and when you're busy it might save power, but it won't save you energy normally) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Paul E. McKenney on 5 Apr 2010 00:30
On Sun, Apr 04, 2010 at 08:44:05PM -0700, Arjan van de Ven wrote: > >And you did mention offlining some CPUs above. The folloiwng patch > >(from Lai Jiangshan) is needed to handle this case. > > btw on x86... don't offline CPUs if you want to save power.. it doesn't. > (at least not during idle.. and when you're busy it might save power, > but it won't save you energy normally) Hmmm... The fact that offlining CPUs doesn't save power could form the basis of an interesting rationalization for my having ignored offlined CPUs in my original patch, I suppose. ;-) So the proper approach is to affinity everything away from the CPUs in question so that they stay in dyntick-idle mode? I must confess that I find this quite counter-intuitive -- and I suspect that I am not the only one who would expect offlined CPUs to drop to the lowest possible power consumption. Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |