A few questions and issues with dynticks, NOHZ and powertop
in [Kernel]
|
Prev: [PATCH 2/4] panic: Add taint flag TAINT_FIRMWARE_WORKAROUND ('I')
Next: [PATCH] raw: fsync method is now required
From: Arjan van de Ven on 5 Apr 2010 10:50 On 4/4/2010 21:22, Paul E. McKenney wrote: > On Sun, Apr 04, 2010 at 08:44:05PM -0700, Arjan van de Ven wrote: >>> And you did mention offlining some CPUs above. The folloiwng patch >>> (from Lai Jiangshan) is needed to handle this case. >> >> btw on x86... don't offline CPUs if you want to save power.. it doesn't. >> (at least not during idle.. and when you're busy it might save power, >> but it won't save you energy normally) > > Hmmm... The fact that offlining CPUs doesn't save power could form > the basis of an interesting rationalization for my having ignored > offlined CPUs in my original patch, I suppose. ;-) > > So the proper approach is to affinity everything away from the CPUs > in question so that they stay in dyntick-idle mode? that is actually equivalently bad ;) > I must confess > that I find this quite counter-intuitive -- and I suspect that I am > not the only one who would expect offlined CPUs to drop to the > lowest possible power consumption. on x86 (other archs might be different), nowadays idle is VERY efficient. (and the way to offline a cpu is to put it into the same deep C state as idle would) the "offline is same power as idle" is only the part where you don't win energy. the part where you lose energy is the part where you realize that you can only really put the memory controller and memory in power saving state if all cpus are in idle. If you offline a cpu (versus leaving it idle), and you have several tasks to run during an activity burst (which are more common the more we group activity to save power), it can happen that there are so many things to do that all remaining cpus can't handle it without the scheduler delaying some tasks. At which point you delay putting the memory (controllers) in low power state and you lose. In general, the winning strategy seems to be to finish things as quickly as you can, eg on as many cpus as you have in the system to then let them all go idle. (this is not always true if you're picking cpu frequency/voltages, but for idle states it tends to be true. there's always cases where it isn't.... power is like that) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Paul E. McKenney on 5 Apr 2010 11:20 On Mon, Apr 05, 2010 at 07:40:23AM -0700, Arjan van de Ven wrote: > On 4/4/2010 21:22, Paul E. McKenney wrote: > >On Sun, Apr 04, 2010 at 08:44:05PM -0700, Arjan van de Ven wrote: > >>>And you did mention offlining some CPUs above. The folloiwng patch > >>>(from Lai Jiangshan) is needed to handle this case. > >> > >>btw on x86... don't offline CPUs if you want to save power.. it doesn't. > >>(at least not during idle.. and when you're busy it might save power, > >>but it won't save you energy normally) > > > >Hmmm... The fact that offlining CPUs doesn't save power could form > >the basis of an interesting rationalization for my having ignored > >offlined CPUs in my original patch, I suppose. ;-) > > > >So the proper approach is to affinity everything away from the CPUs > >in question so that they stay in dyntick-idle mode? > > that is actually equivalently bad ;) > > > I must confess > >that I find this quite counter-intuitive -- and I suspect that I am > >not the only one who would expect offlined CPUs to drop to the > >lowest possible power consumption. > > on x86 (other archs might be different), nowadays idle is VERY efficient. > (and the way to offline a cpu is to put it into the same deep C state as idle would) > > the "offline is same power as idle" is only the part where you don't win energy. > the part where you lose energy is the part where you realize that you can only really put > the memory controller and memory in power saving state if all cpus are in idle. > If you offline a cpu (versus leaving it idle), and you have several tasks to run during an > activity burst (which are more common the more we group activity to save power), it can happen > that there are so many things to do that all remaining cpus can't handle it without the scheduler > delaying some tasks. At which point you delay putting the memory (controllers) in low power state > and you lose. In general, the winning strategy seems to be to finish things as quickly as you can, > eg on as many cpus as you have in the system to then let them all go idle. > > (this is not always true if you're picking cpu frequency/voltages, but for idle states it tends > to be true. there's always cases where it isn't.... power is like that) So the main issue is that for many workloads, it is best to run full bore and get done quickly, thus allowing the entire machine to be powered down? If so, it seems likely that there would be some workloads that were sometimes unable to use all the CPUs, in which case shutting down (idling, offlining, dyntick-idling, whatever) the excess CPUs might nevertheless be the right thing to do. Or am I missing your point? Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Arjan van de Ven on 5 Apr 2010 12:10 On 4/5/2010 8:14, Paul E. McKenney wrote: > So the main issue is that for many workloads, it is best to run full bore > and get done quickly, thus allowing the entire machine to be powered down? yep > > If so, it seems likely that there would be some workloads that were sometimes > unable to use all the CPUs, in which case shutting down (idling, offlining, > dyntick-idling, whatever) the excess CPUs might nevertheless be the right > thing to do. but the point is that the normal scheduler + idle behavior gives you exactly that in a natural way ! If you don't have enough work (tasks) to keep all cores busy, the others are and stay idle. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Arjan van de Ven on 5 Apr 2010 12:30 On 4/5/2010 9:22, Paul E. McKenney wrote: > On Mon, Apr 05, 2010 at 09:07:33AM -0700, Arjan van de Ven wrote: >> On 4/5/2010 8:14, Paul E. McKenney wrote: >>> So the main issue is that for many workloads, it is best to run full bore >>> and get done quickly, thus allowing the entire machine to be powered down? >> >> yep >> >>> If so, it seems likely that there would be some workloads that were sometimes >>> unable to use all the CPUs, in which case shutting down (idling, offlining, >>> dyntick-idling, whatever) the excess CPUs might nevertheless be the right >>> thing to do. >> >> but the point is that the normal scheduler + idle behavior gives you exactly that >> in a natural way ! >> If you don't have enough work (tasks) to keep all cores busy, the others are and stay idle. > > So your earlier objection was not to dyntick-idle as such, but rather > to artificially constraining the scheduler to induce dyntick-idle? my objection was against the notion that offlining cpus helps power/energy ;-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Paul E. McKenney on 5 Apr 2010 12:30
On Mon, Apr 05, 2010 at 09:07:33AM -0700, Arjan van de Ven wrote: > On 4/5/2010 8:14, Paul E. McKenney wrote: > >So the main issue is that for many workloads, it is best to run full bore > >and get done quickly, thus allowing the entire machine to be powered down? > > yep > > >If so, it seems likely that there would be some workloads that were sometimes > >unable to use all the CPUs, in which case shutting down (idling, offlining, > >dyntick-idling, whatever) the excess CPUs might nevertheless be the right > >thing to do. > > but the point is that the normal scheduler + idle behavior gives you exactly that > in a natural way ! > If you don't have enough work (tasks) to keep all cores busy, the others are and stay idle. So your earlier objection was not to dyntick-idle as such, but rather to artificially constraining the scheduler to induce dyntick-idle? Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |