suspend blockers & Android integration
in [Kernel]
|
Prev: How do I ignore the changes made by CVS keyword substitution efficiently?
Next: [PATCH 1/2] x86: make save_stack_address() !CONFIG_FRAME_POINTER friendly
From: Ingo Molnar on 3 Jun 2010 15:40 * tytso(a)mit.edu <tytso(a)mit.edu> wrote: > [...] Not only has the source code been made available, but hundreds of > engineering hours have been made trying to accomodate the demands of LKML > --- and LKML has said no to suspend blockers/wakelocks. I dont think you are being fair here, at all. Firstly, the suspend-blockers feature is not being rejected (fixing and extending suspend is a worthwile goal), it's just that various different schemes have been proposed by the people who'll eventually have to maintain that code down the line. Those reasons seem justified and they are based in praxis that have solved similar problems to what Android tries to solve. Sadly the response from the Android team has been 100% uncompromising: either suspend blockers or nothing. The thing is, if the insertion of 'hundreds of man hours' into discussing a feature was technical grounds for upstream inclusion then we'd today have a Linux kernel with: - STREAMS - a kABI - modularized ipv4 - perfmon - two dozen CPU schedulers - zero-copy stupidly pushed to all the file APIs .... and IMO we'd be off much worse technically. Lets realize it, Linux is an engineering effort that has literally cost about ten thousand man years. That's about a _85 million_ man hours. It takes effort to keep that kind of work valuable! Also, why did the Android team start its contributions with such a difficult and controversial kernel feature? There is absolutely _zero_ technical reason why the Android team should present this as as an all-or-nothing effort. Why not merge hw drivers first (with suspend blockers commented or stubbed out), to reduce the fork distance? Really, i myself have controversial kernel features pending all the time. They dont go upstream for a few kernel releases - over a year sometimes - and sometimes they never go upstream. But the fact that some feature of mine is pending doesnt give me the right to go away sulking, it doesnt mean i will block the whole flow of patches in retaliation (as you seem to suggest Google will now have the right to do) - i simply try to work it out. Lets be reasonable and work it all out, ok? Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Brian Swetland on 3 Jun 2010 16:00 On Thu, Jun 3, 2010 at 12:30 PM, Ingo Molnar <mingo(a)elte.hu> wrote: > > Sadly the response from the Android team has been 100% uncompromising: either > suspend blockers or nothing. Well, we're willing to accept something that gives us the same functionality (thus rewriting the api several times to meet various objections, current discussions around constraint-based-implementations / pm-qos, etc). We believe we're solving a real problem here and have not seen a counter-proposal that accomplishes the same. Suggestions such as "just yell at developers for writing bad apps" or "it's the user's fault if they install a lousy app" or "make your app marketplace more restrictive" are not helpful. The technical discussions around alternatives are more so (though I do feel like we're going in circles in places), which again is why we're still here talking about this (that and Arve is about a billion times more patient and persistent than I am). We're not interested in massively rearchitecting our userspace to accomplish this (and the "rewrite your userspace!" proposals I've seen have had race conditions and/or significant more complexity than the wakelock model). .... > Also, why did the Android team start its contributions with such a difficult > and controversial kernel feature? We started here because it's possibly the only api level change we have -- almost everything else is driver or subarch type work or controversial but entirely self-contained (like the binder, which I would be shocked to see ever hit mainline). Assertions have been made that because the "android kernel" (not a term I like -- linux is linux, we have some assorted patches on top) has this feature it represents a difficulty for silicon vendors trying to support both Android projects and OEMs and mainline: See: http://www.kroah.com/log/linux/android-kernel-problems.html and various other rants about the evil terrible android forks, etc. So, we figure, let's sort out the hard problem first and then move on with our lives. > There is absolutely _zero_ technical reason why the Android team should > present this as as an all-or-nothing effort. Why not merge hw drivers first > (with suspend blockers commented or stubbed out), to reduce the fork distance? If that's the case then there is no problem and people could stop yelling at us and just submit their drivers. Awesome. I can't speak for all the nameless silicon vendors Greg represents, that we apparently are preventing from doing this (how? I don't know!), etc, but for my team maintaining multiple versions of drivers is a headache, we'd rather square away the wakelock debate first and figure something out there, as it just seems like a more logical approach. Maybe we're crazy. > Really, i myself have controversial kernel features pending all the time. They > dont go upstream for a few kernel releases - over a year sometimes - and > sometimes they never go upstream. > > But the fact that some feature of mine is pending doesnt give me the right to > go away sulking, it doesnt mean i will block the whole flow of patches in > retaliation (as you seem to suggest Google will now have the right to do) - i > simply try to work it out. We're not blocking anything. Hell, if people want drivers we wrote upstream and we're not fast enough for 'em, we publish everything via android.git.kernel.org, pretty aggressively rebase to follow latest mainline, and release everything under GPLv2, ready-to-go. We have to ship though, and as long as the version we maintain has the features we need to ship and the mainline version doesn't, we're going to ship based on our version, but this really shouldn't be surprising to anyone. > Lets be reasonable and work it all out, ok? We're trying. I do feel like we're suffering from lack of a clear "how do we move forward" path, and in particular from an environment where every time we do a bunch of work to address one set of concerns and entirely new set of people pop up with different concerns (sometimes contradicting the last round of changes we were asked to make, etc, etc). Brian -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 3 Jun 2010 19:30 * Ingo Molnar <mingo(a)elte.hu> wrote: > * tytso(a)mit.edu <tytso(a)mit.edu> wrote: > > > [...] Not only has the source code been made available, but hundreds of > > engineering hours have been made trying to accomodate the demands of LKML > > --- and LKML has said no to suspend blockers/wakelocks. > > I dont think you are being fair here, at all. > > Firstly, the suspend-blockers feature is not being rejected (fixing and > extending suspend is a worthwile goal), it's just that various different > schemes have been proposed by the people who'll eventually have to maintain > that code down the line. Btw., i'd like to summarize the scheduler based suspend scheme proposed by Thomas Gleixner, Peter Zijlstra and myself. I found no good summary of it in the big thread, and there are also new elements of the proposal: - Create a 'deep idle' mode that suspends. This, if all constraints are met, is triggered by the scheduler automatically: just like the other idle modes are triggered currently. This approach fixes the wakeup races because an incoming wakeup event will set need_resched() and abort the suspend. ( This mode can even use the existing suspend code to bring stuff down, therefore it also solves the pending timer problem and works even on PC style x86. ) - Introduce a 'minimum wakeup latency' task attribute (task->latency), settable via a scheduler syscall. This is an ABI that influences the kernel how idle the system can go. (i.e. the equivalent of suspend blockers, just not binary and not system-wide.) - Solve crappy app confinement via the scheduler: A first proposal was to use the existing cgroup mechanism, but we found a different and probably more elegant solution: We can slightly extend the scheduler and introduce another per task 'minimum latency other tasks are allowed to run' scheduling attribute (task->exclude_latency) - set via a scheduler syscall as well. (only settable by privileged tasks - such as the screensaver.) This allows a task to 'exclude' other tasks that dont have low-latency requirements. Crappy apps would have a large latency value, so they'd be idled out when a privileged task sets the exclusion level low enough. In the case of Android, this would for example be used by the screensaver to introduce different levels of runnability/idling. [ Note that this scheme would also be useful in a completely different scenario, for real-time tasks as well: it would allow extreme-RT tasks to quiescence all lower prio tasks in a controlled manner. (even if the RT task is sleeping) ] - Controlled auto-suspend: drivers (such as input) could on wakeup automatically set the 'minimum wakeup latency' value of wakee tasks to a lower value. This automatically prevents another auto-suspend in the near future: up to the point the wakee task increases its latency (via the scheduler syscall) again and allows suspend again. This means there will be no surprise suspends for a task that may take a bit longer than usual to finish its work. [ Detail: this would only be done for tasks that have a non-default (non-infinity) task->latency value - to prevent the input driver from lowering latency values (and preventing future suspends) just because some unaware apps are running and using input drivers. ] All in one, this scheme allows everything without exception that suspend-blockers allows and supports all the important usecases: - allows agressive auto-idling - has no wakeup races - allows crappy-app confinement and other finegrained suspend control - it should be pretty easy to adopt by Android as well, as it goes along similar principles of kernel automatisms combined with user-space controlled task and system attributes. It's straightforward to adapt and it is also more generic, more clean and more flexible than suspend-blockers. Please mention any remaining technical issues that may still be are unaddressed. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds on 3 Jun 2010 19:50 On Fri, 4 Jun 2010, Ingo Molnar wrote: > > This allows a task to 'exclude' other tasks that dont have low-latency > requirements. Crappy apps would have a large latency value, so they'd > be idled out when a privileged task sets the exclusion level low enough. Quite frankly, this sounds fundamentally broken. Think deadlock. The high-latency task got a lock, and now you're excluding it because it scheduled away. So from my perspective, putting that kind of logic deep in the system sounds like the _last_ thing we want to do. I think it's much saner to have a very targeted suspend blocker that only blocks the opportunistic suspends and has _zero_ interaction with the rest of the system (certainly none at all with core code like the scheduler). And if somebody then suspends the traditional way (by an actual suspend event, not that opportunistic thing), then the suspend blocker does nothing at all - because it simply doesn't even _exist_ at that level. It's only about the opportunistic suspends. (I'd further suggest that disk wait and running in kernel mode disable any opportunistic suspend anyway - but that's not about suspend blockers as much as it is about just the opportunistic suspend itself). Linus -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 3 Jun 2010 19:50
* Linus Torvalds <torvalds(a)linux-foundation.org> wrote: > On Fri, 4 Jun 2010, Ingo Molnar wrote: > > > > This allows a task to 'exclude' other tasks that dont have low-latency > > requirements. Crappy apps would have a large latency value, so they'd > > be idled out when a privileged task sets the exclusion level low enough. > > Quite frankly, this sounds fundamentally broken. > > Think deadlock. The high-latency task got a lock, and now you're excluding > it because it scheduled away. Mail was a bit too long already so i trimmed it at the wrong place :-/ What you say is absolutely true, hence this would be driven via sched_tick() + TIF notifiers - i.e. only ever treat user-mode tasks as 'idle-able'. This can be done with no overhead to the regular fastpaths. The TIF notifier would be the one scheduling to idle - and would thus do it only to user-mode tasks. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |