Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Andrea Arcangeli on 22 Mar 2010 15:30 On Mon, Mar 22, 2010 at 08:10:28PM +0100, Ingo Molnar wrote: > I posit that it's both: and that priorities can be communicated - if only you > try as a maintainer. All i'm suggesting is to add 'usable, unified user-space' > to the list of unfun priorities, because it's possible and because it matters. IMHO blaming anybody for it but qemu maintainership is very unfair. They intentionally reinveinted a less self contained, inferior, underperforming, underfeatured wheel instead of doing the right thing and just making sure that it as self contained enough as possible to avoid risking destabilizing their existing codebase. What can anybody (without qemu git commit access) do about it unless qemu git maintainer change attitude, dumps its qemu/kvm-all.c nosense for good, and do the right thing so we can unify for real? We need to move forward, including multithread the qemu core and be ready to include desktop virtualization protocol when they're ready for submission without being suggested to extend vnc instead to gain a similar speedup (i.e. yet another inferior wheel). Unification means that _all_ qemu users, pure research, theoretical interest, Xen, virtualbox, weird pure software architecture, will be able to push their stuff in for the common good, but that also shall apply to KVM! It has to become clear that reinveinting inferior wheels instead of merging the real thing, is absolutely time wasteful, unnecessary, and it won't make any difference as far as KVM is concerned, proof is that 0% of userbase runs qemu git to run KVM (except the kvm-all.c developers to test it perhaps or somebody by mistake not adding -kvm prefix to command line maybe). I don't pretend to rate KVM as more important than all the rest of niche usages for qemu but it shall be _as_ important as the rest and it'd be nice one day to be able to install only qemu on a system and get something actually usable in production. I very much like that qemu gets contributions from everywhere, it's also nice it can run without KVM (that is purely useful as a debugging tool to me but still...). I think it can all happen and unification should be the object for the gain of everyone in both qemu/kvm and even xen and all the rest. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 22 Mar 2010 15:30 * Avi Kivity <avi(a)redhat.com> wrote: > > Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by > > Anthony. There's numerous ways that this can break: > > I don't like it either. We have libvirt for enumerating guests. Which has pretty much the same problems to the ${HOME}/.qemu/qmp/ solution, obviously. > > - Those special files can get corrupted, mis-setup, get out of sync, or can > > be hard to discover. > > > > - The ${HOME}/.qemu/qmp/ solution suggested by Anthony has a very obvious > > design flaw: it is per user. When i'm root i'd like to query _all_ current > > guest images, not just the ones started by root. A system might not even > > have a notion of '${HOME}'. > > > > - Apps might start KVM vcpu instances without adhering to the > > ${HOME}/.qemu/qmp/ access method. > > - it doesn't work with nfs. So out of a list of 4 disadvantages your reply is that you agree with 3? > > - There is no guarantee for the Qemu process to reply to a request - while > > the kernel can always guarantee an enumeration result. I dont want 'perf > > kvm' to hang or misbehave just because Qemu has hung. > > If qemu doesn't reply, your guest is dead anyway. Erm, but i'm talking about a dead tool here. There's a world of a difference between 'kvm top' not showing new entries (because the guest is dead), and 'perf kvm top' hanging due to Qemu hanging. So it's essentially 4 our of 4. Yet your reply isnt "Ingo you are right" but "hey, too bad" ? > > Really, for such reasons user-space is pretty poor at doing system-wide > > enumeration and resource management. Microkernels lost for a reason. > > Take a look at your desktop, userspace is doing all of that everywhere, from > enumerating users and groups, to deciding how your disks are named. The > kernel only provides the bare facilities. We dont do that for robust system instrumentation, for heaven's sake! By your argument it would be perfectly fine to implement /proc purely via user-space, correct? > > You are committing several grave design mistakes here. > > I am committing on the shoulders of giants. Really, this is getting outright ridiculous. You agree with me that Anothony suggested a technically inferior solution, yet you even seem to be proud of it and are joking about it? And _you_ are complaining about lkml-style hard-talk discussions? Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 22 Mar 2010 15:30 On 03/22/2010 09:10 PM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >> On 03/22/2010 06:32 PM, Ingo Molnar wrote: >> >>> So, what do you think creates code communities and keeps them alive? >>> Developers and code. And the wellbeing of developers are primarily >>> influenced by the repository structure and by the development/maintenance >>> process - i.e. by the 'fun' aspect. (i'm simplifying things there but >>> that's the crux of it.) >>> >> There is nothing fun about having one repository or two. Who cares about >> this anyway? >> >> tools/kvm/ probably will draw developers, simply because of the glory >> associated with kernel work. That's a bug, not a feature. It means that >> effort is not distributed according to how it's needed, but because of >> irrelevant considerations. >> > And yet your solution to that is to ... do all your work in the kernel space > and declare the tooling as something that does not interest you? ;-) > I have done plenty of userspace work in qemu. I don't have a lack of interest in qemu, just in a desktop GUI. I'm not a GUI person and my employer doesn't have a desktop-on-desktop virtualization product that I know of. >> Something I've wanted for a long time is to port kvm_stat to use tracepoints >> instead of the home-grown instrumentation. But that is unrelated to this >> new tracepoint. Other than that we're satisfied with ftrace. >> > Despite it being another in-kernel subsystem that by your earlier arguments > should be done via a user-space package? ;-) > I'm satisfied with it as a user. Architecturally, I'd have preferred it to be a userspace tool. It might have improved usability as well to have something with --help instead of a set of debugfs files. But I'm a lot happier with ftrace existing as a kernel component than not at all. >>> You should realize that naturally developers will gravitate towards the >>> most 'fun' aspects of a project. It is the task of the maintainer to keep >>> the balance between fun and utility, bugs and features, quality and >>> code-rot. >>> >> There are plenty of un-fun tasks (like fixing bugs and providing RAS >> features) that we're doing. We don't do this for fun but to satisfy our >> users. >> > So which one is it, KVM developers are volunteers that do fun stuff and cannot > be told about project priorities, or KVM developers are pros who do unfun > stuff because they can be told about priorities? > From my point of view as maintainer, all contributors are volunteers, I can't tell any of them what to do. From the point of view of many of these volunteer's employers, they are wage slaves who do as they're told or else. So: when someone sends me a patch I gratefully accept if it is good or point out the issues if not. At the secret Red Hat headquarters and the kvm weekly conference call I participate in deciding priorities and task assignments. > I posit that it's both: and that priorities can be communicated - if only you > try as a maintainer. All i'm suggesting is to add 'usable, unified user-space' > to the list of unfun priorities, because it's possible and because it matters. > So: I require a volunteer to write some GUI code before I accept a patch. Back at the Red Hat lair, we think of what features we drop from the product because the kvm maintainer has gone nuts. The 'unified' part of your suggestion is not a requirement, but an implementation detail. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 22 Mar 2010 15:40 On 03/22/2010 02:31 PM, Daniel P. Berrange wrote: > On Mon, Mar 22, 2010 at 02:15:35PM -0500, Anthony Liguori wrote: > >> On 03/22/2010 12:55 PM, Avi Kivity wrote: >> >>>> Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by >>>> Anthony. >>>> There's numerous ways that this can break: >>>> >>> I don't like it either. We have libvirt for enumerating guests. >>> >> We're stuck in a rut with libvirt and I think a lot of the >> dissatisfaction with qemu is rooted in that. It's not libvirt that's >> the probably, but the relationship between qemu and libvirt. >> >> We add a feature to qemu and maybe after six month it gets exposed by >> libvirt. Release time lines of the two projects complicate the >> situation further. People that write GUIs are limited by libvirt >> because that's what they're told to use and when they need something >> simple, they're presented with first getting that feature implemented in >> qemu, then plumbed through libvirt. >> > That is somewhat unfair as a blanket statement! > Sorry, you're certainly correct. Some features appear quickly, but others can take an awfully long time. >> It wouldn't be so bad if libvirt was basically a passthrough interface >> to qemu but it tries to model everything in a generic way which is more >> or less doomed to fail when you're adding lots of new features (as we are). >> >> The list of things that libvirt doesn't support and won't any time soon >> is staggering. >> > As previously discussed, we want to improve both the set of features > supported, and make it much easier to support new features promptly. > The QMP& qdev stuff has been a very good step forward in making it > easier to support QEMU management. There have been a proposals from > several people, yourself included, on how to improve libvirt's support > for the full range of QEMU features. We're committed to looking at this > and figuring out which proposals are practical to support, so we can > improve QEMU& libvirt interaction for everyone. > Regards, Anthony Liguori > Regards, > Daniel > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 22 Mar 2010 15:50
On 03/22/2010 09:22 PM, Ingo Molnar wrote: > >> Transitive had a product that was using a KVM context to run their >> binary translator which allowed them full access to the host >> processes virtual address space range. In this case, there is no >> kernel and there are no devices. >> > And your point is that such vcpus should be excluded from profiling just > because they fall outside the Qemu/libvirt umbrella? > > That is a ridiculous position. > > Non-guest vcpus will not be able to provide Linux-style symbols. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |