Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Ingo Molnar on 22 Mar 2010 15:20 * Avi Kivity <avi(a)redhat.com> wrote: > On 03/22/2010 06:32 PM, Ingo Molnar wrote: > > > > So, what do you think creates code communities and keeps them alive? > > Developers and code. And the wellbeing of developers are primarily > > influenced by the repository structure and by the development/maintenance > > process - i.e. by the 'fun' aspect. (i'm simplifying things there but > > that's the crux of it.) > > There is nothing fun about having one repository or two. Who cares about > this anyway? > > tools/kvm/ probably will draw developers, simply because of the glory > associated with kernel work. That's a bug, not a feature. It means that > effort is not distributed according to how it's needed, but because of > irrelevant considerations. And yet your solution to that is to ... do all your work in the kernel space and declare the tooling as something that does not interest you? ;-) > Something I've wanted for a long time is to port kvm_stat to use tracepoints > instead of the home-grown instrumentation. But that is unrelated to this > new tracepoint. Other than that we're satisfied with ftrace. Despite it being another in-kernel subsystem that by your earlier arguments should be done via a user-space package? ;-) > > You should realize that naturally developers will gravitate towards the > > most 'fun' aspects of a project. It is the task of the maintainer to keep > > the balance between fun and utility, bugs and features, quality and > > code-rot. > > There are plenty of un-fun tasks (like fixing bugs and providing RAS > features) that we're doing. We don't do this for fun but to satisfy our > users. So which one is it, KVM developers are volunteers that do fun stuff and cannot be told about project priorities, or KVM developers are pros who do unfun stuff because they can be told about priorities? I posit that it's both: and that priorities can be communicated - if only you try as a maintainer. All i'm suggesting is to add 'usable, unified user-space' to the list of unfun priorities, because it's possible and because it matters. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 22 Mar 2010 15:20 On 03/22/2010 12:55 PM, Avi Kivity wrote: >> Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by >> Anthony. >> There's numerous ways that this can break: > > I don't like it either. We have libvirt for enumerating guests. We're stuck in a rut with libvirt and I think a lot of the dissatisfaction with qemu is rooted in that. It's not libvirt that's the probably, but the relationship between qemu and libvirt. We add a feature to qemu and maybe after six month it gets exposed by libvirt. Release time lines of the two projects complicate the situation further. People that write GUIs are limited by libvirt because that's what they're told to use and when they need something simple, they're presented with first getting that feature implemented in qemu, then plumbed through libvirt. It wouldn't be so bad if libvirt was basically a passthrough interface to qemu but it tries to model everything in a generic way which is more or less doomed to fail when you're adding lots of new features (as we are). The list of things that libvirt doesn't support and won't any time soon is staggering. libvirt serves an important purpose, but we need to do a better job in qemu with respect to usability. We can't just punt to libvirt. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 22 Mar 2010 15:30 On 03/22/2010 09:20 PM, Joerg Roedel wrote: > >> Why doesnt it solve the bisectability problem? The kernel repo is supposed to >> be bisectable so that problem would be solved. >> > Because Marcelo and Avi try to keep as close to upstream qemu as > possible. So the qemu repo is regularly merged in qemu-kvm and if you > want to bisect you may end up somewhere in the middle of the qemu > repository which has only very minimal kvm-support. > The problem here is that two qemu repositorys exist. But the current > effort of Anthony is directed to create a single qemu repository. But > thats not done overnight. > It's in fact possible to bisect qemu-kvm.git. If you end up in qemu.git, do a 'git bisect skip'. If you end up in a merge, call the merge point A, bisect A^1..A^2, each time merging A^1 before compiling (the merge is always trivial due to the way we do it). Not fun, but it works. When we complete merging kvm integration into qemu.git, this problem will disappear. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 22 Mar 2010 15:30 On 03/22/2010 02:22 PM, Ingo Molnar wrote: >> Transitive had a product that was using a KVM context to run their >> binary translator which allowed them full access to the host >> processes virtual address space range. In this case, there is no >> kernel and there are no devices. >> > And your point is that such vcpus should be excluded from profiling just > because they fall outside the Qemu/libvirt umbrella? > You don't instrument it the way you'd instrument an operating system so no, you don't want it to show up in perf kvm top. Regards, Anthony Liguori > Ingo > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 22 Mar 2010 15:30
* Anthony Liguori <anthony(a)codemonkey.ws> wrote: > On 03/22/2010 12:34 PM, Ingo Molnar wrote: > >* Avi Kivity<avi(a)redhat.com> wrote: > > > >>>>> - Easy default reference to guest instances, and a way for tools to > >>>>> reference them symbolically as well in the multi-guest case. Preferably > >>>>> something trustable and kernel-provided - not some indirect information > >>>>> like a PID file created by libvirt-manager or so. > >>>>Usually 'layering violation' is trotted out at such suggestions. > >>>>[...] > >>>That's weird, how can a feature request be a 'layering violation'? > >>The 'something trustable and kernel-provided'. The kernel knows nothing > >>about guest names. > >The kernel certainly knows about other resources such as task names or network > >interface names or tracepoint names. This is kernel design 101. > > > >>>If something that users find straightforward and usable is a layering > >>>violation to you (such as easily being able to access their own files on > >>>the host as well ...) then i think you need to revisit the definition of > >>>that term instead of trying to fix the user. > >>Here is the explanation, you left it quoted: > >> > >>>>[...] I don't like using the term, because sometimes the layers are > >>>>incorrect and need to be violated. But it should be done explicitly, not > >>>>as a shortcut for a minor feature (and profiling is a minor feature, most > >>>>users will never use it, especially guest-from-host). > >>>> > >>>>The fact is we have well defined layers today, kvm virtualizes the cpu > >>>>and memory, qemu emulates devices for a single guest, libvirt manages > >>>>guests. We break this sometimes but there has to be a good reason. So > >>>>perf needs to talk to libvirt if it wants names. Could be done via > >>>>linking, or can be done using a pluging libvirt drops into perf. > >This is really just the much-discredited microkernel approach for keeping > >global enumeration data that should be kept by the kernel ... > > > >Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. > >There's numerous ways that this can break: > > > > - Those special files can get corrupted, mis-setup, get out of sync, or can > > be hard to discover. > > > > - The ${HOME}/.qemu/qmp/ solution suggested by Anthony has a very obvious > > design flaw: it is per user. When i'm root i'd like to query _all_ current > > guest images, not just the ones started by root. A system might not even > > have a notion of '${HOME}'. > > > > - Apps might start KVM vcpu instances without adhering to the > > ${HOME}/.qemu/qmp/ access method. > > Not all KVM vcpus are running operating systems. But we want to allow developers to instrument all of them ... > Transitive had a product that was using a KVM context to run their > binary translator which allowed them full access to the host > processes virtual address space range. In this case, there is no > kernel and there are no devices. And your point is that such vcpus should be excluded from profiling just because they fall outside the Qemu/libvirt umbrella? That is a ridiculous position. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |