Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Avi Kivity on 18 Mar 2010 10:30 On 03/18/2010 03:57 PM, Ingo Molnar wrote: > >> [...] RHEL 5 is still on 2.6.18, for example. Users >> don't like their kernels updated unless absolutely necessary, with >> good reason. >> > Nope - RHEL 5 is on a 2.6.18 base for entirely different reasons. > All the reasons have 'stability' in them. >> Kernel updates = reboots. >> > If you check the update frequency of RHEL 5 kernels you'll see that it's > comparable to that of Fedora. > I'm sorry to say that's pretty bad. Users don't want to update their kernels. >>> - Qemu on the other hand is not upgraded with (nearly) that level of urgency. >>> Completely new versions will generally have to wait for the next distro >>> release. >>> >> F12 recently updated to 2.6.32. This is probably due to 2.6.31.stable >> dropping away, and no capacity at Fedora to maintain it on their own. So >> they are caught in a bind - stay on 2.6.31 and expose users to security >> vulnerabilities or move to 2.6.32 and cause regressions. Not a happy >> choice. >> > Happy choice or not, this is what i said is the distro practice these days. (i > dont know all the distros that well so i'm sure there's differences) > So in addition to all the normal kernel regressions, you want to force tools/kvm/ regressions on users. >> I don't mind at all if rawhide users run on the latest and greatest, but >> release users deserve a little more stability. >> > What are you suggesting, that released versions of KVM are not reliable? Of > course any tools/ bits are release engineered just as much as the rest of KVM > ... > No, I am suggesting qemu-kvm.git is not as stable as released versions (and won't get fixed backported). Keep in mind that unlike many userspace applications, qemu exposes an ABI to guests which we must keep compatible. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 18 Mar 2010 10:40 On 03/18/2010 04:09 PM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >>> That is not what i said. I said they are closely related, and where >>> technologies are closely related, project proximity turns into project >>> unification at a certain stage. >>> >> I really don't see how. So what if both qemu and kvm implement an i8254? >> They can't share any code since the internal APIs are so different. [...] >> > I wouldnt jump to assumptions there. perf shares some facilities with the > kernel on the source code level - they can be built both in the kernel and in > user-space. > > But my main thought wasnt even to actually share the implementation - but to > actually synchronize when a piece of device emulation moves into the kernel. > It is arguably bad for performance in most cases when Qemu handles a given > device - so all the common devices should be kernel accelerated. > > The version and testing matrix would be simplified significantly as well: as > kernel and qemu goes hand in hand, they are always on the same version. > So, you propose to allow running tools/kvm/ only on the kernel it was shipped with? Otherwise the testing matrix isn't simplified. >> [...] Even worse for the x86 emulator as qemu and kvm are fundamentally >> different. >> > So is it your argument that the difference and the duplication in x86 > instruction emulation is a good thing? Of course it isn't a good thing, but it is unavoidable. Qemu compiles code just-in-time to avoid interpretation overhead, while kvm emulates one instruction at a time. No caching is possible, especially with ept/npt, since the guest is free to manipulate memory with no notification to the host. Qemu also supports the full instruction set while kvm only implements what is necessary. Qemu is a multi-source/multi-target translator while kvm's emulator is x86 specific. > You said it some time ago that > the kvm x86 emulator was very messy and you wish it was cleaner. > It's still messy but is being cleaned up. > While qemu's is indeed rather different (it's partly a translator/JIT), i'm > sure the decoder logic could be shared - and qemu has a slow-path > full-emulation fallback in any case, which is similar to what in-kernel > emulator does (IIRC ...). > > That might have changed meanwhile. > IIUC it only ever translates. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 10:40 * Avi Kivity <avi(a)redhat.com> wrote: > > Happy choice or not, this is what i said is the distro practice these > > days. (i dont know all the distros that well so i'm sure there's > > differences) > > So in addition to all the normal kernel regressions, you want to force > tools/kvm/ regressions on users. So instead you force a NxN compatibility matrix [all versions of qemu combined with all versions of the kernel] instead of a linear N versions matrix with a clear focus on the last version. Brilliant engineering i have to say ;-) Also, by your argument the kernel should be split up into a micro-kernel, with different packages for KVM, scheduler, drivers, upgradeable separately. That would be a nightmare. (i can detail many facets of that nightmare if you insist but i'll spare the electrons for now) Fortunately few kernel developers share your views about this. > > > I don't mind at all if rawhide users run on the latest and greatest, but > > > release users deserve a little more stability. > > > > What are you suggesting, that released versions of KVM are not reliable? > > Of course any tools/ bits are release engineered just as much as the rest > > of KVM ... > > No, I am suggesting qemu-kvm.git is not as stable as released versions (and > won't get fixed backported). Keep in mind that unlike many userspace > applications, qemu exposes an ABI to guests which we must keep compatible. I think you still dont understand it: if a tool moves to the kernel repo, then it is _released stable_ together with the next stable kernel. I.e. you'd get a stable qemu-2.6.34 in essence, when v2.6.34 is released. You get minor updates with 2.6.34.1, 2.6.34.2, 2.6.34.3, etc - while development continues. I.e. you get _more_ stability, because a matching kernel is released with a matching Qemu. Qemu might have a different release schedule. Which, i argue, is not a good thing for exactly that reason :-) If it moved to tools/kvm/ it would get the same 90 days release frequency, merge window and stabilization window treatment as the upstream kernel. Furthermore, users can also run experimental versions of qemu together with experimental versions of the kernel, by running something like 2.6.34-rc1 on Rawhide. Even if they dont download the latest qemu git and build it. I.e. clearly _more_ is possible in such a scheme. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 18 Mar 2010 10:40 On 03/17/2010 03:10 AM, Ingo Molnar wrote: > * Anthony Liguori<anthony(a)codemonkey.ws> wrote: > > >> On 03/16/2010 12:39 PM, Ingo Molnar wrote: >> >>>> If we look at the use-case, it's going to be something like, a user is >>>> creating virtual machines and wants to get performance information about >>>> them. >>>> >>>> Having to run a separate tool like perf is not going to be what they would >>>> expect they had to do. Instead, they would either use their existing GUI >>>> tool (like virt-manager) or they would use their management interface >>>> (either QMP or libvirt). >>>> >>>> The complexity of interaction is due to the fact that perf shouldn't be a >>>> stand alone tool. It should be a library or something with a programmatic >>>> interface that another tool can make use of. >>>> >>> But ... a GUI interface/integration is of course possible too, and it's being >>> worked on. >>> >>> perf is mainly a kernel developer tool, and kernel developers generally dont >>> use GUIs to do their stuff: which is the (sole) reason why its first ~850 >>> commits of tools/perf/ were done without a GUI. We go where our developers >>> are. >>> >>> In any case it's not an excuse to have no proper command-line tooling. In fact >>> if you cannot get simpler, more atomic command-line tooling right then you'll >>> probably doubly suck at doing a GUI as well. >>> >> It's about who owns the user interface. >> >> If qemu owns the user interface, than we can satisfy this in a very simple >> way by adding a perf monitor command. If we have to support third party >> tools, then it significantly complicates things. >> > Of course illogical modularization complicates things 'significantly'. > Ok. Then apply this to the kernel. I'm then happy to take patches. Regards, Anthony Liguori
From: Jes Sorensen on 18 Mar 2010 10:50
On 03/18/10 15:22, Ingo Molnar wrote: > > * Jes Sorensen<Jes.Sorensen(a)redhat.com> wrote: >> Both perf and oprofile are still relatively small projects in comparison to >> QEMU. > > So is your argument that the unification does not make sense due to size? > Would a smaller Qemu be more appropriate for this purpose? As I have stated repeatedly in this discussion, a unification would hurt the QEMU development process because it would alienate a large number of QEMU developers who are *not* Linux kernel users. QEMU is a lot more complex than you let on. >> Well I think we are just going to agree to disagree on this one. I am not >> against merging projects where it makes sense, but in this particular case I >> am strongly convinced the loss would be much greater than the gain. > > I wish you said that based on first hand negative experience with > unifications, not based on just pure speculation. > > (and yes, i speculate too, but at least with some basis) You still haven't given us a *single* example of unification of something that wasn't purely linked to the Linux kernel. perf/ oprofile is 100% linked to the Linux kernel, QEMU is not. I wish you would actually look at what users use QEMU for. As long as you continue to purely speculate on this, to use your own words, your arguments are not holding up. And you are not being consistent either. You have conveniently continue to ignore my questions about why the file system tools are not to be merged into the Linux kernel source tree? Jes -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |