Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Anthony Liguori on 18 Mar 2010 10:50 On 03/17/2010 03:10 AM, Ingo Molnar wrote: > - move a clean (and minimal) version of the Qemu code base to tools/kvm/, in > the upstream kernel repo, and work on that from that point on. > QEMU is about 600k LOC. We have a mechanism to compile out portions of the code but a lot things are tied together in an intimate way. In the long run, we're working on adding stronger interfaces such that we can split components out into libraries that are consumable by other applications. Simplying forking the device model won't work. Well more than half of our contributors are not coming from KVM developers/users. If you just fork the device models, you start to lose a ton of fixes (look at Xen and VirtualBox). So feel free to either 1) apply my previous patch and then start working on a "clean (and minimal)" QEMU or 2) wait to commit my previous patch and start sending patches to clean up QEMU. Absolute none of this is going to give you a VirtualBox like GUI for QEMU. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 18 Mar 2010 11:00 On 03/18/2010 04:36 PM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >>> Happy choice or not, this is what i said is the distro practice these >>> days. (i dont know all the distros that well so i'm sure there's >>> differences) >>> >> So in addition to all the normal kernel regressions, you want to force >> tools/kvm/ regressions on users. >> > So instead you force a NxN compatibility matrix [all versions of qemu combined > with all versions of the kernel] instead of a linear N versions matrix with a > clear focus on the last version. Brilliant engineering i have to say ;-) > Thanks. In fact with have an QxKxGxT compatibility matrix since we need to keep compatibility with guests and with tools. Since the easiest interface to keep compatible is the qemu/kernel interface, allowing the kernel and qemu to change independently allows reducing the compatibility matrix while still providing some improvements. Regardless of that I'd keep binary compatibility anyway. Not everyone is on the update treadmill with everything updating every three months and those people appreciate stability. I intend to keep providing it. > Also, by your argument the kernel should be split up into a micro-kernel, with > different packages for KVM, scheduler, drivers, upgradeable separately. > Some kernels do provide some of that facility (without being microkernels), for example the Windows and RHEL kernels. So it seems people want it. > That would be a nightmare. (i can detail many facets of that nightmare if you > insist but i'll spare the electrons for now) Fortunately few kernel developers > share your views about this. > I'm not sure you know my views about this. >>>> I don't mind at all if rawhide users run on the latest and greatest, but >>>> release users deserve a little more stability. >>>> >>> What are you suggesting, that released versions of KVM are not reliable? >>> Of course any tools/ bits are release engineered just as much as the rest >>> of KVM ... >>> >> No, I am suggesting qemu-kvm.git is not as stable as released versions (and >> won't get fixed backported). Keep in mind that unlike many userspace >> applications, qemu exposes an ABI to guests which we must keep compatible. >> > I think you still dont understand it: if a tool moves to the kernel repo, then > it is _released stable_ together with the next stable kernel. > I was confused by the talk about 2.6.34-rc1, which isn't stable. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 18 Mar 2010 11:00 On 03/18/2010 06:48 AM, Ingo Molnar wrote: > * Avi Kivity<avi(a)redhat.com> wrote: > > >> On 03/18/2010 12:50 PM, Ingo Molnar wrote: >> >>> * Avi Kivity<avi(a)redhat.com> wrote: >>> >>> >>>>> The moment any change (be it as trivial as fixing a GUI detail or as >>>>> complex as a new feature) involves two or more packages, development speed >>>>> slows down to a crawl - while the complexity of the change might be very >>>>> low! >>>>> >>>> Why is that? >>>> >>> It's very simple: because the contribution latencies and overhead compound, >>> almost inevitably. >>> >> It's not inevitable, if the projects are badly run, you'll have high >> latencies, but projects don't have to be badly run. >> > So the 64K dollar question is, why does Qemu still suck? > Why does Linux AIO still suck? Why do we not have a proper interface in userspace for doing asynchronous file system operations? Why don't we have an interface in userspace to do zero-copy transmit and receive of raw network packets? The lack of a decent userspace API for asynchronous file system operations is a huge usability problem for us. Take a look at the complexity of our -drive option. It's all because the kernel gives us sucky interfaces. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on 18 Mar 2010 11:10 On 03/18/2010 08:00 AM, Ingo Molnar wrote: >> [...] kvm in fact knows nothing about vga, to take your last >> example. [...] >> > Look at the VGA dirty bitmap optimization a'ka the KVM_GET_DIRTY_LOG ioctl. > > See qemu/kvm-all.c's kvm_physical_sync_dirty_bitmap(). > > It started out as a VGA optimization (also used by live migration) and even > today it's mostly used by the VGA drivers - albeit a weak one. > > I wish there were stronger VGA optimizations implemented, copying the dirty > bitmap is not a particularly performant solution. (although it's certainly > better than full emulation) Graphics performance is one of the more painful > aspects of KVM usability today. > We have to maintain a dirty bitmap because we don't have a paravirtual graphics driver. IOW, someone needs to write an Xorg driver. Ideally, we could just implement a Linux framebuffer device, right? Well, we took that approach in Xen and that sucks even worse because the Xorg framebuffer driver doesn't implement any of the optimizations that the Linux framebuffer supports and the Xorg driver does not provide use the kernel's interfaces for providing update regions. Of course, we need to pull in X into the kernel to fix this, right? Any sufficiently complicated piece of software is going to interact with a lot of other projects. The solution is not to pull it all into one massive repository. It's to build relationships and to find ways to efficiently work with the various communities. And we're working on this with X. We'll have a paravirtual graphics driver very soon. There are no magic solutions. We need more developers working on the hard problems. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 11:20
* Anthony Liguori <anthony(a)codemonkey.ws> wrote: > On 03/18/2010 08:00 AM, Ingo Molnar wrote: > >> > >> [...] kvm in fact knows nothing about vga, to take your last example. > >> [...] > > > > Look at the VGA dirty bitmap optimization a'ka the KVM_GET_DIRTY_LOG > > ioctl. > > > > See qemu/kvm-all.c's kvm_physical_sync_dirty_bitmap(). > > > > It started out as a VGA optimization (also used by live migration) and > > even today it's mostly used by the VGA drivers - albeit a weak one. > > > > I wish there were stronger VGA optimizations implemented, copying the > > dirty bitmap is not a particularly performant solution. (although it's > > certainly better than full emulation) Graphics performance is one of the > > more painful aspects of KVM usability today. > > We have to maintain a dirty bitmap because we don't have a paravirtual > graphics driver. IOW, someone needs to write an Xorg driver. > > Ideally, we could just implement a Linux framebuffer device, right? No, you'd want to interact with DRM. ( Especially as you want to write guest accelerators passing guest-space OpenGL requests straight to the kernel DRM level. ) Especially if you want to do things like graphics card virtualization, with aspects of the graphics driver passed through to the guest OS. There are all kernel space projects, going through Xorg would be a horrible waste of performance for full-screen virtualization. It's fine for the windowed or networked case (and good as a compatibility fallback), but very much not fine for local desktop use. > Well, we took that approach in Xen and that sucks even worse because the > Xorg framebuffer driver doesn't implement any of the optimizations that the > Linux framebuffer supports and the Xorg driver does not provide use the > kernel's interfaces for providing update regions. > > Of course, we need to pull in X into the kernel to fix this, right? FYI, this part of X has already been pulled into the kernel, it's called DRM. If then it's being expanded. > Any sufficiently complicated piece of software is going to interact with a > lot of other projects. The solution is not to pull it all into one massive > repository. It's to build relationships and to find ways to efficiently > work with the various communities. That's my whole point with this thread: the kernel side of KVM and qemu, but all practical purposes should not be two 'separate communities'. They should be one and the same thing. Separation makes sense where the relationship is light or strictly hierarchical - here it's neither. KVM and Qemu is interconnected, quite fundamentally so. > And we're working on this with X. We'll have a paravirtual graphics driver > very soon. There are no magic solutions. We need more developers working > on the hard problems. The thing is, writing up a DRM connector to a guest Linux OS could be done in no time. It could be deployed to users in no time as well, with the proper development model. That after years and years of waiting proper GX support is _still_ not implemented in KVM is really telling of the efficiency of development based on such disjoint 'communities'. Maybe put up a committee as well to increase efficiency? ;-) Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |