Unify KVM kernel-space and user-space code into a single project
in [Kernel]
|
Prev: Irish 2010 Grant Winner
Next: [PATCH] staging: winbond: mds_f.h whitespace and CamelCase corrections.
From: Avi Kivity on 18 Mar 2010 13:20 On 03/18/2010 07:02 PM, Ingo Molnar wrote: > > I find the 'KVM mostly cares about the server, not about the desktop' attitude > expressed in this thread troubling. > It's not kvm, just it's developers (and their employers, where applicable). If you post desktop oriented patches I'm sure they'll be welcome. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 13:20 * Avi Kivity <avi(a)redhat.com> wrote: > On 03/18/2010 04:09 PM, Ingo Molnar wrote: > >* Avi Kivity<avi(a)redhat.com> wrote: > > > >>> That is not what i said. I said they are closely related, and where > >>> technologies are closely related, project proximity turns into project > >>> unification at a certain stage. > >> > >> I really don't see how. So what if both qemu and kvm implement an i8254? > >> They can't share any code since the internal APIs are so different. [...] > > > > I wouldnt jump to assumptions there. perf shares some facilities with the > > kernel on the source code level - they can be built both in the kernel and > > in user-space. > > > > But my main thought wasnt even to actually share the implementation - but > > to actually synchronize when a piece of device emulation moves into the > > kernel. It is arguably bad for performance in most cases when Qemu handles > > a given device - so all the common devices should be kernel accelerated. > > > > The version and testing matrix would be simplified significantly as well: > > as kernel and qemu goes hand in hand, they are always on the same version. > > So, you propose to allow running tools/kvm/ only on the kernel it was > shipped with? No, but i propose concentrating on that natural combination. > Otherwise the testing matrix isn't simplified. It is, because testing is more focused and more people are testing the combination that developers tested as well. (and not some random version combination picked by the distributor or the user) Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 13:30 * Avi Kivity <avi(a)redhat.com> wrote: > On 03/18/2010 07:02 PM, Ingo Molnar wrote: > > > > I find the 'KVM mostly cares about the server, not about the desktop' > > attitude expressed in this thread troubling. > > It's not kvm, just it's developers (and their employers, where applicable). > If you post desktop oriented patches I'm sure they'll be welcome. Just such a patch-set was posted in this very thread: 'perf kvm'. There were two negative reactions immediately, both showed a fundamental server versus desktop bias: - you did not accept that the most important usecase is when there is a single guest running. - the reaction to the 'how do we get symbols out of the guest' sub-question was, paraphrased: 'we dont want that due to <unspecified> security threat to XYZ selinux usecase with lots of guests'. Anyone being aware of how Linux and KVM is being used on the desktop will know how detached that attitude is from the typical desktop usecase ... Usability _never_ sucks because of lack of patches or lack of suggestions. I bet if you made the next server feature contingent on essential usability fixes they'd happen overnight - for God's sake there's been 1000 commits in the last 3 months in the Qemu repository so there's plenty of manpower... Usability suckage - and i'm not going to be popular for saying this out loud - almost always shows a basic maintainer disconnect with the real world. See your very first reactions to my 'KVM usability' observations. Read back your and Anthony's replies: total 'sure, patches welcome' kind of indifference. It is _your project_, not some other project down the road ... So that is my first-hand experience about how you are welcoming these desktop issues, in this very thread. I suspect people try a few times with suggestions, then get shot down like our suggestions were shot down and then give up. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 13:50 * Anthony Liguori <anthony(a)codemonkey.ws> wrote: > On 03/18/2010 06:48 AM, Ingo Molnar wrote: > >* Avi Kivity<avi(a)redhat.com> wrote: > > > >>On 03/18/2010 12:50 PM, Ingo Molnar wrote: > >>>* Avi Kivity<avi(a)redhat.com> wrote: > >>> > >>>>>The moment any change (be it as trivial as fixing a GUI detail or as > >>>>>complex as a new feature) involves two or more packages, development speed > >>>>>slows down to a crawl - while the complexity of the change might be very > >>>>>low! > >>>>Why is that? > >>>It's very simple: because the contribution latencies and overhead compound, > >>>almost inevitably. > >>It's not inevitable, if the projects are badly run, you'll have high > >>latencies, but projects don't have to be badly run. > >So the 64K dollar question is, why does Qemu still suck? > > Why does Linux AIO still suck? Why do we not have a proper interface in > userspace for doing asynchronous file system operations? Good that you mention it, i think it's an excellent example. The suckage of kernel async IO is for similar reasons: there's an ugly package separation problem between the kernel and between glibc - and between the apps that would make use of it. ( With the separated libaio it was made worse: there were 3 libraries to work with, and even less applications that could make use of it ... ) So IMO klibc is an arguably good idea - eventually hpa will get around posting it for upstream merging again. Then we could offer both new libraries much faster, and could offer things like comprehensive AIO used pervasively within existing APIs. > Why don't we have an interface in userspace to do zero-copy transmit and > receive of raw network packets? > > The lack of a decent userspace API for asynchronous file system operations > is a huge usability problem for us. Take a look at the complexity of our > -drive option. It's all because the kernel gives us sucky interfaces. If you had your bits in tools/kvm/ you could make a strong case for a good kaio implementation - coupled with an actual, working use-case. ( You could use the raw syscall even without klibc. ) We could see the arguments on lkml turn from: 'do we want this and it will take years to propagate this into apps' into something like: ' Exactly how much faster does kvm go? and I'd get is straight away with my next kernel update tomorrow? Wow! ' Ok, i exaggerated a bit - but you get the idea. It's a much different picture when kernel developers and maintainers see an actual use-case, _right in the kernel repo they work with every day_. Currently there's a wall between kernel developers and user-space developers, and there's somewhat of an element of fear and arrogance on both sides. For efficient technology such walls needs torn down and people need a bit more experience with each other's areas. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 18 Mar 2010 14:10
* Anthony Liguori <anthony(a)codemonkey.ws> wrote: > On 03/18/2010 10:17 AM, Ingo Molnar wrote: > >* Anthony Liguori<anthony(a)codemonkey.ws> wrote: > > > >>On 03/18/2010 08:00 AM, Ingo Molnar wrote: > >>>>[...] kvm in fact knows nothing about vga, to take your last example. > >>>>[...] > >>>Look at the VGA dirty bitmap optimization a'ka the KVM_GET_DIRTY_LOG > >>>ioctl. > >>> > >>>See qemu/kvm-all.c's kvm_physical_sync_dirty_bitmap(). > >>> > >>>It started out as a VGA optimization (also used by live migration) and > >>>even today it's mostly used by the VGA drivers - albeit a weak one. > >>> > >>>I wish there were stronger VGA optimizations implemented, copying the > >>>dirty bitmap is not a particularly performant solution. (although it's > >>>certainly better than full emulation) Graphics performance is one of the > >>>more painful aspects of KVM usability today. > >>We have to maintain a dirty bitmap because we don't have a paravirtual > >>graphics driver. IOW, someone needs to write an Xorg driver. > >> > >>Ideally, we could just implement a Linux framebuffer device, right? > >No, you'd want to interact with DRM. > > Using DRM doesn't help very much. You still need an X driver and most of > the operations you care about (video rendering, window movement, etc) are > not operations that need to go through DRM. You stripped out this bit from my reply: > > There are all kernel space projects, going through Xorg would be a > > horrible waste of performance for full-screen virtualization. It's fine > > for the windowed or networked case (and good as a compatibility fallback), > > but very much not fine for local desktop use. For the full-screen case (which is a very common mode of using a guest OS on the desktop) there's not much of window management needed. You need to save/restore as you switch in/out. > 3D graphics virtualization is extremely difficult in the non-passthrough > case. It really requires hardware support that isn't widely available today > (outside a few NVIDIA chipsets). Granted it's difficult in the general case. > >>Xorg framebuffer driver doesn't implement any of the optimizations that the > >>Linux framebuffer supports and the Xorg driver does not provide use the > >>kernel's interfaces for providing update regions. > >> > >>Of course, we need to pull in X into the kernel to fix this, right? > > > > FYI, this part of X has already been pulled into the kernel, it's called > > DRM. If then it's being expanded. > > It doesn't provide the things we need to a good user experience. You need > things like an absolute input device, host driven display resize, RGBA > hardware cursors. None of these go through DRI and it's those things that > really provide the graphics user experience. With KSM the display resize is in the kernel. Cursor management is not. Yet: i think it would be a nice feature as the cursor could move even if Xorg is blocked or busy with other things. > >> Any sufficiently complicated piece of software is going to interact with > >> a lot of other projects. The solution is not to pull it all into one > >> massive repository. It's to build relationships and to find ways to > >> efficiently work with the various communities. > > > > That's my whole point with this thread: the kernel side of KVM and qemu, > > but all practical purposes should not be two 'separate communities'. They > > should be one and the same thing. > > I don't know why you keep saying this. The people who are in these > "separate communities" keep claiming that they don't feel this way. If you are not two separate communities but one community, then why do you go through the (somewhat masochistic) self-punishing excercise of keeping the project in two different pieces? In a distant past Qemu was a separate project and KVM was just a newcomer who used it for fancy stuff. Today as you say(?) the two communities are one and the same. Why not bring it to its logical conclusion? > I'm not just saying this to be argumentative. Many of the people in the > community have thought this same thing, and tried it themselves, and we've > all come to the same conclusion. > > It's certainly possible that we just missed the obvious thing to do but > we'll never know that unless someone shows us. I'm not aware of anyone in the past having attempted to move qemu to tools/kvm/ in the uptream kernel repo, and having reported on the experiences with such a contribution setup. (obviously it's not possible at all without heavy cooperation and acceptance from you and Avi, so this will probably remain a thought experiment forever) If then you must refer to previous attempts to 'strip down' Qemu, right? Those attempts didnt really solve the fundamental problem of project code base separation. Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |