From: Ingo Molnar on

* Zachary Amsden <zamsden(a)redhat.com> wrote:

> On 03/18/2010 11:15 AM, Ingo Molnar wrote:
> >* Zachary Amsden<zamsden(a)redhat.com> wrote:
> >
> >>On 03/18/2010 12:50 AM, Ingo Molnar wrote:
> >>>* Avi Kivity<avi(a)redhat.com> wrote:
> >>>
> >>>>>The moment any change (be it as trivial as fixing a GUI detail or as
> >>>>>complex as a new feature) involves two or more packages, development speed
> >>>>>slows down to a crawl - while the complexity of the change might be very
> >>>>>low!
> >>>>Why is that?
> >>>It's very simple: because the contribution latencies and overhead compound,
> >>>almost inevitably.
> >>>
> >>>If you ever tried to implement a combo GCC+glibc+kernel feature you'll know
> >>>...
> >>>
> >>>Even with the best-run projects in existence it takes forever and is very
> >>>painful - and here i talk about first hand experience over many years.
> >>Ingo, what you miss is that this is not a bad thing. Fact of the
> >>matter is, it's not just painful, it downright sucks.
> >Our experience is the opposite, and we tried both variants and report about
> >our experience with both models honestly.
> >
> >You only have experience about one variant - the one you advocate.
> >
> >See the assymetry?
> >
> >>This is actually a Good Thing (tm). It means you have to get your
> >>feature and its interfaces well defined and able to version forwards
> >>and backwards independently from each other. And that introduces
> >>some complexity and time and testing, but in the end it's what you
> >>want. You don't introduce a requirement to have the feature, but
> >>take advantage of it if it is there.
> >>
> >>It may take everyone else a couple years to upgrade the compilers,
> >>tools, libraries and kernel, and by that time any bugs introduced by
> >>interacting with this feature will have been ironed out and their
> >>patterns well known.
> >Sorry, but this is pain not true. The 2.4->2.6 kernel cycle debacle has taught
> >us that waiting long to 'iron out' the details has the following effects:
> >
> > - developer pain
> > - user pain
> > - distro pain
> > - disconnect
> > - loss of developers, testers and users
> > - grave bugs discovered months (years ...) down the line
> > - untested features
> > - developer exhaustion
> >
> >It didnt work, trust me - and i've been around long enough to have suffered
> >through the whole 2.5.x misery. Some of our worst ABIs come from that cycle as
> >well.
>
> You're talking about a single project and comparing it to my argument about
> multiple independent projects. In that case, I see no point in the
> discussion. If you want to win the argument by strawman, you are welcome to
> do so.

The kernel is a very complex project with many ABI issues, so all those
arguments apply to it as well. The description you gave:

| This is actually a Good Thing (tm). It means you have to get your feature
| and its interfaces well defined and able to version forwards and backwards
| independently from each other. And that introduces some complexity and
| time and testing, but in the end it's what you want. You don't introduce a
| requirement to have the feature, but take advantage of it if it is there.

matches the kernel too. We have many such situations. (Furthermore, the
tools/perf/ situation, which relates to ABIs and user-space/kernel-space
interactions is similar as well.)

Do you still think i'm making a straw-man argument?

> > Sorry, but i really think you are really trying to rationalize a
> > disadvantage here ...
>
> This could very well be true, but until someone comes forward with
> compelling numbers (as in, developers committed to working on the project,
> number of patches and total amount of code contribution), there is no point
> in having an argument, there really isn't anything to discuss other than
> opinion. My opinion is you need a really strong justification to have a
> successful fork and I don't see that justification.

I can give you rough numbers for tools/perf - if that counts for you.

For the first four months of its existence, when it was a separate project, i
had a single external contributor IIRC.

The moment it went into the kernel repo the number of contributors and
contributions skyrocketed and basically all contributions were top-notch. We
are at 60+ separate contributors now (after about 8 months upstream) - which
is still small compared to the kernel or to Qemu, but huge for a relatively
isolated project like instrumentation.

So in my estimation tools/kvm/ would certainly be popular. Whether it would be
more popular than current Qemu is hard to tell - it would be pure speculation.

Any reliable numbers for the other aspect, whether a split project creates a
more fragile and less developed ABI would be extremely hard to get. I believe
it to be true, but that's my opinion based on my experience with other
projects, extrapolated to KVM/Qemu.

Anyway, the issue is moot as there's clear opposition to the unification idea.

Too bad - there was heavy initial opposition to the arch/x86 unification as
well [and heavy opposition to tools/perf/ as well], still both worked out
extremely well :-)

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on
On 03/19/2010 12:44 AM, Ingo Molnar wrote:
>
> Too bad - there was heavy initial opposition to the arch/x86 unification as
> well [and heavy opposition to tools/perf/ as well], still both worked out
> extremely well :-)
>

Did you forget that arch/x86 was a merging of a code fork that happened
several years previously? Maybe that fork shouldn't have been done to
begin with.

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on
On 03/18/2010 07:28 PM, Ingo Molnar wrote:
> * Avi Kivity<avi(a)redhat.com> wrote:
>
>
>> On 03/18/2010 07:02 PM, Ingo Molnar wrote:
>>
>>> I find the 'KVM mostly cares about the server, not about the desktop'
>>> attitude expressed in this thread troubling.
>>>
>> It's not kvm, just it's developers (and their employers, where applicable).
>> If you post desktop oriented patches I'm sure they'll be welcome.
>>
> Just such a patch-set was posted in this very thread: 'perf kvm'.
>
> There were two negative reactions immediately, both showed a fundamental
> server versus desktop bias:
>
> - you did not accept that the most important usecase is when there is a
> single guest running.
>

Well, it isn't.

> - the reaction to the 'how do we get symbols out of the guest' sub-question
> was, paraphrased: 'we dont want that due to<unspecified> security threat
> to XYZ selinux usecase with lots of guests'.
>

When I review a patch, I try to think of the difficult cases, not just
the easy case.

> Anyone being aware of how Linux and KVM is being used on the desktop will know
> how detached that attitude is from the typical desktop usecase ...
>
> Usability _never_ sucks because of lack of patches or lack of suggestions. I
> bet if you made the next server feature contingent on essential usability
> fixes they'd happen overnight - for God's sake there's been 1000 commits in
> the last 3 months in the Qemu repository so there's plenty of manpower...
>

First of all I am not a qemu maintainer. Second, from my point of view
all contributors are volunteers (perhaps their employer volunteered
them, but there's no difference from my perspective). Asking them to
repaint my apartment as a condition to get a patch applied is abuse. If
a patch is good, it gets applied.

> Usability suckage - and i'm not going to be popular for saying this out loud -
> almost always shows a basic maintainer disconnect with the real world. See
> your very first reactions to my 'KVM usability' observations. Read back your
> and Anthony's replies: total 'sure, patches welcome' kind of indifference. It
> is _your project_, not some other project down the road ...
>

I could drop everything and write a gtk GUI for qemu. Is that what you
want?

If someone is truly interested in a qemu usability, it's up to them to
write the patches. Personally I've never missed the eject button.

As to disconnect from the real world, most products based on kvm and
qemu (and Linux) are server based. Perhaps that's the reason people
emphasise that? Maybe if Linux had 10-20% desktop market penetration,
there would be more interest in a bells and whistles qemu GUI.

> So that is my first-hand experience about how you are welcoming these desktop
> issues, in this very thread. I suspect people try a few times with
> suggestions, then get shot down like our suggestions were shot down and then
> give up.
>

I don't recall anyone trying this much less being shot down. Perhaps
people are concentrating on virt-manager and the like and leaving qemu
alone.

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* Avi Kivity <avi(a)redhat.com> wrote:

> > There were two negative reactions immediately, both showed a fundamental
> > server versus desktop bias:
> >
> > - you did not accept that the most important usecase is when there is a
> > single guest running.
>
> Well, it isn't.

Erm, my usability points are _doubly_ true when there are multiple guests ...

The inconvenience of having to type:

perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms \
--guestmodules=/home/ymzhang/guest/modules top

is very obvious even with a single guest. Now multiply that by more guests ...

The crux is: we are working on improving KVM instrumentation. There are
working patches posted to this thread and we would like to have/implement an
automatism to allow the discovery of all this information. The information
should be available to the developer who wants it, and easily/transparently so
- in true Linux fashion.

> > - the reaction to the 'how do we get symbols out of the guest' sub-question
> > was, paraphrased: 'we dont want that due to<unspecified> security threat
> > to XYZ selinux usecase with lots of guests'.
>
> When I review a patch, I try to think of the difficult cases, not
> just the easy case.

You havent articulated an actionable reason and you have suggested no solution
either, you just passive-agressive backed the claim that giving developers
access to the symbol space is some sort of vague 'security threat'.

If that is not so i'd be glad to be proven wrong.

> > Anyone being aware of how Linux and KVM is being used on the desktop will
> > know how detached that attitude is from the typical desktop usecase ...
> >
> > Usability _never_ sucks because of lack of patches or lack of suggestions.
> > I bet if you made the next server feature contingent on essential
> > usability fixes they'd happen overnight - for God's sake there's been 1000
> > commits in the last 3 months in the Qemu repository so there's plenty of
> > manpower...
>
> First of all I am not a qemu maintainer. [...]

That is the crux of the matter. My experience in these threads was that no-one
really seems to feel in charge of the whole thing. Should we really wonder why
KVM usability sucks?

> [...] Second, from my point of view all contributors are volunteers (perhaps
> their employer volunteered them, but there's no difference from my
> perspective). Asking them to repaint my apartment as a condition to get a
> patch applied is abuse. If a patch is good, it gets applied.

This is one of the weirdest arguments i've seen in this thread. Almost all the
time do we make contributions conditional on the general shape of the project.
Developers dont get to do just the fun stuff.

This is a basic quid pro quo: new features introduce risks and create
additional workload not just to the originating developer but on the rest of
the community as well. You should check how Linus has pulled new features in
the past 15 years: he very much requires the existing code to first be
top-notch before he accepts new features for a given area of functionality.

Doing that and insisting on developers to see those imbalances as well is
absolutely essential to code quality: otherwise everyone would be running
around implementing just the features they are interested in, without regard
for the general health of the project.

Of course, if you keep the project in two halves (KVM and Qemu), and pretend
that they are separate and have little relation, imbalances of quality can
mount up and you can throw your hands up and say that it's "too bad, I'm not
maintaining that". It is your basic duty as a Linux maintainer to keep
balances of quality. I do it all day, other maintainers do it all day.

> > Usability suckage - and i'm not going to be popular for saying this out
> > loud - almost always shows a basic maintainer disconnect with the real
> > world. See your very first reactions to my 'KVM usability' observations.
> > Read back your and Anthony's replies: total 'sure, patches welcome' kind
> > of indifference. It is _your project_, not some other project down the
> > road ...
>
> I could drop everything and write a gtk GUI for qemu. Is that what you
> want?

No, my suggestion to you (it's up to you whether you give my opinion any
weight) is to accept your mistakes and improve, and to not stand in the way of
people who'd like to improve the situation. You are happy with the server
features and you also made it clear that you dont feel responsible for the
rest of the package - which is a big mistake IMO.

Also, you have demonstrated it in this thread that you have near zero
technical clue about basic desktop and development usability matters - for
example your stance on symbol space access and your stance on how to enumerate
guests symbolically are outright bizarre.

Thanks,

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Anthony Liguori on
On 03/19/2010 03:53 AM, Ingo Molnar wrote:
> * Avi Kivity<avi(a)redhat.com> wrote:
>
>
>>> There were two negative reactions immediately, both showed a fundamental
>>> server versus desktop bias:
>>>
>>> - you did not accept that the most important usecase is when there is a
>>> single guest running.
>>>
>> Well, it isn't.
>>
> Erm, my usability points are _doubly_ true when there are multiple guests ...
>
> The inconvenience of having to type:
>
> perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms \
> --guestmodules=/home/ymzhang/guest/modules top
>
> is very obvious even with a single guest. Now multiply that by more guests ...
>

If you want to improve this, you need to do the following:

1) Add a userspace daemon that uses vmchannel that runs in the guest and
can fetch kallsyms and arbitrary modules. If that daemon lives in
tools/perf, that's fine.
2) Add a QMP interface in qemu to interact with such daemon
3) Add a default QMP port in a well known location[1]
4) Modify the perf tool to look for a default QMP port. In the case of
a single guest, there's one port. If there are multiple guests, then
you will have to connect to each port, find the name or any other
identifying information, and let the user choose.

Patches are certainly welcome.

[1] I've written up this patch and will send it out some time today.

Regards,

Anthony Liguori

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/