From: Pekka Enberg on
On Fri, Mar 5, 2010 at 8:49 AM, Ingo Molnar <mingo(a)elte.hu> wrote:
> The conclusion is crystal clear, breaking an ABI via a "flag day"
> cleanup/feature/etc is:
>
> �- wrong
>
> �- harmful
>
> �- limits the developer base
>
> �- limits the tester base
>
> �- wastes time and effort. (fewer developers/testers means that while _this_
> � feature was easier to add, all your _future_ features will be a bit harder
> � to do. It compounds up.)
>
> �- so it hurts even the very developer who is most convinced that this was the
> � right thing to do
>
> It's a bad technical decision throughout. It's masochistic and often suicidal
> to just about any project in essence. I've seen projects that did it once and
> died just due to that single act of stupidity. I've seen projects that have
> done it a few times and took the usage hit, limped along with the wounds and
> never grew to the size they could have achieved. I've seen projects that did
> it once, took the hit, learned from it and never did it again.

Agreed. What bothers me in this discussion is that people keep
bringing up the fact that nouveau is mostly developed by volunteers
and thus it doesn't make sense to make sure it's backwards (or
forwards) compatible. But the way I see it, it's the complete
opposite. It's _more_ important to support ABIs for community-driven
efforts because you're relying on people who by definition don't have
time to waste. While the nouveau people might have good intentions,
I'm afraid they might be severely limiting their developer and tester
base because they're not focused on real world problems (like the ones
Linus is seeing).

Pekka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: C. Bergström on
Pekka Enberg wrote:
> On Fri, Mar 5, 2010 at 8:49 AM, Ingo Molnar <mingo(a)elte.hu> wrote:
>
>> The conclusion is crystal clear, breaking an ABI via a "flag day"
>> cleanup/feature/etc is:
>>
>> - wrong
>>
>> - harmful
>>
>> - limits the developer base
>>
>> - limits the tester base
>>
>> - wastes time and effort. (fewer developers/testers means that while _this_
>> feature was easier to add, all your _future_ features will be a bit harder
>> to do. It compounds up.)
>>
>> - so it hurts even the very developer who is most convinced that this was the
>> right thing to do
>>
>> It's a bad technical decision throughout. It's masochistic and often suicidal
>> to just about any project in essence. I've seen projects that did it once and
>> died just due to that single act of stupidity. I've seen projects that have
>> done it a few times and took the usage hit, limped along with the wounds and
>> never grew to the size they could have achieved. I've seen projects that did
>> it once, took the hit, learned from it and never did it again.
>>
>
> Agreed. What bothers me in this discussion is that people keep
> bringing up the fact that nouveau is mostly developed by volunteers
> and thus it doesn't make sense to make sure it's backwards (or
> forwards) compatible. But the way I see it, it's the complete
> opposite. It's _more_ important to support ABIs for community-driven
> efforts because you're relying on people who by definition don't have
> time to waste. While the nouveau people might have good intentions,
> I'm afraid they might be severely limiting their developer and tester
> base because they're not focused on real world problems (like the ones
> Linus is seeing).
staging != stable

Nobody guaranteed a stable API for staging and in fact it was stated
previously it needed to be changed. Please lets just get back to work
and stop declaring the sky is falling.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* Pekka Enberg <penberg(a)cs.helsinki.fi> wrote:

> On Fri, Mar 5, 2010 at 8:49 AM, Ingo Molnar <mingo(a)elte.hu> wrote:
> > The conclusion is crystal clear, breaking an ABI via a "flag day"
> > cleanup/feature/etc is:
> >
> > ?- wrong
> >
> > ?- harmful
> >
> > ?- limits the developer base
> >
> > ?- limits the tester base
> >
> > ?- wastes time and effort. (fewer developers/testers means that while _this_
> > ? feature was easier to add, all your _future_ features will be a bit harder
> > ? to do. It compounds up.)
> >
> > ?- so it hurts even the very developer who is most convinced that this was the
> > ? right thing to do
> >
> > It's a bad technical decision throughout. It's masochistic and often suicidal
> > to just about any project in essence. I've seen projects that did it once and
> > died just due to that single act of stupidity. I've seen projects that have
> > done it a few times and took the usage hit, limped along with the wounds and
> > never grew to the size they could have achieved. I've seen projects that did
> > it once, took the hit, learned from it and never did it again.
>
> Agreed. What bothers me in this discussion is that people keep bringing up
> the fact that nouveau is mostly developed by volunteers and thus it doesn't
> make sense to make sure it's backwards (or forwards) compatible. But the way
> I see it, it's the complete opposite. It's _more_ important to support ABIs
> for community-driven efforts because you're relying on people who by
> definition don't have time to waste. While the nouveau people might have
> good intentions, I'm afraid they might be severely limiting their developer
> and tester base because they're not focused on real world problems (like the
> ones Linus is seeing).

Yeah. I've seen a few other bad arguments as well:

'exploding test matrix'

This is often the result of _another_ bad technical decision:
over-modularization.

Xorg, mesa/libdrm and the kernel DRM drivers pretty share this signature:

- it's developed by the same tightly knit developer base who often cross
between these packages. Features often need changes in each component.

- a developer to be able to do real work has to have the latest sources
of all these components.

- a user just uses whatever horizontal version cut the distro did and never
truly 'mixes' these components as a conscious decision.

- distros just try to get the latest and most capable but still stable
version. Desperately so. Often they will create a version mix that was
never tested by developers in that form. They'll expose users to ABI
combinations that were never really intended. They have trouble
bootstrapping and stabilizing those essentially random combinations and
then have trouble applying stability and security fixes.

The thing is, if development has such characteristics then it's pretty clearly
not 3-4 separate projects but _one_ abstract project. [*]

So the 'exploding test matrix' is simply the result of: creating ABIs between
3-4 _artificial components of the same project_ and then going through
developer hell living with that mistake. [**]

It's a bit as if we split up the kernel into 'microkernel' components, did a
VFS ABI, MM ABI, drivers ABI, scheduler ABI, networking ABI and arch ABIs, and
then tried to develop them as separate components.

If we did then then Linux kernel development would slow down massively while
in reality everyone would _still_ have to have the latest and greatest source
checked out to do some real development work and to be able to implement
features that affect the whole kernel ...

Linux would become an epic fail of historic proportions if we ever did that.

Ingo

[*] I realize that it's possibly hard for Xorg to merge with mesa and the
kernel for license reasons, but my technical observation still stands.

[**] I realize that modularization and many small packages were a clear
advantage when we were still all downloading bits via 14.4k modems, but
in this day and age of megabit connections that has become a non-issue.
Rampant over-modularization and the resulting loss of focus on the end
result (and the resulting explosion of a test matrix) is hurting us _far
more_ than the disadvantages of any monolithic could ever hurt. We are
seeing the proof of that all day with a 10+ MLOC kernel.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Dave Airlie on
On Fri, Mar 5, 2010 at 5:44 PM, Ingo Molnar <mingo(a)elte.hu> wrote:
>
> * Pekka Enberg <penberg(a)cs.helsinki.fi> wrote:
>
>> On Fri, Mar 5, 2010 at 8:49 AM, Ingo Molnar <mingo(a)elte.hu> wrote:
>> > The conclusion is crystal clear, breaking an ABI via a "flag day"
>> > cleanup/feature/etc is:
>> >
>> > ?- wrong
>> >
>> > ?- harmful
>> >
>> > ?- limits the developer base
>> >
>> > ?- limits the tester base
>> >
>> > ?- wastes time and effort. (fewer developers/testers means that while _this_
>> > ? feature was easier to add, all your _future_ features will be a bit harder
>> > ? to do. It compounds up.)
>> >
>> > ?- so it hurts even the very developer who is most convinced that this was the
>> > ? right thing to do
>> >
>> > It's a bad technical decision throughout. It's masochistic and often suicidal
>> > to just about any project in essence. I've seen projects that did it once and
>> > died just due to that single act of stupidity. I've seen projects that have
>> > done it a few times and took the usage hit, limped along with the wounds and
>> > never grew to the size they could have achieved. I've seen projects that did
>> > it once, took the hit, learned from it and never did it again.
>>
>> Agreed. What bothers me in this discussion is that people keep bringing up
>> the fact that nouveau is mostly developed by volunteers and thus it doesn't
>> make sense to make sure it's backwards (or forwards) compatible. But the way
>> I see it, it's the complete opposite. It's _more_ important to support ABIs
>> for community-driven efforts because you're relying on people who by
>> definition don't have time to waste. While the nouveau people might have
>> good intentions, I'm afraid they might be severely limiting their developer
>> and tester base because they're not focused on real world problems (like the
>> ones Linus is seeing).
>
> Yeah. I've seen a few other bad arguments as well:
>
> � 'exploding test matrix'
>
> This is often the result of _another_ bad technical decision:
> over-modularization.
>
> Xorg, mesa/libdrm and the kernel DRM drivers pretty share this signature:
>
> �- it's developed by the same tightly knit developer base who often cross
> � between these packages. Features often need changes in each component.
>
> �- a developer to be able to do real work has to have the latest sources
> � of all these components.
>
> �- a user just uses whatever horizontal version cut the distro did and never
> � truly 'mixes' these components as a conscious decision.
>
> �- distros just try to get the latest and most capable but still stable
> � version. Desperately so. Often they will create a version mix that was
> � never tested by developers in that form. They'll expose users to ABI
> � combinations that were never really intended. They have trouble
> � bootstrapping and stabilizing those essentially random combinations and
> � then have trouble applying stability and security fixes.
>
> The thing is, if development has such characteristics then it's pretty clearly
> not 3-4 separate projects but _one_ abstract project. [*]
>
> So the 'exploding test matrix' is simply the result of: creating ABIs between
> 3-4 _artificial components of the same project_ and then going through
> developer hell living with that mistake. [**]
>
> It's a bit as if we split up the kernel into 'microkernel' components, did a
> VFS ABI, MM ABI, drivers ABI, scheduler ABI, networking ABI and arch ABIs, and
> then tried to develop them as separate components.
>
> If we did then then Linux kernel development would slow down massively while
> in reality everyone would _still_ have to have the latest and greatest source
> checked out to do some real development work and to be able to implement
> features that affect the whole kernel ...
>
> Linux would become an epic fail of historic proportions if we ever did that.

The other option that we used to have was out-of-tree kernel modules, that
shipped in X.org along with Mesa all in one big tree. This wasn't satisfactory
and we were pretty much told to put the drm modules into the kernel at
that point,

Other suggestion from Luc has been to stablise drm/mesa/X.org APIs a
lot more and
ship driver sources for all 3 bit separately, but that doesn't seem
workable either,
since the Mesa API is infinitely broad (its effectively OpenGL), and
changes way too
often, as does the kernel API, though the drm component is probably okay.

You'll find groups like Intel are releasing things at fairly similiar
times every quarter
and recommending those groupings for distros to take as tested together.

You also have the BSD options where they just create a base OS system and screw
the whole pick-n-mix decisions.

Dave
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* "C. Bergstr?m" <cbergstrom(a)pathscale.com> wrote:

> Pekka Enberg wrote:
> >On Fri, Mar 5, 2010 at 8:49 AM, Ingo Molnar <mingo(a)elte.hu> wrote:
> >>The conclusion is crystal clear, breaking an ABI via a "flag day"
> >>cleanup/feature/etc is:
> >>
> >> - wrong
> >>
> >> - harmful
> >>
> >> - limits the developer base
> >>
> >> - limits the tester base
> >>
> >> - wastes time and effort. (fewer developers/testers means that while _this_
> >> feature was easier to add, all your _future_ features will be a bit harder
> >> to do. It compounds up.)
> >>
> >> - so it hurts even the very developer who is most convinced that this was the
> >> right thing to do
> >>
> >>It's a bad technical decision throughout. It's masochistic and often suicidal
> >>to just about any project in essence. I've seen projects that did it once and
> >>died just due to that single act of stupidity. I've seen projects that have
> >>done it a few times and took the usage hit, limped along with the wounds and
> >>never grew to the size they could have achieved. I've seen projects that did
> >>it once, took the hit, learned from it and never did it again.
> >
> >Agreed. What bothers me in this discussion is that people keep
> >bringing up the fact that nouveau is mostly developed by volunteers
> >and thus it doesn't make sense to make sure it's backwards (or
> >forwards) compatible. But the way I see it, it's the complete
> >opposite. It's _more_ important to support ABIs for community-driven
> >efforts because you're relying on people who by definition don't have
> >time to waste. While the nouveau people might have good intentions,
> >I'm afraid they might be severely limiting their developer and tester
> >base because they're not focused on real world problems (like the ones
> >Linus is seeing).
> staging != stable
>
> Nobody guaranteed a stable API for staging and in fact it was stated
> previously it needed to be changed. Please lets just get back to work and
> stop declaring the sky is falling.

I dont think you understood the argument.

The (very simple) argument was: no matter how a project is developed, whether
it's been freshly announced (not even in staging), in staging or been upstream
for years, breaking ABIs is _technically wrong_.

No ifs and when. A released ABI that is in use cannot be so messy to make it
worth breaking. You've got users. You've got developers. You've got yourself.

You can still phase it out gradually (and even do that quickly), one or two
stable releases down the road you can even print out the final ABI removal
patch on paper, make a bonfire out of it and jump on its ashes in joy, but if
you are interested in running a successful OSS project then the current ABI is
sacrosanct.

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/