[9fans] linux stats in last year from linuxcon

[9fans] linux stats in last year from linuxcon

[ron minnich]

2.7M lines last year
10K lines added a day.
5K lines deleted per day.

I keep thinking this can't be sustained. What happens next?

At the same time, well, as pointed out, we all use it all the time.
I'm sending this from gmail.

Or you can use Linux by googling these stats :-)

ron

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 12:22 PM, ron minnich wrote:

> 2.7M lines last year
> 10K lines added a day.
> 5K lines deleted per day.

At least by what i've seen, a good number of these submits have been
fixing the same area, over and over again. How much of this is
actually good development anyways (i.e. The "does this really belong
here?" comments).
>
> I keep thinking this can't be sustained. What happens next?

Of course it can be sustained, but the question is should it be.
>
> At the same time, well, as pointed out, we all use it all the time.
> I'm sending this from gmail.
>
> Or you can use Linux by googling these stats :-)

Yup, and it works good! Of corse you can also use BSD by doing similar.
>
> ron
>

Re: [9fans] linux stats in last year from linuxcon

[tlaronde]

On Mon, Sep 21, 2009 at 09:22:56AM -0700, ron minnich wrote:
> 2.7M lines last year
> 10K lines added a day.
> 5K lines deleted per day.
>
> I keep thinking this can't be sustained. What happens next?

Are there stats indicating where the lines are added? If this is new
hardware (drivers), the accumulation is not a problem---if the API stays
stable; if one needs to rework all the drivers because the API does not
stabilized...

The only time I had to dive in the Linux kernel code, I was disappointed
by the "entropy" of the style and ended grep'ing or awk'ing all around
to extract a (partial) list of PCI identifiers and drivers. (This was
long ago now. 2002 ?)

I wonder if a software project will some day be an example of a black
hole: collapsing from its own size, the work needed to just make it work
being greater than the resources available and the gain to have it
work; and the inability to understand the whole (too much, too long)
resulting in the impossibility to evolve...
--
Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
                 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> At least by what i've seen, a good number of these submits have been
> fixing the same area, over and over again. How much of this is
> actually good development anyways (i.e. The "does this really belong
> here?" comments).
[...]
> Yup, and it works good! Of corse you can also use BSD by doing similar.

in aeronautical circles, it's said that the f4 is proof that given
enough thrust even a brick will fly.

linux is the f4 of computing?

- erik

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 1:02 PM, tlaronde@polynum.com wrote:

> On Mon, Sep 21, 2009 at 09:22:56AM -0700, ron minnich wrote:
>> 2.7M lines last year
>> 10K lines added a day.
>> 5K lines deleted per day.
>>
>> I keep thinking this can't be sustained. What happens next?
>
> Are there stats indicating where the lines are added? If this is new
> hardware (drivers), the accumulation is not a problem---if the API
> stays
> stable; if one needs to rework all the drivers because the API does
> not
> stabilized...
>
> The only time I had to dive in the Linux kernel code, I was
> disappointed
> by the "entropy" of the style and ended grep'ing or awk'ing all around
> to extract a (partial) list of PCI identifiers and drivers. (This was
> long ago now. 2002 ?)
>
> I wonder if a software project will some day be an example of a black
> hole: collapsing from its own size, the work needed to just make it
> work
> being greater than the resources available and the gain to have it
> work; and the inability to understand the whole (too much, too long)
> resulting in the impossibility to evolve...

I wouldn't doubt it, it's a monolithic kernel, It's bound to get too
large, At one of Googles recent conferences one the the Linux guys
said along the lines of, If your codes outside of the kernel, get it
in there. So... building everything into the kernel. Where have I
heard that was a terrible thing to do... oh yes, everywhere.

A software black-hole certainly seems to be possible. I can't help but
question, how good a Monolithic kernel is if you can't maintain it any
longer.
> --
> Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
>                 http://www.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 1:04 PM, erik quanstrom wrote:

>> At least by what i've seen, a good number of these submits have been
>> fixing the same area, over and over again. How much of this is
>> actually good development anyways (i.e. The "does this really belong
>> here?" comments).
> [...]
>> Yup, and it works good! Of corse you can also use BSD by doing
>> similar.
>
> in aeronautical circles, it's said that the f4 is proof that given
> enough thrust even a brick will fly.
>
> linux is the f4 of computing?

I'm not sure I understand your question. I was just saying Linux's
development is rather chaotic, and in some cases, crack jobs. It still
manages to work, and work well, but I'm not really sure for how long.
>
> - erik
>

Re: [9fans] linux stats in last year from linuxcon

[Jack Norton]

ron minnich wrote:
> 2.7M lines last year
> 10K lines added a day.
> 5K lines deleted per day.
>
> I keep thinking this can't be sustained. What happens next?
>
> At the same time, well, as pointed out, we all use it all the time.
> I'm sending this from gmail.
>
> Or you can use Linux by googling these stats :-)
>
> ron
>
>
Here is a little related tidbit:
http://lwn.net/Articles/222773/
It shows employer/company vs. changed lines/contributions etc...
I think this has as much to do with the state of  the linux kernel as
the overall design and ideal therein.  It defines the 'new' open
source.   I don't think something this large can benifit anymore from
open source (as in open 'all the time' to anyone, everywhere -- as
opposed to let's say apple's version of open source dev).  The
development scheme just doesn't scale.
In any event, I'm still waiting for the damn thing to fork...

-Jack

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 1:41 PM, Jack Norton wrote:

> ron minnich wrote:
>> 2.7M lines last year
>> 10K lines added a day.
>> 5K lines deleted per day.
>>
>> I keep thinking this can't be sustained. What happens next?
>>
>> At the same time, well, as pointed out, we all use it all the time.
>> I'm sending this from gmail.
>>
>> Or you can use Linux by googling these stats :-)
>>
>> ron
>>
>>
> Here is a little related tidbit:
> http://lwn.net/Articles/222773/
> It shows employer/company vs. changed lines/contributions etc...
> I think this has as much to do with the state of  the linux kernel
> as the overall design and ideal therein.  It defines the 'new' open
> source.   I don't think something this large can benifit anymore
> from open source (as in open 'all the time' to anyone, everywhere --
> as opposed to let's say apple's version of open source dev).  The
> development scheme just doesn't scale.
> In any event, I'm still waiting for the damn thing to fork...

Fork... That's true, everything under the sun has forked, except the
Linux kernel...

>
> -Jack

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

On Mon, Sep 21, 2009 at 10:53 AM, Patrick Kelly <kameo76890@gmail.com>wrote:

>
> On Sep 21, 2009, at 1:41 PM, Jack Norton wrote:
>
>  ron minnich wrote:
>>
>>> 2.7M lines last year
>>> 10K lines added a day.
>>> 5K lines deleted per day.
>>>
>>> I keep thinking this can't be sustained. What happens next?
>>>
>>> At the same time, well, as pointed out, we all use it all the time.
>>> I'm sending this from gmail.
>>>
>>> Or you can use Linux by googling these stats :-)
>>>
>>> ron
>>>
>>>
>>>  Here is a little related tidbit:
>> http://lwn.net/Articles/222773/
>> It shows employer/company vs. changed lines/contributions etc...
>> I think this has as much to do with the state of  the linux kernel as the
>> overall design and ideal therein.  It defines the 'new' open source.   I
>> don't think something this large can benifit anymore from open source (as in
>> open 'all the time' to anyone, everywhere -- as opposed to let's say apple's
>> version of open source dev).  The development scheme just doesn't scale.
>> In any event, I'm still waiting for the damn thing to fork...
>>
>
> Fork... That's true, everything under the sun has forked, except the Linux
> kernel...
>

Except for the times when the linux kernel was forked for PPC support :-).

Or the fork for running linux on L4.

or....




>
>
>> -Jack
>>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2351 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]


On Sep 21, 2009, at 3:32 PM, David Leimbach wrote:

>
>
> On Mon, Sep 21, 2009 at 10:53 AM, Patrick Kelly
> <kameo76890@gmail.com> wrote:
>
> On Sep 21, 2009, at 1:41 PM, Jack Norton wrote:
>
> ron minnich wrote:
> 2.7M lines last year
> 10K lines added a day.
> 5K lines deleted per day.
>
> I keep thinking this can't be sustained. What happens next?
>
> At the same time, well, as pointed out, we all use it all the time.
> I'm sending this from gmail.
>
> Or you can use Linux by googling these stats :-)
>
> ron
>
>
> Here is a little related tidbit:
> http://lwn.net/Articles/222773/
> It shows employer/company vs. changed lines/contributions etc...
> I think this has as much to do with the state of  the linux kernel
> as the overall design and ideal therein.  It defines the 'new' open
> source.   I don't think something this large can benifit anymore
> from open source (as in open 'all the time' to anyone, everywhere --
> as opposed to let's say apple's version of open source dev).  The
> development scheme just doesn't scale.
> In any event, I'm still waiting for the damn thing to fork...
>
> Fork... That's true, everything under the sun has forked, except the
> Linux kernel...
>
> Except for the times when the linux kernel was forked for PPC
> support :-).
>
> Or the fork for running linux on L4.
>
> or....

Last thing I new all of these ports were merged back into the main
tree. Although I guess that would still be considered a port.
>
>
>
>
>
> -Jack
>
>
>


[-- Attachment #2: Type: text/html, Size: 2953 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[andrey mirtchovski]

it's on slashdot, it must be true:

"During a roundtable discussion at LinuxCon in Portland, Oregon this
afternoon, moderator and Novell distinguished engineer James Bottomley
asked Tovalds whether Linux kernel features were being released too
fast, before the kernel is stabilized.

Citing an internal Intel study that tracked kernel releases, Bottomley
said Linux performance had dropped about two per centage points at
every release, for a cumulative drop of about 12 per cent over the
last ten releases. "Is this a problem?" he asked.

"We're getting bloated and huge. Yes, it's a problem," said Torvalds."

well, not really slashdot:
http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/

Ron, did you throw anything at Linus while you were there? :)

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 9:12 PM, andrey mirtchovski wrote:

> it's on slashdot, it must be true:
>
> "During a roundtable discussion at LinuxCon in Portland, Oregon this
> afternoon, moderator and Novell distinguished engineer James Bottomley
> asked Tovalds whether Linux kernel features were being released too
> fast, before the kernel is stabilized.
>
> Citing an internal Intel study that tracked kernel releases, Bottomley
> said Linux performance had dropped about two per centage points at
> every release, for a cumulative drop of about 12 per cent over the
> last ten releases. "Is this a problem?" he asked.
>
> "We're getting bloated and huge. Yes, it's a problem," said Torvalds."

So may be Tanenbaum was right, after all, there's a reason we make
things modular.
>
> well, not really slashdot:
> http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/
>
> Ron, did you throw anything at Linus while you were there? :)
>

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> > "We're getting bloated and huge. Yes, it's a problem," said Torvalds."
>
> So may be Tanenbaum was right, after all, there's a reason we make
> things modular.

rob, presotto, ken and phil did not agree with tanenbaum's
ideas about modular kernels.

this was a direct response to ast many years ago.  it was
hard to dig up when i did so in 2006.  perhaps someone
has a better link:

   - Microkernels are the way to go
        False unless your only goal is to get papers published.
        Plan 9's kernel is a fraction of the size of any microkernel
        we know and offers more functionality and comparable
        or often better performance.

- erik

Re: [9fans] linux stats in last year from linuxcon

[andrey mirtchovski]

uriel keeps a copy of everything:

http://harmful.cat-v.org/software/andy_tanenbaum

>
> this was a direct response to ast many years ago.  it was
> hard to dig up when i did so in 2006.  perhaps someone
> has a better link:

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 21, 2009, at 10:33 PM, erik quanstrom wrote:

>>> "We're getting bloated and huge. Yes, it's a problem," said
>>> Torvalds."
>>
>> So may be Tanenbaum was right, after all, there's a reason we make
>> things modular.
>
> rob, presotto, ken and phil did not agree with tanenbaum's
> ideas about modular kernels.
>
> this was a direct response to ast many years ago.  it was
> hard to dig up when i did so in 2006.  perhaps someone
> has a better link:
>
>   - Microkernels are the way to go
>        False unless your only goal is to get papers published.
>        Plan 9's kernel is a fraction of the size of any microkernel
>        we know and offers more functionality and comparable
>        or often better performance.

It was somewhat sarcastic. I'm aware each type of kernel has it's
specific areas it excels in. I was poking fun at the Mono v Micro
debate.

We as people don't always agree on a subject. I can't agree completely
with some of the Bell labs staff on some subjects. C or C++ for
example, I strongly prefer Ada. Does that mean I don't agree with Plan
9's design? Far from it, almost every machine I've set up either runs
Plan 9 or is an embedded system.

>
> - erik
>

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> We as people don't always agree on a subject. I can't agree completely
> with some of the Bell labs staff on some subjects. C or C++ for
> example, I strongly prefer Ada.

that's cool.  i like to do embedded work in object cobol.

- erik

Re: [9fans] linux stats in last year from linuxcon

[ron minnich]

I skipped the panel. Those things are never interesting. I was working
outside and heard lots of laughter however.

ron

Re: [9fans] linux stats in last year from linuxcon

[ron minnich]

Having just read the register article, I'm now doubly glad I wasn't
there I was working on Plan 9 :-)

ron

Re: [9fans] linux stats in last year from linuxcon

[Tim Newsham]

> this was a direct response to ast many years ago.  it was
> hard to dig up when i did so in 2006.  perhaps someone
> has a better link:
>
>   - Microkernels are the way to go
>        False unless your only goal is to get papers published.
>        Plan 9's kernel is a fraction of the size of any microkernel
>        we know and offers more functionality and comparable
>        or often better performance.

not intending to pour gas on the flames, but there have been a number of
ukernels since that are a fraction of the size of p9 (and less functional,
by design).  Some with very good performance.

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] linux stats in last year from linuxcon

[Eric Van Hensbergen]

On Sep 21, 2009, at 9:33 PM, erik quanstrom wrote:

>>> "We're getting bloated and huge. Yes, it's a problem," said
>>> Torvalds."
>>
>> So may be Tanenbaum was right, after all, there's a reason we make
>> things modular.
>
> rob, presotto, ken and phil did not agree with tanenbaum's
> ideas about modular kernels.
>
> this was a direct response to ast many years ago.  it was
> hard to dig up when i did so in 2006.  perhaps someone
> has a better link:
>
>   - Microkernels are the way to go
>        False unless your only goal is to get papers published.
>        Plan 9's kernel is a fraction of the size of any microkernel
>        we know and offers more functionality and comparable
>        or often better performance.
>

IMHO, that statement applies to existing microkernel implementations
(at the time? perhaps still?) -- its not clear to me that they
inherently must be that way.
Likely their use as "fuel for papers and PhD's" contributed to their
bloat.

       -eric

Re: [9fans] linux stats in last year from linuxcon

[J.R. Mauro]

On Mon, Sep 21, 2009 at 9:12 PM, andrey mirtchovski
<mirtchovski@gmail.com> wrote:
> it's on slashdot, it must be true:
>
> "During a roundtable discussion at LinuxCon in Portland, Oregon this
> afternoon, moderator and Novell distinguished engineer James Bottomley
> asked Tovalds whether Linux kernel features were being released too
> fast, before the kernel is stabilized.
>
> Citing an internal Intel study that tracked kernel releases, Bottomley
> said Linux performance had dropped about two per centage points at
> every release, for a cumulative drop of about 12 per cent over the
> last ten releases. "Is this a problem?" he asked.
>
> "We're getting bloated and huge. Yes, it's a problem," said Torvalds."

And instead of revising the design, I think we're going to see them go
for the 2-percent speedups here and there.

Another thing they won't consider is having separate versions for
high-end servers and PCs. I don't understand why Torvalds thinks Linux
has to be all things to all people.

>
> well, not really slashdot:
> http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/
>
> Ron, did you throw anything at Linus while you were there? :)
>
>

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> >   - Microkernels are the way to go
> >        False unless your only goal is to get papers published.
> >        Plan 9's kernel is a fraction of the size of any microkernel
> >        we know and offers more functionality and comparable
> >        or often better performance.
>
> not intending to pour gas on the flames, but there have been a number of
> ukernels since that are a fraction of the size of p9 (and less functional,
> by design).  Some with very good performance.

i'm not sure what "good performance" means.  is there enough
functionality in current µkernels to even benchmark real work
against plan 9?

the original problem posed was the "scalability of linux development".
how does l4 help with linux' development problems?

i tend to think the problem is in goals and mgmt.
as a case in point,

> And instead of revising the design, I think we're going to see them go
> for the 2-percent speedups here and there.

from what i've seen, there's a funny math to
linux optimizations.  a number of 2% µoptimizations
lead to global 12% pessimization.

- erik

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

On Mon, Sep 21, 2009 at 7:33 PM, erik quanstrom <quanstro@quanstro.net>wrote:

> > > "We're getting bloated and huge. Yes, it's a problem," said Torvalds."
> >
> > So may be Tanenbaum was right, after all, there's a reason we make
> > things modular.
>
> rob, presotto, ken and phil did not agree with tanenbaum's
> ideas about modular kernels.
>
> this was a direct response to ast many years ago.  it was
> hard to dig up when i did so in 2006.  perhaps someone
> has a better link:
>
>   - Microkernels are the way to go
>        False unless your only goal is to get papers published.
>        Plan 9's kernel is a fraction of the size of any microkernel
>        we know and offers more functionality and comparable
>        or often better performance.
>
>
This does not mean they were saying microkernels can't be done well :-).  At
least I hope not, because there's a few counterexamples out there.

L4 based systems are quite impressive, though their idea of a microkernel is
a bit more like an advanced hypervisor, but more for something along the
lines of paravirtualization.  QNX is another good example of a microkernel
that works quite well in practice.

If by microkernel you mean Mach, and hopefully no one does anymore, then
you're pretty much dead on ;-).

Even the latest versions of Minix really aren't too shabby and still
microkernel based.  (small memory footprint, decent functionality, lots of
Unix stuff work on it, and they have an interesting choice of compilers.)

Are these systems more complex to reason about though?   Probably :-).  But
when you've only got 7 system calls (per the original L4 specifications I've
read over) you don't really have a lot to debug.  Just gotta make sure you
chose the correct primitives to compose all the software you need to write
on the system.

I think, in that sense, Plan 9 and some microkernels are pretty similar.

Dave


> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 2674 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

On Tue, Sep 22, 2009 at 6:48 AM, Eric Van Hensbergen <ericvh@gmail.com>wrote:

> On Sep 21, 2009, at 9:33 PM, erik quanstrom wrote:
>
>  "We're getting bloated and huge. Yes, it's a problem," said Torvalds."
>>>>
>>>
>>> So may be Tanenbaum was right, after all, there's a reason we make
>>> things modular.
>>>
>>
>> rob, presotto, ken and phil did not agree with tanenbaum's
>> ideas about modular kernels.
>>
>> this was a direct response to ast many years ago.  it was
>> hard to dig up when i did so in 2006.  perhaps someone
>> has a better link:
>>
>>  - Microkernels are the way to go
>>       False unless your only goal is to get papers published.
>>       Plan 9's kernel is a fraction of the size of any microkernel
>>       we know and offers more functionality and comparable
>>       or often better performance.
>>
>>
> IMHO, that statement applies to existing microkernel implementations (at
> the time? perhaps still?) -- its not clear to me that they inherently must
> be that way.
> Likely their use as "fuel for papers and PhD's" contributed to their bloat.
>
>      -eric
>
>
>
At that time, and even today, microkernels are "academically bloated".
 However some of the more practical academics (yeah I know it's like jumbo
shrimp or military intelligence) have spun very interesting things off like
 OKL4, which is running in several cellular telephones, and on Qualcomm
equipment, possibly with a Linux personality ported to it.

[-- Attachment #2: Type: text/html, Size: 2292 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1898 bytes --]

On Tue, Sep 22, 2009 at 7:17 AM, J.R. Mauro <jrm8005@gmail.com> wrote:

> On Mon, Sep 21, 2009 at 9:12 PM, andrey mirtchovski
> <mirtchovski@gmail.com> wrote:
> > it's on slashdot, it must be true:
> >
> > "During a roundtable discussion at LinuxCon in Portland, Oregon this
> > afternoon, moderator and Novell distinguished engineer James Bottomley
> > asked Tovalds whether Linux kernel features were being released too
> > fast, before the kernel is stabilized.
> >
> > Citing an internal Intel study that tracked kernel releases, Bottomley
> > said Linux performance had dropped about two per centage points at
> > every release, for a cumulative drop of about 12 per cent over the
> > last ten releases. "Is this a problem?" he asked.
> >
> > "We're getting bloated and huge. Yes, it's a problem," said Torvalds."
>
> And instead of revising the design, I think we're going to see them go
> for the 2-percent speedups here and there.
>
> Another thing they won't consider is having separate versions for
> high-end servers and PCs. I don't understand why Torvalds thinks Linux
> has to be all things to all people.
>
>
Having used uClinux on an embedded platform, my only advice is "for the love
of god don't do that!!!".

It's not that good software can't be written for it, it's that people think
they're writing linux programs that the compatible APIs seem to almost
encourage one to write, and the net result is chaos and debugging
nightmares.

If I ever get to sound off on an embedded platform choice again, uClinux is
not going to be near the top of my list.

If you want linux, make sure the target has an MMU.  That's all :-)

Dave


> >
> > well, not really slashdot:
> >
> http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/
> >
> > Ron, did you throw anything at Linus while you were there? :)
> >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 2826 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> Are these systems more complex to reason about though?   Probably :-).  But
> when you've only got 7 system calls (per the original L4 specifications I've
> read over) you don't really have a lot to debug.  Just gotta make sure you
> chose the correct primitives to compose all the software you need to write
> on the system.

that functionality doesn't disappear, does it?  where ever it goes,
the bugs will follow.

if the argument is that it's easier to debug if it's not in the kernel,
i think that argument requires some proof.

> However some of the more practical academics (yeah I know it's like jumbo
> shrimp or military intelligence) have spun very interesting things off like

we used to call these people research fellows at corporate
labs.  sadly, their astroid has landed.

- erik

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 3393 bytes --]

On Tue, Sep 22, 2009 at 7:47 AM, erik quanstrom <quanstro@quanstro.net>wrote:

> > Are these systems more complex to reason about though?   Probably :-).
>  But
> > when you've only got 7 system calls (per the original L4 specifications
> I've
> > read over) you don't really have a lot to debug.  Just gotta make sure
> you
> > chose the correct primitives to compose all the software you need to
> write
> > on the system.
>
> that functionality doesn't disappear, does it?  where ever it goes,
> the bugs will follow.
>

That's absolutely correct.  However, if you can test a piece in isolation
from other noise, you can rule out certain areas as being suspect in the
process of diagnosing issues.


>
> if the argument is that it's easier to debug if it's not in the kernel,
> i think that argument requires some proof.
>
>
The argument is that if something is logically separable from a larger
system, and independently testable, then once you've verified it is correct,
and that the "glue" is correct that is used to compose a larger system, that
you can more readily decide where to look for problem sources.

This is actually the basis of pure functional programming.  Pure functions
can not deviate in the values they produce because they have a property
called referential transparency.  Not all code can be written this way
obviously, but if you can build a small system, test that it works, then
compose with another small system to do more processing, you can "glue"
those things together to build something more complex.

Think Unix/Plan 9 pipelines of small commands and you get the point.  I
believe this concept took a little mind-bending to get to as well, but seems
almost obvious now. You can think of sort, uniq, and grep as pure functional
routines, which you run some I/O through.  If the inputs are the same, the
outputs (damn well better) be the same.

Back in the "old days" (20 years ago?) Microkernels were going to make it so
that all software was organized this way.  It's just not practical to do so
though.  They wanted servers for disk access, servers for network etc.
 Basically it looked like what GNU Hurd was after... and we see how well
that's done.

You'll find systems based on the L4 microkernels today that implement a
single address space (like Inferno I suppose? or Singularity) and you'll
find stuff like L4-Linux, mostly being used as a way to bridge other OSes to
Linux's drivers, or just to run multiple instances of Linux as in a
virtualization like system like VMWare's high end products provide (they're
basically using a microkernel too for their high-end hypervisors).


> > However some of the more practical academics (yeah I know it's like jumbo
> > shrimp or military intelligence) have spun very interesting things off
> like
>
> we used to call these people research fellows at corporate
> labs.  sadly, their astroid has landed.
>

Yeah :-(.   It's a sad time.  Microsoft, for as much as they've been bashed,
is keeping that dream alive for some.  They seem to understand that R&D is
actually important to keeping on top of things.

I feel a little "icky" to admit it but I really like Visual Studio and F#.
 I was amazed at how quickly I was able to learn C# and write GUI programs
compared to say, Xcode and Cocoa.

Dave








>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 4527 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

On Tue Sep 22 11:06:37 EDT 2009, leimy2k@gmail.com wrote:
> The argument is that if something is logically separable from a larger
> system, and independently testable, then once you've verified it is correct,
> and that the "glue" is correct that is used to compose a larger system, that
> you can more readily decide where to look for problem sources.
>
> This is actually the basis of pure functional programming.

i thought that was called "unit testing", and i don't think
unit testing is the exclusive domain of functional programming
or microkernels.

- erik

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]

On Tue, Sep 22, 2009 at 8:14 AM, erik quanstrom <quanstro@quanstro.net>wrote:

> On Tue Sep 22 11:06:37 EDT 2009, leimy2k@gmail.com wrote:
> > The argument is that if something is logically separable from a larger
> > system, and independently testable, then once you've verified it is
> correct,
> > and that the "glue" is correct that is used to compose a larger system,
> that
> > you can more readily decide where to look for problem sources.
> >
> > This is actually the basis of pure functional programming.
>
> i thought that was called "unit testing", and i don't think
> unit testing is the exclusive domain of functional programming
> or microkernels.
>
>
I never claimed exclusivity.  I simply said "we've seen this before" and why
it's good, in response to you're "this remains to be proven that isolation
is a good thing".  I gave examples.

I don't understand why we're arguing and in agreement at the same time.   I
guess it's fun, but I've got better things to do right now that I've got to
get back to :-)

Peace!

Dave


> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 1771 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[Tim Newsham]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 954 bytes --]

>> not intending to pour gas on the flames, but there have been a number of
>> ukernels since that are a fraction of the size of p9 (and less functional,
>> by design).  Some with very good performance.
>
> i'm not sure what "good performance" means.  is there enough
> functionality in current µkernels to even benchmark real work
> against plan 9?

you can microbenchmark the ukernel itself and run macrobenchmarks
on operating systems sitting atop the ukernel.

btw, there's even been one ukernel recently that has a formal
proof of correctness (against its specification and some containment
properties).  Roughly a 10 man-year effort for about 7.5kloc.
Not something you'd likely be able to do yet against something linux-
sized.

> the original problem posed was the "scalability of linux development".
> how does l4 help with linux' development problems?

no idea.

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] linux stats in last year from linuxcon

[Richard Uhtenwoldt]

J.R. Mauro writes:
>Another thing they won't consider is having separate versions for
>high-end servers and PCs. I don't understand why Torvalds thinks Linux
>has to be all things to all people.

the Linux running on a high-end server is probably compiled from
the same (evolving over time) source tree as the Linux running on
a desktop.

but cannot the same be said of Windows now that most desktops run
Windows XP or a later version of Windows?  cannot the same be
said of OS X?

Richard Uhtenwoldt
http://sonic.net/~sielskr

Re: [9fans] linux stats in last year from linuxcon

[Jack Norton]

Richard Uhtenwoldt wrote:
> J.R. Mauro writes:
>
>> Another thing they won't consider is having separate versions for
>> high-end servers and PCs. I don't understand why Torvalds thinks Linux
>> has to be all things to all people.
>>
>
> the Linux running on a high-end server is probably compiled from
> the same (evolving over time) source tree as the Linux running on
> a desktop.
>
> but cannot the same be said of Windows now that most desktops run
> Windows XP or a later version of Windows?  cannot the same be
> said of OS X?
>
> Richard Uhtenwoldt
> http://sonic.net/~sielskr
>
>
The big topic for me is the realtime patch (the one mentioned at
rt.wiki.kernel.org).  I dabble in computer based audio, and this patch
is mandatory for low latency audio.  There is a big debate as to why
this isn't pushed into the main kernel source and/or forked in the name
of such things.  All I will say is that on OSX I can use jack daemon and
get low latency audio right out of the box and on windows I can use low
latency drivers such as ASIO and the newer WaveRT.   It's even more
tragic as there are tons of great linux audio tools, but they are a hard
sale because you need to apply the rt-patch (which for a musician is
like performing open heart surgery).
In the end I don't care what the linux devs do, but they need to come up
with a game plan and either fork (server, desktop linux) or include it
all and try and make everyone happy (the latter will end in chaos me
thinks).

What I just described is the number one topic that brings up the 'fork
linux' debate (at least it's the one I always pay attention to).

Speaking of realtime, I am trying my hardest to port some of our custom
control applications that we use around my engineering lab to inferno
(anyone doing something similar?  inferno list is not exactly a popular
place apparently).  Right now I spit out a python script on the fly for
everything (quick turnaround) and it's getting old (plus I want to be
able to control anything in the lab from any machine in the lab -- i.e.
a perfect place for some inferno installs)

-jack

Re: [9fans] linux stats in last year from linuxcon

[Chad Brown]

On Sep 22, 2009, at 7:17 AM, J.R. Mauro wrote:
> Another thing they won't consider is having separate versions for
> high-end servers and PCs. I don't understand why Torvalds thinks Linux
> has to be all things to all people.

Back when I cared about linux for servers (not high-end hardware, but
large numbers of easily repalceable dedicated servers), our system was
based on a `server-leaning' kernel from one of the public forks
(originally the `-ac' series, from Alan Cox, but we switched to
someone else's kernel fork at some point).

I don't know what things are like since the millennium, but from 1998
until about 2000 there were both server-focused kernel trees and well-
known kernel forks of `linux', and that doesn't include the wider-
ranging stuff like RTLinux or Mosix (to pick two).

*Chad

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

On Sep 22, 2009, at 4:47 PM, Jack Norton wrote:

> Richard Uhtenwoldt wrote:
>> J.R. Mauro writes:
>>
>>> Another thing they won't consider is having separate versions for
>>> high-end servers and PCs. I don't understand why Torvalds thinks
>>> Linux
>>> has to be all things to all people.
>>>
>>
>> the Linux running on a high-end server is probably compiled from
>> the same (evolving over time) source tree as the Linux running on
>> a desktop.
>>
>> but cannot the same be said of Windows now that most desktops run
>> Windows XP or a later version of Windows?  cannot the same be
>> said of OS X?
>>
>> Richard Uhtenwoldt
>> http://sonic.net/~sielskr
>>
>>
> The big topic for me is the realtime patch (the one mentioned at
> rt.wiki.kernel.org).  I dabble in computer based audio, and this
> patch is mandatory for low latency audio.  There is a big debate as
> to why this isn't pushed into the main kernel source and/or forked
> in the name of such things.  All I will say is that on OSX I can use
> jack daemon and get low latency audio right out of the box and on
> windows I can use low latency drivers such as ASIO and the newer
> WaveRT.   It's even more tragic as there are tons of great linux
> audio tools, but they are a hard sale because you need to apply the
> rt-patch (which for a musician is like performing open heart surgery).
> In the end I don't care what the linux devs do, but they need to
> come up with a game plan and either fork (server, desktop linux) or
> include it all and try and make everyone happy (the latter will end
> in chaos me thinks).

Funny when you consider the only music oriented systems still alive
are Gentoo, and 64Studio. Everything else either has dropped out, or
hasn't been updated in forever. It even looks like this might happen
for 64Studio.
I don't know if it's just for my couple sound cards (emu10k1, and a
couple RME's) but it seems audio quality has gotten worse since I
first dabbled in Linux audio.

> What I just described is the number one topic that brings up the
> 'fork linux' debate (at least it's the one I always pay attention to).
> Speaking of realtime, I am trying my hardest to port some of our
> custom control applications that we use around my engineering lab to
> inferno (anyone doing something similar?  inferno list is not
> exactly a popular place apparently).  Right now I spit out a python
> script on the fly for everything (quick turnaround) and it's getting
> old (plus I want to be able to control anything in the lab from any
> machine in the lab -- i.e. a perfect place for some inferno installs)
>
> -jack
>

Re: [9fans] linux stats in last year from linuxcon

[David Arnold]

On 22/09/2009, at 4:47 PM, Jack Norton wrote:

> In the end I don't care what the linux devs do, but they need to
> come up with a game plan and either fork (server, desktop linux) or
> include it all and try and make everyone happy (the latter will end
> in chaos me thinks).

There are several Linux distributions aimed at digital audio
workstation usage.  They come with an appropriate kernel configuration/
patchset (and audio daemons).

The notion that Linux should fork ignores the reality that most
distributions have different kernel configurations and source
already.  The OpenMoko or Android kernel is quite different to that
running on an SGI Altix (and that's ignoring stuff like uClinux, RT-
Linux, etc).



d

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

On Tue, Sep 22, 2009 at 9:55 PM, David Arnold <davida@pobox.com> wrote:

> On 22/09/2009, at 4:47 PM, Jack Norton wrote:
>
> In the end I don't care what the linux devs do, but they need to come up
>> with a game plan and either fork (server, desktop linux) or include it all
>> and try and make everyone happy (the latter will end in chaos me thinks).
>>
>
> There are several Linux distributions aimed at digital audio workstation
> usage.  They come with an appropriate kernel configuration/patchset (and
> audio daemons).
>

Yes, but as I said, most of them haven't been updated in forever, and mostly
run on hardware from 1-2 years ago (debian philosophy?)

Gentoo seems to be the only one regularly updated, and keeping itself up to
date. That or build your own kernel, with patches, which as stated, most
musicians aren't going to want to do.

If there is a distrobution thats still around and regularly updated, please,
let us know.

>
> The notion that Linux should fork ignores the reality that most
> distributions have different kernel configurations and source already.  The
> OpenMoko or Android kernel is quite different to that running on an SGI
> Altix (and that's ignoring stuff like uClinux, RT-Linux, etc).
>
>
>
> d
>
>
>

[-- Attachment #2: Type: text/html, Size: 1891 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[Jacob Todd]

[-- Attachment #1: Type: text/plain, Size: 1510 bytes --]

On Thu, Sep 24, 2009 at 10:21:35AM -0400, Patrick Kelly wrote:
> On Tue, Sep 22, 2009 at 9:55 PM, David Arnold <davida@pobox.com> wrote:
> 
> > On 22/09/2009, at 4:47 PM, Jack Norton wrote:
> >
> > In the end I don't care what the linux devs do, but they need to come up
> >> with a game plan and either fork (server, desktop linux) or include it all
> >> and try and make everyone happy (the latter will end in chaos me thinks).
> >>
> >
> > There are several Linux distributions aimed at digital audio workstation
> > usage.  They come with an appropriate kernel configuration/patchset (and
> > audio daemons).
> >
> 
> Yes, but as I said, most of them haven't been updated in forever, and mostly
> run on hardware from 1-2 years ago (debian philosophy?)
> 
> Gentoo seems to be the only one regularly updated, and keeping itself up to
> date. That or build your own kernel, with patches, which as stated, most
> musicians aren't going to want to do.
> 
> If there is a distrobution thats still around and regularly updated, please,
> let us know.
> 
> >
> > The notion that Linux should fork ignores the reality that most
> > distributions have different kernel configurations and source already.  The
> > OpenMoko or Android kernel is quite different to that running on an SGI
> > Altix (and that's ignoring stuff like uClinux, RT-Linux, etc).
> >
> >
> >
> > d
> >
> >
> >

There's 64 studio (http://64studio.com).

-- 
Jake Todd
// If it isn't broke, tweak it!

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[Iruata Souza]

On Thu, Sep 24, 2009 at 11:21 AM, Patrick Kelly <kameo76890@gmail.com> wrote:
>
>
> On Tue, Sep 22, 2009 at 9:55 PM, David Arnold <davida@pobox.com> wrote:
>>
>> On 22/09/2009, at 4:47 PM, Jack Norton wrote:
>>
>>> In the end I don't care what the linux devs do, but they need to come up
>>> with a game plan and either fork (server, desktop linux) or include it all
>>> and try and make everyone happy (the latter will end in chaos me thinks).
>>
>> There are several Linux distributions aimed at digital audio workstation
>> usage.  They come with an appropriate kernel configuration/patchset (and
>> audio daemons).
>
>
> Yes, but as I said, most of them haven't been updated in forever, and mostly
> run on hardware from 1-2 years ago (debian philosophy?)
>
> Gentoo seems to be the only one regularly updated, and keeping itself up to
> date. That or build your own kernel, with patches, which as stated, most
> musicians aren't going to want to do.
>
> If there is a distrobution thats still around and regularly updated, please,
> let us know.
>

wrong door, sir.
search for linux in the door with the penguin.
thanks

Re: [9fans] linux stats in last year from linuxcon

[Patrick Kelly]

-----Original Message-----
From: 9fans-bounces@9fans.net [mailto:9fans-bounces@9fans.net] On Behalf Of Iruata Souza
Sent: Thursday, September 24, 2009 11:56 AM
To: Fans of the OS Plan 9 from Bell Labs
Subject: Re: [9fans] linux stats in last year from linuxcon

On Thu, Sep 24, 2009 at 11:21 AM, Patrick Kelly <kameo76890@gmail.com> wrote:
>
>
> On Tue, Sep 22, 2009 at 9:55 PM, David Arnold <davida@pobox.com> wrote:
>>
>> On 22/09/2009, at 4:47 PM, Jack Norton wrote:
>>
>>> In the end I don't care what the linux devs do, but they need to 
>>> come up with a game plan and either fork (server, desktop linux) or 
>>> include it all and try and make everyone happy (the latter will end in chaos me thinks).
>>
>> There are several Linux distributions aimed at digital audio 
>> workstation usage.  They come with an appropriate kernel 
>> configuration/patchset (and audio daemons).
>
>
> Yes, but as I said, most of them haven't been updated in forever, and 
> mostly run on hardware from 1-2 years ago (debian philosophy?)
>
> Gentoo seems to be the only one regularly updated, and keeping itself 
> up to date. That or build your own kernel, with patches, which as 
> stated, most musicians aren't going to want to do.
>
> If there is a distrobution thats still around and regularly updated, 
> please, let us know.
>

wrong door, sir.
search for linux in the door with the penguin.
Thanks

It was a question not meant to be answered. I should have stressed that somehow.

Re: [9fans] linux stats in last year from linuxcon

[Ethan Grammatikidis]

On Mon, 21 Sep 2009 19:02:02 +0200
tlaronde@polynum.com wrote:

> On Mon, Sep 21, 2009 at 09:22:56AM -0700, ron minnich wrote:
> > 2.7M lines last year
> > 10K lines added a day.
> > 5K lines deleted per day.
> >
> > I keep thinking this can't be sustained. What happens next?
>
> Are there stats indicating where the lines are added? If this is new
> hardware (drivers), the accumulation is not a problem---if the API stays
> stable; if one needs to rework all the drivers because the API does not
> stabilized...

The API has not stabilized.  I noticed the API changing twice within a
couple of months this year, and on questioning this strange happening
(as I saw it) I was told that the driver API hasn't been stable since
2.0.0.  What seems to be happening is that the driver API needs to be
altered to plug security holes.  I think it is also changed to
implement ideas of performance enhancement.  As to further details I
am quite in the dark, and for once in my life happy to remain so.

--
Ethan Grammatikidis

Those who are slower at parsing information must
necessarily be faster at problem-solving.

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> wrong door, sir.
> search for linux in the door with the penguin.
> Thanks

they no longer have a door.  they have a two-door interlock
system.  you can enter the space any time you'd like, but you
may never leave.  that's the hotel linux for ya.

- erik

Re: [9fans] linux stats in last year from linuxcon

[Jason Catena]

> There were no proofs about the behavior of the C compiler or the underlying cpu.

In related news, there is a verified Clight compiler out there for
PowerPC machines.

Leroy, X.  2009.  Formal verification of a realistic compiler.
Commun. ACM 52, 7 (Jul. 2009), 107-115.  DOI=
http://doi.acm.org/10.1145/1538788.1538814

This paper reports on the development and formal verification
(proof of semantic preservation) of CompCert, a
compiler from Clight (a large subset of the C programming
language) to PowerPC assembly code, using the Coq proof
assistant both for programming the compiler and for proving
its correctness. Such a verified compiler is useful in the
context of critical software and its formal verification: the
verification of the compiler guarantees that the safety properties
proved on the source code hold for the executable
compiled code as well.

> Tim Newsham

Jason Catena

Re: [9fans] linux stats in last year from linuxcon

[Tim Newsham]

>> i'm not clear on what all functional correctness entails.  can

I thought I'd go into a little more detail about what they did since my
last email probably doesnt clear it up very much.  They wrote a model of
their operating system in a high level language (Haskell).  They then
translated the model into a language suitable for theorem proving
(Isabelle/HOL) and they proved certain properties about the model.  I
don't know specifics on what all was proven, but they did mention that
they processes are properly isolated (one process cannot write to the
memory of another process except through certain specific and controlled
message passing primitives). They then manually translated the spec into C
and made proofs that the model and the C implementation behave the same
way. There were no proofs about the behavior of the C compiler or the
underlying cpu.  So there should be no defects relating to the properties
they proved unless they are hardware or compiler bugs.  There can still be
bugs for properties which they did not prove.  Also the proof is over the
microkernel only, its obviously still possible to write an OS on top of
the kernel that has its own bugs.

You can find out a lot more from their papers and websites:
http://ertos.nicta.com.au/research/sel4/
http://ertos.nicta.com.au/research/l4.verified/
http://ertos.nicta.com.au/research/l4.verified/proof.pml

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] linux stats in last year from linuxcon

[Tim Newsham]

> i'm not clear on what all functional correctness entails.  can
> a functionally correct program suffer from deadlock or livelock?

Yes.  It depends on if that property was stated as part of the
specification of what correctness means.
That is definitely something that can be stated and proven.
I'm not sure if their spec stated that correctness means no
deadlocks or if they explicitely stated that as a property
and then proved it.  The spec is publically available but I havent
studied it.  The proofs are not yet available.  There are some
papers that talk about some of what they have done.

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] linux stats in last year from linuxcon

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On Tue, Sep 22, 2009 at 10:13 AM, erik quanstrom <quanstro@quanstro.net>wrote:

> > btw, there's even been one ukernel recently that has a formal
> > proof of correctness (against its specification and some containment
> > properties).  Roughly a 10 man-year effort for about 7.5kloc.
> > Not something you'd likely be able to do yet against something linux-
> > sized.
>
> the other way of looking at this is all the complex and error
> prone stuff is not prooved.
>
> i'm not clear on what all functional correctness entails.  can
> a functionally correct program suffer from deadlock or livelock?
>
>
I'm not sure... I can tell you when I spawn threads in haskell that, while
testing, the runtime seems to detect certain deadlock states, tells me about
them, and then the program crashes.  I like that a little better than
deadlock :-)

How that all works is a lit

Dave


> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 1481 bytes --]

Re: [9fans] linux stats in last year from linuxcon

[erik quanstrom]

> btw, there's even been one ukernel recently that has a formal
> proof of correctness (against its specification and some containment
> properties).  Roughly a 10 man-year effort for about 7.5kloc.
> Not something you'd likely be able to do yet against something linux-
> sized.

the other way of looking at this is all the complex and error
prone stuff is not prooved.

i'm not clear on what all functional correctness entails.  can
a functionally correct program suffer from deadlock or livelock?

- erik