* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:14 ` [9fans] Modern development language for Plan 9, WAS: " smiley
@ 2011-02-02 5:37 ` Scott Sullivan
2011-02-02 5:38 ` EBo
` (3 subsequent siblings)
4 siblings, 0 replies; 126+ messages in thread
From: Scott Sullivan @ 2011-02-02 5:37 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On 02/02/2011 12:14 AM, smiley@zenzebra.mv.com wrote:
> [...] though I would hesitate to use ANY code written by
> Google without a thorough audit.
This is where I point out that GO isn't so much written by Google, as
more it's written by Rob Pike and Ken Thompson who now work at Google.
--
Scott Sullivan
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:14 ` [9fans] Modern development language for Plan 9, WAS: " smiley
2011-02-02 5:37 ` Scott Sullivan
@ 2011-02-02 5:38 ` EBo
2011-02-02 15:54 ` Anthony Sorace
2011-02-02 12:54 ` erik quanstrom
` (2 subsequent siblings)
4 siblings, 1 reply; 126+ messages in thread
From: EBo @ 2011-02-02 5:38 UTC (permalink / raw)
To: 9fans
On Wed, 02 Feb 2011 05:14:54 +0000, smiley@zenzebra.mv.com wrote:
> Can Libmo be compiled to native machine code?
>
> There was some mention that, during the history of Plan 9, developers
> had difficulty maintaining two different languages on the system. I
> wonder how much of that difficulty would still apply today. Although
> the kernel could concievably be translated to a modern compiled
> language, I doubt it could be written in Go. If Go were used, then,
> there would still have to be two languages/compilers/development
> environments on the system.
along those lines, can a Libmo byte code be used as an intermediate
language and be compiles/assembled into machine code instead of
interpreted?
EBo --
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:38 ` EBo
@ 2011-02-02 15:54 ` Anthony Sorace
0 siblings, 0 replies; 126+ messages in thread
From: Anthony Sorace @ 2011-02-02 15:54 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
Just to address the unanswered Limbo questions:
The only Limbo compilers extant compile to a portable bytecode for the Dis virtual machine. The only first-class Dis implementation is built into Inferno. Dis can be either interpreted or just-in-time compiled. The historical claim was a that the JIT gave performance about 1.5x slower than compiled C, although I've not seen that benchmarked in about a decade. Years ago, Russ did a sort of first draft of getting Dis to run directly under Plan 9 (which I believe is still available), and I have some vague recollection of someone extending that a bit.
Limbo remains my favorite language to write in, but given Go's surprisingly rapid uptake and current momentum, I somewhat suspect the community would be better served by assisting in those porting efforts.
As an aside, the comments about Alef's demise aren't really relevant. Alef had no significant development community outside the Labs, only ran on one other platform afaik, and all the work to support it had to be done by the same group doing "core" Plan 9. A community-provided port of a language with an existing language with its own community wouldn't fit those circumstances.
Anthony
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:14 ` [9fans] Modern development language for Plan 9, WAS: " smiley
2011-02-02 5:37 ` Scott Sullivan
2011-02-02 5:38 ` EBo
@ 2011-02-02 12:54 ` erik quanstrom
2011-02-02 13:48 ` Devon H. O'Dell
2011-02-02 17:47 ` David Leimbach
2011-02-02 17:44 ` David Leimbach
2011-02-18 5:23 ` ron minnich
4 siblings, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 12:54 UTC (permalink / raw)
To: 9fans
> There was some mention that, during the history of Plan 9, developers
> had difficulty maintaining two different languages on the system. I
> wonder how much of that difficulty would still apply today. Although
> the kernel could concievably be translated to a modern compiled
> language, I doubt it could be written in Go. If Go were used, then,
> there would still have to be two languages/compilers/development
> environments on the system.
although the proof is in the putting, i don't see why a kernel
in principle, can't be written in go, or a slightly restricted subset
of go.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 12:54 ` erik quanstrom
@ 2011-02-02 13:48 ` Devon H. O'Dell
2011-02-02 17:47 ` David Leimbach
1 sibling, 0 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-02 13:48 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/2 erik quanstrom <quanstro@quanstro.net>:
>> There was some mention that, during the history of Plan 9, developers
>> had difficulty maintaining two different languages on the system. I
>> wonder how much of that difficulty would still apply today. Although
>> the kernel could concievably be translated to a modern compiled
>> language, I doubt it could be written in Go. If Go were used, then,
>> there would still have to be two languages/compilers/development
>> environments on the system.
>
> although the proof is in the putting, i don't see why a kernel
> in principle, can't be written in go, or a slightly restricted subset
> of go.
There existed part of the tree called "pchw," renamed "tiny" and then
removed due to lack of maintenance that used the xv6 bootloader and
implemented a tiny "Hello, World" kernel. It's clear that some changes
would have to be made for a serious kernel (ensuring not blocking in
an interrupt handler for instance), but it's certainly possible -- and
has been done -- with the language as-is.
--dho
> - erik
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 12:54 ` erik quanstrom
2011-02-02 13:48 ` Devon H. O'Dell
@ 2011-02-02 17:47 ` David Leimbach
2011-02-02 17:53 ` erik quanstrom
2011-02-02 18:07 ` tlaronde
1 sibling, 2 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 17:47 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 822 bytes --]
On Wed, Feb 2, 2011 at 4:54 AM, erik quanstrom <quanstro@quanstro.net>wrote:
> > There was some mention that, during the history of Plan 9, developers
> > had difficulty maintaining two different languages on the system. I
> > wonder how much of that difficulty would still apply today. Although
> > the kernel could concievably be translated to a modern compiled
> > language, I doubt it could be written in Go. If Go were used, then,
> > there would still have to be two languages/compilers/development
> > environments on the system.
>
> although the proof is in the putting, i don't see why a kernel
> in principle, can't be written in go, or a slightly restricted subset
> of go.
>
Wait, isn't it "the proof is in the *pudding*"? YOU MEAN WE DON'T GET
FRENCH BENEFITS!?!
>
> - erik
>
>
[-- Attachment #2: Type: text/html, Size: 1328 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:47 ` David Leimbach
@ 2011-02-02 17:53 ` erik quanstrom
2011-02-02 18:07 ` tlaronde
1 sibling, 0 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 17:53 UTC (permalink / raw)
To: 9fans
> Wait, isn't it "the proof is in the *pudding*"? YOU MEAN WE DON'T GET
> FRENCH BENEFITS!?!
sadly, no. the work week is still 100hrs and we get -3 holidays/decade.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:47 ` David Leimbach
2011-02-02 17:53 ` erik quanstrom
@ 2011-02-02 18:07 ` tlaronde
2011-02-02 18:26 ` David Leimbach
1 sibling, 1 reply; 126+ messages in thread
From: tlaronde @ 2011-02-02 18:07 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wed, Feb 02, 2011 at 09:47:01AM -0800, David Leimbach wrote:
>
> Wait, isn't it "the proof is in the *pudding*"? YOU MEAN WE DON'T GET
> FRENCH BENEFITS!?!
Please explain.
--
Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:07 ` tlaronde
@ 2011-02-02 18:26 ` David Leimbach
2011-02-02 18:48 ` tlaronde
2011-02-03 0:39 ` Charles Forsyth
0 siblings, 2 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 18:26 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]
On Wed, Feb 2, 2011 at 10:07 AM, <tlaronde@polynum.com> wrote:
> On Wed, Feb 02, 2011 at 09:47:01AM -0800, David Leimbach wrote:
> >
> > Wait, isn't it "the proof is in the *pudding*"? YOU MEAN WE DON'T GET
> > FRENCH BENEFITS!?!
>
> Please explain.
>
I was just pointing out something that happens a lot in our speech that can
translate into text and I think most every american I've ever met falls into
:-).
Sometimes we americans say a lot of things that aren't quite right but
sound close like my ex girlfriend who used to say "supposably" instead of
"supposedly". "Fringe" is close enough to "French" that it's often heard in
it's place. Another one is "He couldn't care a less" for "He couldn't care
less".
A fringe benefit is pretty well described here:
http://www.wisegeek.com/what-are-fringe-benefits.htm and you'll hear people
call them "French Benefits".
As for me, I wasn't really sure if the proof was in the pudding or the
putting, so I was trying to poke fun at myself.
Dave
> --
> Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
> http://www.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
>
>
[-- Attachment #2: Type: text/html, Size: 2021 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:26 ` David Leimbach
@ 2011-02-02 18:48 ` tlaronde
2011-02-02 19:26 ` Nick LaForge
2011-02-03 0:39 ` Charles Forsyth
1 sibling, 1 reply; 126+ messages in thread
From: tlaronde @ 2011-02-02 18:48 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wed, Feb 02, 2011 at 10:26:34AM -0800, David Leimbach wrote:
> On Wed, Feb 2, 2011 at 10:07 AM, <tlaronde@polynum.com> wrote:
>
> > On Wed, Feb 02, 2011 at 09:47:01AM -0800, David Leimbach wrote:
> > >
> > > Wait, isn't it "the proof is in the *pudding*"? YOU MEAN WE DON'T GET
> > > FRENCH BENEFITS!?!
> >
> > Please explain.
> >
>
> [...]
> A fringe benefit is pretty well described here:
> http://www.wisegeek.com/what-are-fringe-benefits.htm and you'll hear people
> call them "French Benefits".
> [...]
Sorry, I'm quite edgy at the moment ;) And I'm not the only one...
--
Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:48 ` tlaronde
@ 2011-02-02 19:26 ` Nick LaForge
0 siblings, 0 replies; 126+ messages in thread
From: Nick LaForge @ 2011-02-02 19:26 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
"BCPL makes C look like a very high-level language and provides
absolutely no type checking or run-time support."
B. Stroustrup, The Design and Evolution of C++, 1994
"C++ was designed to be used in a rather traditional compilation and
run-time environment, the C programming environment on the UNIX
system. Facilities such as exception handling or concurrent
programming that require nontrivial loader and run-time support are
not included in C++. Consequently, a C++ implementation can be very
easily ported."
B. Stroustrup, The C++ Programming Language, 1986
"Except for the new, delete, typeid, dynamic_cast, and throw operators
and the try-block, individual C++ expressions and statements need no
run-time support."
B. Stroustrup, The C++ Programming Language, 3rd ed., 2000
Nick
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:26 ` David Leimbach
2011-02-02 18:48 ` tlaronde
@ 2011-02-03 0:39 ` Charles Forsyth
1 sibling, 0 replies; 126+ messages in thread
From: Charles Forsyth @ 2011-02-03 0:39 UTC (permalink / raw)
To: 9fans
>you'll hear people call [fringe benefits] "French Benefits".
i did not expect that! i'd have guessed: `cheese'.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:14 ` [9fans] Modern development language for Plan 9, WAS: " smiley
` (2 preceding siblings ...)
2011-02-02 12:54 ` erik quanstrom
@ 2011-02-02 17:44 ` David Leimbach
2011-02-02 17:50 ` erik quanstrom
2011-02-02 18:03 ` erik quanstrom
2011-02-18 5:23 ` ron minnich
4 siblings, 2 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 17:44 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 5412 bytes --]
On Tue, Feb 1, 2011 at 9:14 PM, <smiley@zenzebra.mv.com> wrote:
> ron minnich <rminnich@gmail.com> writes:
>
> > I think you should set your sights higher than the macro approach you
> > propose. At least in my opinion it's a really ugly idea.
>
> You might be surprised to hear that I agree. :) It's far from an ideal
> solution. I am certainly open to alternatives!
>
> > You could make a lasting contribution by bringing a good modern
> > language to Plan 9.
>
> Maybe. My first criterion for such a language would be that it compile
> to native machine code. Although requiring such may be presumptive, it
> seems appropriate that the core OS applications (file servers, command
> line utilities, etc.) be in native machine code. On the other hand, on
> Inferno, Limbo compiles to architecture-independent bytecode,
> eliminating the need for the /$objtype directories on Plan 9, while
> enabling easier sharing of object code. What are all your thoughts' on
> this "compiled vs interpreted" design decision?
>
You can already write Limbo programs for Plan 9. The line between the "OS"
of Inferno and the VM of Inferno is small. You should be able to access
your plan 9 resources from Inferno just fine. Just like you can access most
of what you'd want from an operating system from Java or Erlang. It's not
very different, except that Inferno has shells and editors and a GUI that
run in it's VM.
>
> The Go language (from Google? sigh. Evil, evil.) appears to compile to
> native machine code. The Go web site (http://golang.org), however,
> claims that Go requires a "small" runtime... which causes me to wonder
> just how fully "compiled" it is. Anyone know the scoop on what this
> "runtime" is all about?
>
Even C has a runtime. Perhaps you should look more into how programming
languages are implemented :-). C++ has one too, especially in the wake of
exceptions and such.
>
> Go is also a garbage-collected language. I'm also a bit leery of using
> a GC language for coding core OS applications. I've generally thought
> of GC as being for lazy programmers (/me runs and hides under his desk,
> peeks out...) and incurring somewhat of a performance hit. I'm not sure
> if that would be appropriate for core applications. Then again, it
> seems to be what's done on Inferno. Thoughts on this?
>
GC can incur performance hits in some families of applications where timing
guarantees are needed and make writing code for hard realtime applications
basically impossible, unless you can get some guarantees from the GC that it
won't interrupt your processing that must complete by a particular deadline.
>
> Wikipedia says that Go doesn't support safe concurrency. However, the
> Go web site claims that "goroutines" (which are kinda like threads)
> coordinate through explicit synchronization. Isn't that how the Plan 9
> threading library works, too? I'm not sure why the Wikipedia article
> would make a claim like that. Thoughts on the relative merits of
> concurrency in Go vs Plan 9 C would also be welcome.
>
The memory model is very clear on how changes become visible across
goroutines. One must either synchronize with channels or synchronize via
some locking mechanism to guarantee that updates to shared data are visible.
Go encourages a CSP style of concurrency that promotes using channels for
both synchronization and update of shared data.
This is something you could learn by reading more about it yourself, or
trying it out. There's even an in-browser sandbox you can use.
>
> On an implementation note, it sounds like Go can be bootstrapped from C,
> with a little bit of assembly. It might not be so monumental a task to
> port Go to Plan 9, though I would hesitate to use ANY code written by
> Google without a thorough audit.
>
People already have a Go cross compiler to Plan 9. You could verify these
"sounds like" factoids yourself though by checking it out and trying it.
>
> > I'll say it again, I don't think a cpp-based approach will be well
>
> Did you mean what you wrote, "cpp" or did you mean C++?
>
C pre-processor probably.
>
> > Or even native Limbo, that one is frequently requested.
>
> Can Libmo be compiled to native machine code?
>
> There was some mention that, during the history of Plan 9, developers
> had difficulty maintaining two different languages on the system. I
> wonder how much of that difficulty would still apply today. Although
> the kernel could concievably be translated to a modern compiled
> language, I doubt it could be written in Go. If Go were used, then,
> there would still have to be two languages/compilers/development
> environments on the system.
>
Where did your C compiler come from? Someone probably compiled it with a C
compiler. Bootstrapping is a fact of life as a new compiler can't just be
culled from /dev/random or willed into existence otherwise. It takes a plan
9 system to build plan 9 right? (This was not always true for infinitely
recursive reasons)
>
> --
> +---------------------------------------------------------------+
> |E-Mail: smiley@zenzebra.mv.com PGP key ID: BC549F8B|
> |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
> +---------------------------------------------------------------+
>
>
[-- Attachment #2: Type: text/html, Size: 7144 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:44 ` David Leimbach
@ 2011-02-02 17:50 ` erik quanstrom
2011-02-02 18:15 ` Jonathan Cast
2011-02-02 18:21 ` David Leimbach
2011-02-02 18:03 ` erik quanstrom
1 sibling, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 17:50 UTC (permalink / raw)
To: 9fans
> Even C has a runtime. Perhaps you should look more into how programming
> languages are implemented :-). C++ has one too, especially in the wake of
> exceptions and such.
really? what do you consider to be the c runtime?
i don't think that the asm goo that gets you to main
really counts as "runtime" and neither does the c
library, because neither implement language features.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:50 ` erik quanstrom
@ 2011-02-02 18:15 ` Jonathan Cast
2011-02-02 18:21 ` erik quanstrom
2011-02-02 18:21 ` David Leimbach
1 sibling, 1 reply; 126+ messages in thread
From: Jonathan Cast @ 2011-02-02 18:15 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wed, 2011-02-02 at 12:50 -0500, erik quanstrom wrote:
> > Even C has a runtime. Perhaps you should look more into how programming
> > languages are implemented :-). C++ has one too, especially in the wake of
> > exceptions and such.
>
> really? what do you consider to be the c runtime?
> i don't think that the asm goo that gets you to main
> really counts as "runtime" and neither does the c
> library, because neither implement language features.
A runtime system is just a library whose entry points are language
keywords.[1] In go, dynamic allocation, threads, channels, etc. are
accessed via language features, so the libraries that implement those
things are considered part of the RTS. That's a terminological
difference only from Plan 9 C, which has the same features[2] but
accesses them through ordinary library entry points so the libraries
that implement them aren't called `runtimes'. But I think complaining
about a library only because its entry point is a keyword is kind of
silly.
jcc
[1] Or other syntactic features of the language. I'm not aware of any
other simplification in this statement; correct me if I'm wrong.
[2] Well, C has somewhat less useful versions of the same features. The
difference has no significant impact on the size of the relevant
libraries.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:15 ` Jonathan Cast
@ 2011-02-02 18:21 ` erik quanstrom
2011-02-02 18:36 ` David Leimbach
2011-02-02 19:15 ` Jonathan Cast
0 siblings, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 18:21 UTC (permalink / raw)
To: 9fans
> A runtime system is just a library whose entry points are language
> keywords.[1] In go, dynamic allocation, threads, channels, etc. are
> accessed via language features, so the libraries that implement those
> things are considered part of the RTS. That's a terminological
> difference only from Plan 9 C, which has the same features[2] but
> accesses them through ordinary library entry points so the libraries
> that implement them aren't called `runtimes'. But I think complaining
> about a library only because its entry point is a keyword is kind of
> silly.
i think this glosses over a key difference. a runtime can do things
that are not invoked by function call. the canonical example is
garbage collection.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:21 ` erik quanstrom
@ 2011-02-02 18:36 ` David Leimbach
2011-02-02 18:38 ` erik quanstrom
2011-02-02 19:15 ` Jonathan Cast
1 sibling, 1 reply; 126+ messages in thread
From: David Leimbach @ 2011-02-02 18:36 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 1379 bytes --]
On Wed, Feb 2, 2011 at 10:21 AM, erik quanstrom <quanstro@quanstro.net>wrote:
> > A runtime system is just a library whose entry points are language
> > keywords.[1] In go, dynamic allocation, threads, channels, etc. are
> > accessed via language features, so the libraries that implement those
> > things are considered part of the RTS. That's a terminological
> > difference only from Plan 9 C, which has the same features[2] but
> > accesses them through ordinary library entry points so the libraries
> > that implement them aren't called `runtimes'. But I think complaining
> > about a library only because its entry point is a keyword is kind of
> > silly.
>
> i think this glosses over a key difference. a runtime can do things
> that are not invoked by function call. the canonical example is
> garbage collection.
>
> - erik
>
> An excellent example would also be the scheduling of goroutines. I do not
believe there's anything in the language specification that says that
goroutines could not one day be pre-emptive.
Also, from this point of view, could pthreads be considered runtime for C?
Depends on the implementation I suppose. You've got thread local storage,
which is not handled by any explicit C code, but by a coordinated effort
between the kernel and the pthreads library. So the kernel is a C runtime
too :-).
Dave
[-- Attachment #2: Type: text/html, Size: 1782 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:36 ` David Leimbach
@ 2011-02-02 18:38 ` erik quanstrom
2011-02-02 18:46 ` David Leimbach
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 18:38 UTC (permalink / raw)
To: 9fans
> Also, from this point of view, could pthreads be considered runtime for C?
no. then every library/os function ever bolted onto
c would be "part of the c runtime". clearly this isn't
the case and pthreads are not specified in the c standard.
it might be part of /a/ runtime, but not the c runtime.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:38 ` erik quanstrom
@ 2011-02-02 18:46 ` David Leimbach
0 siblings, 0 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 18:46 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wednesday, February 2, 2011, erik quanstrom <quanstro@quanstro.net> wrote:
>> Also, from this point of view, could pthreads be considered runtime for C?
>
> no. then every library/os function ever bolted onto
> c would be "part of the c runtime". clearly this isn't
> the case and pthreads are not specified in the c standard.
>
> it might be part of /a/ runtime, but not the c runtime.
>
> - erik
>
>
You are right. I suppose in C only the stack space is really needed
for function calls and that may be pushing it too.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:21 ` erik quanstrom
2011-02-02 18:36 ` David Leimbach
@ 2011-02-02 19:15 ` Jonathan Cast
2011-02-02 19:31 ` erik quanstrom
1 sibling, 1 reply; 126+ messages in thread
From: Jonathan Cast @ 2011-02-02 19:15 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wed, 2011-02-02 at 13:21 -0500, erik quanstrom wrote:
> > A runtime system is just a library whose entry points are language
> > keywords.[1] In go, dynamic allocation, threads, channels, etc. are
> > accessed via language features, so the libraries that implement those
> > things are considered part of the RTS. That's a terminological
> > difference only from Plan 9 C, which has the same features[2] but
> > accesses them through ordinary library entry points so the libraries
> > that implement them aren't called `runtimes'. But I think complaining
> > about a library only because its entry point is a keyword is kind of
> > silly.
>
> i think this glosses over a key difference. a runtime can do things
> that are not invoked by function call. the canonical example is
> garbage collection.
I don't follow. Garbage collection certainly can be done in a library
(e.g., Boehm). GC is in my experience normally triggered by
* Allocation --- which is a function call in C
* Explicit call to the `garbage collect now' entry point in the
standard library. A function call in every language.
What other events canonically would trigger garbage collection, but not
be invoked by function calls?
jcc
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 19:15 ` Jonathan Cast
@ 2011-02-02 19:31 ` erik quanstrom
2011-02-02 19:48 ` Jeff Sickel
2011-02-02 20:07 ` Jonathan Cast
0 siblings, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 19:31 UTC (permalink / raw)
To: 9fans
> I don't follow. Garbage collection certainly can be done in a library
> (e.g., Boehm). GC is in my experience normally triggered by
>
> * Allocation --- which is a function call in C
> * Explicit call to the `garbage collect now' entry point in the
> standard library. A function call in every language.
>
> What other events canonically would trigger garbage collection, but not
> be invoked by function calls?
i should have said automatic garbage collection.
i think of it this way, the janitor doesn't insist that the factory shut
down so he can sweep. he waits for the factory to be idle, and then
sweeps.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 19:31 ` erik quanstrom
@ 2011-02-02 19:48 ` Jeff Sickel
2011-02-02 20:07 ` Jonathan Cast
1 sibling, 0 replies; 126+ messages in thread
From: Jeff Sickel @ 2011-02-02 19:48 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Feb 2, 2011, at 1:31 PM, erik quanstrom wrote:
> i think of it this way, the janitor doesn't insist that the factory shut
> down so he can sweep. he waits for the factory to be idle, and then
> sweeps.
Clearly I've been working on the wrong floors. That or all the janitors I know are using the Java GC model: never around for long periods of time, then right in the midst of a critical section it's stop what you're doing, stand up, and move out of the way. Don't forget to take your chair and any other items you might want to be around when you can get back to the task. Otherwise those important bits will be in the dust bin on route to the garbage chute before taking the long trek out to the land fill.
-jas
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 19:31 ` erik quanstrom
2011-02-02 19:48 ` Jeff Sickel
@ 2011-02-02 20:07 ` Jonathan Cast
2011-02-02 20:11 ` erik quanstrom
1 sibling, 1 reply; 126+ messages in thread
From: Jonathan Cast @ 2011-02-02 20:07 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Wed, 2011-02-02 at 14:31 -0500, erik quanstrom wrote:
> > I don't follow. Garbage collection certainly can be done in a library
> > (e.g., Boehm). GC is in my experience normally triggered by
> >
> > * Allocation --- which is a function call in C
> > * Explicit call to the `garbage collect now' entry point in the
> > standard library. A function call in every language.
> >
> > What other events canonically would trigger garbage collection, but not
> > be invoked by function calls?
>
> i should have said automatic garbage collection.
> i think of it this way, the janitor doesn't insist that the factory shut
> down so he can sweep. he waits for the factory to be idle, and then
> sweeps.
I think factory owners get pretty upset when their factories idle.
I still think the program needs to call into the threading library
(whether you call it part of the RTS or not) to idle. So the only
benefit you have is that putting threading and the garbage collector
into the RTS allows you to ignore the abstraction barriers between the
two systems, which makes it easier for the thread system to signal the
garbage collector. I mean, if the thread does this (making up syntax on
the spot):
start := now();
while (now() < start + 2hours);
You don't expect GC to be able to trigger, right?
jcc
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:50 ` erik quanstrom
2011-02-02 18:15 ` Jonathan Cast
@ 2011-02-02 18:21 ` David Leimbach
1 sibling, 0 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 18:21 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 1127 bytes --]
On Wed, Feb 2, 2011 at 9:50 AM, erik quanstrom <quanstro@quanstro.net>wrote:
> > Even C has a runtime. Perhaps you should look more into how programming
> > languages are implemented :-). C++ has one too, especially in the wake
> of
> > exceptions and such.
>
> really? what do you consider to be the c runtime?
>
i don't think that the asm goo that gets you to main
> really counts as "runtime" and neither does the c
> library, because neither implement language features.
>
>
How about setting up stack space in the code for an operating system kernel?
That's something you don't explicitly write in C that must be there
somehow, for example in an operating system kernel. You end up changing
that runtime bit and then all your C code has different stack space
available. I suppose you could group that into the kernel's runtime, but
since the operating system I'm thinking of is coded in C, that kind of line
drawing seems silly ;-)
I agree that C has a really really minimal need for any "help" to run on raw
metal, but some level of support is still necessary.
Dave
> - erik
>
>
[-- Attachment #2: Type: text/html, Size: 1812 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 17:44 ` David Leimbach
2011-02-02 17:50 ` erik quanstrom
@ 2011-02-02 18:03 ` erik quanstrom
2011-02-02 18:30 ` David Leimbach
1 sibling, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-02 18:03 UTC (permalink / raw)
To: 9fans
> Where did your C compiler come from? Someone probably compiled it with a C
> compiler. Bootstrapping is a fact of life as a new compiler can't just be
> culled from /dev/random or willed into existence otherwise. It takes a plan
> 9 system to build plan 9 right? (This was not always true for infinitely
> recursive reasons)
ah, but where did your go compiler come from?
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 18:03 ` erik quanstrom
@ 2011-02-02 18:30 ` David Leimbach
0 siblings, 0 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-02 18:30 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]
On Wed, Feb 2, 2011 at 10:03 AM, erik quanstrom <quanstro@labs.coraid.com>wrote:
> > Where did your C compiler come from? Someone probably compiled it with a
> C
> > compiler. Bootstrapping is a fact of life as a new compiler can't just
> be
> > culled from /dev/random or willed into existence otherwise. It takes a
> plan
> > 9 system to build plan 9 right? (This was not always true for infinitely
> > recursive reasons)
>
> ah, but where did your go compiler come from?
>
> - erik
>
> Well my Go compiler came from a plan 9 C compiler that came from a gcc
compiler, that came from the operating system distribution CD that shipped
with Mac OS X.
Someone at apple presumably bootstrapped that gcc build for Mac OS X from
another GCC build for Mac OS X, and that one probably goes back to some
version of OpenStep, all the way back to NeXTStep, and before that some
version of Unix most likely that bootstrapped NeXTStep.
A lot of that lineage was a guess. It's really difficult, for instance, to
bootstrap the GHC (Haskell) compiler from the intermediate C files it
generates these days, and you pretty much need a port of Haskell to your
platform in order get a port of haskell to your platform. It's a bit of an
undocumented black art as far as I can tell, but it was supposed to be
simpler :-).
Many lisp compilers/systems need a lisp compiler or system in place in order
to bootstrap them too.
Dave
[-- Attachment #2: Type: text/html, Size: 1856 bytes --]
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-02 5:14 ` [9fans] Modern development language for Plan 9, WAS: " smiley
` (3 preceding siblings ...)
2011-02-02 17:44 ` David Leimbach
@ 2011-02-18 5:23 ` ron minnich
2011-02-18 5:34 ` Paul Lalonde
4 siblings, 1 reply; 126+ messages in thread
From: ron minnich @ 2011-02-18 5:23 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
I was looking at another fine example of modern programming from glibc
and just had to share it.
Where does the getpid happen? It's anyone's guess. This is just so
readable too ... I'm glad they want to such effort to optimize getpid.
ron
#ifndef NOT_IN_libc
static inline __attribute__((always_inline)) pid_t really_getpid (pid_t oldval);
static inline __attribute__((always_inline)) pid_t
really_getpid (pid_t oldval)
{
if (__builtin_expect (oldval == 0, 1))
{
pid_t selftid = THREAD_GETMEM (THREAD_SELF, tid);
if (__builtin_expect (selftid != 0, 1))
return selftid;
}
INTERNAL_SYSCALL_DECL (err);
pid_t result = INTERNAL_SYSCALL (getpid, err, 0);
/* We do not set the PID field in the TID here since we might be
called from a signal handler while the thread executes fork. */
if (oldval == 0)
THREAD_SETMEM (THREAD_SELF, tid, result);
return result;
}
#endif
pid_t
__getpid (void)
{
#ifdef NOT_IN_libc
INTERNAL_SYSCALL_DECL (err);
pid_t result = INTERNAL_SYSCALL (getpid, err, 0);
#else
pid_t result = THREAD_GETMEM (THREAD_SELF, pid);
if (__builtin_expect (result <= 0, 0))
result = really_getpid (result);
#endif
return result;
}
libc_hidden_def (__getpid)
weak_alias (__getpid, getpid)
libc_hidden_def (getpid)
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 5:23 ` ron minnich
@ 2011-02-18 5:34 ` Paul Lalonde
2011-02-18 13:29 ` erik quanstrom
0 siblings, 1 reply; 126+ messages in thread
From: Paul Lalonde @ 2011-02-18 5:34 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
Best recent c99 example:
int foo[] = {
[0] = 1,
[1] = 2,
[2] = 4,
[3] = 8,
[4] = 16,
[5] = 32
};
I shudder to think about foo[6].
Paul
On Thursday, February 17, 2011, ron minnich <rminnich@gmail.com> wrote:
> I was looking at another fine example of modern programming from glibc
> and just had to share it.
>
> Where does the getpid happen? It's anyone's guess. This is just so
> readable too ... I'm glad they want to such effort to optimize getpid.
>
> ron
>
> #ifndef NOT_IN_libc
> static inline __attribute__((always_inline)) pid_t really_getpid (pid_t oldval);
>
> static inline __attribute__((always_inline)) pid_t
> really_getpid (pid_t oldval)
> {
> if (__builtin_expect (oldval == 0, 1))
> {
> pid_t selftid = THREAD_GETMEM (THREAD_SELF, tid);
> if (__builtin_expect (selftid != 0, 1))
> return selftid;
> }
>
> INTERNAL_SYSCALL_DECL (err);
> pid_t result = INTERNAL_SYSCALL (getpid, err, 0);
>
> /* We do not set the PID field in the TID here since we might be
> called from a signal handler while the thread executes fork. */
> if (oldval == 0)
> THREAD_SETMEM (THREAD_SELF, tid, result);
> return result;
> }
> #endif
>
> pid_t
> __getpid (void)
> {
> #ifdef NOT_IN_libc
> INTERNAL_SYSCALL_DECL (err);
> pid_t result = INTERNAL_SYSCALL (getpid, err, 0);
> #else
> pid_t result = THREAD_GETMEM (THREAD_SELF, pid);
> if (__builtin_expect (result <= 0, 0))
> result = really_getpid (result);
> #endif
> return result;
> }
>
> libc_hidden_def (__getpid)
> weak_alias (__getpid, getpid)
> libc_hidden_def (getpid)
>
>
--
I'm migrating my email. plalonde@telus.net will soon be disconnected.
Please use paul.a.lalonde@gmail.com from now on.
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 5:34 ` Paul Lalonde
@ 2011-02-18 13:29 ` erik quanstrom
2011-02-18 13:45 ` dexen deVries
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 13:29 UTC (permalink / raw)
To: 9fans
so this is a complete waste of time if forks > getpids.
and THREAD_GETMEM must allocate memory. so
the first call isn't exactly cheep. aren't they optimizing
for bad programming?
not only that, ... from getpid(2)
NOTES
Since glibc version 2.3.4, the glibc wrapper function for getpid()
caches PIDs, so as to avoid additional system calls when a process
calls getpid() repeatedly. Normally this caching is invisible, but its
correct operation relies on support in the wrapper functions for
fork(2), vfork(2), and clone(2): if an application bypasses the glibc
wrappers for these system calls by using syscall(2), then a call to
getpid() in the child will return the wrong value (to be precise: it
will return the PID of the parent process). See also clone(2) for dis-
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 13:29 ` erik quanstrom
@ 2011-02-18 13:45 ` dexen deVries
2011-02-18 14:54 ` David Leimbach
2011-02-18 15:15 ` Devon H. O'Dell
0 siblings, 2 replies; 126+ messages in thread
From: dexen deVries @ 2011-02-18 13:45 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Friday, February 18, 2011 02:29:54 pm erik quanstrom wrote:
> so this is a complete waste of time if forks > getpids.
> and THREAD_GETMEM must allocate memory. so
> the first call isn't exactly cheep. aren't they optimizing
> for bad programming?
>
> not only that, ... from getpid(2)
>
> NOTES
> Since glibc version 2.3.4, the glibc wrapper function for
> getpid() caches PIDs, so as to avoid additional system calls when a
> process calls getpid() repeatedly. Normally this caching is invisible,
> but its correct operation relies on support in the wrapper
> functions for fork(2), vfork(2), and clone(2): if an application bypasses
> the glibc wrappers for these system calls by using syscall(2), then a
> call to getpid() in the child will return the wrong value (to be
> precise: it will return the PID of the parent process). See also
> clone(2) for dis-
which boggles my mind: why would getpid() need to be optimized for in the first
place?
Konqueror 4.5.5 (browser) calls it once per short session (few tabs)
Firefox 4 (browser) calls it about once per tab
openssh calls it once or twice per session
bash calls it once
lsof, find do not call it at all.
what does call getpid() often? @_@
anyway, it looks a bit like library lock-in to me: ``your app better perform
_every_ syscall through glibc, or else'' -- or else strange things may happen,
eh?
--
dexen deVries
[[[↓][→]]]
> how does a C compiler get to be that big? what is all that code doing?
iterators, string objects, and a full set of C macros that ensure
boundary conditions and improve interfaces.
ron minnich, in response to Charles Forsyth
http://9fans.net/archive/2011/02/90
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 13:45 ` dexen deVries
@ 2011-02-18 14:54 ` David Leimbach
2011-02-18 15:15 ` Devon H. O'Dell
1 sibling, 0 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-18 14:54 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs; +Cc: Fans of the OS Plan 9 from Bell Labs
Sent from my iPhone
On Feb 18, 2011, at 5:45 AM, dexen deVries <dexen.devries@gmail.com> wrote:
> On Friday, February 18, 2011 02:29:54 pm erik quanstrom wrote:
>> so this is a complete waste of time if forks > getpids.
>> and THREAD_GETMEM must allocate memory. so
>> the first call isn't exactly cheep. aren't they optimizing
>> for bad programming?
>>
>> not only that, ... from getpid(2)
>>
>> NOTES
>> Since glibc version 2.3.4, the glibc wrapper function for
>> getpid() caches PIDs, so as to avoid additional system calls when a
>> process calls getpid() repeatedly. Normally this caching is invisible,
>> but its correct operation relies on support in the wrapper
>> functions for fork(2), vfork(2), and clone(2): if an application bypasses
>> the glibc wrappers for these system calls by using syscall(2), then a
>> call to getpid() in the child will return the wrong value (to be
>> precise: it will return the PID of the parent process). See also
>> clone(2) for dis-
>
> which boggles my mind: why would getpid() need to be optimized for in the first
> place?
LMbench?
>
> Konqueror 4.5.5 (browser) calls it once per short session (few tabs)
> Firefox 4 (browser) calls it about once per tab
> openssh calls it once or twice per session
> bash calls it once
> lsof, find do not call it at all.
>
> what does call getpid() often? @_@
>
>
> anyway, it looks a bit like library lock-in to me: ``your app better perform
> _every_ syscall through glibc, or else'' -- or else strange things may happen,
> eh?
>
>
> --
> dexen deVries
>
> [[[↓][→]]]
>
>> how does a C compiler get to be that big? what is all that code doing?
>
> iterators, string objects, and a full set of C macros that ensure
> boundary conditions and improve interfaces.
>
> ron minnich, in response to Charles Forsyth
>
> http://9fans.net/archive/2011/02/90
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 13:45 ` dexen deVries
2011-02-18 14:54 ` David Leimbach
@ 2011-02-18 15:15 ` Devon H. O'Dell
2011-02-18 16:11 ` erik quanstrom
` (2 more replies)
1 sibling, 3 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 15:15 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 dexen deVries <dexen.devries@gmail.com>:
> On Friday, February 18, 2011 02:29:54 pm erik quanstrom wrote:
>> so this is a complete waste of time if forks > getpids.
>> and THREAD_GETMEM must allocate memory. so
>> the first call isn't exactly cheep. aren't they optimizing
>> for bad programming?
>>
>> not only that, ... from getpid(2)
>>
>> NOTES
>> Since glibc version 2.3.4, the glibc wrapper function for
>> getpid() caches PIDs, so as to avoid additional system calls when a
>> process calls getpid() repeatedly. Normally this caching is invisible,
>> but its correct operation relies on support in the wrapper
>> functions for fork(2), vfork(2), and clone(2): if an application bypasses
>> the glibc wrappers for these system calls by using syscall(2), then a
>> call to getpid() in the child will return the wrong value (to be
>> precise: it will return the PID of the parent process). See also
>> clone(2) for dis-
>
> which boggles my mind: why would getpid() need to be optimized for in the first
> place?
>
> Konqueror 4.5.5 (browser) calls it once per short session (few tabs)
> Firefox 4 (browser) calls it about once per tab
> openssh calls it once or twice per session
> bash calls it once
> lsof, find do not call it at all.
>
> what does call getpid() often? @_@
Benchmark utilities to measure the overhead of syscalls. It's cheating
to do for getpid, but for other things like gettimeofday, it's
*extremely* nice. Linux's gettimeofday(2) beats the socks off of the
rest of the time implementations. About the only faster thing is to
get CPU speed and use rdtsc. Certainly no other OS allows you to get
the timestamp faster with a syscall.
>
> anyway, it looks a bit like library lock-in to me: ``your app better perform
> _every_ syscall through glibc, or else'' -- or else strange things may happen,
> eh?
I know we're fond of bashing people who need to eek performance out of
systems, and a lot of time it's all in good fun. There's little
justification for getpid, but getpid isn't the only implementor of
this functionality. For other interfaces, it definitely makes sense to
speed up the system to speed up applications. Argue about it all you
want, but without this sort of mentality, we also wouldn't have
non-blocking I/O or kernel thread support.
Yes, processors are fast enough. Except when they aren't.
--dho
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 15:15 ` Devon H. O'Dell
@ 2011-02-18 16:11 ` erik quanstrom
2011-02-18 16:28 ` Devon H. O'Dell
2011-02-18 16:53 ` dexen deVries
2011-02-19 10:34 ` Steve Simon
2 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 16:11 UTC (permalink / raw)
To: 9fans
> I know we're fond of bashing people who need to eek performance out of
> systems, and a lot of time it's all in good fun. There's little
> justification for getpid, but getpid isn't the only implementor of
> this functionality. For other interfaces, it definitely makes sense to
> speed up the system to speed up applications. Argue about it all you
> want, but without this sort of mentality, we also wouldn't have
> non-blocking I/O or kernel thread support.
define "we". there's no non-blocking io in plan 9.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:11 ` erik quanstrom
@ 2011-02-18 16:28 ` Devon H. O'Dell
2011-02-18 16:49 ` erik quanstrom
2011-02-18 17:05 ` andrey mirtchovski
0 siblings, 2 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 16:28 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 erik quanstrom <quanstro@labs.coraid.com>:
>> I know we're fond of bashing people who need to eek performance out of
>> systems, and a lot of time it's all in good fun. There's little
>> justification for getpid, but getpid isn't the only implementor of
>> this functionality. For other interfaces, it definitely makes sense to
>> speed up the system to speed up applications. Argue about it all you
>> want, but without this sort of mentality, we also wouldn't have
>> non-blocking I/O or kernel thread support.
>
> define "we". there's no non-blocking io in plan 9.
I didn't mean "we" in the context of Plan 9. I meant "we" in the
context of computer science and software engineering. Someone thought
there was a problem with an interface and came up with a solution.
Plan 9 has a different approach to solving the problem by providing a
different means to addressing it entirely.
Arguing that performance is unimportant is counterintuitive. It
certainly is. Arguing that it is unimportant if it causes unnecessary
complexity has merit. Defining when things become "unnecessarily
complex" is important to the argument. Applications with timers (or
doing lots of logging) using gettimeofday(2) being instantaneously
improved by *very* measurable amounts due to such changes seems like a
good idea to me, and it doesn't seem too complex. Doing it for
getpid(2) seems pretty dumb.
I think it's time that we do some real-world style benchmarks on
multiple systems for Plan 9 versus other systems. I'd be interested in
seeing what we could come up with, how we address it, and the relative
"ease" for each solution. Anybody want to work together to put
something like that together?
--dho
> - erik
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:28 ` Devon H. O'Dell
@ 2011-02-18 16:49 ` erik quanstrom
2011-02-18 17:10 ` Devon H. O'Dell
2011-02-18 18:43 ` Joseph Stewart
2011-02-18 17:05 ` andrey mirtchovski
1 sibling, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 16:49 UTC (permalink / raw)
To: 9fans
> Arguing that performance is unimportant is counterintuitive. It
> certainly is. Arguing that it is unimportant if it causes unnecessary
> complexity has merit. Defining when things become "unnecessarily
> complex" is important to the argument. Applications with timers (or
> doing lots of logging) using gettimeofday(2) being instantaneously
> improved by *very* measurable amounts due to such changes seems like a
> good idea to me, and it doesn't seem too complex. Doing it for
> getpid(2) seems pretty dumb.
i take a different view of performance.
performance is like scotch. you always want better scotch,
but you only upgrade if the stuff you're drinking is a problem.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:49 ` erik quanstrom
@ 2011-02-18 17:10 ` Devon H. O'Dell
2011-02-18 17:32 ` erik quanstrom
2011-02-18 18:43 ` Joseph Stewart
1 sibling, 1 reply; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 17:10 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 erik quanstrom <quanstro@quanstro.net>:
>> Arguing that performance is unimportant is counterintuitive. It
>> certainly is. Arguing that it is unimportant if it causes unnecessary
>> complexity has merit. Defining when things become "unnecessarily
>> complex" is important to the argument. Applications with timers (or
>> doing lots of logging) using gettimeofday(2) being instantaneously
>> improved by *very* measurable amounts due to such changes seems like a
>> good idea to me, and it doesn't seem too complex. Doing it for
>> getpid(2) seems pretty dumb.
>
> i take a different view of performance.
>
> performance is like scotch. you always want better scotch,
> but you only upgrade if the stuff you're drinking is a problem.
I really like this viewpoint. Unfortunately in software engineering,
we are more the creators and purveyors of scotch, and our customers
constantly request better scotch. Sometimes you have to say "live with
it" -- but sometimes you really do need to upgrade what you are
providing.
I'd like to think my viewpoint maps equally well to libations. A good
wine matures with age, becoming ever more complex in flavor. If you
don't keep it right, that complexity turns right into vinegar. :)
I agree with your point. At the same time, we have large customers who
constantly push the limits of our mail server, and they have extremely
good performance with it. Likely better than they can get with any
competitor's implementation. If you ask an ISP or large social network
if they would like to do more with less, then answer will always be
"yes." Ergo there is always a perceived problem -- even if you're the
de-facto leader in your industry.
I'd be surprised if things were dissimilar for you at Coraid -- and I
certainly *am not* implying that you guys have poor performance. I'm
just saying if you went to your customers and asked, "Given the choice
between something that is the same as what you have now, and something
that's faster, and both have the same reliability, which do you want?"
you probably wouldn't have many people who wouldn't take advantage of
improved performance.
--dho
> - erik
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:10 ` Devon H. O'Dell
@ 2011-02-18 17:32 ` erik quanstrom
2011-02-18 17:44 ` ron minnich
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 17:32 UTC (permalink / raw)
To: 9fans
> I'd be surprised if things were dissimilar for you at Coraid -- and I
> certainly *am not* implying that you guys have poor performance. I'm
> just saying if you went to your customers and asked, "Given the choice
> between something that is the same as what you have now, and something
> that's faster, and both have the same reliability, which do you want?"
> you probably wouldn't have many people who wouldn't take advantage of
> improved performance.
wire speed is generally considered "good enough". ☺
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:32 ` erik quanstrom
@ 2011-02-18 17:44 ` ron minnich
2011-02-18 19:28 ` Devon H. O'Dell
0 siblings, 1 reply; 126+ messages in thread
From: ron minnich @ 2011-02-18 17:44 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, Feb 18, 2011 at 9:32 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> wire speed is generally considered "good enough". ☺
depends on field of use. In my biz everyone hits wire speed, and the
question from there is: how much of the CPU are you eating to get that
wire speed.
It's a very tangled thicked.
ron
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:44 ` ron minnich
@ 2011-02-18 19:28 ` Devon H. O'Dell
2011-02-18 19:33 ` erik quanstrom
0 siblings, 1 reply; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 19:28 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 ron minnich <rminnich@gmail.com>:
> On Fri, Feb 18, 2011 at 9:32 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>
>> wire speed is generally considered "good enough". ☺
Touche.
> depends on field of use. In my biz everyone hits wire speed, and the
> question from there is: how much of the CPU are you eating to get that
> wire speed.
>
> It's a very tangled thicked.
Indeed. It's very difficult to do SMTP anywhere close to wire speed
with the protocol-required persistent I/O overhead, the typical
content analysis stuff that ISPs, ESPs, and large content providers
tend to want to do. Add on RBL lookups, crypto-related stuff (e.g.
DKIM), etc., it's just not really feasible on commodity hardware. (Of
course, these days, operating systems and RAID controllers with
battery-backed caches make it impossible to guarantee that your
message ever ends up in persistent storage, but that's still a small
part of the processing overhead for a given message.)
--dho
> ron
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:28 ` Devon H. O'Dell
@ 2011-02-18 19:33 ` erik quanstrom
2011-02-18 19:49 ` Devon H. O'Dell
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 19:33 UTC (permalink / raw)
To: 9fans
> DKIM), etc., it's just not really feasible on commodity hardware. (Of
> course, these days, operating systems and RAID controllers with
> battery-backed caches make it impossible to guarantee that your
> message ever ends up in persistent storage, but that's still a small
bb cache is persistent storage.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:33 ` erik quanstrom
@ 2011-02-18 19:49 ` Devon H. O'Dell
0 siblings, 0 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 19:49 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 erik quanstrom <quanstro@quanstro.net>:
>> DKIM), etc., it's just not really feasible on commodity hardware. (Of
>> course, these days, operating systems and RAID controllers with
>> battery-backed caches make it impossible to guarantee that your
>> message ever ends up in persistent storage, but that's still a small
>
> bb cache is persistent storage.
My bad, indeed. :-).
Still, modern filesystems still raise questions as to whether the data
even winds up there. Modern data stores (like Mongo) raise questions
as to whether the data makes it to VFS. "We" make a lot of assumptions
about quite volatile systems in the name of performance. And that's
usually ok, until it isn't. It's kind of an addendum to Rob's point,
actually: you can quite easily decrease the reliability / efficacy /
correctness of your implementation without (or with!) the right
information.
And of course, there's also something to be said for eeking
performance out of a system that can't perform because it's poorly
designed. (I'm looking at you, Twitter.) The amount of resources they
have and the amount of downtime they have are really quite
extraordinary. For their popularity, you'd really expect to see an
inverse correlation in one way or the other.
I think I've digressed further from the original point discussed
(which wasn't even the original topic) and contributed way more than I
intended to, though. Food for thought.
--dho
> - erik
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:49 ` erik quanstrom
2011-02-18 17:10 ` Devon H. O'Dell
@ 2011-02-18 18:43 ` Joseph Stewart
1 sibling, 0 replies; 126+ messages in thread
From: Joseph Stewart @ 2011-02-18 18:43 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
> i take a different view of performance.
>
> performance is like scotch. you always want better scotch,
> but you only upgrade if the stuff you're drinking is a problem.
>
> - erik
Awesome. That quote is going on my office door below the Tanenbaum
quote on bandwidth and station wagons!
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:28 ` Devon H. O'Dell
2011-02-18 16:49 ` erik quanstrom
@ 2011-02-18 17:05 ` andrey mirtchovski
2011-02-18 17:12 ` Devon H. O'Dell
1 sibling, 1 reply; 126+ messages in thread
From: andrey mirtchovski @ 2011-02-18 17:05 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
> I think it's time that we do some real-world style benchmarks on
> multiple systems for Plan 9 versus other systems. I'd be interested in
Ron did work measuring syscall costs and latencies in plan9.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:05 ` andrey mirtchovski
@ 2011-02-18 17:12 ` Devon H. O'Dell
0 siblings, 0 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 17:12 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 andrey mirtchovski <mirtchovski@gmail.com>:
>> I think it's time that we do some real-world style benchmarks on
>> multiple systems for Plan 9 versus other systems. I'd be interested in
>
> Ron did work measuring syscall costs and latencies in plan9.
I would love to duplicate that across multiple systems doing similar
tests. I'd also like to do real-world benchmarking -- not just
microbenchmarking.
--dho
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 15:15 ` Devon H. O'Dell
2011-02-18 16:11 ` erik quanstrom
@ 2011-02-18 16:53 ` dexen deVries
2011-02-18 16:59 ` Devon H. O'Dell
2011-02-19 10:34 ` Steve Simon
2 siblings, 1 reply; 126+ messages in thread
From: dexen deVries @ 2011-02-18 16:53 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Friday, February 18, 2011 04:15:10 pm you wrote:
> Benchmark utilities to measure the overhead of syscalls. It's cheating
> to do for getpid, but for other things like gettimeofday, it's
> *extremely* nice. Linux's gettimeofday(2) beats the socks off of the
> rest of the time implementations. About the only faster thing is to
> get CPU speed and use rdtsc. Certainly no other OS allows you to get
> the timestamp faster with a syscall.
Would you mind explaining what technique is used by Linux to speed up the
gettimeofday()? I'd guess it's not per-process caching... and if it's not,
then it involves two context-switches; not the fastest thing in my books.
As for performance in general, some speculative fiction:
in general, drivers are kept in kernel for two reasons -- to protect resources
from processes going rogue and to provide common, infrequently changing API to
diverse hardware. The later reason is pretty much security-insensitive and
serves to aid cross-platform development.
In principle, the read-only parts of some drivers could be embedded in
processes (things like system timer, rather than harddrive). Is there any OS
out there that actually lets processes embed the read-only parts of drivers to
avoid context switches for going through kernel?
The closest thing I can think of is Google's Native Client, which lets
untrusted code execute (within a trusted `host process') with constrained,
readexecute-only access to trusted code so it can execute hand-picked syscalls
& communication with the host process.
Perhaps a *constrained* read-write driver for harddrive (and filesystem) access
could perhaps also be held rx-only in virtual memory of untrusted code...
--
dexen deVries
[[[↓][→]]]
> how does a C compiler get to be that big? what is all that code doing?
iterators, string objects, and a full set of C macros that ensure
boundary conditions and improve interfaces.
ron minnich, in response to Charles Forsyth
http://9fans.net/archive/2011/02/90
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:53 ` dexen deVries
@ 2011-02-18 16:59 ` Devon H. O'Dell
2011-02-18 17:07 ` erik quanstrom
2011-02-18 17:12 ` andrey mirtchovski
0 siblings, 2 replies; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 16:59 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 dexen deVries <dexen.devries@gmail.com>:
> On Friday, February 18, 2011 04:15:10 pm you wrote:
>> Benchmark utilities to measure the overhead of syscalls. It's cheating
>> to do for getpid, but for other things like gettimeofday, it's
>> *extremely* nice. Linux's gettimeofday(2) beats the socks off of the
>> rest of the time implementations. About the only faster thing is to
>> get CPU speed and use rdtsc. Certainly no other OS allows you to get
>> the timestamp faster with a syscall.
>
> Would you mind explaining what technique is used by Linux to speed up the
> gettimeofday()? I'd guess it's not per-process caching... and if it's not,
> then it involves two context-switches; not the fastest thing in my books.
The high level overview is that it is stored in a shared page, mapped
into each new process's memory space at start-up. The kernel is never
entered; there are no context switches. The kernel has a timer that
updates this page atomically.
--dho
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:59 ` Devon H. O'Dell
@ 2011-02-18 17:07 ` erik quanstrom
2011-02-18 17:11 ` Devon H. O'Dell
2011-02-18 17:16 ` Russ Cox
2011-02-18 17:12 ` andrey mirtchovski
1 sibling, 2 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 17:07 UTC (permalink / raw)
To: 9fans
> The high level overview is that it is stored in a shared page, mapped
> into each new process's memory space at start-up. The kernel is never
> entered; there are no context switches. The kernel has a timer that
> updates this page atomically.
i wonder if that is uniformly faster. consider that
making reads of that page coherent enough on a
big multiprocessor and making sure there's not too
much interprocesser skew might be slower than a
system call.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:07 ` erik quanstrom
@ 2011-02-18 17:11 ` Devon H. O'Dell
2011-02-18 17:21 ` erik quanstrom
2011-02-18 17:16 ` Russ Cox
1 sibling, 1 reply; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 17:11 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 erik quanstrom <quanstro@quanstro.net>:
>> The high level overview is that it is stored in a shared page, mapped
>> into each new process's memory space at start-up. The kernel is never
>> entered; there are no context switches. The kernel has a timer that
>> updates this page atomically.
>
> i wonder if that is uniformly faster. consider that
> making reads of that page coherent enough on a
> big multiprocessor and making sure there's not too
> much interprocesser skew might be slower than a
> system call.
Real world tests show that it is consistently faster. It's probably
cached anyway.
> - erik
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:11 ` Devon H. O'Dell
@ 2011-02-18 17:21 ` erik quanstrom
2011-02-18 17:52 ` John Floren
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 17:21 UTC (permalink / raw)
To: 9fans
> > i wonder if that is uniformly faster. consider that
> > making reads of that page coherent enough on a
> > big multiprocessor and making sure there's not too
> > much interprocesser skew might be slower than a
> > system call.
>
> Real world tests show that it is consistently faster. It's probably
> cached anyway.
as andrey points out that's exactly the problem if you want
an accurate time. also, while making gettimeofday() faster,
you potentially invalidate BY2PG/BY2CL cachelines on
*every* processor in the system. this has the real potential
for being a problem on, say, a 256 processor system and making
everything else on the system slower.
linux optimization is a ratrace. you are only judged on
the immediate effect on your subsystem, not the system
as a whole. so unless you play the game, your system will
appear to regress over time as other optimizers take resources
away from you.
there aren't many processor arm systems yet, but on such
a system, the os will need to do the equivalent of cachedinvse()
and l2cacheuwbinvse often enough to make the timing look
reasonable.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:21 ` erik quanstrom
@ 2011-02-18 17:52 ` John Floren
2011-02-18 18:46 ` Rob Pike
0 siblings, 1 reply; 126+ messages in thread
From: John Floren @ 2011-02-18 17:52 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, Feb 18, 2011 at 9:21 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> linux optimization is a ratrace. you are only judged on
> the immediate effect on your subsystem, not the system
> as a whole. so unless you play the game, your system will
> appear to regress over time as other optimizers take resources
> away from you.
On the other hand, how many optimizations have been put into the Plan
9 kernel recently? If Linux adds 100 optimizations a year, I bet at
least a few of them are going to actually improve things. Is it better
to have optimized and failed than never to have optimized at all?
John
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:52 ` John Floren
@ 2011-02-18 18:46 ` Rob Pike
2011-02-18 19:15 ` Bakul Shah
2011-02-18 19:15 ` Devon H. O'Dell
0 siblings, 2 replies; 126+ messages in thread
From: Rob Pike @ 2011-02-18 18:46 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
The more you optimize, the better the odds you slow your program down.
Optimization adds instructions and often data, in one of the
paradoxes of engineering. In time, then, what you gain by
"optimizing" increases cache pressure and slows the whole thing down.
C++ inlines a lot because microbenchmarks improve, but inline every
modest function in a big program and you make the binary much bigger
and blow the i-cache.
-rob
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 18:46 ` Rob Pike
@ 2011-02-18 19:15 ` Bakul Shah
2011-02-18 19:26 ` erik quanstrom
2011-02-18 19:35 ` David Leimbach
2011-02-18 19:15 ` Devon H. O'Dell
1 sibling, 2 replies; 126+ messages in thread
From: Bakul Shah @ 2011-02-18 19:15 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, 18 Feb 2011 10:46:51 PST Rob Pike <robpike@gmail.com> wrote:
> The more you optimize, the better the odds you slow your program down.
> Optimization adds instructions and often data, in one of the
> paradoxes of engineering. In time, then, what you gain by
> "optimizing" increases cache pressure and slows the whole thing down.
You need a feedback loop. Uncontrolled anything is a recipe
for disaster. Optimizations need to be `judicious' but that
requires experience, profiling and understanding but the
trend seems to be away from that.....
On a slightly different tangent, 9p is simple but it doesn't
handle latency very well. To make efficient use of long fat
pipes you need more complex mechanisms -- there is no getting
around that fact. rsync & hg in spite of their complexity
beat the pants off replica. Their cache behavior is not very
relevant here. Similarly file readahead is usually a win.
> C++ inlines a lot because microbenchmarks improve, but inline every
> modest function in a big program and you make the binary much bigger
> and blow the i-cache.
That's a compiler fault. Surely modern compilers need to be
cache aware? ideally a smart compiler treats `inline' as a hint
at most, just like `register'.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:15 ` Bakul Shah
@ 2011-02-18 19:26 ` erik quanstrom
2011-02-18 19:46 ` Bakul Shah
2011-02-18 19:35 ` David Leimbach
1 sibling, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 19:26 UTC (permalink / raw)
To: 9fans
> On a slightly different tangent, 9p is simple but it doesn't
> handle latency very well. To make efficient use of long fat
> pipes you need more complex mechanisms -- there is no getting
> around that fact. rsync & hg in spite of their complexity
> beat the pants off replica. Their cache behavior is not very
> relevant here. Similarly file readahead is usually a win.
i don't think that it makes sense to say that since replica
is slow and hg/rsync are fast, it follows that 9p is slow.
9p is a file protocol, and the rest are programs. it's apples
and giraffes. as a child might tell you, an apple is just as
fast as a giraffe if the apple is inside the giraffe.
similarly, you blame c++ compilers for excessive inlining.
i might blame the mount driver for its 1 oustanding policy.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:26 ` erik quanstrom
@ 2011-02-18 19:46 ` Bakul Shah
2011-02-18 20:15 ` erik quanstrom
0 siblings, 1 reply; 126+ messages in thread
From: Bakul Shah @ 2011-02-18 19:46 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, 18 Feb 2011 14:26:32 EST erik quanstrom <quanstro@quanstro.net> wrote:
> > On a slightly different tangent, 9p is simple but it doesn't
> > handle latency very well. To make efficient use of long fat
> > pipes you need more complex mechanisms -- there is no getting
> > around that fact. rsync & hg in spite of their complexity
> > beat the pants off replica. Their cache behavior is not very
> > relevant here. Similarly file readahead is usually a win.
>
> i don't think that it makes sense to say that since replica
> is slow and hg/rsync are fast, it follows that 9p is slow.
It is the other way around. 9p can't handle latency so on
high latency pipes programs using 9p won't be as fast as
programs using streaming (instead of rpc). Granted that there
are many other factors when it comes to hg & replica but
latency is a major one.
> similarly, you blame c++ compilers for excessive inlining.
I am suggesting modern compilers should ignore the inline
keyword and be cache aware. For the same reason as why the
register keyword is mostly ignored. People are wont to use
inlining in the hope of improving performance (just as they
used register). Sometime it is better to fix a program than
try educating the hordes.
Actually what I'd really like to suggest is C++ shouldn't
be used at all :-)
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:46 ` Bakul Shah
@ 2011-02-18 20:15 ` erik quanstrom
2011-02-18 21:06 ` John Floren
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-18 20:15 UTC (permalink / raw)
To: 9fans
> > i don't think that it makes sense to say that since replica
> > is slow and hg/rsync are fast, it follows that 9p is slow.
>
> It is the other way around. 9p can't handle latency so on
> high latency pipes programs using 9p won't be as fast as
> programs using streaming (instead of rpc). Granted that there
> are many other factors when it comes to hg & replica but
> latency is a major one.
you're still comparing apples and girraffes. rsync/hg have
protocols ment for syncing. replica uses 9p, which is not a
protocol designed for syncing. it's designed for regular file
access. it would be similarly difficult to use rsync's protocol
directly for file access.
while 9p can and should be improved upon, this case doesn't
seem like a real motivator. the nfs guys don't complain similarly
about nfs loosing to rsync.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 20:15 ` erik quanstrom
@ 2011-02-18 21:06 ` John Floren
2011-02-18 22:21 ` Bakul Shah
2011-02-19 15:36 ` erik quanstrom
0 siblings, 2 replies; 126+ messages in thread
From: John Floren @ 2011-02-18 21:06 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, Feb 18, 2011 at 12:15 PM, erik quanstrom <quanstro@quanstro.net> wrote:
>> > i don't think that it makes sense to say that since replica
>> > is slow and hg/rsync are fast, it follows that 9p is slow.
>>
>> It is the other way around. 9p can't handle latency so on
>> high latency pipes programs using 9p won't be as fast as
>> programs using streaming (instead of rpc). Granted that there
>> are many other factors when it comes to hg & replica but
>> latency is a major one.
>
> you're still comparing apples and girraffes. rsync/hg have
> protocols ment for syncing. replica uses 9p, which is not a
> protocol designed for syncing. it's designed for regular file
> access. it would be similarly difficult to use rsync's protocol
> directly for file access.
>
So why does replica use 9P? Because it's *The Plan 9 Protocol*. If
*The Plan 9 Protocol* turns out to not serve our needs, we need to
figure out why.
You like to put forward devmnt's penchant for only having one
Tread/Twrite per process in flight at one time. I agree that this is a
problem, now, how do we fix it? All it needs is somebody willing to
rewrite devmnt... I think you may just have to rewrite mntrdwr to be
just a little smarter. Any takers?
9P as specified in the documentation might not necessarily be the
problem, but the implementation apparently is.
> while 9p can and should be improved upon, this case doesn't
> seem like a real motivator. the nfs guys don't complain similarly
> about nfs loosing to rsync.
>
> - erik
I don't see the NFS guys pretending that NFS is a good protocol for
transferring large files across high-latency links... I've only ever
heard of it being used in LAN environments. Yet 9P, which is not
really that dissimilar for NFS in basic concept, gets presented as
something we ought to be using over the Internet, for example to keep
our systems updated.
John
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 21:06 ` John Floren
@ 2011-02-18 22:21 ` Bakul Shah
2011-02-19 10:26 ` Steve Simon
` (2 more replies)
2011-02-19 15:36 ` erik quanstrom
1 sibling, 3 replies; 126+ messages in thread
From: Bakul Shah @ 2011-02-18 22:21 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, 18 Feb 2011 13:06:43 PST John Floren <john@jfloren.net> wrote:
> On Fri, Feb 18, 2011 at 12:15 PM, erik quanstrom <quanstro@quanstro.net> wr=
> ote:
> >> > i don't think that it makes sense to say that since replica
> >> > is slow and hg/rsync are fast, it follows that 9p is slow.
> >>
> >> It is the other way around. 9p can't handle latency so on
> >> high latency pipes programs using 9p won't be as fast as
> >> programs using streaming (instead of rpc). Granted that there
> >> are many other factors when it comes to hg & replica but
> >> latency is a major one.
> >
> > you're still comparing apples and girraffes. =A0rsync/hg have
> > protocols ment for syncing. =A0replica uses 9p, which is not a
> > protocol designed for syncing. =A0it's designed for regular file
> > access. =A0it would be similarly difficult to use rsync's protocol
> > directly for file access.
>
> So why does replica use 9P? Because it's *The Plan 9 Protocol*. If
> *The Plan 9 Protocol* turns out to not serve our needs, we need to
> figure out why.
The point I was trying to make (but clearly not clearly) was
that simplicity and performance are often at cross purposes
and a simple solution is not always "good enough". RPC
(which is what 9p is) is simpler and perfectly fine when
latencies are small but not when there is a lot of latency in
relation to the amount of work doable with each rpc call.
Instead of reading/writing in small chunks, you want to
minimize the number of request/response round trips by
conveying information at a more abstract level (which is
what rsync does).
> 9P as specified in the documentation might not necessarily be the
> problem, but the implementation apparently is.
It is inherent to 9p (and RPC).
The wikipedia page on plan9 says "Plan 9 was engineered for
modern distributed environments, designed from the start to
be a networked operating system." -- but it _is_ curious that
a networked/distributed OS does not handle latency well. This
may be a heretical thing to say but there it is :-)
I think it is worth looking at a successor protocol instead
of just minimally fixing up 9p (a clean slate approach frees
up your mind. You can then merge the two later).
^ permalink raw reply [flat|nested] 126+ messages in thread* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 22:21 ` Bakul Shah
@ 2011-02-19 10:26 ` Steve Simon
2011-02-19 15:09 ` erik quanstrom
2011-02-19 16:57 ` Skip Tavakkolian
2 siblings, 0 replies; 126+ messages in thread
From: Steve Simon @ 2011-02-19 10:26 UTC (permalink / raw)
To: 9fans
> So why does replica use 9P? Because it's *The Plan 9 Protocol*. If
> *The Plan 9 Protocol* turns out to not serve our needs, we need to
> figure out why.
I really don't get this, what is the problem with replica's speed?
I run replica once every week or two and it typically runs for about
30 secconds.
This is not a major part of my life, it doesn't stop me working
in another window.
9p being slow over high latency links is occasionally an annoyance
but replica is not a good example of this.
-Steve
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 22:21 ` Bakul Shah
2011-02-19 10:26 ` Steve Simon
@ 2011-02-19 15:09 ` erik quanstrom
2011-02-19 20:09 ` Bakul Shah
2011-02-19 16:57 ` Skip Tavakkolian
2 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-19 15:09 UTC (permalink / raw)
To: 9fans
> The point I was trying to make (but clearly not clearly) was
> that simplicity and performance are often at cross purposes
> and a simple solution is not always "good enough". RPC
> (which is what 9p is) is simpler and perfectly fine when
> latencies are small but not when there is a lot of latency in
> relation to the amount of work doable with each rpc call.
this is a pretty general claim. but i don't see the argument.
the difference between, e.g. tcp and an explicitly rpc protocol
like 9p is basically 2 relevant things
1. 9p requires 1:1 message and reply; tcp can combine replies
2. 9p sends a set of tagged messages; tcp sends an ordered byte
stream.
i don't see how either one is necessarly going to slow 9p down.
the structure of 9p (tagged rpc) is the same for reads & writes
as aoe. aoe is always talking over a long fat network
with large jitter, as disk drives are slow and sometimes misseek
but have relatively large bandwidth. i've had no trouble filling
even 10gbe pipes with aoe traffic.
> Instead of reading/writing in small chunks, you want to
> minimize the number of request/response round trips by
> conveying information at a more abstract level (which is
> what rsync does).
that's not what rsync does. rsync calculates and sends the
differences. that's not on the same planet as 9p.
by the way, rsync is fundamentally broken. it takes unbounded
memory to do its job.
> It is inherent to 9p (and RPC).
please defend this. i don't see any evidence for this bald claim.
> I think it is worth looking at a successor protocol instead
> of just minimally fixing up 9p (a clean slate approach frees
> up your mind. You can then merge the two later).
what is the goal? without a clear problem to solve that you
can build a system around, i don't see the point. making replica
fast doesn't seem like sufficient motivation to me at all.
as to the problem of latency, i think there are two things
i note
1. plan 9 can work well with latency.
most days i do some link that has 25-2000ms latency.
the way plan 9 is structured, this latency is not very noticable
until it reaches ~750ms, and then only when i'm saving files,
or sometimes, due to an oversite or bug in acme, when searching.
clearly a kernel mk over a 2000ms link will take nearly forever,
but you don't have to do that. you can cpu where you need to
be and work there.
2. at cross-u.s. latencies of 80ms, serial file operations like
mk can be painful. if bringing the work to the data doesn't work
or still takes too many rtts, the only solution i see is to cache everything.
and try to manage coherence vs. latency.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-19 15:09 ` erik quanstrom
@ 2011-02-19 20:09 ` Bakul Shah
2011-02-19 21:15 ` erik quanstrom
0 siblings, 1 reply; 126+ messages in thread
From: Bakul Shah @ 2011-02-19 20:09 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Sat, 19 Feb 2011 10:09:08 EST erik quanstrom <quanstro@quanstro.net> wrote:
> > It is inherent to 9p (and RPC).
>
> please defend this. i don't see any evidence for this bald claim.
We went over latency issues multiple times in the past but
let us take your 80ms latency. You can get 12.5 rpc calls
through in 1 sec even if you take 0 seconds to generate &
process each request & response. If each call transfers 64K,
at most you get a throughput of 800KB/sec. If you pipeline
your requests, without waiting for each reply, you are using
streaming. To avoid `streaming' you can setup N parallel
connections but that is again adding a lot of complexity to a
relatively simple problem.
> > I think it is worth looking at a successor protocol instead
> > of just minimally fixing up 9p (a clean slate approach frees
> > up your mind. You can then merge the two later).
>
> what is the goal?
Better handling of latency at a minimum? If I were to do
this I would experiment with extending the channel concept.
> without a clear problem to solve that you
> can build a system around, i don't see the point. making replica
> fast doesn't seem like sufficient motivation to me at all.
No. I just use Ron's hg repo now.
> 2. at cross-u.s. latencies of 80ms, serial file operations like
> mk can be painful. if bringing the work to the data doesn't work
> or still takes too many rtts, the only solution i see is to cache everything.
> and try to manage coherence vs. latency.
For things like remote copy of a whole bunch of files caching
is not going to help you much but streaming will. So will
increasing parllelism (upto a point). Compression might.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-19 20:09 ` Bakul Shah
@ 2011-02-19 21:15 ` erik quanstrom
2011-02-20 23:49 ` Bakul Shah
0 siblings, 1 reply; 126+ messages in thread
From: erik quanstrom @ 2011-02-19 21:15 UTC (permalink / raw)
To: 9fans
On Sat Feb 19 15:10:58 EST 2011, bakul+plan9@bitblocks.com wrote:
> On Sat, 19 Feb 2011 10:09:08 EST erik quanstrom <quanstro@quanstro.net> wrote:
> > > It is inherent to 9p (and RPC).
> >
> > please defend this. i don't see any evidence for this bald claim.
>
> We went over latency issues multiple times in the past but
> let us take your 80ms latency. You can get 12.5 rpc calls
> through in 1 sec even if you take 0 seconds to generate &
> process each request & response. If each call transfers 64K,
> at most you get a throughput of 800KB/sec. If you pipeline
> your requests, without waiting for each reply, you are using
> streaming. To avoid `streaming' you can setup N parallel
> connections but that is again adding a lot of complexity to a
> relatively simple problem.
i think that your analysis of 9p rpc is missing the fact that each
read() or write() can have 1 out standing through the mount driver.
so the mountpt can have > 800kb/s in your example if multiple
reads or writes are oustanding.
also, sending concurrent rpc doesn't seem the the same to me
as a stream since there's no requirement that the rpcs are sent,
arrive or are processed in order. in plan 9, one would assume
they would be sent in order. (wikipedia's definition.)
and i don't really see that this is complicated. the same
logic is in devaoe(3). the logic is trivial.
> > what is the goal?
>
> Better handling of latency at a minimum? If I were to do
> this I would experiment with extending the channel concept.
hmm. let me try again ... do you have a concrete goal?
it's hard to know why a new file protocol would be necessary
given the very abstract goal of "reduced latency".
for example, i would like to use 1 file server at 3 locations.
2 on the east coast, 1 on the west coast. the rtt is a:b 80ms
a:c 20ms b:c 100ms. i don't think any sort of streaming will
help that much. 100ms is a long time. i think (pre)caching will
need to be the answer.
> For things like remote copy of a whole bunch of files caching
> is not going to help you much but streaming will. So will
> increasing parllelism (upto a point). Compression might.
that depends entirely on your setup. it may make sense to
copy the files before anyone asks for them.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-19 21:15 ` erik quanstrom
@ 2011-02-20 23:49 ` Bakul Shah
2011-02-21 14:47 ` erik quanstrom
0 siblings, 1 reply; 126+ messages in thread
From: Bakul Shah @ 2011-02-20 23:49 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Sat, 19 Feb 2011 16:15:47 EST erik quanstrom <quanstro@quanstro.net> wrote:
> > > what is the goal?
> >
> > Better handling of latency at a minimum? If I were to do
> > this I would experiment with extending the channel concept.
>
> hmm. let me try again ... do you have a concrete goal?
> it's hard to know why a new file protocol would be necessary
> given the very abstract goal of "reduced latency".
Not reduced latency but increased throughput by streaming.
Obviously a new protocol is not "necessary" but something
worth exploring is what I said.
> for example, i would like to use 1 file server at 3 locations.
> 2 on the east coast, 1 on the west coast. the rtt is a:b 80ms
> a:c 20ms b:c 100ms. i don't think any sort of streaming will
> help that much. 100ms is a long time. i think (pre)caching will
> need to be the answer.
Caching is definitely worth doing but you don't always have
the opportunity to do it. If you are copying a lot of files
across, it would help quite a bit if you can just pipeline
requests (or send fewer bundled requests). If you are copying
very large files, streaming would help. When copying large
amounts of data from various sources to a local file server,
caching is not very relevant.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-20 23:49 ` Bakul Shah
@ 2011-02-21 14:47 ` erik quanstrom
0 siblings, 0 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-21 14:47 UTC (permalink / raw)
To: 9fans
> Caching is definitely worth doing but you don't always have
> the opportunity to do it. If you are copying a lot of files
> across, it would help quite a bit if you can just pipeline
> requests (or send fewer bundled requests). If you are copying
> very large files, streaming would help. When copying large
> amounts of data from various sources to a local file server,
> caching is not very relevant.
i'm not sure why the focus on copying a bunch of files. i
wouldn't go to the effort to better stand-alone ftp/scp/http.
the application is a file server and you choose to operate
synchronously, between amdahl's law and the speed of light,
you're going to be in quite a box.
as a contrived example, imagine two nodes of a distributed
fs 80ms apart. now imagine both nodes writing to
/sys/log/timesync. naively copying and locking is going to
be so slow that one imagines that timesync will drift off course.
:-)
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 22:21 ` Bakul Shah
2011-02-19 10:26 ` Steve Simon
2011-02-19 15:09 ` erik quanstrom
@ 2011-02-19 16:57 ` Skip Tavakkolian
2 siblings, 0 replies; 126+ messages in thread
From: Skip Tavakkolian @ 2011-02-19 16:57 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
it seems to me that trying Op (Octopus) on Plan 9 would be a logical first step.
On Fri, Feb 18, 2011 at 2:21 PM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:
> On Fri, 18 Feb 2011 13:06:43 PST John Floren <john@jfloren.net> wrote:
>> On Fri, Feb 18, 2011 at 12:15 PM, erik quanstrom <quanstro@quanstro.net> wr=
>> ote:
>> >> > i don't think that it makes sense to say that since replica
>> >> > is slow and hg/rsync are fast, it follows that 9p is slow.
>> >>
>> >> It is the other way around. 9p can't handle latency so on
>> >> high latency pipes programs using 9p won't be as fast as
>> >> programs using streaming (instead of rpc). Granted that there
>> >> are many other factors when it comes to hg & replica but
>> >> latency is a major one.
>> >
>> > you're still comparing apples and girraffes. =A0rsync/hg have
>> > protocols ment for syncing. =A0replica uses 9p, which is not a
>> > protocol designed for syncing. =A0it's designed for regular file
>> > access. =A0it would be similarly difficult to use rsync's protocol
>> > directly for file access.
>>
>> So why does replica use 9P? Because it's *The Plan 9 Protocol*. If
>> *The Plan 9 Protocol* turns out to not serve our needs, we need to
>> figure out why.
>
> The point I was trying to make (but clearly not clearly) was
> that simplicity and performance are often at cross purposes
> and a simple solution is not always "good enough". RPC
> (which is what 9p is) is simpler and perfectly fine when
> latencies are small but not when there is a lot of latency in
> relation to the amount of work doable with each rpc call.
>
> Instead of reading/writing in small chunks, you want to
> minimize the number of request/response round trips by
> conveying information at a more abstract level (which is
> what rsync does).
>
>> 9P as specified in the documentation might not necessarily be the
>> problem, but the implementation apparently is.
>
> It is inherent to 9p (and RPC).
>
> The wikipedia page on plan9 says "Plan 9 was engineered for
> modern distributed environments, designed from the start to
> be a networked operating system." -- but it _is_ curious that
> a networked/distributed OS does not handle latency well. This
> may be a heretical thing to say but there it is :-)
>
> I think it is worth looking at a successor protocol instead
> of just minimally fixing up 9p (a clean slate approach frees
> up your mind. You can then merge the two later).
>
>
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 21:06 ` John Floren
2011-02-18 22:21 ` Bakul Shah
@ 2011-02-19 15:36 ` erik quanstrom
1 sibling, 0 replies; 126+ messages in thread
From: erik quanstrom @ 2011-02-19 15:36 UTC (permalink / raw)
To: 9fans
> So why does replica use 9P? Because it's *The Plan 9 Protocol*. If
> *The Plan 9 Protocol* turns out to not serve our needs, we need to
> figure out why.
i appreciate the sentiment, but i think that's just taking it a wee bit
overboard. we don't pretend that 9p replaces http, ftp, smtp, etc.
venti/fossil don't even use 9p. nor does factotum, secstore nor the
auth server. these last few might be mistakes, but it's clear that the
thinking was never, we've got a 9p hammer, and all the world's a nail.
that being said, i agree with steve. replica is a fine tool when used
as intended. i think it assumes size(corpus) >> size(differences).
when you use it as a file transfer mechanism, it doesn't work as well.
note the bulk of plan 9 is distributed as an iso through http.
this was necessary because before you have the iso, you don't have
replica. (pre p9p, anyway.) so i don't see why bulk transfer would
have been optimized.
it's a simple tool for a narrow problem. and there are many
ways to speed up replica if anyone cares to solve the problem.
> You like to put forward devmnt's penchant for only having one
> Tread/Twrite per process in flight at one time. I agree that this is a
> problem, now, how do we fix it? All it needs is somebody willing to
> rewrite devmnt... I think you may just have to rewrite mntrdwr to be
> just a little smarter. Any takers?
i think there are two things required. first, the mount driver needs
to have the ability to send 1..n T(read write) messages at once (and
therefore the ability to manage the number of outstanding)
and a mount flag enabling multiple outstanding.
bls suggssted that a mode bit could be added to each file.
this would allow, e.g., exportfs to maintain multiple outstanding
on some files but not others, but that would require
an additional open mode that could be or'd with OREAD/OWRITE.
- erik
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:15 ` Bakul Shah
2011-02-18 19:26 ` erik quanstrom
@ 2011-02-18 19:35 ` David Leimbach
2011-02-18 20:10 ` Bakul Shah
2011-02-18 23:55 ` Federico G. Benavento
1 sibling, 2 replies; 126+ messages in thread
From: David Leimbach @ 2011-02-18 19:35 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs; +Cc: Fans of the OS Plan 9 from Bell Labs
Sent from my iPhone
On Feb 18, 2011, at 11:15 AM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:
> On Fri, 18 Feb 2011 10:46:51 PST Rob Pike <robpike@gmail.com> wrote:
>> The more you optimize, the better the odds you slow your program down.
>> Optimization adds instructions and often data, in one of the
>> paradoxes of engineering. In time, then, what you gain by
>> "optimizing" increases cache pressure and slows the whole thing down.
>
> You need a feedback loop. Uncontrolled anything is a recipe
> for disaster. Optimizations need to be `judicious' but that
> requires experience, profiling and understanding but the
> trend seems to be away from that.....
>
> On a slightly different tangent, 9p is simple but it doesn't
> handle latency very well. To make efficient use of long fat
> pipes you need more complex mechanisms -- there is no getting
> around that fact. rsync & hg in spite of their complexity
> beat the pants off replica. Their cache behavior is not very
> relevant here. Similarly file readahead is usually a win.
>
>> C++ inlines a lot because microbenchmarks improve, but inline every
>> modest function in a big program and you make the binary much bigger
>> and blow the i-cache.
>
> That's a compiler fault. Surely modern compilers need to be
> cache aware? ideally a smart compiler treats `inline' as a hint
> at most, just like `register'.
>
Well how does template expansion affect all of this? I've heard in conversations that C++ is pretty register hungry which makes me think lots of inlining happens behind the scenes. Then again that's an implementation detail, except maybe for templates.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:35 ` David Leimbach
@ 2011-02-18 20:10 ` Bakul Shah
2011-02-18 21:03 ` ron minnich
2011-02-18 23:55 ` Federico G. Benavento
1 sibling, 1 reply; 126+ messages in thread
From: Bakul Shah @ 2011-02-18 20:10 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Fri, 18 Feb 2011 11:35:18 PST David Leimbach <leimy2k@gmail.com> wrote:
> >> C++ inlines a lot because microbenchmarks improve, but inline every
> >> modest function in a big program and you make the binary much bigger
> >> and blow the i-cache.
> >
> > That's a compiler fault. Surely modern compilers need to be
> > cache aware? ideally a smart compiler treats `inline' as a hint
> > at most, just like `register'.
>
> Well how does template expansion affect all of this? I've heard in conversa=
> tions that C++ is pretty register hungry which makes me think lots of inlini=
> ng happens behind the scenes. Then again that's an implementation detail, e=
> xcept maybe for templates.=
Templates encourage inlining. There is at least one template
libraries where the bulk of code is implemented in separate
.cc files (using void* tricks), used by some embedded
products. But IIRC the original STL from sgi was all in .h
files and things don't seem to have changed much -- but I avoid
them so who knows.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 20:10 ` Bakul Shah
@ 2011-02-18 21:03 ` ron minnich
0 siblings, 0 replies; 126+ messages in thread
From: ron minnich @ 2011-02-18 21:03 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs; +Cc: Bakul Shah
On Fri, Feb 18, 2011 at 12:10 PM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:
> Templates encourage inlining. There is at least one template
> libraries where the bulk of code is implemented in separate
> .cc files (using void* tricks), used by some embedded
> products. But IIRC the original STL from sgi was all in .h
> files and things don't seem to have changed much -- but I avoid
> them so who knows.
Very little of Boost libraries are libraries -- they are include
files. If I have 100 files, and they include a lot of boost stuff,
then I get to recompile the same Boost files many, many times.
I spent several hours yesterday watching Boost "build" and then
install -- 7000+ files in all. I guess it's all very useful. And
modern.
There was a C++ package called Pooma. It introduced the notion of 38
MB symbol tables and symbols so long (due to use of templates and so
on) that they caused almost every extant C++ compiler to core dump in
1999 -- 256 characters is such a limitation on symbol name length ...
the fix was to issue lots of money to people to "fix" their compiler
to handle multi-thousand-character symbol names.
ron
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:35 ` David Leimbach
2011-02-18 20:10 ` Bakul Shah
@ 2011-02-18 23:55 ` Federico G. Benavento
1 sibling, 0 replies; 126+ messages in thread
From: Federico G. Benavento @ 2011-02-18 23:55 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
afaik, templates might be inlined, static or shared... depending on
the compiler and the flags.
for gcc see:
http://gcc.gnu.org/onlinedocs/gcc/Template-Instantiation.html
On Fri, Feb 18, 2011 at 4:35 PM, David Leimbach <leimy2k@gmail.com> wrote:
>
>
> Sent from my iPhone
>
> On Feb 18, 2011, at 11:15 AM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:
>
>> On Fri, 18 Feb 2011 10:46:51 PST Rob Pike <robpike@gmail.com> wrote:
>>> The more you optimize, the better the odds you slow your program down.
>>> Optimization adds instructions and often data, in one of the
>>> paradoxes of engineering. In time, then, what you gain by
>>> "optimizing" increases cache pressure and slows the whole thing down.
>>
>> You need a feedback loop. Uncontrolled anything is a recipe
>> for disaster. Optimizations need to be `judicious' but that
>> requires experience, profiling and understanding but the
>> trend seems to be away from that.....
>>
>> On a slightly different tangent, 9p is simple but it doesn't
>> handle latency very well. To make efficient use of long fat
>> pipes you need more complex mechanisms -- there is no getting
>> around that fact. rsync & hg in spite of their complexity
>> beat the pants off replica. Their cache behavior is not very
>> relevant here. Similarly file readahead is usually a win.
>>
>>> C++ inlines a lot because microbenchmarks improve, but inline every
>>> modest function in a big program and you make the binary much bigger
>>> and blow the i-cache.
>>
>> That's a compiler fault. Surely modern compilers need to be
>> cache aware? ideally a smart compiler treats `inline' as a hint
>> at most, just like `register'.
>>
>
> Well how does template expansion affect all of this? I've heard in conversations that C++ is pretty register hungry which makes me think lots of inlining happens behind the scenes. Then again that's an implementation detail, except maybe for templates.
>
--
Federico G. Benavento
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 18:46 ` Rob Pike
2011-02-18 19:15 ` Bakul Shah
@ 2011-02-18 19:15 ` Devon H. O'Dell
2011-02-21 5:08 ` smiley
1 sibling, 1 reply; 126+ messages in thread
From: Devon H. O'Dell @ 2011-02-18 19:15 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
2011/2/18 Rob Pike <robpike@gmail.com>:
> The more you optimize, the better the odds you slow your program down.
> Optimization adds instructions and often data, in one of the
> paradoxes of engineering. In time, then, what you gain by
> "optimizing" increases cache pressure and slows the whole thing down.
>
> C++ inlines a lot because microbenchmarks improve, but inline every
> modest function in a big program and you make the binary much bigger
> and blow the i-cache.
I think what I've been trying to say in this thread doesn't clash with
anything that you, Erik, or others have said. Understanding the
system, the complexity you are introducing, and carefully measuring
the net effect are all important parts of the optimization process.
You can't just switch from one data structure to another. Skip lists
are a really great example: they have really amazing properties, but
you trash your cache when you use them, and gathering entropy to
determine where a node is placed is *not* cheap. In the end, an
optimization that slows things down is not an optimization at all. You
can't do it if you don't understand what you're doing, and you don't
understand the overall effect.
I don't think that Linux's gettimeofday(2) optimization falls into
this category, though I do think that some of the similar
optimizations they've done using the same approach do. In this
specific case, it is an easy optimization that is cheap and works
quite well. It provides measurable performance improvements in the
general case, as well as in special cases. As mentioned before, people
who need more accurate times can still use rdtsc.
> -rob
--dho
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 19:15 ` Devon H. O'Dell
@ 2011-02-21 5:08 ` smiley
0 siblings, 0 replies; 126+ messages in thread
From: smiley @ 2011-02-21 5:08 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
"Devon H. O'Dell" <devon.odell@gmail.com> writes:
> determine where a node is placed is *not* cheap. In the end, an
> optimization that slows things down is not an optimization at all. You
There are many different kinds of optimization one can perform. One may
optimize compiled code for size, speed, simplicity, or reliability. One
may optimize communications for throughput, latency, reliability, or
bandwidth efficiency. Operations may be optimized to peform within
guaranteed constraints, such as with real-time applications, bandwith
contracting, etc.
Very often, optimizing for one property sacrifices optimization in one
or more of the others. For example, code optimized for speed may occupy
more memory; code optimized for simplicity may run more slowly; etc.
Which optimization (implementation) is chosen for a particular
application depends on the intended use of the application. Ultimately,
the resultant properties of the application should be reflected in its
documentation. If an app like rsync is intended for -- and good at --
synchronizing files, but not as fast as wget for copying them, it would
only be appropriate for rsync's documentation to indicate that design
assumption. That way, the user can choose the application with the
desired properties for the job.
It's not possible simply to "optimize" code. It always has to be
optimized for some specific set of intended uses. The design of a
protocol (such as 9P) for a certain set of circumstances entails that it
will perform better under some circumstances than others.
--
+---------------------------------------------------------------+
|E-Mail: smiley@zenzebra.mv.com PGP key ID: BC549F8B|
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
+---------------------------------------------------------------+
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 17:07 ` erik quanstrom
2011-02-18 17:11 ` Devon H. O'Dell
@ 2011-02-18 17:16 ` Russ Cox
1 sibling, 0 replies; 126+ messages in thread
From: Russ Cox @ 2011-02-18 17:16 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs; +Cc: erik quanstrom
On Fri, Feb 18, 2011 at 12:07 PM, erik quanstrom <quanstro@quanstro.net> wrote:
>> The high level overview is that it is stored in a shared page, mapped
>> into each new process's memory space at start-up. The kernel is never
>> entered; there are no context switches. The kernel has a timer that
>> updates this page atomically.
>
> i wonder if that is uniformly faster. consider that
> making reads of that page coherent enough on a
> big multiprocessor and making sure there's not too
> much interprocesser skew might be slower than a
> system call.
are you claiming that
enter system call
look at kernel data page to figure out time
exit system call
could be faster than
look at kernel data page to figure out time
?
either way the memory accesses in the middle
are the same.
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 16:59 ` Devon H. O'Dell
2011-02-18 17:07 ` erik quanstrom
@ 2011-02-18 17:12 ` andrey mirtchovski
1 sibling, 0 replies; 126+ messages in thread
From: andrey mirtchovski @ 2011-02-18 17:12 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
> The kernel has a timer that
> updates this page atomically.
which timer updates the page even when nobody is interested in knowing
what the time is, increasing the noise in the system[1]. i still keep
graphs of a full-blown plan9 cpu server with users logged in and close
to 200 running processes exhibiting very little deviation:
http://mirtchovski.com/screenshots/ftq.jpg
people are doing work negating the timer "optimization":
http://ozlabs.org/~anton/linux/osjitter/
andrey
[1] http://www.computer.org/portal/web/csdl/doi/10.1109/CLUSTR.2004.1392636
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-18 15:15 ` Devon H. O'Dell
2011-02-18 16:11 ` erik quanstrom
2011-02-18 16:53 ` dexen deVries
@ 2011-02-19 10:34 ` Steve Simon
2011-02-19 17:25 ` dexen deVries
2 siblings, 1 reply; 126+ messages in thread
From: Steve Simon @ 2011-02-19 10:34 UTC (permalink / raw)
To: 9fans
> Benchmark utilities to measure the overhead of syscalls. It's cheating
> to do for getpid, but for other things like gettimeofday, it's
> *extremely* nice. Linux's gettimeofday(2) beats the socks off of the
> rest of the time implementations. About the only faster thing is to
> get CPU speed and use rdtsc. Certainly no other OS allows you to get
> the timestamp faster with a syscall.
Here is where my memory gets hazy, however Solaris 2 had a very fast
implementation of gettimeofday(), it was still a syscall I think but
had a shortcut in the kernel.
This was added (If I rembember correctly) to get a database (Sybase
I think) to run on Solaris 2 as fast as it always used to run on SunOS.
This was commented in the code as a special, ugly hack as a result of
extreme pressure from an important customer.
I wonder if Linux inherited the hack from Solaris?
-Steve
^ permalink raw reply [flat|nested] 126+ messages in thread
* Re: [9fans] Modern development language for Plan 9, WAS: Re: RESOLVED: recoving important header file rudely
2011-02-19 10:34 ` Steve Simon
@ 2011-02-19 17:25 ` dexen deVries
0 siblings, 0 replies; 126+ messages in thread
From: dexen deVries @ 2011-02-19 17:25 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Saturday 19 of February 2011 11:34:19 Steve Simon wrote:
> > Benchmark utilities to measure the overhead of syscalls. It's cheating
> > to do for getpid, but for other things like gettimeofday, it's
> > *extremely* nice. Linux's gettimeofday(2) beats the socks off of the
> > rest of the time implementations. About the only faster thing is to
> > get CPU speed and use rdtsc. Certainly no other OS allows you to get
> > the timestamp faster with a syscall.
>
> Here is where my memory gets hazy, however Solaris 2 had a very fast
> implementation of gettimeofday(), it was still a syscall I think but
> had a shortcut in the kernel.
>
> This was added (If I rembember correctly) to get a database (Sybase
> I think) to run on Solaris 2 as fast as it always used to run on SunOS.
> This was commented in the code as a special, ugly hack as a result of
> extreme pressure from an important customer.
>
> I wonder if Linux inherited the hack from Solaris?
Perhaps the concept of providing very fast gettimeofday() at the cost of using
an uncommon implementation, but not necessarily the actual implementation.
Linux' gettimeofday() /is not a syscall/ at all.
Devon posted the following earlier:
> The high level overview is that it is stored in a shared page, mapped
> into each new process's memory space at start-up. The kernel is never
> entered; there are no context switches. The kernel has a timer that
> updates this page atomically.
>
> --dho
--
dexen deVries
``One can't proceed from the informal to the formal by formal means.''
^ permalink raw reply [flat|nested] 126+ messages in thread
