Re: [9fans] Interested in improving networking in Plan 9

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

personally, i think the best contributions come
from people who have a real personal need or
better want to solve a problem, solve it and
contribute the  solution back to the community.

i think that's why unix and plan 9 exist at all.

so i would encourage folks who would like to
contribute to find stuff they're intereted in.

i also think that it may be a good idea for
us to figure out how to help folks who would
like to take on projects like this do so outside
the framework of gsoc.

i'm willing to contribute time to that end.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Vinu Rajashekhar]

On Sun, Aug 30, 2009 at 10:52 PM, erik quanstrom<quanstro@quanstro.net> wrote:
> personally, i think the best contributions come
> from people who have a real personal need or
> better want to solve a problem, solve it and
> contribute the  solution back to the community.
>

Yes, I do agree with that.

> i think that's why unix and plan 9 exist at all.
>
> so i would encourage folks who would like to
> contribute to find stuff they're intereted in.
>
> i also think that it may be a good idea for
> us to figure out how to help folks who would
> like to take on projects like this do so outside
> the framework of gsoc.

I wasn't thinking about doing this as a GSOC project,
I wanted to do something for my master's project which
was a hardcore open-source implementation, that's why I
was going through the gsoc ideas page.

>
> i'm willing to contribute time to that end.
>
> - erik
>
>



--
Vinu Rajashekhar,
5th Year Dual Degree Student,
Deptt of Computer Science & Engg,
IIT Kharagpur,
India.

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> I wasn't thinking about doing this as a GSOC project,
> I wanted to do something for my master's project which
> was a hardcore open-source implementation, that's why I
> was going through the gsoc ideas page.

makes sense to me.  i'd incourage you to work a bit with
the community.  part of open source is fitting in with
the community.

also, there are two related ip projects that might be of
interest to you: nat (non existent) and ip6 (needs work).

- erik

[9fans] Interested in improving networking in Plan 9

[Vinu Rajashekhar]

Hi,
 I was looking for some open-source implementation work to be done as my
master's project when I chanced upon the Plan 9 GSOC projects page.
My interest is in networking, so I was particularly interested in projects about
adding zeroconf networking and firewall support to Plan 9.

 I think I have sufficient knowledge of C/C++, but I haven't ever
worked or used Plan 9, but have heard of it, so I would like to take this
opportunity to get to know the system better and contribute to the
community. I have a year with me to do the project, though the coursework
is interlaced with classes, so I won't be able to dedicate 30-40 hrs per week
 like the GSOC students do ! So a year is a sufficient enough time I guess.

 Can someone please discuss with me how to proceed, and what are the things
I should learn before starting on this ? I have already started
reading the papers on Plan 9 and is starting to install the plan 9
on virtual box on my machine.



--
Vinu Rajashekhar,
5th Year Dual Degree Student,
Deptt of Computer Science & Engg,
IIT Kharagpur,
India.

Re: [9fans] Interested in improving networking in Plan 9

[ron minnich]

I think your first, best bet is to try to find out what the Plan 9
community needs, rather than adding on something that might not be the
that important. I have not heard anyone express a need for zeroconf in
Plan 9, but maybe I'm missing something.

Simple example: file systems on Plan 9 are slow. Why is that? How do
they work? How would you go about finding out how to make them faster?

Linux has oprofile and (now) ftrace. Would either of these be helpful?

Another simple example: networking on Plan 9 could be faster. What
makes it less than it could be? How would you figure out how to fix
it?

Another example: we don't have much in the way of current graphics
drivers. Why is that? How would you fix it? One mechanism we use to
support graphics is VESA mode, which does not work in SMP. Why is
that? Could you fix VESA code to work in SMP?

This last project (VESA) is pretty self-contained and easily defined,
you might want to start there.

I think you need to spend some time climbing the learning curve before
picking a project.

ron

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Sun, Aug 30, 2009 at 9:21 AM, ron minnich<rminnich@gmail.com> wrote:
> I think your first, best bet is to try to find out what the Plan 9
> community needs, rather than adding on something that might not be the
> that important. I have not heard anyone express a need for zeroconf in
> Plan 9, but maybe I'm missing something.
>

He was mining the GSoC idea archives -- and I had expressed a desire
for zeroconf -- I'd really like to get rid of /lib/ndb/local as much
as such a thing would be possible.  We also need it for Blue Gene and
Cloud environments since IP addresses are subject to change and need
discovery.  The zeroconf support itself is actually kind of trivial --
its coming up with a reasonable synthetic file system interface (for
discovery as well as publishing) which would be the real contribution.

      -eric

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> Simple example: file systems on Plan 9 are slow. Why is that? How do
> they work? How would you go about finding out how to make them faster?

which ones?  there are quite a number to choose from.
i've found that ken's fs beats the pants off nfs on similar
hardware with literally one cpu tied behind its back.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Venkatesh Srinivas]

On Sun, Aug 30, 2009 at 1:07 PM, erik quanstrom<quanstro@quanstro.net> wrote:
>> Simple example: file systems on Plan 9 are slow. Why is that? How do
>> they work? How would you go about finding out how to make them faster?
>
> which ones?  there are quite a number to choose from.
> i've found that ken's fs beats the pants off nfs on similar
> hardware with literally one cpu tied behind its back.

Try this - build the source to charon over a 200ms link over 9p. Then
try again over sshfs.

Also, look at a single terminal with a local fossil install. Trace the
path of an 'ls /'. Count the number of copies and context switches.

Having the fastest file server in the world means nothing for file
system performance when your path to it is a maze of twisty passages,
all alike.

-- vs

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> Try this - build the source to charon over a 200ms link over 9p. Then
> try again over sshfs.

why would you do this?  why not run the compile closer to
the source.  this is the power of plan 9.

> Also, look at a single terminal with a local fossil install. Trace the
> path of an 'ls /'. Count the number of copies and context switches.
>
> Having the fastest file server in the world means nothing for file
> system performance when your path to it is a maze of twisty passages,
> all alike.

again, this is not a typical install for a performance-sensitive
system.  and i don't know that anyone ever claimed fossil
to be a high-performance file system.  running a file system in user
space is a matter of convienence, not performance.

at coraid, we run a stand-alone ken's file server.  ken's file
server has no user space.  so while there are context switches,
those are really speedy.  they take maybe a few hundred cycles.
the real win is that no data is copied from/to user space and
there is one global page table that is never changed.  there are
no tlb flushes.  those can hurt.

before the switch to nupas, we averaged 200MB/s of fileserver traffic
during the day to >50 clients on a single, fairly pedestrian
xeon 5000 machine with 3.5gb of usable memory.

perhaps my standards are low and i'm out of touch, but i don't
think nfs could do that well with that little.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[ron minnich]

I see your point. It does sound like zeroconf would be useful to some
people. I wonder if it could be done with a 9p orientation as eric
suggested.

I don't recall what the security issues are with zeroconf, but, if
it's the microsoft-inspired version I'm thinking of, I would guess
there are many. I wonder if Plan 9 could provide some angle to make it
more secure?

Anyway, I did not mean to discourage your enthusiasm; go for it and
let us know what we can do to help.

ron

Re: [9fans] Interested in improving networking in Plan 9

[ron minnich]

One way to make this kind of interesting is to address how you'd do a
reasonable zeroconf effort given that you need to boot 1m+ machines.
We've booted 4400*250 VMs on a machine at sandia, and, let me tell
you, it was a pain. It is amusing to watch the programs traverse
million line /etc/hosts file -- for a while.

So we're working on approaches in which files -- e.g. /etc/hosts --
are replace with programs. Key is that there can be no configuration
files that are linearly related in size to the number of nodes --
which kills dhcpd, hosts, and so on. 9P servers make this sort of
thing bearable.

Just FYI.

ron

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

On Sun Aug 30 14:37:29 EDT 2009, rminnich@gmail.com wrote:
> One way to make this kind of interesting is to address how you'd do a
> reasonable zeroconf effort given that you need to boot 1m+ machines.
> We've booted 4400*250 VMs on a machine at sandia, and, let me tell
> you, it was a pain. It is amusing to watch the programs traverse
> million line /etc/hosts file -- for a while.

how does this apply to plan 9?  ndb already provides
for indexed databases.  but i have not tried this with
millions of entries.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Aug 30, 2009, at 10:34 PM, erik quanstrom <quanstro@quanstro.net>
wrote:

> On Sun Aug 30 14:37:29 EDT 2009, rminnich@gmail.com wrote:
>> One way to make this kind of interesting is to address how you'd do a
>> reasonable zeroconf effort given that you need to boot 1m+ machines.
>> We've booted 4400*250 VMs on a machine at sandia, and, let me tell
>> you, it was a pain. It is amusing to watch the programs traverse
>> million line /etc/hosts file -- for a while.
>
> how does this apply to plan 9?  ndb already provides
> for indexed databases.  but i have not tried this with
> millions of entries.
>

I think there are a few issues beyond will it scale - of course with
128k nodes scaling is a baseline prereq for us.  On BG we have a
segmented network to deal with -- but it's likely you'll want some
form of hierarchy regardless.

I have done much with dynamic service registry using DNS in plan 9 -
maybe it's easy and just not well documented.

Then there's issue of locality - finding the registered service that
is closest to you.

Then there is load balancing -  finding the closest service whch isn't
heavily loaded.

Then there is dealing with reliability, security, and so forth and so
on.

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> I think there are a few issues beyond will it scale - of course with
> 128k nodes scaling is a baseline prereq for us.  On BG we have a
> segmented network to deal with -- but it's likely you'll want some
> form of hierarchy regardless.
>
> I have done much with dynamic service registry using DNS in plan 9 -
> maybe it's easy and just not well documented.

i'm sure there are.

could you explan why you're focused on dns?
a more natural way to use plan 9 would be to use
ndb and cs directly.  wouldn't it?

by the way, ndb/cs is already a program that
replaces a static file.  why couldn't you use
it to do these interesting lookups?  it already
has some specialized knowledge of protocols.
it seems like the place for these things.

i've probablly just given away my vast ignorance.
please set me straight.  :-)

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Vinu Rajashekhar]

"You can implement a NAT by mounting a /net from a perimeter machine
with a public IP, while connecting to it from an internal network of private
IP addresses, using the Plan 9 protocol 9P in the internal network."

This is from the wikipedia page on Plan 9 OS.

Is something like iptables like in linux needed to be implemented for
Plan 9 ?

Again forgive me if I came out looking like an idiot !

On Mon, Aug 31, 2009 at 6:21 PM, erik quanstrom<quanstro@quanstro.net> wrote:
>> I think there are a few issues beyond will it scale - of course with
>> 128k nodes scaling is a baseline prereq for us.  On BG we have a
>> segmented network to deal with -- but it's likely you'll want some
>> form of hierarchy regardless.
>>
>> I have done much with dynamic service registry using DNS in plan 9 -
>> maybe it's easy and just not well documented.
>
> i'm sure there are.
>
> could you explan why you're focused on dns?
> a more natural way to use plan 9 would be to use
> ndb and cs directly.  wouldn't it?
>
> by the way, ndb/cs is already a program that
> replaces a static file.  why couldn't you use
> it to do these interesting lookups?  it already
> has some specialized knowledge of protocols.
> it seems like the place for these things.
>
> i've probablly just given away my vast ignorance.
> please set me straight.  :-)
>
> - erik
>
>



-- 
Vinu Rajashekhar,
5th Year Dual Degree Student,
Deptt of Computer Science & Engg,
IIT Kharagpur,
India.

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> "You can implement a NAT by mounting a /net from a perimeter machine
> with a public IP, while connecting to it from an internal network of private
> IP addresses, using the Plan 9 protocol 9P in the internal network."
>
> This is from the wikipedia page on Plan 9 OS.
>
> Is something like iptables like in linux needed to be implemented for
> Plan 9 ?
>
> Again forgive me if I came out looking like an idiot !

not at all.  of course this only works for computers that can
import /net.  just theoretically suppose that your gateway
is a plan 9 machine and someone in the house has a windows
box.  nat would be useful for supporting the windows box
that can't import /net.

some people would argue that a lot of modems can do the
nat so why bother.  the reason to bother is to (a) avoid
buggy modem firmware and (b) to integrate nat into the
rest of one's network services so things like logging, etc.
are uniform.

and you don't have to deal with the modems dumb http
interface.  :-)

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Anthony Sorace]

that wiki writeup isn't really right. importing /net isn't NAT in any
sort of technical sense; rather, it's what plan 9 does instead.
there's no "translation" of ports or addresses, it's more
(conceptually) like a straight multiplexing.

Re: [9fans] Interested in improving networking in Plan 9

[Devon H. O'Dell]

2009/8/31 Vinu Rajashekhar <vinutheraj@gmail.com>:
> "You can implement a NAT by mounting a /net from a perimeter machine
> with a public IP, while connecting to it from an internal network of private
> IP addresses, using the Plan 9 protocol 9P in the internal network."
>
> This is from the wikipedia page on Plan 9 OS.
>
> Is something like iptables like in linux needed to be implemented for
> Plan 9 ?

My student's summer of code project, which was quite unfortunately not
completed, was to implement support for NAT in Plan 9, and to
implement a firewalling infrastructure. I think it would be good to
implement something like this, and several people thought that
implementing NAT, if done correctly, would be quite useful for people
running Plan 9. I'm still interested in providing guidance and info
about this if it's something you're interested in pursuing -- I have
quite a few ideas on how it should work.

Kind regards,

Devon H. O'Dell

Re: [9fans] Interested in improving networking in Plan 9

[Vinu Rajashekhar]

On Mon, Aug 31, 2009 at 8:21 PM, Devon H. O'Dell<devon.odell@gmail.com> wrote:
> 2009/8/31 Vinu Rajashekhar <vinutheraj@gmail.com>:
>> "You can implement a NAT by mounting a /net from a perimeter machine
>> with a public IP, while connecting to it from an internal network of private
>> IP addresses, using the Plan 9 protocol 9P in the internal network."
>>
>> This is from the wikipedia page on Plan 9 OS.
>>
>> Is something like iptables like in linux needed to be implemented for
>> Plan 9 ?
>
> My student's summer of code project, which was quite unfortunately not
> completed, was to implement support for NAT in Plan 9, and to
> implement a firewalling infrastructure. I think it would be good to
> implement something like this, and several people thought that
> implementing NAT, if done correctly, would be quite useful for people
> running Plan 9. I'm still interested in providing guidance and info
> about this if it's something you're interested in pursuing -- I have
> quite a few ideas on how it should work.
>

 Yea I did see that it was accepted as a project, but I couldn't see the
actual sources or anything, so I assumed it may have got stalled. I am
interested in it, from an overall point of view, but right now I am don't know
enough about Plan 9 to give any constructive ideas. Still climbing the learning
curve ! Thanks for responding though.

> Kind regards,
>
> Devon H. O'Dell
>
>



--
Vinu Rajashekhar,
5th Year Dual Degree Student,
Deptt of Computer Science & Engg,
IIT Kharagpur,
India.

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 7:51 AM, erik quanstrom<quanstro@quanstro.net> wrote:
> could you explan why you're focused on dns?
> a more natural way to use plan 9 would be to use
> ndb and cs directly.  wouldn't it?
>

Of course.  My use of DNS was really just in abstract to refer to the
suite of existing services for name and service resolution under Plan
9.  However, I think the current interfaces for ndb and cs are very
limiting and the single file based query mechanisms don't really match
the hierarchical style of the rest of the system.  If there is a
masters project in here, its to look at a revision of the whole suite
and extension of the relevant file system based interfaces.  The other
side of this is then to integrate this new service
registration/discovery mechanism to existing applications.  When I
export, a flag should make that export visible to zeroconf resolution,
etc.

      -eric

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> Of course.  My use of DNS was really just in abstract to refer to the
> suite of existing services for name and service resolution under Plan
> 9.  However, I think the current interfaces for ndb and cs are very
> limiting and the single file based query mechanisms don't really match
> the hierarchical style of the rest of the system.  If there is a
> masters project in here, its to look at a revision of the whole suite
> and extension of the relevant file system based interfaces.  The other
> side of this is then to integrate this new service

i definately think there's room for improvement, but
i would hate for ndb to morph into the windows registry.

the main situation where ndb is weak is when there are rapid changes.
otherwise ndb/cs and ndb/mkhash should make it unnecessary to scan whole
files.

given the database= option, if one could confine rapid changes to
smaller files, one could teach ndb to only reread changed files.

> registration/discovery mechanism to existing applications.  When I
> export, a flag should make that export visible to zeroconf resolution,
> etc.

what do you mean by export?

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 9:04 AM, erik quanstrom<quanstro@quanstro.net> wrote:
>
> given the database= option, if one could confine rapid changes to
> smaller files, one could teach ndb to only reread changed files.
>

Why not have a synthetic file system interface to ndb that allows it
to update its own files?  I think this is my primary problem.
Granular modification to static files is a PITA to manage -- we should
be using synthetic file system interfaces to to help manage and gate
modifications.  Most of the services I have in mind may be transient
and task specific, so there are elements of scope to consider and you
may not want to write anything out to static storage.

>> registration/discovery mechanism to existing applications.  When I
>> export, a flag should make that export visible to zeroconf resolution,
>> etc.
>
> what do you mean by export?
>

When I publish a service, in the Plan 9 case primarily by exporting a
synthetic file system.  I shouldn't have to have static configuration
for file servers, it should be much more fluid.  I'm not arguing for a
microsoft style registry -- but the network discovery environment on
MacOSX is much nicer than what we have today within Plan 9.  An even
better example is the environment on the OLPC, where many of the
applications are implicitly networked and share resources based on
Zeroconf pub/sub interfaces.

       -eric

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> >
> > given the database= option, if one could confine rapid changes to
> > smaller files, one could teach ndb to only reread changed files.
> >
>
> Why not have a synthetic file system interface to ndb that allows it
> to update its own files?  I think this is my primary problem.
> Granular modification to static files is a PITA to manage -- we should
> be using synthetic file system interfaces to to help manage and gate
> modifications.  Most of the services I have in mind may be transient
> and task specific, so there are elements of scope to consider and you
> may not want to write anything out to static storage.

i can see in principle how this could be a good idea (no more
comments, though).  could you elaborate, though.  i have found
editing /lib/ndb/local works well at the scales i see.

i also don't know what you mean by "transient, task specific services".
i can only think of things like ramfs or cdfs.  but they live in my
namespace so ndb doesn't enter into the picture.

could you give an example?

> When I publish a service, in the Plan 9 case primarily by exporting a
> synthetic file system.  I shouldn't have to have static configuration
> for file servers, it should be much more fluid.  I'm not arguing for a
> microsoft style registry -- but the network discovery environment on
> MacOSX is much nicer than what we have today within Plan 9.  An even
> better example is the environment on the OLPC, where many of the
> applications are implicitly networked and share resources based on
> Zeroconf pub/sub interfaces.

sounds interesting.  but i don't understand what you're talking about
exactly.  maybe you're thinking that cpu could be rigged so that
cpu with no host specifier would be equivalent to cpu -h '$boredcpu'
where '$boredcpu' would be determined by cs via dynamic mapping?
or am i just confused?

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 9:36 AM, erik quanstrom<quanstro@quanstro.net> wrote:
>
> i can see in principle how this could be a good idea (no more
> comments, though).  could you elaborate, though.  i have found
> editing /lib/ndb/local works well at the scales i see.
>

I think the main issue with just editing /lib/ndb/local is a
combination of scale and number of writers.  Writing static config
files could work fine in a "typical" plan 9 network of hundreds of
machines, even with multiple admins.  I have a feeling it starts to
break down with thousands of machines, particularly in an environment
where machines are appearing and disappearing at regular intervals
(clouds, HPC partitioning, or Blue Gene).  Hundreds of thousands of
nodes with this sort of behavior probably makes it impractical.  Of
course -- this won't effect the casual user, but its something that
effects us.  Its also possible that such a dynamic environment would
better support a 9grid style environment.

>
> i also don't know what you mean by "transient, task specific services".
> i can only think of things like ramfs or cdfs.  but they live in my
> namespace so ndb doesn't enter into the picture.
>

There is the relatively mundane configuration examples of publishing
multiple file servers, authentication servers, and cpu servers.
There's a slightly more interesting example of more pervasive sharing
(imagine sharing portions of your ACME file system to collaborate with
several co-authors, or for code review).  The more applications which
export synthetic file systems, the more opportunity there is for
sharing and requiring a broader pub/sub interface.

There is another option here which I'm side-stepping because its
something I'm actively working on -- which is instead of doing such a
pub/sub interface within ndb and CS, extending srv to provide a
registry for cluster/grid/cloud.  However, underneath it may still be
nice to have zeroconf as a pub/sub for interoperation with non-Plan 9
systems.

>
>> When I publish a service, in the Plan 9 case primarily by exporting a
>> synthetic file system.  I shouldn't have to have static configuration
>> for file servers, it should be much more fluid.  I'm not arguing for a
>> microsoft style registry -- but the network discovery environment on
>> MacOSX is much nicer than what we have today within Plan 9.  An even
>> better example is the environment on the OLPC, where many of the
>> applications are implicitly networked and share resources based on
>> Zeroconf pub/sub interfaces.
>
> sounds interesting.  but i don't understand what you're talking about
> exactly.  maybe you're thinking that cpu could be rigged so that
> cpu with no host specifier would be equivalent to cpu -h '$boredcpu'
> where '$boredcpu' would be determined by cs via dynamic mapping?
> or am i just confused?
>

Actually, the idea would be that cpu's default behavior would be to go
grab $boredcpu -- but that's part of the idea.  That would make adding
cpu servers as easy as booting them -- they'd publish their service
with zeroconf and everyone would automatically pick it up and be able
to query them for additional attributes related to utilization,
authentication, or even fee-structures.  Of course, as a I alluded to
earlier, I think its much more interesting in the presence of
pervasive network services exported through file systems.  I'd suggest
those who haven't to go grab "sugar on a stick" from the OLPC folks
and run it under vmware or whatever.  I'm not broadly endorsing every
aspect of their environment, but I really liked their loosely coupled
sharing/collaboration framework built upon zeroconf and their mesh
networks (it may be a bit hard to see this on sugar on a stick, but
there were in the past several public OLPC zeroconf servers you could
attach to and play around.

       -eric

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> > i can see in principle how this could be a good idea (no more
> > comments, though).  could you elaborate, though.  i have found
> > editing /lib/ndb/local works well at the scales i see.
[...]
> machines, even with multiple admins.  I have a feeling it starts to
> break down with thousands of machines, particularly in an environment
> where machines are appearing and disappearing at regular intervals
> (clouds, HPC partitioning, or Blue Gene).  Hundreds of thousands of
> nodes with this sort of behavior probably makes it impractical.  Of
> course -- this won't effect the casual user, but its something that
> effects us.

so plunkers like us with a few hundred machines are just "casual users"?
i'd hate for plan 9 to become harder to use outside a hpc environment.
it would be good to be flexable enough to support fairly degnerate cases
like just flat files.

> > i also don't know what you mean by "transient, task specific services".
> > i can only think of things like ramfs or cdfs.  but they live in my
> > namespace so ndb doesn't enter into the picture.
> >
>
> There is the relatively mundane configuration examples of publishing
> multiple file servers, authentication servers, and cpu servers.

how many file servers and authentication servers are you running?

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 10:52 AM, erik quanstrom<quanstro@coraid.com> wrote:
>
> so plunkers like us with a few hundred machines are just "casual users"?
> i'd hate for plan 9 to become harder to use outside a hpc environment.
> it would be good to be flexable enough to support fairly degnerate cases
> like just flat files.
>

I don't disagree.  Worst case, this is a complementary platform for
large-scale deployments, best case is that its an alternative
interface that also improves the experience for the "casual" user -- I
think the main benefit here will be in establishing better mechanisms
for collaboration amongst "casual" users.  If any aspect of this makes
things more complicated, we are doing something wrong.  The whole
point of going zeroconf is to make configuration simple.

>> > i also don't know what you mean by "transient, task specific services".
>> > i can only think of things like ramfs or cdfs.  but they live in my
>> > namespace so ndb doesn't enter into the picture.
>> >
>>
>> There is the relatively mundane configuration examples of publishing
>> multiple file servers, authentication servers, and cpu servers.
>
> how many file servers and authentication servers are you running?
>

>From a file server perspective on blue gene, a full scale rack will
have thousands.  we're currently operating without auth (in part due
to configuration issues), so I don't know how well it will scale.  The
other aspect here is that in current configurations, every "run" has a
different machine configuration based on what you request from the job
scheduler and what you actually get.  We pretty much get different IP
addresses every time, with different front ends, different file
servers, etc. etc.
Again though - the idea is to use file systems more pervasively within
the applications as well -- so there may be multiple file servers per
node providing different services depending on workload needs at the
particular point of computation.  Read our MTAGS paper from last
year's supercomputing conference to get a bigger picture view on how
we view services coming, going, migrating, and adapting to changing
application usage and failure.

      -eric

Re: [9fans] Interested in improving networking in Plan 9

[Francisco J Ballesteros]

Hmmm. we did that for FS processes on Plan B. I mean, keep a
dynamic version of a registry. It kept the list of volumes available at a
central place.

I think it can be used as is on Plan 9, without changes.

There was a program (I think it was called adsrv; not sure, it´s on the
Plan B man pages) were file servers could keep an open file as long as
they were alive.

We didn´t do load balancing but it shouldn´t be hard to add that to
this program.

If there´s interest I can dig in our worm (although it should be also
on sources).

On Mon, Aug 31, 2009 at 4:25 PM, Eric Van Hensbergen<ericvh@gmail.com> wrote:
> On Mon, Aug 31, 2009 at 9:04 AM, erik quanstrom<quanstro@quanstro.net> wrote:
>>
>> given the database= option, if one could confine rapid changes to
>> smaller files, one could teach ndb to only reread changed files.
>>
>
> Why not have a synthetic file system interface to ndb that allows it
> to update its own files?  I think this is my primary problem.
> Granular modification to static files is a PITA to manage -- we should
> be using synthetic file system interfaces to to help manage and gate
> modifications.  Most of the services I have in mind may be transient
> and task specific, so there are elements of scope to consider and you
> may not want to write anything out to static storage.
>
>>> registration/discovery mechanism to existing applications.  When I
>>> export, a flag should make that export visible to zeroconf resolution,
>>> etc.
>>
>> what do you mean by export?
>>
>
> When I publish a service, in the Plan 9 case primarily by exporting a
> synthetic file system.  I shouldn't have to have static configuration
> for file servers, it should be much more fluid.  I'm not arguing for a
> microsoft style registry -- but the network discovery environment on
> MacOSX is much nicer than what we have today within Plan 9.  An even
> better example is the environment on the OLPC, where many of the
> applications are implicitly networked and share resources based on
> Zeroconf pub/sub interfaces.
>
>       -eric
>
>

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 9:55 AM, Francisco J Ballesteros<nemo@lsub.org> wrote:
> Hmmm. we did that for FS processes on Plan B. I mean, keep a
> dynamic version of a registry. It kept the list of volumes available at a
> central place.
>
> I think it can be used as is on Plan 9, without changes.
>
> There was a program (I think it was called adsrv; not sure, it´s on the
> Plan B man pages) were file servers could keep an open file as long as
> they were alive.
>
> We didn´t do load balancing but it shouldn´t be hard to add that to
> this program.
>
> If there´s interest I can dig in our worm (although it should be also
> on sources).
>

While that sounds interesting and may be useful in its own right, a
centralized server isn't really desirable -- part of the nice thing of
zeroconf is moving to a decentralized environment, and ideally doing
it in a scalable fashion (which isn't trivial on hundreds of thousands
of cores, we certainly don't want unrestricted multicast in such an
environment lest we drown in our own vomit of multicast queries and
responses).  Inferno also has a dynamic registry service available as
another example implementation.  However -- I think embracing some
internet standards wouldn't be a bad thing -- DNS is certainly an
existing example of an external protocol we support even though we
could have invented our own.  Extending the DNS support to mDNS and
DNS-SD shouldn't be that big of a deal -- and most of the hard work is
in defining the Plan 9 interface not actually writing the protocol
support.  It would allow us to play nice with other systems, which may
be very beneficial in xcpu environments (which also currently suffers
from a static configuration), and in particular on our Blue Gene work
where front-end nodes are typically Linux or MacOSX workstations.

         -eric

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> While that sounds interesting and may be useful in its own right, a
> centralized server isn't really desirable -- part of the nice thing of
> zeroconf is moving to a decentralized environment, and ideally doing
> it in a scalable fashion (which isn't trivial on hundreds of thousands
> of cores, we certainly don't want unrestricted multicast in such an
> environment lest we drown in our own vomit of multicast queries and
> responses).  Inferno also has a dynamic registry service available as
> another example implementation.  However -- I think embracing some
> internet standards wouldn't be a bad thing -- DNS is certainly an
> existing example of an external protocol we support even though we
> could have invented our own.  Extending the DNS support to mDNS and
> DNS-SD shouldn't be that big of a deal -- and most of the hard work is
> in defining the Plan 9 interface not actually writing the protocol
> support.  It would allow us to play nice with other systems, which may
> be very beneficial in xcpu environments (which also currently suffers
> from a static configuration), and in particular on our Blue Gene work
> where front-end nodes are typically Linux or MacOSX workstations.

dns is a bad protocol to build on.  it's not really secure
and the security measures are onerous.

not to be confused with a technical argument for or
against building on top of dns, but it's worth noting
that ndb/dns has been pushed just about as far as it
can go.  it tends to suffer from cache funnies and
can leak memory.

i'm hoping that i can get a chance to build a thread
library based dns server in the near future.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Bakul Shah]

On Mon, 31 Aug 2009 09:25:36 CDT Eric Van Hensbergen <ericvh@gmail.com>  wrote:
>
> Why not have a synthetic file system interface to ndb that allows it
> to update its own files?  I think this is my primary problem.
> Granular modification to static files is a PITA to manage -- we should
> be using synthetic file system interfaces to to help manage and gate
> modifications.  Most of the services I have in mind may be transient
> and task specific, so there are elements of scope to consider and you
> may not want to write anything out to static storage.

ndb maps directly to a list of lisp's association lists but
how would you map this to a synthetic fs? Something like
<attr>/<value> to yield a tuple? For example:

% cat ndb/ip/198.41.0.4 # same as ndbquery ip 198.41.0.4
dom=A.ROOT-SERVERS.NET ip=198.41.0.4
% cat ndb/dom/A.ROOT-SERVERS.NET
dom=A.ROOT-SERVERS.NET ip=198.41.0.4

But this is nasty!
% cat ndb/dom/'' # same as ndbquery dom ''
dom= ns=A.ROOT-SERVERS.NET ns=B.ROOT-SERVERS.NET ns=C.ROOT-SERVERS.NET ns=D.ROOT-SERVERS.NET ns=E.ROOT-SERVERS.NET ns=F.ROOT-SERVERS.NET ns=G.ROOT-SERVERS.NET ns=H.ROOT-SERVERS.NET ns=I.ROOT-SERVERS.NET ns=J.ROOT-SERVERS.NET ns=K.ROOT-SERVERS.NET ns=L.ROOT-SERVERS.NET ns=M.ROOT-SERVERS.NET

And it is not clear how you would map
% ndbquery attr value rattr ...

Another alternative is to map each tuple to a directory:
% ls ndb/dom/A.ROOT-SERVERS.NET # just show the attributes!
dom ip

% grep '' ndb/dom/A.ROOT-SERVERS.NET/*
dom:A.ROOT-SERVERS.NET
ip:198.41.0.4

An intriguing idea that can point toward a synth fs interface
to a dbms or search results....  But I don't think this would
be a lightweight interface.

Re: [9fans] Interested in improving networking in Plan 9

[Eric Van Hensbergen]

On Mon, Aug 31, 2009 at 11:16 AM, Bakul Shah<bakul+plan9@bitblocks.com> wrote:
>
> ndb maps directly to a list of lisp's association lists but
> how would you map this to a synthetic fs? Something like
> <attr>/<value> to yield a tuple? For example:
>

My current intuition in these situations is to allow for both
interfaces -- top level aggregated queries as well as granular
hierarchical queries.  Both make sense for different reasons.

>
> An intriguing idea that can point toward a synth fs interface
> to a dbms or search results....  But I don't think this would
> be a lightweight interface.
>

The fact that its not immediately clear is what makes it a good
research topic.  It will likely take several iterations to identify a
"best fit" interface with the likely result that multiple
interfaces/views are required.  I think there are precious little
information on synthetic file system design, given its continuing
popularity in the non-Plan-9 world, we could use more published
wisdom/experiences from the evolution of various file-system based
interfaces.

          -eric

Re: [9fans] Interested in improving networking in Plan 9

[Bakul Shah]

On Mon, 31 Aug 2009 11:33:13 CDT Eric Van Hensbergen <ericvh@gmail.com>  wrote:
> On Mon, Aug 31, 2009 at 11:16 AM, Bakul Shah<bakul+plan9@bitblocks.com> wrote:
> >
> > An intriguing idea that can point toward a synth fs interface
> > to a dbms or search results.... But I don't think this would
> > be a lightweight interface.
> >
>
> The fact that its not immediately clear is what makes it a good
> research topic.  It will likely take several iterations to identify a
> "best fit" interface with the likely result that multiple
> interfaces/views are required.  I think there are precious little
> information on synthetic file system design, given its continuing
> popularity in the non-Plan-9 world, we could use more published
> wisdom/experiences from the evolution of various file-system based
> interfaces.

Oh I don't know.... Shoehorning a DB interface into a FS
interface doesn't feel right but stranger things have
happened.

Re: [9fans] Interested in improving networking in Plan 9

[Robert Raschke]

[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]

On Wed, Sep 2, 2009 at 5:50 PM, Bakul Shah
<bakul+plan9@bitblocks.com<bakul%2Bplan9@bitblocks.com>
> wrote:

> On Mon, 31 Aug 2009 11:33:13 CDT Eric Van Hensbergen <ericvh@gmail.com>
>  wrote:
> > On Mon, Aug 31, 2009 at 11:16 AM, Bakul Shah<bakul+plan9@bitblocks.com<bakul%2Bplan9@bitblocks.com>>
> wrote:
> > >
> > > An intriguing idea that can point toward a synth fs interface
> > > to a dbms or search results.... But I don't think this would
> > > be a lightweight interface.
> > >
> >
> > The fact that its not immediately clear is what makes it a good
> > research topic.  It will likely take several iterations to identify a
> > "best fit" interface with the likely result that multiple
> > interfaces/views are required.  I think there are precious little
> > information on synthetic file system design, given its continuing
> > popularity in the non-Plan-9 world, we could use more published
> > wisdom/experiences from the evolution of various file-system based
> > interfaces.
>
> Oh I don't know.... Shoehorning a DB interface into a FS
> interface doesn't feel right but stranger things have
> happened.
>
>
There are some folk about who are working on a distributed spreadsheet (
hypernumbers.com, there used to be a trial system up and lots of documents,
but I'm not sure where that all went, and I can't be bothereed signing up
just now, they're presenting at ICFP CUFP on Friday, so the company/project
should still be properly alive).

Anyway, the idea they've got is that every cell in a spreadsheet has a URL
and you can reference and use cells across a distributed space. Interesting
idea, but I'm not entirely sure what to do with it, being
spreadsheet-challenged myself.

Robby

[-- Attachment #2: Type: text/html, Size: 2277 bytes --]

Re: [9fans] Interested in improving networking in Plan 9

[matt]

>Oh I don't know.... Shoehorning a DB interface into a FS
>interface doesn't feel right but stranger things have
>happened.
>
>
>
I've done it a few ways. echo commit > /n/db/0/ctl is kind of where one
ends up

for my limbo postgres module I never got round to the fs part. i just
wrap the sql bits in their own adt

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> I've done it a few ways. echo commit > /n/db/0/ctl is kind of where one
> ends up
>
> for my limbo postgres module I never got round to the fs part. i just
> wrap the sql bits in their own adt

i would think that rather than use an adt, one would want
to make the language the communication's protocol.  it
seems so straightforward to just send formatted sql or
pl/sql to the engine and get normally formatted output.
one could just con to the engine and type away for testing.

even if you've degenerated the fs to a pipe like /net/tcp/nn,
it's still quite useful to see who is connected and you can import
the connection easily.

this isn't too far from what sql*net does, except they need
to deal with sockets.  they may also do wierd formatting of
the data stream.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Anthony Sorace]

i've not used matt's sql module itself (i should check it out) so i
can't comment on his implementation, but... SQL is really ugly. it's
not hard to construct something that provides the same functionality
in a much more palatable form. aesthetics aside, if you're dealing
with a database-heavy app, it can make the code much easier to read.

somewhat harder is creating such a module in a way that's not specific
to a specific database, or even sql. this is similar (in theory) to
what the various php database modules do. this can be important if
your application is delivered/deployed in an environment where you
don't have control over what database is being used (although if the
database is important for your app, i think not having control over
the database is a poor position to be in).

not any harder but a lot less general is the option of wrapping the
database around a higher-level abstraction. you can do things like
data constraints and validations in the application code, rather than
in the sql database itself, which always feels like this random
bolt-on to the application logic. in the last large database-centered
project i was involved in, owing in large part to the lead
programmer's linguistics background, we called this the ontology
layer: it defined the things that exist. the rest of the app only had
to deal with what was in there, and it took care of mapping things to
a backing store.

at the layer of ontology, a fs-like interface can work pretty well.
i'm less convinced it's worth doing much with a lower-level database
beyond a slightly smart data channel.

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> [...] but... SQL is really ugly. it's
> not hard to construct something that provides the same functionality
> in a much more palatable form. aesthetics aside, if you're dealing
> with a database-heavy app, it can make the code much easier to read.

could you explain what in particular is objectionable about
sql?  i wrote about 100k lines of sql and pl/sql for a 1tb database
in the late ninties.  (doesn't sound very big anymore.)  i found
it easy to develop, read and maintain.  it almost felt like cheating.

there was about 100k lines of application / web stuff that went
along with it.

> not any harder but a lot less general is the option of wrapping the
> database around a higher-level abstraction. you can do things like
> data constraints and validations in the application code, rather than
> in the sql database itself, which always feels like this random
> bolt-on to the application logic.

of course this is a difficult roe to hoe.  the database can change
table.name from varchar(10) to varchar(32) without consulting
the application code, or vice versa.  seperating the constraining
fact from the constraint seems like a fundamental error to me.

alternatively, one can just as easily use stored procedures to validate input.
then the validation can work against the live constraints.  checks
like "the document.id must be a valid document.id" can be done
race-free in a stored procedure without doing a dance between
the application and db while holding a transaction lock.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Robert Raschke]

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

On Thu, Sep 3, 2009 at 5:13 PM, Anthony Sorace <anothy@gmail.com> wrote:

> i've not used matt's sql module itself (i should check it out) so i
> can't comment on his implementation, but... SQL is really ugly. it's
> not hard to construct something that provides the same functionality
> in a much more palatable form. aesthetics aside, if you're dealing
> with a database-heavy app, it can make the code much easier to read.
>
>
SQL in itself is actually pretty elegant for what it is intended, querying
relational tables.

The problem with databases is more one of really bad design. A badly
designed database makes SQL use awkward. Using a more general programming
language allows you to create abstractions that hide the hideous database.
It ends up as a veneer that usually grows and grows and grows trying to
"fix" problems that actually exist at the database level itself.

The fact that using SQL against your database is a pain, should really
signal alarm bells and make you investigate what may be wrong with its
design.

Robby

[-- Attachment #2: Type: text/html, Size: 1371 bytes --]

Re: [9fans] Interested in improving networking in Plan 9

[Daniel Lyons]

On Sep 3, 2009, at 10:13 AM, Anthony Sorace wrote:

> you can do things like
> data constraints and validations in the application code, rather than
> in the sql database itself, which always feels like this random
> bolt-on to the application logic.


I think it's useful to think of relational databases as integration  
technology rather than merely storage technology. Having everything  
important in the application layer means you have to go through the  
application layer to do anything. Part of the point of SQL and RDBMSes  
being so complex is that it liberates the data from any one particular  
application that wants to access the data. This fact is often  
forgotten these days because most databases are only accessed through  
the web via one app. But I usually find that eventually one needs to  
get at the data directly, for reporting, maintenance or some other  
task that isn't intrinsically part of the website. Then the database  
can take on the data management role and make sure you don't make your  
data incomprehensible, as well as provide you with a little security.

Of course, it's best of all if you have a database that supports  
things like views; otherwise you will need application code to have  
any abstractions at all.

—
Daniel Lyons

Re: [9fans] Interested in improving networking in Plan 9

[Steve Simon]

> it
> seems so straightforward to just send formatted sql or
> pl/sql to the engine and get normally formatted output.

I did somthing like this for mysql to access our
corperate telephone database.

I took the inferno odbcfs as an example:

	http://www.vitanuova.com/inferno/man/10/odbc.html

if interested see the steve/mysqlfs contrib package

-Steve

Re: [9fans] Interested in improving networking in Plan 9

[Devon H. O'Dell]

2009/8/31 Bakul Shah <bakul+plan9@bitblocks.com>:
> But this is nasty!
> % cat ndb/dom/'' # same as ndbquery dom ''

No, the nasty part is really that the file should be called `.' and
the filesystem reserves dot as the reference to the current directory.
You could probably call the file `dot' or `root' (cat ndb/dom/dot or
cat ndb/dom/root) as something that shouldn't ever conflict with
anything else -- but the root of DNS is not an empty string.

--dho

> dom= ns=A.ROOT-SERVERS.NET ns=B.ROOT-SERVERS.NET ns=C.ROOT-SERVERS.NET ns=D.ROOT-SERVERS.NET ns=E.ROOT-SERVERS.NET ns=F.ROOT-SERVERS.NET ns=G.ROOT-SERVERS.NET ns=H.ROOT-SERVERS.NET ns=I.ROOT-SERVERS.NET ns=J.ROOT-SERVERS.NET ns=K.ROOT-SERVERS.NET ns=L.ROOT-SERVERS.NET ns=M.ROOT-SERVERS.NET
>
> And it is not clear how you would map
> % ndbquery attr value rattr ...
>
> Another alternative is to map each tuple to a directory:
> % ls ndb/dom/A.ROOT-SERVERS.NET # just show the attributes!
> dom ip
>
> % grep '' ndb/dom/A.ROOT-SERVERS.NET/*
> dom:A.ROOT-SERVERS.NET
> ip:198.41.0.4
>
> An intriguing idea that can point toward a synth fs interface
> to a dbms or search results....  But I don't think this would
> be a lightweight interface.
>
>

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> 2009/8/31 Bakul Shah <bakul+plan9@bitblocks.com>:
> > But this is nasty!
> > % cat ndb/dom/'' # same as ndbquery dom ''
>
> No, the nasty part is really that the file should be called `.' and
> the filesystem reserves dot as the reference to the current directory.
> You could probably call the file `dot' or `root' (cat ndb/dom/dot or
> cat ndb/dom/root) as something that shouldn't ever conflict with
> anything else -- but the root of DNS is not an empty string.

aren't you being a little bit pedantic?  quoting is a fact
of life.  we don't say that it's evil to need to quote or
transform things in rc or smtp to deal with local requirements.
why would it be evil to quote '.'?  and why would be calling
'.' 'root' or 'dot' rather than '' be any less evil?

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Devon H. O'Dell]

2009/8/31 erik quanstrom <quanstro@coraid.com>:
>> 2009/8/31 Bakul Shah <bakul+plan9@bitblocks.com>:
>> > But this is nasty!
>> > % cat ndb/dom/'' # same as ndbquery dom ''
>>
>> No, the nasty part is really that the file should be called `.' and
>> the filesystem reserves dot as the reference to the current directory.
>> You could probably call the file `dot' or `root' (cat ndb/dom/dot or
>> cat ndb/dom/root) as something that shouldn't ever conflict with
>> anything else -- but the root of DNS is not an empty string.
>
> aren't you being a little bit pedantic?  quoting is a fact
> of life.  we don't say that it's evil to need to quote or
> transform things in rc or smtp to deal with local requirements.
> why would it be evil to quote '.'?  and why would be calling
> '.' 'root' or 'dot' rather than '' be any less evil?

It's (in my opinion) slightly less evil because if(!strlen(name))
seems like a pretty poor way to determine that you're looking at the
root zone. It's also more intuitive and easier to document that you're
looking at the root than saying `to find root, look for a file named
as an empty string'. So: less evil because it makes code more
intuitive and it makes documentation easier.

Also, I think I should state that I really don't care about
implementation as long as it's documented, and that the root is not an
empty string. I don't particularly care about quoting at all, and I
don't consider that any reason for one method to be more obtuse than
another.

--dho

> - erik
>
>

Re: [9fans] Interested in improving networking in Plan 9

[erik quanstrom]

> It's (in my opinion) slightly less evil because if(!strlen(name))
> seems like a pretty poor way to determine that you're looking at the
> root zone. It's also more intuitive and easier to document that you're
> looking at the root than saying `to find root, look for a file named
> as an empty string'. So: less evil because it makes code more
> intuitive and it makes documentation easier.

having the root be '' makes sense to me.  having it
be otherwise, say "dot" would mean that one would
properly request example.net.dot not example.net
and implies that there are other roots than dot.

- erik

Re: [9fans] Interested in improving networking in Plan 9

[Federico G. Benavento]

Reposting this to 9fans:

hola,

First of all, I'm really glad you are considering Plan 9 for your project,
thanks.

>  Can someone please discuss with me how to proceed, and what are the things
> I should learn before starting on this ?
>

you can start by reading nemo's intro "Introduction to Operating
Systems Abstractions Using Plan 9 from Bell Labs" which covers pretty
much
everything you need to get started plus some advanced topics like
how to implement a synthetic fileserver and others.

http://lsub.org/who/nemo/9.intro.pdf

if you need any specific help, feel free to get in touch with me.


--
Federico G. Benavento

Re: [9fans] Interested in improving networking in Plan 9

[Vinu Rajashekhar]

On Mon, Aug 31, 2009 at 9:47 AM, Federico G.
Benavento<benavento@gmail.com> wrote:
> Reposting this to 9fans:
>
> hola,
>
> First of all, I'm really glad you are considering Plan 9 for your project,
> thanks.
>
>>  Can someone please discuss with me how to proceed, and what are the things
>> I should learn before starting on this ?
>>
>
> you can start by reading nemo's intro "Introduction to Operating
> Systems Abstractions Using Plan 9 from Bell Labs" which covers pretty
> much
> everything you need to get started plus some advanced topics like
> how to implement a synthetic fileserver and others.
>
> http://lsub.org/who/nemo/9.intro.pdf
>

 Thank you, I am going through "Plan 9 from Bell Labs" and
"The Organization of Networks in Plan 9" right now.

> if you need any specific help, feel free to get in touch with me.
>
>
  Will do !

> --
> Federico G. Benavento
>
>