Re: [9fans] Do we have a catalog of 9P servers?

[Eris Discordia <eris.discordia@gmail.com>]

Thanks a lot, Nathaniel Filardo. Your clean and detailed explanation is
very much appreciated. Although, Micah Stetson did post a similar, albeit
far more concise, explanation before.

Despite the impression I seem to have made, I understand--as of a few days
ago, at least--why a Plan 9 gateway can be transparent and hassle-free. I
even pointed out on a previous post that comparable forms of transparency
are provided at different layers of the network by tunnelling protocols
such as L2TP and PPTP. I also explained why there's an oxymoronic quality
to the transparency provided in Plan 9, i.e. that network layer operations
are delegated to application layer. Many network devices, if implemented as
they ought to be, should operate at specific layers of the stack--a
full-blown network host should in principle be different from a router
which in turn should be different from a bridge. An application layer is a
luxury not always granted. Also, the delegation of a network layer
operation to higher layers has an undertone of blasphemy, of violating the
principles of layered design.

Anyhow, the issue is resolved. I believe we understand each other.

--------------------------------------------------------------------------------

Regarding two specific claims:

> ... you get the idea.  It does this with no special tools, no complex
> code, and a very minimal per-connection overhead (just the IC and G
> kernels and G's exportfs tracking a file descriptor).

I haven't seen--and even if I did I wouldn't understand--the source code
required for:

a. Plan 9's abstraction of a network into a filesystem or part of a
filesystem,
b. a UNIX-clone NAT implementation,

so I really can't comment on, or offhandedly accept the comments on, the
complexity of these codes.

On the other hand, the one unresolved issue on this thread was a factual
comparison of NAT overhead compared to the transport+application layer
overheads generated on a Plan 9 gateway. I remain rather convinced that NAT
overhead will be meaningfully smaller _if_ the implementation is sane. Why?
Because the problem solved is the same and as a rule of thumb the effort
required to solve a problem is distributed between the computer and the
programmer's mind. If the  programmer's mind does more effort to solve the
problem then the computer will be relieved of some of the effort required
to carry out the solution (I'm not taking into account "quantum leaps," by
the way, I doubt they actually exist). Apparently, NAT takes more
brainpower to implement. The other, probably more agreeable, reason for
NAT's lesser overhead is that it operates (mostly) at network layer while
things that go through a hypothetical Plan 9 gateway necessarily involve
transport and application layer interaction. Abstraction costs,
doubtlessly, but I am willing to concede that the cost may well prove
tolerable in view of the hassle of creating and deploying a more "concrete"
solution.

> There can, in fact, be multiple IP stacks on a Plan 9 box, bound to
> multiple Ethernet cards, as you claim.  (In fact, one can import another
> computer's ethernet cards and run snoopy, or build a network stack, using
> those instead.)  I don't think that's relevant to the example at hand,
> though, as should be clear from the above.

This was pointed out on another thread by Ron Minnich. I actually was
surprised by the fact that *BSDs are (normally) limited to one instance of
network stack. And I later used this same difference to justify the claim
that each /net imported from a Plan 9 gateway will necessarily require a
new copy of the gateway's protocol stack. This was rebuked and your
explanation makes it even more clear why.

Nevertheless, the same machinations that allow for transparency in Plan 9
disallow certain functions that can be naturally provided by a NAT
implementation, or any of a number of software categories that involve
packet filtering/rewriting/inspection. For example, the one I referred to
in the posting you have quoted in your response: load balancing. Add to the
list: rate control, intrusion detection, QoS earmarking, honeynetting, et
cetera ad [put you favorite -um, -am here].

--------------------------------------------------------------------------------

--On Tuesday, November 18, 2008 8:59 PM -0500 Nathaniel W Filardo
<nwf@cs.jhu.edu> wrote:

> On Sun, Nov 16, 2008 at 09:24:19PM +0000, Eris Discordia wrote:
>>> That isn't happening.  All we have is one TCP connection and one small
>>> program exporting file service.
>>
>> I see. But then, is it the "small program exporting file service" that
>> does the multiplexing? I mean, if two machines import a gateway's /net
>> and both run HTTP servers binding to and listening on *:80 what takes
>> care of which packet belongs to which HTTP server?
>
> I don't think you've quite got it yet.... also I swore I wouldn't post in
> this thread.  Oh well, here goes.
>
> First, let me draw a picture, just to introduce the characters:
>
> +----------+              +---------+
>| Internal |<--Ethernet-->| Gateway |<--Ethernet-->(Internet)
>| Computer |   ^          +---------+
> +----------+   |
>                |
> +----------+   |
>|  Other   |<--+
>| Internal |
>| Computer |
> +----------+
>
> OK.  Here, we have two internal computers (IC and OIC) and a gateway G.
> There are two Ethernet networks in flight, one connecting IC, OIC, and G,
> and the other connecting G to the internet at large, somehow (e.g. ADSL).
>
> IC and OIC both initialize somehow, say using DHCP from G, and bring their
> network stacks up using this information.  Their kernels now provide the
> services that will be mounted, by convention, at /net, again using this
> private information.  G initializes statically, bringing its IP stack up
> using two interfaces, with two routes: one for internal traffic, and one
> default route.
>
> So far, it's all very similar between Plan 9 and Linux.  Here's where
> our story diverges.
>
> In Linux, IC and OIC are both taught that they have default routes to the
> Internet At Large by sending IP datagrams to G's internal ethernet
> interface.  How this works in more detail, they don't care.  G will also
> send them corresponding IP datagrams from its interface; that's all they
> care about.  G works furiously to decode the network traffic bound for the
> internet and NA(P)T it out to its gateway, maintaining very detailed
> tables about TCP connections, UDP datagrams, etc, etc.  How could it be
> but any other way?
>
> Let's redraw the picture, as things stand now, under Plan 9, just so
> things are clear:
>
>                                                   (OIC similar to IC)
> +----Internal Computer--------------------+               |
>| bind '#I' /net                          |               |
>| # I : Kernel IP stack (192.168.1.2)      |               |
>| # l : Kernel ethernet driver             |<---Ethernet---+
> +-----------------------------------------+               |
>                                                           |
> +----Gateway------------------------------+               |
>| bind '#I' /net                          |               |
>| # I : Kernel IP stack (192.168.1.1)      |               |
>|                      (4.2.2.2)          |               |
>| # l : Kernel ethernet driver  (ether0)   |<---Ethernet---+
>|                              (ether1)   |<---Ethernet----->(Internet)
> +-----------------------------------------+
>
> In Plan 9, G behaves like any other machine, building a /net out of pieces
> exported by its kernel, including the bits that know how to reach the
> internet at large through the appropriate interface.  Good so far?
>
> Let's have G run an exportfs, exposing its /net on the internal IP
> address. This /net knows how to talk to the internal addresses and the
> external ones.
>
> Meanwhile, IC can reach out and import G's /net, binding it at /net.alt,
> let's say.  Now, programs can talk to the Internet by opening files in
> /net.alt.  These open requests will be carried by IC's mount driver, and
> then IC's network stack, to G, whereupon the exportfs (in G's userland)
> will forward them to its idea of /net (by open()ing, read()ing,
> write()ing, etc.), which is the one built on G's kernel, which knows how
> to reach the Internet.  Tada!  Picture time:
>
>                                                         (OIC)
> +----Internal Computer-------------------------+          |
>| abaco: open /net.alt/tcp/clone               |          |
>|                                              |          |
>| import tcp!192.168.1.1!9fs /net.alt (devmnt) |          |
>| bind '#I' /net                               |          |
>| # I : Kernel IP stack (192.168.1.2)           |          |
>| # l : Kernel ethernet driver                  |<---------+
> +----------------------------------------------+          |
>                                                           |
> +----Gateway------------------------------+               |
>| exportfs -a -r /net                     |               |
>|                                         |               |
>| bind '#I' /net                          |               |
>| # I : Kernel IP stack (192.168.1.1)      |               |
>|                      (4.2.2.2)          |               |
>| # l : Kernel ethernet driver  (ether0)   |<---Ethernet---+
>|                              (ether1)   |<---Ethernet----->(Internet)
> +-----------------------------------------+
>
> This works perfectly for making connections: IC's IP stack is aware only
> of devmnt requests, and G's IP stack is aware of some trafic to&from a
> normal process called exportfs, and that that process happens to be
> making network requests via #I bound at /net.
>
> The beauty of this design is just how well it works, everywhere, for
> everything you'd ever want.
>
> Now, suppose IC goes to listen on TCP:80, by opening /net.alt/tcp/clone.
> The same flow of events happen, and to a certain extent, G's network stack
> thinks that the exportfs program (running on G) is listening on TCP:80.
> exportfs dutifully copies the /net data back to its client.
>
> Naturally, if another program on G were already listening on TCP:80, or
> the same program (possibly exportfs) attempted to listen twice (if, say,
> OIC played the same game and also tried to listen on G's TCP:80), it
> would be told that the port was busy.  This error would be carried back
> along the exportfs path just as any other.
>
> So as you see, there is no need to take care of "which packet belongs to
> which server" since there can, of course, be only one server listening on
> TCP:80.  That server is running on G, and behaves like any other program.
> That it just so happens to be an exportfs program, relaying data to and
> from another computer, is immaterial.
>
> This also works for FTP, and UDP, and ESP (which are notorious problems in
> the NAT world), and for IL, and for IPv6, and for ethernet frames (!), and
> ... you get the idea.  It does this with no special tools, no complex
> code, and a very minimal per-connection overhead (just the IC and G
> kernels and G's exportfs tracking a file descriptor).
>
> There are no connection tracking tables anywhere in this design.  There
> are just normal IP requests over normal ethernet frames, and a few more
> TCP (or IL) connections transporting 9P data.
>
>> On a UNIX clone, or on Windows, because there is exactly one TCP/IP
>> protocol stack in usual setups no two programs can bind to the same port
>> at the same time. I thought Plan 9's approach eliminated that by keeping
>> a distinct instance of the stack for each imported /net.
>
> There can, in fact, be multiple IP stacks on a Plan 9 box, bound to
> multiple Ethernet cards, as you claim.  (In fact, one can import another
> computer's ethernet cards and run snoopy, or build a network stack, using
> those instead.)  I don't think that's relevant to the example at hand,
> though, as should be clear from the above.
>
> --nwf;