[9fans] channels across machines

[9fans] channels across machines

[Bakul Shah]

Has anyone extended the idea of channels where the
sender/receiver are on different machines (or at least in
different processes)?  A netcat equivalent for channels!

Actual plumbing seems easy: one can add a `proxy' thread in
each process to send a message via whatever inter process
mechanism is available. One issue would be for the two sides
to identify a specific channel.  I imagine something like the
following would work.

	// on the client:
	chan = chanconnect("<host>:<port>", elemsize, nelem);

	// on the server:
	x = chanbind("<port>", elemsize, nelem);
	chan = chanaccept(x); // to allow multiple connections

Or one can build this on top of a socket or file descriptor.

Another issue is endianness (unless all processors are the
same type). Yet another issue is sending variable size
things. In a single address space you can pass a pointer +
may be a size but that trick doesn't work across process (or
machine boundaries) -- so pretty much have to add some
marshalling/unmarshalling that knows about types.

Or is there a better idea?  This certainly seems preferable
to RPC or plain byte pipes for communicating structured
values.

Thanks!

--bakul

Re: [9fans] channels across machines

[erik quanstrom]

On Sat Jul 18 03:46:01 EDT 2009, bakul+plan9@bitblocks.com wrote:
> Has anyone extended the idea of channels where the
> sender/receiver are on different machines (or at least in
> different processes)?  A netcat equivalent for channels!

i think the general idea is that if you want to do this between
arbitrary machines, you provide a 9p interface.  you can think
of 9p as a channel with a predefined set of messages.  acme
does this.  kernel devices do this.

however inferno provides file2chan
http://www.vitanuova.com/inferno/man/2/sys-file2chan.html.
of course, somebody has to provide the 9p interface, even
if that's just posting a fd to /srv.

if you wanted to do something like file2chan in plan 9 and c, you're
going to have to marshal your data.  this means that chanconnect
as specified is impossible.  (acme avoids this by using text only.
the kernel devices keep this under control by using a defined
byte order and very few binary interfaces.)  i don't know how you would
solve the naming problems e.g. acme would have using dial-string
addressing  (i assume that's what you ment; host-port addressing is
ip specific.).  what would be the port-numbering huristic for allowing
multiple acmes on the same cpu server?

after whittling away problem cases, i think one is left with pipes,
and it seems pretty clear how to connect things so that
chan <-> pipe <-> chan.  one could generalize to multiple
machines by using tools like cpu(1).

- erik

Re: [9fans] channels across machines

[Mechiel Lukkien]

On Sat, Jul 18, 2009 at 06:25:19AM -0400, erik quanstrom wrote:
> i think the general idea is that if you want to do this between
> arbitrary machines, you provide a 9p interface.  you can think
> of 9p as a channel with a predefined set of messages.  acme
> does this.  kernel devices do this.
>
> however inferno provides file2chan
> http://www.vitanuova.com/inferno/man/2/sys-file2chan.html.
> of course, somebody has to provide the 9p interface, even
> if that's just posting a fd to /srv.
>
> if you wanted to do something like file2chan in plan 9 and c, you're
> going to have to marshal your data.  this means that chanconnect
> as specified is impossible.
[...]
>
> after whittling away problem cases, i think one is left with pipes,
> and it seems pretty clear how to connect things so that
> chan <-> pipe <-> chan.  one could generalize to multiple
> machines by using tools like cpu(1).

inferno's file2chan is local too, just giving a simple interface to
handling plain reads & writes on a file.  unless i've been using it
wrong.

what i like about file2chan is that you can return your own error
strings.  as far is i know, that's not possible with pipes.  if it is
possible, i'm very interested to learn how it's done.

mjl

Re: [9fans] channels across machines

[erik quanstrom]

> inferno's file2chan is local too, just giving a simple interface to
> handling plain reads & writes on a file.  unless i've been using it
> wrong.

i assume that import and srv could be used to export a fd?

- erik

Re: [9fans] channels across machines

[Skip Tavakkolian]

> Or is there a better idea?  This certainly seems preferable
> to RPC or plain byte pipes for communicating structured
> values.

i have some incomplete ideas that are tangentially related to this --
more for handling interfaces.

it seems one could write a compiler that translates an interface
definition (e.g.  IDL) into a server and a client library; 9p(2) and
ioproc(2) can be readily used in the generated code to do the tricky
stuff.  the main part then becomes how to translate the calls across.

conventionally the server for an interface with an ID of X (e.g.
GUID) is posted to /srv/X.

for a given method bar in interface foo, writing to /n/foo/bar (
/srv/X mounted on /n/foo) would invoke foo:bar wherever it is.
obviously when the reader/writer are generated, the types and sizes of
parameters are known.

-Skip

Re: [9fans] channels across machines

[J.R. Mauro]

On Sat, Jul 18, 2009 at 1:20 PM, Skip Tavakkolian<9nut@9netics.com> wrote:
>> Or is there a better idea?  This certainly seems preferable
>> to RPC or plain byte pipes for communicating structured
>> values.
>
> i have some incomplete ideas that are tangentially related to this --
> more for handling interfaces.
>
> it seems one could write a compiler that translates an interface
> definition (e.g.  IDL) into a server and a client library; 9p(2) and
> ioproc(2) can be readily used in the generated code to do the tricky
> stuff.  the main part then becomes how to translate the calls across.
>
> conventionally the server for an interface with an ID of X (e.g.
> GUID) is posted to /srv/X.
>
> for a given method bar in interface foo, writing to /n/foo/bar (
> /srv/X mounted on /n/foo) would invoke foo:bar wherever it is.
> obviously when the reader/writer are generated, the types and sizes of
> parameters are known.

Reminds me of the http interface to venti, which has things like
/set/variable and /plot/variable

Re: [9fans] channels across machines

[Bakul Shah]

On Sat, 18 Jul 2009 06:25:19 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> On Sat Jul 18 03:46:01 EDT 2009, bakul+plan9@bitblocks.com wrote:
> > Has anyone extended the idea of channels where the
> > sender/receiver are on different machines (or at least in
> > different processes)?  A netcat equivalent for channels!
>
> i think the general idea is that if you want to do this between
> arbitrary machines, you provide a 9p interface.  you can think
> of 9p as a channel with a predefined set of messages.  acme
> does this.  kernel devices do this.
>
> however inferno provides file2chan
> http://www.vitanuova.com/inferno/man/2/sys-file2chan.html.
> of course, somebody has to provide the 9p interface, even
> if that's just posting a fd to /srv.
>
> if you wanted to do something like file2chan in plan 9 and c, you're
> going to have to marshal your data.  this means that chanconnect
> as specified is impossible.  (acme avoids this by using text only.
> the kernel devices keep this under control by using a defined
> byte order and very few binary interfaces.)  i don't know how you would
> solve the naming problems e.g. acme would have using dial-string
> addressing  (i assume that's what you ment; host-port addressing is
> ip specific.).  what would be the port-numbering huristic for allowing
> multiple acmes on the same cpu server?
>
> after whittling away problem cases, i think one is left with pipes,
> and it seems pretty clear how to connect things so that
> chan <-> pipe <-> chan.  one could generalize to multiple
> machines by using tools like cpu(1).

First of all, thanks for all the responses!

I should've mentioned this won't run on top of plan9 (or
Unix).  What I really want is alt!  I brought up RPC but
really, this is just Limbo's "chan of <type>" idea.  In the
concurrent application where I want to use this, it would
factor out a bunch of stuff and simplify communication.

The chanconnect("<host>:<port>") syntax was to get the idea
across.  A real building block might look something like
this:

	int fd = <get a pipe end to something somehow>
	Chan c = fd2chan(fd, <marshalling functions>);

Ideally I'd generate relevant struct definitions and
marshalling functions from the same spec.

-- bakul

Re: [9fans] channels across machines

[Eric Van Hensbergen]

On Sat, Jul 18, 2009 at 1:52 PM, Bakul Shah<bakul+plan9@bitblocks.com> wrote:
>
> I should've mentioned this won't run on top of plan9 (or
> Unix).  What I really want is alt!  I brought up RPC but
> really, this is just Limbo's "chan of <type>" idea.  In the
> concurrent application where I want to use this, it would
> factor out a bunch of stuff and simplify communication.
>
> The chanconnect("<host>:<port>") syntax was to get the idea
> across.  A real building block might look something like
> this:
>
>        int fd = <get a pipe end to something somehow>
>        Chan c = fd2chan(fd, <marshalling functions>);
>

getting a pipe end to something somehow is why you really want to
leverage the namespace as a 9p file system.  <host>:<port> is really a
legacy concept that often locks you into a way of thinking or a
particular transport.  The typed channels (or for that matter file
system interfaces) is an interesting topic, and there have been some
folks thinking about it, but I'm not sure they are ready to share
their thoughts yet.

     -eric

Re: [9fans] channels across machines

[Bakul Shah]

On Sat, 18 Jul 2009 10:20:11 PDT Skip Tavakkolian <9nut@9netics.com>  wrote:
> > Or is there a better idea?  This certainly seems preferable
> > to RPC or plain byte pipes for communicating structured
> > values.
>
> i have some incomplete ideas that are tangentially related to this --
> more for handling interfaces.
>
> it seems one could write a compiler that translates an interface
> definition (e.g.  IDL) into a server and a client library; 9p(2) and
> ioproc(2) can be readily used in the generated code to do the tricky
> stuff.  the main part then becomes how to translate the calls across.

I did something like this (map a file of ascii struct defns
to C++ classes that included serialization/deserialization)
for the only company both of us have worked in!

These days I am likely to just use s-exprs if the choice is mine.
The joy of sexpr :-)

On Sat, 18 Jul 2009 14:32:30 CDT Eric Van Hensbergen <ericvh@gmail.com>  wrote:
> getting a pipe end to something somehow is why you really want to
> leverage the namespace as a 9p file system.

Indeed but it is a separable concern (just like authentication).

Re: [9fans] channels across machines

[maht]

It sounds to me like you have one specific problem you are trying to
generalize by sending ADTs across the wire. Microsoft and IBM "solved"
it with Com & Corba - later picked up by GNOME for the same job -
though, of course, they added RPC into the mix.
XML, S-expr, JSON all try and solve the same problem and add REST / SOAP
/ crap de jour for RPC.

Plan9 chose 9p. Seeing as your endpoints already know the layout of the
struct, they present a directory tree to write into.

You mentioned you wanted ALT, your server can do that for you on Clunk.

Re: [9fans] channels across machines

[erik quanstrom]

> On Sat, Jul 18, 2009 at 4:38 AM, Akshat
> Kumar<akumar@mail.nanosouffle.net> wrote:
> > The idea seems inviting at first, but have you
> > given a thought to using plumber(4) for
> > "interprocess messaging" (which is what you
> > want, from what I understand)? This seems
> > more appropriate for communication amongst
> > processes alien to one another than something
> > so code-level like a chan extension.
>
> Acme does this.

the plumber is pretty far from a channel.  a channel
has one input and one output.  the plumber has
one input and many ouputs and uses a dynamic
ruleset to route messages.

let's not confuse them.

however, the fact that plumber implements a fs
and the import(4) and cpu(1) illustrates how one
could use a pipe to build a inter-machine chan-like
mechanism.  i say chan-like because this ignores
data marshalling.

- erik

Re: [9fans] channels across machines

[J.R. Mauro]

On Sat, Jul 18, 2009 at 4:38 AM, Akshat
Kumar<akumar@mail.nanosouffle.net> wrote:
> The idea seems inviting at first, but have you
> given a thought to using plumber(4) for
> "interprocess messaging" (which is what you
> want, from what I understand)? This seems
> more appropriate for communication amongst
> processes alien to one another than something
> so code-level like a chan extension.

Acme does this.

>
>
> at least, in terms of Plan 9
> ak
>
>

[9fans] channels across machines

[Akshat Kumar]

The idea seems inviting at first, but have you
given a thought to using plumber(4) for
"interprocess messaging" (which is what you
want, from what I understand)? This seems
more appropriate for communication amongst
processes alien to one another than something
so code-level like a chan extension.


at least, in terms of Plan 9
ak