[9fans] Regarding 9p based "protocols" message framing

[Ciprian Dorin Craciun <ciprian.craciun@gmail.com>]

    Hello all!

    I'm reading through the nice `factotum` paper [1], and the section
describing the `rpc` file behaviour (section "2.6. Factotum
transactions") states:

~~~~
Programs authenticate with factotum by writing a request to the rpc
file and reading back the reply; this sequence is called an RPC
transaction. Requests and replies have the same format: a textual verb
possibly followed by arguments, which may be textual or binary. [...]
Once started, an RPC conversation usually consists of a sequence of
read and write transactions.
[...]
PS -> FS: start proto=apop role=server
FS -> PS: ok
~~~~

    Now my puzzle was related to what does actually "sequence of read
and write transactions" mean. To be more clearly:
    * the cited paragraph clearly involves two protocols: one is the
factotum RPC protocol which is layered on-top of the 9p protocol;
    * if we take only the factotum RPC protocol (in abstraction of 9p)
we **might** have the impression that each "request" and "response" is
sent as an individual message whose framing is that of an text line,
and 9p is used merely as a transport; (but this framing is not clearly
expressed, see below...);
    * thus if we take only this factotum RPC protocol into
consideration the implementor might have the impression that the
implementation should roughly be: keep receiving write requests until
we have an entire request (pseudo-C):
~~~~
handle_write (context, data) {
  append (context->buffer, data);
  if (request_ready (context) {
    handle_request (context);
  }
  return (succeed);
~~~~
    * but if we look at how it is implemented [2] it is actually: one
request and reply must map over exactly one 9p read or write
operation.
~~~~
handle_write (context, data) {
  if (parse_request (context, data))
    return (handle_request (context));
  else
    return (failed);
~~~~

    Why was I puzzled: because as a non Plan9 user / developer, I
usually think of the underlaying transport technology (be it sockets
or 9p) as a stream of bytes without explicit framing. Meanwhile I
think that the Plan9 usage of 9p is:
    * sometimes as a bidirectional message pipe (i.e. read == pull
message, write == push message, similar to what ZeroMQ library
offers);
    * sometimes as an stateless RPC (like HTTP): open file, write
request data in maybe multiple write operations, read reply data in
maybe multiple read operations; (like the `ctl` file, as I guess `cat
>ctl` makes one write per read line);
    * sometimes as an un-seekable stream of data (like the `log` file);

    Now I see here advantages and disadvantages... Advantages:
    * the developer doesn't need to bother anymore with framing -- it
is done directly by the FS; (and gosh is this a relief, as every other
protocol invented its own framing: length followed by data, line feed
terminated, empty line terminated, random token ended, etc.)
    * concurrent outstanding requests -- just call write in parallel;
(I've read the protocol and the specification doesn't say its
forbidden, but I've read something a few months ago on this mailing
list which suggested it should not be done???)

    Disadvantages:
    * the tools must be aware of this mapping between 1 request == 1
write operation; thus if for example my request would be composed of
multiple lines, and I use `cat >./some-file` as an "interface"; then
`cat` would read one line and immediately send it as an individual
write operation which of course is not respecting the "protocol" built
ontop of 9p;


    Thus I would like to hear the position of the people on this
mailing list about:
    * the explicitly mapping of one request to one 9p operation;
    * the "recommended" usage "patterns" of 9p when building "control"
protocols ontop;

    Thanks,
    Ciprian.


    [1] Security in Plan 9 -- http://plan9.bell-labs.com/sys/doc/auth.html
    [2] http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/auth/factotum/rpc.c