[9fans] Porting to plan9 / PF_UNIX

[9fans] Porting to plan9 / PF_UNIX

[Lluís Batlle]

I'm the developer of a Unix program that may be useful in plan9, and
as an exercise, I'd like it to build in plan9 native.

Now, I've written it assuming POSIX (int signals, PF_UNIX available,
BSD sockets API, ...). I know more or less some details on plan9 (per
process namespaces, network not in a library to link with, notes,
...), but I barely know the 'common' use of the environment.

My concern is that I need some local processes to communicate to a
local server. Now I use a SOCK_STREAM PF_UNIX in a path in /tmp, where
I suppose I can write. I think it's something I can suppose in a POSIX
system, but not in plan9.
How would you connect local processes (which can be launched at any
time) with a central server in plan9? Making that central server serve
a filesystem, where the clients write/read?
If I knew what's the path I should follow, I could think more easily
on a proper API for POSIX and Plan9 for my program.

I may not have not worked in plan9 enough to deserve your attention,
but I hope a quick answer (even in the form of "man ___") will not
take much time.

btw, the program I'd like to be able to run in plan9 is
http://vicerveza.homeunix.net/~viric/soft/ts/ . I wrote it because I
couldn't find anything similar to fit my needs, and maybe in plan9 you
already have something quick-and-useful for that purpose, and 'ts'
would not make any sense in p9 now. That's also something I'd be glad
to know.

Thanks in advance,
Lluís.

Re: [9fans] Porting to plan9 / PF_UNIX

[W B Hacker]

Lluís Batlle wrote:
> I'm the developer of a Unix program that may be useful in plan9, and
> as an exercise, I'd like it to build in plan9 native.
> 
> Now, I've written it assuming POSIX (int signals, PF_UNIX available,
> BSD sockets API, ...). I know more or less some details on plan9 (per
> process namespaces, network not in a library to link with, notes,
> ...), but I barely know the 'common' use of the environment.
> 
> My concern is that I need some local processes to communicate to a
> local server. Now I use a SOCK_STREAM PF_UNIX in a path in /tmp, where
> I suppose I can write. I think it's something I can suppose in a POSIX
> system, but not in plan9.
> How would you connect local processes (which can be launched at any
> time) with a central server in plan9? Making that central server serve
> a filesystem, where the clients write/read?
> If I knew what's the path I should follow, I could think more easily
> on a proper API for POSIX and Plan9 for my program.
> 
> I may not have not worked in plan9 enough to deserve your attention,
> but I hope a quick answer (even in the form of "man ___") will not
> take much time.
> 
> btw, the program I'd like to be able to run in plan9 is
> http://vicerveza.homeunix.net/~viric/soft/ts/ . I wrote it because I
> couldn't find anything similar to fit my needs, and maybe in plan9 you
> already have something quick-and-useful for that purpose, and 'ts'
> would not make any sense in p9 now. That's also something I'd be glad
> to know.
> 
> Thanks in advance,
> Lluís.
> 

LLuis,

Read the description.  Am confused.

You connect to s server with a 'terminal' (the client?)

The 'client' awaits permission from the server to run the task.

- But the task actually runs on the *client*?

Shall we presume it is the equivalent of a Plan9 'CPU' then? Or?

(i.e. plenty of RAM and CPU-cycles, but short on storage space.)

- using the server for scheduling control & spooling the output?

AFAIK Plan9 can do many "similar" things already.

But I am missing an example or three of what you would actually use this for..
370-155's aside, there may be sound reason why there is nothing similar...

;-)

Bill

Re: [9fans] Porting to plan9 / PF_UNIX

[Lluís Batlle]

2007/4/14, W B Hacker <wbh@conducive.org>:
> LLuis,
>
> Read the description.  Am confused.
>
> You connect to s server with a 'terminal' (the client?)
No. The client is also 'ts', as the server. Two processes from the
same program, doing different tasks. The server maintains a batch
queue of the commands queued by 'ts' clients.
>
> The 'client' awaits permission from the server to run the task.
The server maintains the queue of the tasks to be run (in a simple list).
>
> - But the task actually runs on the *client*?
The task (the queued command) will be forked and execed from the
client, once it received the server message allowing the task to be
run (so, all previous tasks in the queue were accomplished)
>
> Shall we presume it is the equivalent of a Plan9 'CPU' then? Or?
I don't think it's any similar at all.
>
> (i.e. plenty of RAM and CPU-cycles, but short on storage space.)
>
> - using the server for scheduling control & spooling the output?
The server knows to what file the output is stored. And it 'spools'
the tasks. I wouldn't use the verb 'schedule' for spooling.
>
> AFAIK Plan9 can do many "similar" things already.
>
> But I am missing an example or three of what you would actually use this for..
> 370-155's aside, there may be sound reason why there is nothing similar...
What are 370 and 155?

Maybe you could look at the manpage of 'ts' for more information.

Regards,
Lluís.

Re: [9fans] Porting to plan9 / PF_UNIX

[Russ Cox]

There is nothing in Plan 9 like a named socket.
There is /srv, which is like named pipes (the 
difference is that named sockets distinguish
multiple connections to them).  

If ts were a Plan 9 program, it would present a
file tree that could be manipulated to add things
to the queue or remove them or check their status,
instead of inventing a custom RPC protocol.
Then it would post a fd in /srv for 9P service,
and you'd mount /srv/ts /mnt/ts and then interact
with /mnt/ts via file operations.  (Multiple programs
can mount /srv/ts because the kernel acts as a 
multiplexor for 9P services -- the server sees only
one 9P conversation.)

Russ

Re: [9fans] Porting to plan9 / PF_UNIX

[Lluís Batlle]

2007/4/14, Russ Cox <rsc@swtch.com>:
> There is nothing in Plan 9 like a named socket.
> There is /srv, which is like named pipes (the
> difference is that named sockets distinguish
> multiple connections to them).
That agrees with what I knew. I could not fit the sockets' multiple
connections with srv.
>
> If ts were a Plan 9 program, it would present a
> file tree that could be manipulated to add things
> to the queue or remove them or check their status,
> instead of inventing a custom RPC protocol.
> Then it would post a fd in /srv for 9P service,
> and you'd mount /srv/ts /mnt/ts and then interact
> with /mnt/ts via file operations.  (Multiple programs
> can mount /srv/ts because the kernel acts as a
> multiplexor for 9P services -- the server sees only
> one 9P conversation.)
That's exactly the kind of answer I was asking for. Thank you.

In 'ts', the server does some tasks (has the task list, results.,) and
the client does some others. The most important task (fork and exec
the queued commad) must be necessarily be run from the client, so all
the environment/namespace are kept.

What I feel common from the plan9 file servers I've seen by now is
that the server does all the processing, and mostly provides the
filesystem as an interface to the user. Thus, being the server the
process doing all the concerned tasks.

Now I still cannot image what kind of files/directories 'ts' should
serve to offer a good interface to the user, and how 'enqueuing'
should work, getting a nice 'RPC' through that filesystem.
I'll think of it, sure.

Thanks!

Re: [9fans] Porting to plan9 / PF_UNIX

[ron minnich]

On 4/14/07, Lluís Batlle <viriketo@gmail.com> wrote:

>
> Now I still cannot image what kind of files/directories 'ts' should
> serve to offer a good interface to the user, and how 'enqueuing'
> should work, getting a nice 'RPC' through that filesystem.
> I'll think of it, sure.


Liuis, I did something like this for the 9grid, for taskbags. I've
also written similar tools for Unix over the years, so I think I see
what you are up to. It's not identical to ts, just similar in nature.

The server I wrote for Plan 9 was originally a port of the TNT work
described here:
http://arxiv.org/PS_cache/astro-ph/pdf/9912/9912134v1.pdf. The TNT
server maintained task state describing work to be done, and
mutiplexed that work to clients. One of the nice things about TNT, not
mentioned in the paper, is that it supports "third party" or middlemen
clients, i.e. clients can become servers and grab a bunch of tasks to
be handed out. This distribution can be important if you have enough
nodes, and turn tasks over very frequently. Also, of course, clients
can put new work back into the bag of tasks. The work to be done was
communicated to the server to/from clients via RPC (SunRPC). So, I
began to do a straight conversion of this Unix socket-based server to
Plan 9; about 10 minutes into the job, I realized I was not thinking
about the problem correctly.

I thought about what I would do if starting from scratch, on Plan 9.
Plan 9 makes writing servers so easy that I went that route. I dumped
the RPC-based model and created a server that represented the state of
the work as directories and files. Basically, the server has three
directories, questions, working, and answers. You put questions into
the questions directory by opening and writing a file. To pass your
'namespace', you could do an ns command into the questions file you
created, then put the work in as a set of commands or rc script into
that file. Each new piece of work is a new file.
Clients start working by opening a questions file. Once a client has
opened a questions file, subsequent 'ls' commands will show that file
in the working directory, not the questions directory.  To see what
work is being done, you ls the working directory. Once a process is
done, it closes the file; at that point, the file will appear in the
answers directory, IF it has been written to. If the file was not
written to, it reappears in the questions directory (this is to handle
the case of clients that die). To see the answers to date, ls answers
or cat answers/*. If you want programs to find out when work is
completed,you could extend the server by having it create a 'status'
file at the top level; programs can read that and block on it, and
your server can distribute status info to as jobs are done.

With this new Plan 9 server, I was able to do distributed computation
entirely with shell scripts.

One old problem with task bags related to multiple users. What if more
than one user writes the task bag? This has been solved in complex
ways on Unixen, with authentication happening on the server so that
only the 'right' people get to the socket. This is solved trivially in
Plan 9 -- each user, needing this service, starts a task bag server --
problem solved.

The beauty of a server is that you can stop thinking in terms of RPC!
The "files" create a structure that is normally done in RPC. So, on
Plan 9, try to avoid falling into old Unix patterns. If you start
thinking in terms of a 'server' socket, and not a 9p server, and if
you start writing RPC code, the odds are good that you are not
approaching the problem correctly.

I ported a few HPC computations to my taskbag server; it was pretty
trivial. I *think* I left the code on sources (man 9fs to see what
this statement means) in the 9grid directory; you can look.

BTW, this sort of 'file-based' task bag idea has been done to death as
well, usually on NFS and with a huge boatload of scripts to go with
it. One system I saw, for MPEG encoding, had a 100-page (or so) script
to use NFS files for holding tasks. 99 pages were for dealing with the
problems that come with using NFS for this purpose ... not being able
to guarantee exclusive create, open, and remove makes these things
messy.

It's not just the idea of using files to describe work that is so
effective; it's the fact that the operations on the files are under
control of your server, and hence they're not really files, but a
representation of the work, shown to you as files. This disctinction
is very important. It's why a Plan 9 task bag server can work, and the
NFS-based task bag servers fail, and fail badly.

It's nice to have you involved in Plan 9; welcome aboard. Don't
hesitate to ask questions. It will be nice to see what you create.

thanks

ron

Re: [9fans] Porting to plan9 / PF_UNIX

[Lluís Batlle]

2007/4/14, ron minnich <rminnich@gmail.com>:
> On 4/14/07, Lluís Batlle <viriketo@gmail.com> wrote:
>
> >
> > Now I still cannot image what kind of files/directories 'ts' should
> > serve to offer a good interface to the user, and how 'enqueuing'
> > should work, getting a nice 'RPC' through that filesystem.
> > I'll think of it, sure.
>
>
> The server I wrote for Plan 9 was originally a port of the TNT work
> described here:
> http://arxiv.org/PS_cache/astro-ph/pdf/9912/9912134v1.pdf. The TNT
> server maintained task state describing work to be done, and
> mutiplexed that work to clients. One of the nice things about TNT, not
> mentioned in the paper, is that it supports "third party" or middlemen
> clients, i.e. clients can become servers and grab a bunch of tasks to
> be handed out. This distribution can be important if you have enough
> nodes, and turn tasks over very frequently. Also, of course, clients
> can put new work back into the bag of tasks. The work to be done was
> communicated to the server to/from clients via RPC (SunRPC). So, I
> began to do a straight conversion of this Unix socket-based server to
> Plan 9; about 10 minutes into the job, I realized I was not thinking
> about the problem correctly.
I was almost certain that I should rethink the whole program again,
and although I may use some kind of RPC in unix (for avoiding linking
against p9p libs), I should be able to provide an API independant from
RPC or file servers.
But having some code written in that direction would make me more sure
of my words. :)
>
> I thought about what I would do if starting from scratch, on Plan 9.
> Plan 9 makes writing servers so easy that I went that route. I dumped
> the RPC-based model and created a server that represented the state of
> the work as directories and files. Basically, the server has three
> directories, questions, working, and answers. You put questions into
> the questions directory by opening and writing a file. To pass your
> .....
I'd like to know what kind of file the "task submitter" should open.
Did you have several names in the questions directory, and each
submitter took one at random? It should be a name not colliding with
others in working or answers...
Or maybe the submitter opened a file with _any_ name (altough no 'ls'
would show it in questions), and that was created with the same name
in working?
>
> One old problem with task bags related to multiple users. What if more
> than one user writes the task bag? This has been solved in complex
> ways on Unixen, with authentication happening on the server so that
> only the 'right' people get to the socket. This is solved trivially in
> Plan 9 -- each user, needing this service, starts a task bag server --
> problem solved.
Also in my 'ts' each user has (in general) its own server. It depends
on the socket path they use. On agreement among users, they can use a
common socket with proper unix socket permissions in the filesystem.
>
> The beauty of a server is that you can stop thinking in terms of RPC!
> The "files" create a structure that is normally done in RPC. So, on
> Plan 9, try to avoid falling into old Unix patterns. If you start
> thinking in terms of a 'server' socket, and not a 9p server, and if
> you start writing RPC code, the odds are good that you are not
> approaching the problem correctly.
I agree. But RPC may be as awful as lazy I am, and the file structure
requires more thinking to find a nice solution - it will be probably
published to users :)
>
> I ported a few HPC computations to my taskbag server; it was pretty
> trivial. I *think* I left the code on sources (man 9fs to see what
> this statement means) in the 9grid directory; you can look.
I looked at /n/sources/9grid, but I could only find some code for xcpu
there. Nothing named 'task bags' at first glance.
>
> BTW, this sort of 'file-based' task bag idea has been done to death as
> well, usually on NFS and with a huge boatload of scripts to go with
> it. One system I saw, for MPEG encoding, had a 100-page (or so) script
> to use NFS files for holding tasks. 99 pages were for dealing with the
> problems that come with using NFS for this purpose ... not being able
> to guarantee exclusive create, open, and remove makes these things
> messy.
Well, my purpose on ts has never been multiuser queues for different
computers. I thought it for the case a user knows the
multitasking-limits of the resources of a single computer. Using hard
disk heavily in two applications may slowdown both. Using at once two
applications each requiring the amount of physical ram in the
computer, may also slow/bring down the system. That's what ts was
meant for.
>
> It's not just the idea of using files to describe work that is so
> effective; it's the fact that the operations on the files are under
> control of your server, and hence they're not really files, but a
> representation of the work, shown to you as files. This disctinction
> is very important. It's why a Plan 9 task bag server can work, and the
> NFS-based task bag servers fail, and fail badly.
Sure. I barely can believe someone tried such a thing over NFS.
>
> It's nice to have you involved in Plan 9; welcome aboard. Don't
> hesitate to ask questions. It will be nice to see what you create.
Having seen code in the operating system, read some works from Pike
and Kernighan, watched the discussions in this list from a long time,
it'd be a honor contributing a bit to the OS.
But I cannot guarantee code in short term. My university project is
entirely for another computing field and I should work on it, my job,
and other studies grab my time in own elected priority. I can't say
that I _need_ ts in plan9 (I mostly use p9p and not the OS). But it's
a direction I'd like to move on.

I'll also consider the nice and simple idea from John Stalker.

Thank you for your inspiring and couraging messages,
Lluís

Re: [9fans] Porting to plan9 / PF_UNIX

[W B Hacker]

Lluís Batlle wrote:
> 2007/4/14, W B Hacker <wbh@conducive.org>:
>> LLuis,
*snip*

>> But I am missing an example or three of what you would actually use 
>> this for..
>> 370-155's aside, there may be sound reason why there is nothing 
>> similar...
> What are 370 and 155?

One of a family of IBM systems for turning batch processing into cash.
Quite a lot of it from 1964 to the present day.

> 
> Maybe you could look at the manpage of 'ts' for more information.
>

You describe a method, but not a specific application that benefits from it.

Some - pehaps *all* of what you need to accomplish should be 'doable' within the 
existing Plan9 toolset

- just not in quite the same manner as in either the Unix or '370 compatible' 
worlds..

Bill

Re: [9fans] Porting to plan9 / PF_UNIX

[Francisco J Ballesteros]

If you want a program to serve something for others, there are several ways.
One way would be to write a file server, and register with srv(3) one
end of a pipe
that others could mount. There are plenty of examples in /sys/src/cmd.
I'd suggest to go over the manual seeking for programs that might do something
similar to what you want, and then reading their code.

hth

On 4/14/07, Lluís Batlle <viriketo@gmail.com> wrote:
> I'm the developer of a Unix program that may be useful in plan9, and
> as an exercise, I'd like it to build in plan9 native.
>
> Now, I've written it assuming POSIX (int signals, PF_UNIX available,
> BSD sockets API, ...). I know more or less some details on plan9 (per
> process namespaces, network not in a library to link with, notes,
> ...), but I barely know the 'common' use of the environment.
>
> My concern is that I need some local processes to communicate to a
> local server. Now I use a SOCK_STREAM PF_UNIX in a path in /tmp, where
> I suppose I can write. I think it's something I can suppose in a POSIX
> system, but not in plan9.
> How would you connect local processes (which can be launched at any
> time) with a central server in plan9? Making that central server serve
> a filesystem, where the clients write/read?
> If I knew what's the path I should follow, I could think more easily
> on a proper API for POSIX and Plan9 for my program.
>
> I may not have not worked in plan9 enough to deserve your attention,
> but I hope a quick answer (even in the form of "man ___") will not
> take much time.
>
> btw, the program I'd like to be able to run in plan9 is
> http://vicerveza.homeunix.net/~viric/soft/ts/ . I wrote it because I
> couldn't find anything similar to fit my needs, and maybe in plan9 you
> already have something quick-and-useful for that purpose, and 'ts'
> would not make any sense in p9 now. That's also something I'd be glad
> to know.
>
> Thanks in advance,
> Lluís.
>
>

Re: [9fans] Porting to plan9 / PF_UNIX

[Uriel]

You might want to look at cpu(1) (for Plan 9) and http;//www.xcpu.org
(for Plan 9 and other systems.)

And there is also taskfs by Kenji Arisawa, I think the latest version
of you can find in /n/sources/contrib/arisawa/grid.tgz (Kenji, would
be nice if you added an INDEX file to your sources/contrib dir)

And finally taskbag by Nashi of which the only version I can find is
in http://www.tip9ug.jp/who/nashi/9grid/taskbag.tgz

Nashi: *please*, could you put that stuff in sources/contrib like
everyone else? It is already hard enough to find things as it is.

And as nemo pointed out, srv(1) is probably what you were asking for.

Best wishes

uriel

On 4/14/07, Lluís Batlle <viriketo@gmail.com> wrote:
> I'm the developer of a Unix program that may be useful in plan9, and
> as an exercise, I'd like it to build in plan9 native.
>
> Now, I've written it assuming POSIX (int signals, PF_UNIX available,
> BSD sockets API, ...). I know more or less some details on plan9 (per
> process namespaces, network not in a library to link with, notes,
> ...), but I barely know the 'common' use of the environment.
>
> My concern is that I need some local processes to communicate to a
> local server. Now I use a SOCK_STREAM PF_UNIX in a path in /tmp, where
> I suppose I can write. I think it's something I can suppose in a POSIX
> system, but not in plan9.
> How would you connect local processes (which can be launched at any
> time) with a central server in plan9? Making that central server serve
> a filesystem, where the clients write/read?
> If I knew what's the path I should follow, I could think more easily
> on a proper API for POSIX and Plan9 for my program.
>
> I may not have not worked in plan9 enough to deserve your attention,
> but I hope a quick answer (even in the form of "man ___") will not
> take much time.
>
> btw, the program I'd like to be able to run in plan9 is
> http://vicerveza.homeunix.net/~viric/soft/ts/ . I wrote it because I
> couldn't find anything similar to fit my needs, and maybe in plan9 you
> already have something quick-and-useful for that purpose, and 'ts'
> would not make any sense in p9 now. That's also something I'd be glad
> to know.
>
> Thanks in advance,
> Lluís.
>

Re: [9fans] Porting to plan9 / PF_UNIX

[John Stalker]

I am not sure how I would use this, so my ideas of an appropriate interface
are probably a bit off.  For what it's worth, I would imagine a file system
like the one mounted on /proc.  One subdirectory per process.  The files in
the subdirectory would vary a bit depending on whether the job is finished,
running or queued.  In addition to the numbered directories there would
be a special directory next, containing only the args file.  Writing to
/queue/next/args would create a job, which would then appear as a numbered
directory like all the others.  Feel free to ignore any of this if it
doesn't fit your idea of what the spooler is for.

John

ps Since I started writing this, Ron's answer has appeared.  It may well
be a better idea, since he has actually implemented it and therefore
knows where the pitfalls are.


> I'm the developer of a Unix program that may be useful in plan9, and
> as an exercise, I'd like it to build in plan9 native.
> 
> Now, I've written it assuming POSIX (int signals, PF_UNIX available,
> BSD sockets API, ...). I know more or less some details on plan9 (per
> process namespaces, network not in a library to link with, notes,
> ...), but I barely know the 'common' use of the environment.
> 
> My concern is that I need some local processes to communicate to a
> local server. Now I use a SOCK_STREAM PF_UNIX in a path in /tmp, where
> I suppose I can write. I think it's something I can suppose in a POSIX
> system, but not in plan9.
> How would you connect local processes (which can be launched at any
> time) with a central server in plan9? Making that central server serve
> a filesystem, where the clients write/read?
> If I knew what's the path I should follow, I could think more easily
> on a proper API for POSIX and Plan9 for my program.
> 
> I may not have not worked in plan9 enough to deserve your attention,
> but I hope a quick answer (even in the form of "man ___") will not
> take much time.
> 
> btw, the program I'd like to be able to run in plan9 is
> http://vicerveza.homeunix.net/~viric/soft/ts/ . I wrote it because I
> couldn't find anything similar to fit my needs, and maybe in plan9 you
> already have something quick-and-useful for that purpose, and 'ts'
> would not make any sense in p9 now. That's also something I'd be glad
> to know.
> 
> Thanks in advance,
> Llu=EDs.
-- 
John Stalker
School of Mathematics
Trinity College Dublin
tel +353 1 896 1983
fax +353 1 896 2282