Re: [9fans] xcpu fix for Plan9.c

Re: [9fans] xcpu fix for Plan9.c

[YAMANASHI Takeshi]

On Fri Oct 21 13:31:40 JST 2005, Russ Cox wrote:
> > Does p9p threadpostmountsrv() listen on the network,
> > if it is given a network address as the service name?
> 
> Yes, it does, but that's not 100% by design.
 :
> In Plan 9,
> the kernel is expected to do that, so you'd do:
> 
>     xcpusrv -s myservice
>     aux/listen1 -v tcp!*!20001 /bin/exportfs -S /srv/myservice

Thanks.  It resulted in
	aux/listen1 -tv tcp!*!20001 /bin/exportfs -S /srv/myservice
because myservice was mode 600.

After this, I could mount the xcpusrv by
	9fs tcp!hostname!20001 /n/temp
-- 
"on travel, off the network ... and a fossil in my pocket"

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

Kenji Okamoto wrote:

> term% xcpusrv -sxcpu
> ....
> term% mount -ac /srv/xcpu /mnt/xcpu
> term% openclone /mnt/xcpu
> (then on a newly created rio window)
> term% cat /mnt/xcpu/clone
> 65537term%    <=============here! not 65536!
> term% ls -l /mnt/xcpu
> --rw-rw-rw- M 66 ssh ssh 0 Jan  1  1970 /mnt/xcpu/65536  <========= not 65537!
> --rw-rw-rw- M 66 ssh ssh 0 Jan  1  1970 /mnt/xcpu/clone

weird. I will have to try this on plan 9. I think I know what I am doing 
wrong but want to verify it.

> 
> On the point why only once copy of execution file, I didn't noticed
> the danger of your example /bin/sh execution.  Now, I think why
> you chose that way.   So, in xcpu we should do only batch job, right?

no, if you look at xsh you can see it can be used for interactive jobs 
(in future).

The real issues are these:
Suppose I
cp /bin/date exec
echo exec >ctl
cp /bin/uname exec
echo exec > ctl
cat stdout

OK, what does the output of stdout mean in this case? how do I 
distinguish the two things in a reasonable way? If I want to control the 
process, which one am I controlling? If I get an eof on the stdout file, 
which one did I get an EOF for? Should stdout deliver more than one EOF 
for each process that ends? The whole situation turns into a confusing 
mess if you let more than one process run for each 'exec' file.

> On the xsh.c, I still have some problem.
> Isn't this a program to be used for a cluster environment?

yes, but I make no guarantees. I am still fighting what might be a p9p 
issue on Linux, so xsh is not quite ready yet. I am trying to get this 
thing ready for SC '05, but I am running out of time :-)

> 
> In the file, there is a line of
> 	dirno[nodeno++] = smprint("/%s/%s/xcpu/%s", base, s, buf);
> 
> here, I suppose we should name our cluster's cp server by s,
> such as
> /mnt/xcpu/"s"/xcpu/"number", right?

yes, exactly, I had no idea of how to name things, so this is what I 
came up with. This is done this way to make it fit linux as well. On 
linux, we will have
/mnt/xcpu/"s"/xcpu ==> xcpu server for "s"
/mnt/xcpu/"s"/fs   ==> u9fs server for that node, so we can access files 
on the node "s"

note that in this model, the node is the server, and it exports both the 
xcpu service and its own file system.
> 
> So, we have only one cpu server, then we should use the xsh command
> like 
> term% xsh 0 
> ?

yes.

> 
> Or if we have many cpu server, then
> 
> xsh 0 1 2 3 4 5 6 7
> ?

yes. so to run date on all those nodes:
xsh 0 1 2 3 4 5 6 7 -- /bin/date

> 
> By the way, in your mkfile-plan9
> 
> $O.xsh: xsh.$O P9pshell.$O
> 	$LD -o $target $prereq $LDFLAGS
> 
> should be
> $O.xsh: xsh.$O Plan9shell.$O    <============
> 	$LD -o $target $prereq $LDFLAGS

thanks for that fix, I just installed it.

thanks again

ron

Re: [9fans] xcpu fix for Plan9.c

[Kenji Okamoto]

> the danger of your example /bin/sh execution.  Now, I think why
> you chose that way.   So, in xcpu we should do only batch job, right?

As usual 
s/Now, I think/Now, I think I understand/

Sorry

Kenji

Re: [9fans] xcpu fix for Plan9.c

[Kenji Okamoto]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

Thank you very much, Ron.

I just reconfirmed my expeience also here at work.
(I'm using Geoff's (or Ken's) file server just now.)

In this case, I got more curious result as follows:

This time, I'm doing this all on a terminal.

term% xcpusrv -sxcpu
....
term% mount -ac /srv/xcpu /mnt/xcpu
term% openclone /mnt/xcpu
(then on a newly created rio window)
term% cat /mnt/xcpu/clone
65537term%    <=============here! not 65536!
term% ls -l /mnt/xcpu
--rw-rw-rw- M 66 ssh ssh 0 Jan  1  1970 /mnt/xcpu/65536  <========= not 65537!
--rw-rw-rw- M 66 ssh ssh 0 Jan  1  1970 /mnt/xcpu/clone

On the point why only once copy of execution file, I didn't noticed
the danger of your example /bin/sh execution.  Now, I think why
you chose that way.   So, in xcpu we should do only batch job, right?

On the xsh.c, I still have some problem.
Isn't this a program to be used for a cluster environment?

In the file, there is a line of
	dirno[nodeno++] = smprint("/%s/%s/xcpu/%s", base, s, buf);

here, I suppose we should name our cluster's cp server by s,
such as
/mnt/xcpu/"s"/xcpu/"number", right?

So, we have only one cpu server, then we should use the xsh command
like 
term% xsh 0 
?

Or if we have many cpu server, then

xsh 0 1 2 3 4 5 6 7
?

By the way, in your mkfile-plan9

$O.xsh: xsh.$O P9pshell.$O
	$LD -o $target $prereq $LDFLAGS

should be
$O.xsh: xsh.$O Plan9shell.$O    <============
	$LD -o $target $prereq $LDFLAGS
, I suppose.

Kenji

[-- Attachment #2: Type: message/rfc822, Size: 5255 bytes --]

From: Ronald G Minnich <rminnich@lanl.gov>
To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu>
Subject: Re: [9fans] xcpu fix for Plan9.c
Date: Mon, 24 Oct 2005 08:46:47 -0600
Message-ID: <435CF3D7.60004@lanl.gov>

kokamoto@hera.eonet.ne.jp wrote:

> 1) why I got 131072 instead of 0 or 1?
>     Am I on the right way?

you are. I've had to go to this stilly numbering scheme to handle some 
LInux issues. It's a dentry cache thing, I can find the email and 
describe the problem if you wish. As nice as the dentry cache in Linux 
is, it is perilous when you venture outside its 'modus operandi'.

> 2) why the /mnt/cpu/131072/exec file accepts cp command
>     only once?

this is intentional. Consider the open of the clone file as a 'session'. 
In the original xcpu, as written by Vic Zandy, you could do all kinds of 
execs for a single session:
cp /bin/date exec
echo exec > ctl
cp /bin/whoami exec
echo exec > ctl

I initially copied this style for this implementation of xcpu.

There are lots of problems here, and it turned out to be simpler to just 
allow one and only one exec per session. Note that the child you start 
can also exec lots of things:
cp /bin/sh exec
echo exec > ctl

Please look at xsh.c -- it is what I am hoping people will actually use. 
The interface presented by the server is very low level -- by intent -- 
and xsh is something like what I think people will use.

Andrey starts here tomorrow,so we will be getting his help on this too :-)

Thanks so much for trying this out, and for getting your comments out to 
the list.

ron

Re: [9fans] xcpu fix for Plan9.c

[Lucio De Re]

> Quite frankly, as a completely uneducated spectator, I'd be prone to
> think:
> 
> 	echo date > exec
> 
> instead.  Presumably, the "exec" command is very common?

Looks like I'm waaay off base, here.  Sorry for the noise.

++L

Re: [9fans] xcpu fix for Plan9.c

[Lucio De Re]

[-- Attachment #1: Type: text/plain, Size: 166 bytes --]

Quite frankly, as a completely uneducated spectator, I'd be prone to
think:

	echo date > exec

instead.  Presumably, the "exec" command is very common?

++L

[-- Attachment #2: Type: message/rfc822, Size: 3904 bytes --]

From: Ronald G Minnich <rminnich@lanl.gov>
To: YAMANASHI Takeshi <9.nashi@gmail.com>
Cc: 9fans@cse.psu.edu
Subject: Re: [9fans] xcpu fix for Plan9.c
Date: Mon, 24 Oct 2005 08:40:27 -0600
Message-ID: <435CF25B.4030504@lanl.gov>

YAMANASHI Takeshi wrote:
>>This argv model was in the original xcpu stuff, should we really instead 
>>be doing this:
>>echo exec date > ctl, i.e. just require the args as part of the exec 
>>command? It seems easier to me somehow to do it this way.
> 
> 
> This model looks easier to me too.  I'm curious why the original
> author didn't do it this way though.

I think we were just feeling our way around, trying to understand how 
this ought to work, so that's how Vic implemented it.

I am going to go to this new usage.

ron

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

kokamoto@hera.eonet.ne.jp wrote:

> 1) why I got 131072 instead of 0 or 1?
>     Am I on the right way?

you are. I've had to go to this stilly numbering scheme to handle some 
LInux issues. It's a dentry cache thing, I can find the email and 
describe the problem if you wish. As nice as the dentry cache in Linux 
is, it is perilous when you venture outside its 'modus operandi'.

> 2) why the /mnt/cpu/131072/exec file accepts cp command
>     only once?

this is intentional. Consider the open of the clone file as a 'session'. 
In the original xcpu, as written by Vic Zandy, you could do all kinds of 
execs for a single session:
cp /bin/date exec
echo exec > ctl
cp /bin/whoami exec
echo exec > ctl

I initially copied this style for this implementation of xcpu.

There are lots of problems here, and it turned out to be simpler to just 
allow one and only one exec per session. Note that the child you start 
can also exec lots of things:
cp /bin/sh exec
echo exec > ctl

Please look at xsh.c -- it is what I am hoping people will actually use. 
The interface presented by the server is very low level -- by intent -- 
and xsh is something like what I think people will use.

Andrey starts here tomorrow,so we will be getting his help on this too :-)

Thanks so much for trying this out, and for getting your comments out to 
the list.

ron

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

YAMANASHI Takeshi wrote:
>>This argv model was in the original xcpu stuff, should we really instead 
>>be doing this:
>>echo exec date > ctl, i.e. just require the args as part of the exec 
>>command? It seems easier to me somehow to do it this way.
> 
> 
> This model looks easier to me too.  I'm curious why the original
> author didn't do it this way though.

I think we were just feeling our way around, trying to understand how 
this ought to work, so that's how Vic implemented it.

I am going to go to this new usage.

ron

Re: [9fans] xcpu fix for Plan9.c

[YAMANASHI Takeshi]

> 1) why I got 131072 instead of 0 or 1?
>     Am I on the right way?

My observation was the same.  I usualy get 65536 at first.

> 2) why the /mnt/cpu/131072/exec file accepts cp command
>     only once?

"xcpu operation" section in xcpu.pdf describes this.

	A session allows a remote user (or program) to start
	a process or process tree for one binary, once.
	:
	We have adopted this simple rule, one xcpu-started process
	per session, to simplify system management:
-- 
"on travel, off the network ... and a fossil in my pocket"

Re: [9fans] xcpu fix for Plan9.c

[kokamoto]

I'm now at home, and playing with xcpu on two machine Plan 9
system, one for CPU/AUTH/Fossil file server and the other is a
terminal.

On the CPU server, I dispatched the command like:
venus# xcpusrv -s xcpu
then got the answer:
fsnetproc pid is 421
threadpostmountsrv RETURNS!~!!!!!!

Next, I did the command
venus# aux/listen1 -tv tcp!*!20001 /bin/exportfs -S /srv/xcpu
listen started

After this I moved to the terminal, and dispatched the command:
term# 9fs tcp!venus!20001 /mnt/xcpu
post...
Then, I listed the terminal's /mnt/xcpu directory, so

term# ls -l /mnt/xcpu
and got the answer:
--rw--rw--rw M 47 ssh ssh 0 Jan 1 1970 clone

Ok, Next I read the clone file, and got
65536term#   
Oops! I forgot open the clone file, then,

term# openclone /mnt/xcpu/clone&
and got,
term# 131072openclone: child exits, we exit too  <==== not 0/1

then,
term# ls -l /mnt/xcpu
and got
d-r-x-r-x-r-x M 47 ssh ssh 0 Jan 1 1970 131072
--rw-rw-rw- M 47 ssh ssh 0 Jan 1 1970 clone

Ok, then I dispatched the series of command in Ron's xcpu.pdf:

term# cp /bin/date /mnt/xcpu/131072/exec
term# echo date >/mnt/xcpu/131072/argv
term# echo exec >/mnt/xcpu/131072/ctl
term# cat /mnt/xcpu/131072/stdout   <=======

Then, I got the right answer as
Sat Oct 22 11:38:02 JST 2005

Ok, everything worked fine.

Then, I tried to do the above sequence of commands, and
got error such as
  error writing /mnt/xcpu/131072/exec: not connected.

So, my questions are:

1) why I got 131072 instead of 0 or 1?
    Am I on the right way?
2) why the /mnt/cpu/131072/exec file accepts cp command
    only once?

I know Ron has been very busy, and may be still...
I can wait Ron, don't work too hard!

Kenji

Re: [9fans] xcpu fix for Plan9.c

[Russ Cox]

> Does p9p threadpostmountsrv() listen on the network,
> if it is given a network address as the service name?

Yes, it does, but that's not 100% by design.
It's a side effect of the fact that the equivalent
of /srv is already a type of network (unix sockets)
so there is an extra program 9pserve that takes
care of announcing on the network and multiplexing
many clients onto a single 9P connection.  In Plan 9,
the kernel is expected to do that, so you'd do:

    xcpusrv -s myservice
    aux/listen1 -v tcp!*!20001 /bin/exportfs -S /srv/myservice

russ

Re: [9fans] xcpu fix for Plan9.c

[YAMANASHI Takeshi]

> Takeshi, the fixes are up on /n/sources/9grid/xcpu.tar

Thanks.  It compiles and runs fine now on Plan 9.

BTW, how can I make xcpusrv listening on the network?
I found "xcpusrv -s tcp!*!20001" in xcpu.pdf but
this only creates /srv/tcp!*!20001 on Plan 9.

Does p9p threadpostmountsrv() listen on the network,
if it is given a network address as the service name?
-- 
"on travel, off the network ... and a fossil in my pocket"

Re: [9fans] xcpu fix for Plan9.c

[YAMANASHI Takeshi]

> This argv model was in the original xcpu stuff, should we really instead 
> be doing this:
> echo exec date > ctl, i.e. just require the args as part of the exec 
> command? It seems easier to me somehow to do it this way.

This model looks easier to me too.  I'm curious why the original
author didn't do it this way though.
-- 
"on travel, off the network ... and a fossil in my pocket"

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

Takeshi, the fixes are up on /n/sources/9grid/xcpu.tar

ron

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

YAMANASHI Takeshi wrote:
>>sorry about the error, I have put a fix (untested, no time this morning) 
>>on /n/sources/9grid/Plan9.c
> 
> 
> Thanks, but you forgot to put "int kidfds[3]" in, it seems.
> 
> === 
> void runit(void *ac) {
> 	Client *c = ac;
>  :
> 	int kidfds[3];
> === 

thanks, fixed.

> 
> With this addition in Plan9.c, I was able to run xcpusrv on Plan 9
> while /n/xcpu/0/arch always returns "linux". :)

fixed, for 386; will try to put in a read of the right # files but I 
just ran out of time today.

> 
> By the way, tokenize() in runit() will abort if c->argv is nil.
> You can do this by issuing "echo exec > ctl" without
> "echo something > argv" done beforehand.

I think it might be sensible to not let the echo exec > ctl operation 
happen until argv is set? What do you think? I don't see a point in 
letting it run without args.

This argv model was in the original xcpu stuff, should we really instead 
be doing this:
echo exec date > ctl, i.e. just require the args as part of the exec 
command? It seems easier to me somehow to do it this way.

> 
> Oh, one more thing in xcpu.pdf.
> 
> There seems to be a typo in the example for setting up argv on p.12.
> 	Next, argv:
> 	  for i in $NODES
> 	  do
> 	  echo date > /mnt/xcpu/0/xcpu/$i/exec&
> 	  done
> 	  wait
> 
> The file date is echoed should be argv, not exec.

thanks, fixed.

thanks for looking at this. It is much appreciated.

ron

Re: [9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

Thanks for the notes.

It is stupid that I just return  linux, this is laziness on my part, 
which I will fix today.

ron

Re: [9fans] xcpu fix for Plan9.c

[YAMANASHI Takeshi]

> sorry about the error, I have put a fix (untested, no time this morning) 
> on /n/sources/9grid/Plan9.c

Thanks, but you forgot to put "int kidfds[3]" in, it seems.

=== 
void runit(void *ac) {
	Client *c = ac;
 :
	int kidfds[3];
=== 

With this addition in Plan9.c, I was able to run xcpusrv on Plan 9
while /n/xcpu/0/arch always returns "linux". :)

By the way, tokenize() in runit() will abort if c->argv is nil.
You can do this by issuing "echo exec > ctl" without
"echo something > argv" done beforehand.

Oh, one more thing in xcpu.pdf.

There seems to be a typo in the example for setting up argv on p.12.
	Next, argv:
	  for i in $NODES
	  do
	  echo date > /mnt/xcpu/0/xcpu/$i/exec&
	  done
	  wait

The file date is echoed should be argv, not exec.
-- 
"on travel, off the network ... and a fossil in my pocket"

[9fans] xcpu fix for Plan9.c

[Ronald G Minnich]

sorry about the error, I have put a fix (untested, no time this morning) 
on /n/sources/9grid/Plan9.c

ron