[9fans] implementing 9p read

[9fans] implementing 9p read

[Michael Teichgräber]

Hi,

while thinking about whether it would make sense to use 9p for a rather
small embedded device, where Inferno or Plan 9 cannot be run (similar to
styx-on-a-brick), I came to the problem how to implement read access to
dynamic status files, like the `status' named text files on plan 9, or
files like acme's tag or ctl files. An important question seems to be:
When is the textual contents of such a file allowed to change?

For example, a text file like /mnt/acme/1/ctl: If I enter a character
into buffer 1, a certain number within the ctl file will be updated.
What if someone is reading `ctl' (using the same file descriptor)
just before and after the character is inserted - is its contents
allowed to change while the user does subsequent read() calls? This
could give him a corrupted snapshot of ctl, since parts of the text line
likely will be shifted or somehow overwritten; this can actually be
provoked, if one reads e.g. 10 bytes from ctl, enters a character, reads
some more bytes.

Under the condition that a ctl/status file is expected to be read by
`cat' or similar programs only, which read the whole file at once with
a large enough buffer, there will be no problem, as the text will be
consistent.

Is this the way to go? I.e. a 9p Read message arrives, the file server
just `sprint's status data into a buffer, and returns the requested
parts of it. If the user does seek operations, or reads only some bytes
at once, it is his problem if another sprint call results in a different
buffer content.

A different approach could be to allocate a new buffer for each user
opening the status file (i.e. for each fid), and `sprint' status data
only once into each buffer. This way each user always would see a
consistent text file, regardless of whether he performs seek operations
on it or reads only small chunks. Obviously this approach is also more
memory consuming.

Which approach would be preferred on Plan 9? Thanks for any advice,
Michael

Re: [9fans] implementing 9p read

[Sape Mullender]

> Hi,
> 
> while thinking about whether it would make sense to use 9p for a rather
> small embedded device, where Inferno or Plan 9 cannot be run (similar to
> styx-on-a-brick), I came to the problem how to implement read access to
> dynamic status files, like the `status' named text files on plan 9, or
> files like acme's tag or ctl files. An important question seems to be:
> When is the textual contents of such a file allowed to change?
> 
> For example, a text file like /mnt/acme/1/ctl: If I enter a character
> into buffer 1, a certain number within the ctl file will be updated.
> What if someone is reading `ctl' (using the same file descriptor)
> just before and after the character is inserted - is its contents
> allowed to change while the user does subsequent read() calls? This
> could give him a corrupted snapshot of ctl, since parts of the text line
> likely will be shifted or somehow overwritten; this can actually be
> provoked, if one reads e.g. 10 bytes from ctl, enters a character, reads
> some more bytes.
> 
> Under the condition that a ctl/status file is expected to be read by
> `cat' or similar programs only, which read the whole file at once with
> a large enough buffer, there will be no problem, as the text will be
> consistent.
> 
> Is this the way to go? I.e. a 9p Read message arrives, the file server
> just `sprint's status data into a buffer, and returns the requested
> parts of it. If the user does seek operations, or reads only some bytes
> at once, it is his problem if another sprint call results in a different
> buffer content.
> 
> A different approach could be to allocate a new buffer for each user
> opening the status file (i.e. for each fid), and `sprint' status data
> only once into each buffer. This way each user always would see a
> consistent text file, regardless of whether he performs seek operations
> on it or reads only small chunks. Obviously this approach is also more
> memory consuming.
> 
> Which approach would be preferred on Plan 9? Thanks for any advice,
> Michael

Many control files are read using the helper routines readstr and readbuf
(see 9p(2)).
They extract from a string or buffer, the portion needed for the read
command, given seek offset and byte count.  The string buffer is usually
generated anew for each read.  Inconsistencies, therefore, can occur when
the client is a slow reader and the status file changes underfoot.
I don't see how this can be helped other than keeping buffers allocated
on a per open-file basis — and that would be overkill.

	Sape

Re: [9fans] implementing 9p read

[Steve Simon]

> I don't see how this can be helped other than keeping buffers allocated
> on a per open-file basis — and that would be overkill.

We run an embdeed os at work for which we stole many ideas from plan9.

It has a lib9pfile like library to make writing servers easier.
Generally it uses an array of initialised C structures to define the
hierarchy you want to serve, which contains functions to generate the
contents of these files; Each file also has an associated read and write
flag word. Files can be read:

	line at a time - the library assembles the lines into the
		requested buffer's worth

	file at a time - contents generated in malloced memory on open
		and freed on close

	raw access - requests passed direct to underlying function

writing files can be flagged smilarly:

	line at a time - newlines and leading and trailing whitespace stripped

	file at a time - application specific function called only on close

	raw - as above.

This has been useful and I have thought about writing a similar library for
plan9 but have not done so yet, its difficult to decide what is useful often
enough to libraryise and what is just "nice to have".

-Steve

Re: [9fans] implementing 9p read

[Michael Teichgräber]

> line at a time - the library assembles the lines into the requested
> buffer's worth

Nice - this way you may easily generate quite large files using a buffer
covering just one line. This probably also means that Read returns in
most cases less bytes than requested, just as much as needed for the
current line up to \n. The server likely has to keep some information
like the next read-offset expected, and the next line number, so that it
can check whether the next read-request actually asks for the next line
 -- if it doesn't ignore seek/offsets at all, which also seems practicable.

> writing files can be flagged smilarly:
> 
> line at a time - newlines and leading and trailing whitespace
> stripped

I suppose this is adequate for control files waiting for commands. One
can leave the fd open, and send one line after the other as needed.

One thing I like about the idea using 9p even in smaller systems is,
that you can define one file to be something like a "serial terminal".
If one has a lot of such embedded devices connected to a network, one
can easily have a "command line" for each of them without the need to
connect each to a real serial port.

Michael

Re: [9fans] implementing 9p read

[erik quanstrom]

> One thing I like about the idea using 9p even in smaller systems is,
> that you can define one file to be something like a "serial terminal".
> If one has a lot of such embedded devices connected to a network, one
> can easily have a "command line" for each of them without the need to
> connect each to a real serial port.

we use cec(1).  (not in the standard distribution, but in
/n/sources/contrib/quanstro/cec.tar kernel patch in /n/sources/patch/cpu-cec)
it runs directly on ethernet, not requiring 9p, tcp/udp/il or ip.

one could use consolefs(4) to make this very easy on a large scale;
we currently don't have such a need, so unfortunately i haven't had
time to do this.

- erik

[9fans] kvm

[matt]

I notice from the archives that people have had success running plan9 on 
KVM but I don't see it listed

http://kvm.qumranet.com/kvmwiki/Guest_Support_Status

My dual Opteron 270 dual core box gets built next week and I'm looking 
at what I can do with it.


matt

Re: [9fans] kvm

[Paweł Lasek]

On 7/7/07, matt <mattmobile@proweb.co.uk> wrote:
> I notice from the archives that people have had success running plan9 on
> KVM but I don't see it listed
>
> http://kvm.qumranet.com/kvmwiki/Guest_Support_Status
>
> My dual Opteron 270 dual core box gets built next week and I'm looking
> at what I can do with it.

I can say that Plan 9 works fine under KVM. Just follow Qemu
instructions but use the kvm-supplied qemu build. I currently have a
cpu/auth server running on KVM.

The only drawback is that I can't seem to get scsi working under
kvm-24 qemu, which worked fine in kvm-16.

> matt
>

-- 
Paul Lasek

Re: [9fans] kvm

[matt]

Well that certainly sounds promising. I've got qemu images already

Have you come across this open bug :

http://sourceforge.net/tracker/index.php?func=detail&aid=1672286&group_id=180599&atid=893831

Re: [9fans] kvm

[Paweł Lasek]

On 7/7/07, matt <mattmobile@proweb.co.uk> wrote:
> Well that certainly sounds promising. I've got qemu images already
>
> Have you come across this open bug :
>
>http://sourceforge.net/tracker/index.php?func=detail&aid=1672286&group_id=180599&atid=893831
>

I haven't seen something like that, however my installation wasn't
used much, so it might be still hidden somewhere. I'll try that code
the next time I boot it. However those reports are from kvm-12, so I
don't know If it will happen under kvm-24...


-- 
Paul Lasek

Re: [9fans] kvm

[Paweł Lasek]

On 7/7/07, Paweł Lasek <pawel.lasek@gmail.com> wrote:
> On 7/7/07, matt <mattmobile@proweb.co.uk> wrote:
> > Well that certainly sounds promising. I've got qemu images already
> >
> > Have you come across this open bug :
> >
> >http://sourceforge.net/tracker/index.php?func=detail&aid=1672286&group_id=180599&atid=893831
> >
> [cut]

I tried the code they put to test for that bug and nothing happened.
That's with kvm-24, AMD Turion X2 and linux kernel 2.6.19-suspend2-r1
(From Gentoo).

> --
> Paul Lasek
>


-- 
Paul Lasek

Re: [9fans] implementing 9p read

[Steve Simon]

> This probably also means that Read returns in
> most cases less bytes than requested, just as much as needed for the
> current line up to \n. The server likely has to keep some information
> like the next read-offset expected, and the next line number, so that it
> can check whether the next read-request actually asks for the next line
>  -- if it doesn't ignore seek/offsets at all, which also seems practicable.

You cannot do this, the usual idiom is that if read returns less
than the app expected this is treated as EOF. in my library the low level
function generates line at a time data, the library then performs multiple
reads to satisfy the apps read request filling its buffer and rembembering
the partial line for the next read request.

> I suppose this is adequate for control files waiting for commands. One
> can leave the fd open, and send one line after the other as needed.

I don't generally find this is nescessary, the fileserver/device driver holds
internal state so multiple writes mearly modify this state, thusly:

	echo 'cts=1' > /dev/uart/ctl
	echo 'baud=115200' > /dev/uart/ctl

We get a real win using the everything is a file idea in embedded as most of our products
contain multiple PCBs and each PCB has a CPU (to load its xilinxes if nothing else).
Given the remote file protocol running over a a couple of wires (the link layer
distantly related to datakit) we can monitor any cpu from any other cpu and
upgrade all flash files from the cpu that has ethernet.

Its a shame that we didn't have the guts and skill to make it
real plan9 from the begining - hindsignt is a wonderful thing.

-Steve

Re: [9fans] implementing 9p read

[ron minnich]

On 7/6/07, Steve Simon <steve@quintile.net> wrote:

> You cannot do this, the usual idiom is that if read returns less
> than the app expected this is treated as EOF.

Oh not, that's not at all true. Any program that behaves this way is
broken. Unless I totally misunderstand your point.

ron

Re: [9fans] implementing 9p read

[Steve Simon]

> > You cannot do this, the usual idiom is that if read returns less
> > than the app expected this is treated as EOF.
> 
> Oh not, that's not at all true. Any program that behaves this way is
> broken. Unless I totally misunderstand your point.
> 

Ok,

You are probably right, its an assumption I have seen in code
in the past, though not in plan9:

	while((n = fread(buf, 1, sizeof(buf), fp)) != sizeof(buf))
		if(fwrite(buf, 1, n, out) != n)
			sysfatal("write failed");

I have always tried to code defensively around it when writing
fileservers by returning full buffers, it seems I have been
fleeing a mere spectre.

-Steve

Re: [9fans] implementing 9p read

[Charles Forsyth]

> 	while((n = fread(buf, 1, sizeof(buf), fp)) != sizeof(buf))
> 		if(fwrite(buf, 1, n, out) != n)
> 			sysfatal("write failed");

fread is buffered, or similar to readn, and does all it can, hence the difference
from plain read

Re: [9fans] implementing 9p read

[C H Forsyth]

>You cannot do this, the usual idiom is that if read returns less
>than the app expected this is treated as EOF. in my library the low level

a write that returns less than was written is trouble, but end-of-file is a read returning zero.
the count returned by read can be less than the amount requested without marking end-of-file (eg, reads on a pipe
or network connection).  that's why readn exists.

Re: [9fans] implementing 9p read

[ron minnich]

On 7/5/07, Sape Mullender <sape@plan9.bell-labs.com> wrote:

>
> Many control files are read using the helper routines readstr and readbuf
> (see 9p(2)).
> They extract from a string or buffer, the portion needed for the read
> command, given seek offset and byte count.  The string buffer is usually
> generated anew for each read.

another option is to only re-read when offset is 0. I.e., assume a
non-zero offset means 'still reading'.

But that approach has its own problems.

ron

Re: [9fans] implementing 9p read

[Michael Teichgräber]

> I don't see how this can be helped other than keeping buffers allocated
> on a per open-file basis — and that would be overkill.

Thanks. So the simple approach here apparently is also the advisable one.

Michael