[9fans] RFS alternatives (Was: Living with Plan 9)

[9fans] RFS alternatives (Was: Living with Plan 9)

[rbnsw-plan9]

--- On Wed, 8/6/11, I wrote in part:

> I am old enough to remember RFS the Remote File Sharing
> Protocol on SVR4 that offered access to remote devices, but
> I don't have that and I'm not aware of whether there are any
> distributed file  protocols freely available for *nix
> that do that.

Um, does v9fs remote Linux devices? I find it hard to imagine it would remote ioctls but it makes sense *nix to *nix.

If not, are there any other alternatives? Are there any gotchas remoting *nix devices?

If v9fs works in such a configuration it would eliminate most of the issues I have with *nix resource sharing - unless I use a platform without 9fs a 9fs client and servers.

That just leaves my issues with X.

Cheers,

Andrew

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Eric Van Hensbergen]

On Sat, Jun 11, 2011 at 2:26 PM,  <rbnsw-plan9@yahoo.com.au> wrote:
>
> --- On Wed, 8/6/11, I wrote in part:
>
>> I am old enough to remember RFS the Remote File Sharing
>> Protocol on SVR4 that offered access to remote devices, but
>> I don't have that and I'm not aware of whether there are any
>> distributed file  protocols freely available for *nix
>> that do that.
>
> Um, does v9fs remote Linux devices? I find it hard to imagine it would remote ioctls but it makes sense *nix to *nix.
>

Depends on how you configure it.  There is a nodevmap option to the
v9fs mount which will instruct it to just access the remote devices
directly instead of just mapping their major/minor numbers to local
devices.  You are correct in your imagining that we don't go anywhere
near ioctls with a 10 foot pole.  However, many things "just work"
without ioctls these days.

>
> That just leaves my issues with X.
>

Actually, its a bit worse than that.  The physical network devices
aren't file system accessible anymore, so you'd need to remote them as
a service (via Inferno or something) or use the tap device and remote
that and hope that it doesn't require ioctls (and I think it might).

        -eric

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[rbnsw-plan9]

--- On Tue, 21/6/11, Eric Van Hensbergen <ericvh@gmail.com> wrote:
...
>> Um, does v9fs remote Linux devices? I find it hard to
>> imagine it would remote ioctls but it makes sense *nix to
>> *nix.
>>
> 
> Depends on how you configure it.  There is a nodevmap
> option to the v9fs mount which will instruct it to just access 
> the remote devices directly instead of just mapping their major/minor 
> numbers to local devices.  You are correct in your imagining that we
> don't go anywhere near ioctls with a 10 foot pole.  However, many things
> "just work" without ioctls these days.
> 
Thanks for the info, but the devices encumbered with ioctls are the tricky ones and even if they can be sorted out I'm sure there are some other traps out there. Too bad there are no RFS gurus lurking here to offer their wisdom on remoting devices. I have a sneaking suspicion few people would have bothered, since the few devices worth remoting back then were easily handled by rsh/rcmd. Did RFS make it beyond SVR4?
> >
> > That just leaves my issues with X.
> >
> 
> Actually, its a bit worse than that.  The physical
> network devices aren't file system accessible anymore, 
Actually, I'd blotted out of my mind all knowledges of STREAMs devices and the related horror of TLI programming until you reminded me.
> so you'd need to remote them as a service (via Inferno or something) or
> use the tap device and remote that and hope that it doesn't require ioctls (and I think it might).
> 

Oh, I'm not worried about remoting network interfaces. I'm fine with packet forwarding and can live with NAT for now. However, it reminds me of another point in Plan 9's favour that the introductory papers should be updated to be more explicit about - NATs are unnecessary in a pure Plan 9 deployment. Unfortunately, neither Plan 9 routers or decent alternatives to NAT such as RSIP widely available.  
 
>         -eric
>
Andrew

P.S. There's spammers subscribed to this list. Hi there, friends of Khalifa.

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[dave.l]

> Thanks for the info, but the devices encumbered with ioctls are the tricky ones and even if they can be sorted out I'm sure there are some other traps out there. Too bad there are no RFS gurus lurking here to offer their wisdom on remoting devices.

I'm no RFS guru, thank deity, but I did RTFC once and "F" was apposite.

ioctl was handled by having the client "know" exactly what each ioctl "looked like", i.e. it only worked for known cases.
There was a humungous switch which knew the "right stuff" to copy in&out, how to byte swap and align any words, structs etc.
After deciding what the "\x10right stuff" was,
it copied in the "right stuff" from user space,
passed over the call to the server,
the server then executed it and passed back the "right stuff" to the client,
the client then copied out the "right stuff",
then I started losing the will to live.

Dave.

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Charles Forsyth]

>ioctl was handled by having the client "know" exactly what each ioctl "looked like", i.e. it only worked for known cases.

there isn't really any other choice, especially between machines
with different conventions. i think it was one of the bigger parts
of the newcastle connection implementation. that's also why plan 9
doesn't do it that way, but `old dogs' etc.

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[rbnsw-plan9]

--- On Wed, 22/6/11, David Lukes <davelukes@mac.com> wrote:
>...
> I'm no RFS guru, thank deity, but I did RTFC once and "F"
> was apposite.
It took me a little time to figure out RTFC wasn't a remote file system ;-)
> 
> ioctl was handled by having the client "know" exactly what
> each ioctl "looked like", i.e. it only worked for known
> cases.
...

Thanks for pointing out the folly in my thinking. I was hoping for a user/administrators view of RFS but the under the hood perspective is welcome. I'd forgotten that the variable(!) trailing parameters of the ioctl are opaque at the system call level, i.e. their layouts are only known to the application program and device drivers.

So the best that can be done in a generic manner is to send ioctls only between systems with identical binary representations (possibly remapping major/minor device numbers).

Maybe if RFS had been more successful over NFS, (or Linus knew more about Plan 9) the Unix interface would have been fixed by now. I wonder if there is any level of acceptance amongst *nix kernel developers that this is a problem. I can think of schemes to fix this without having to modify every program that issues an ioctl - though that would be a good idea in the long run - /merely/ having to modify every device driver implementing ioctls to assist in the remote translation. While it might seem unfeasible to modify every device driver in this class to support this behaviour, the kernel interfaces change frequently so perhaps this might indeed be possible.

Speaking of device numbers, I was surprised that Plan 9 has a similar notion. However, they are only useful with kernel resident device numbers. Does Plan 9 have some other mechanism that allow one to identify the class of device/file server it belongs to?

In most cases, a name is good enough, but sometimes its not.

And speaking of opaqueness, while Plan 9 might use portable data representation, some interfaces are perhaps unnecessarily opaque in a different way. Consider the uart ctl file with its encoded string of device attributes. If each attribute used a separate file, with meaningful names, the parameters of the interface, although unfortunately not their range of values, would be available. Given that it is not, some other form of automated description is desirable, ideally accessible from the file system itself rather than requiring use of a library. While we have learned (been forced) to live with files that are bags of bytes, I think it would be better not to repeat this with interfaces too.

Oh, and it would be better if the uart device was consistent with the files in net with their separate data and ctl files, if one is going to rely on conventions they should be consistent. Despite the other faults of ioctl there's no doubt where the control interface of a device is.

Andrew

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[ron minnich]

NFS4 on linux has gone with (in addition to everything else) a very
plan 9-like "write commands to files" model that uses text. It's
complex as you can ever imagine, and then some, but at least they seem
to have finally got the idea right to some extent.

The new 802.11 wireless infrastructure, I am told, uses a
non-ioctl-based command setup too with commands echoed into a
synthetic file system. I wouldn't know, I retired most of my linux
laptops ... I was at a linuxconf where this was described and it was
interesting to see how defensive the speaker had to be about not using
an ioctl-like mechanism.

People do learn, it just takes time. It takes way more time than you
might think. I used to think 10 years was a hugely long time to get
changes into the world, now I'm thinking 10 years is remarkably quick,
and 20 is more the rule, unless you hit the right idea at just the
right time. Which means each of us has maybe 4 chances in life to
really push a change into the world. There, I just made your day.

ron

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Jeff Sickel]

On Jun 27, 2011, at 12:20 PM, ron minnich wrote:

> Which means each of us has maybe 4 chances in life to
> really push a change into the world. There, I just made your day.

Great, missed every one of those chances.  Might as well throw in
the towel, stop resisting the inevitable, and chuck any hopes of
doing something great out the window.

Being in the right place at the right time is really hard to do.

-jas

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Skip Tavakkolian]

The essence of being a 9fan: It is more important to be right than to
be popular.
(although it would be nice to eventually be popular -- preferably
during one's lifetime)

-Skip

On Mon, Jun 27, 2011 at 10:45 AM, Jeff Sickel <jas@corpus-callosum.com> wrote:
>
> On Jun 27, 2011, at 12:20 PM, ron minnich wrote:
>
>> Which means each of us has maybe 4 chances in life to
>> really push a change into the world. There, I just made your day.
>
> Great, missed every one of those chances.  Might as well throw in
> the towel, stop resisting the inevitable, and chuck any hopes of
> doing something great out the window.
>
> Being in the right place at the right time is really hard to do.
>
> -jas
>
>
>

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[dexen deVries]

On Monday 27 June 2011 20:18:05 Skip Tavakkolian wrote:
> The essence of being a 9fan: It is more important to be right than to
> be popular.

The essence of being popular on 9fans: being more right than popular elsewhere
;-)



On Monday 27 June 2011 19:20:57 ron minnich wrote:
> (...stuff...) on linux has gone with (in addition to everything else) a very
> plan 9-like "write commands to files" model that uses text.

It's worth mentioning the devpts (the /dev/pts/* thingies) too. You can have
several instances of devpts, by mounting them at various places (with -o
newinstance); the relevant config is CONFIG_DEVPTS_MULTIPLE_INSTANCES


--
dexen deVries

> (...) I never use more than 800Mb of RAM. I am running Linux,
> a browser and a terminal.
rjbond3rd in http://news.ycombinator.com/item?id=2692529

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[erik quanstrom]

> It's worth mentioning the devpts (the /dev/pts/* thingies)

not unless it's 1978.  :-)

- erik

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[dexen deVries]

On Monday 27 June 2011 22:17:49 erik quanstrom wrote:
> > It's worth mentioning the devpts (the /dev/pts/* thingies)
>
> not unless it's 1978.  :-)

oy, i was unclear.

the main point: (in current linux) you can have  several instances of devpts,
by mounting them at various places (with -o  newinstance); the relevant config
is CONFIG_DEVPTS_MULTIPLE_INSTANCES

also, the `protocol' of creating new /dev/pts/* is somewhat sensible, too. no
ioctl(), just open /dev/pmtx, read a textual representation of an integer N
and open /dev/pts/N.

not claiming it's great tech, just that there's some progress in the
linuxland. hopefully more pro- than con-, anyway >_>;;


--
dexen deVries

> (...) I never use more than 800Mb of RAM. I am running Linux,
> a browser and a terminal.
rjbond3rd in http://news.ycombinator.com/item?id=2692529

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Yaroslav]

> also, the `protocol' of creating new /dev/pts/* is somewhat sensible, too. no
> ioctl(), just open /dev/pmtx, read a textual representation of an integer N
> and open /dev/pts/N.

nah, slave name is obtained with ptsname(master) which does
ioctl(TIOCGPTN) on Linux.

...
open("/dev/ptmx", O_RDWR|O_NOCTTY)      = 3
statfs("/dev/pts", {f_type="DEVPTS_SUPER_MAGIC", f_bsize=4096,
f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={0,
0}, f_namelen=255, f_frsize=4096}) = 0
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(3, TIOCGPTN, [8])                 = 0
stat64("/dev/pts/8", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 8), ...}) = 0
...

AIX does too:
...
kopen("/dev/ptc", O_RDWR|O_NOCTTY)		= 3
__loadx(0x01480180, 0x2FF21B50, 0x00000A50, 0xF06AFBB4, 0x00000000) = 0xF0FE4A28
_kgrantpt(0x00000003, 0x00000001, 0xF065F3F8, 0x0000D032, 0x00000003,
0x00000350, 0x00000000, 0x00000000) = 0x00000000
kioctl(3, 536900678, 0x2FF22670, 0x00000000)	= 0
kioctl(3, 536900679, 0x2FF225F0, 0x00000000)	= 0
kioctl(3, 22529, 0x2FF221B4, 0x00000000)	= 0
statx("/dev/pts/27", 0x2FF221D8, 76, 0)		= 0
...

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[Richard Miller]

ron minnich <rminnich@gmail.com>:
> I used to think 10 years was a hugely long time to get
> changes into the world, now I'm thinking 10 years is remarkably quick,
> and 20 is more the rule, unless you hit the right idea at just the
> right time. Which means each of us has maybe 4 chances in life to
> really push a change into the world.

Possibly a few more if you pipeline them ...

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[erik quanstrom]

> Speaking of device numbers, I was surprised that Plan 9 has a similar
> notion.  However, they are only useful with kernel resident device
> numbers.  Does Plan 9 have some other mechanism that allow one to
> identify the class of device/file server it belongs to?
>
> In most cases, a name is good enough, but sometimes its not.

no.  most stereotypical drivers are written in the miniport
style.  (please excuse the reference.)  where one bag of generic
code provides the interface, so one interacts with all ethernet,
serial or storage devices in the same way.

> And speaking of opaqueness, while Plan 9 might use portable data
> representation, some interfaces are perhaps unnecessarily opaque in a
> different way.  Consider the uart ctl file with its encoded string of
> device attributes.

what kind of abstract manipuation of serial devices could you imagine
would be useful?

> Oh, and it would be better if the uart device was consistent with the
> files in net with their separate data and ctl files, if one is going
> to rely on conventions they should be consistent.  Despite the other
> faults of ioctl there's no doubt where the control interface of a
> device is.

but it does.  there's eia0status, eia0ctl and eia0 (the data file).
; ls '#t'
'#t/eia0'
'#t/eia0ctl'
'#t/eia0status'
'#t/eia1'
'#t/eia1ctl'
'#t/eia1status'

- erik

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[rbnsw-plan9]

--- On Tue, 28/6/11, erik quanstrom <quanstro@quanstro.net> wrote:

>> Speaking of device numbers, I was surprised that Plan 9 has a similar
>> notion.  However, they are only useful with kernel resident device
>> numbers.  Does Plan 9 have some other mechanism that allow one to
>> identify the class of device/file server it belongs to?
>> 
>> In most cases, a name is good enough, but sometimes its not.
> 
> no.  most stereotypical drivers are written in the
> miniport style.  (please excuse the reference.) where one
> bag of generic code provides the interface, so one interacts with all
> ethernet, serial or storage devices in the same way.

Ok, then I misunderstand what the stat man page means by "Type identifies the server type,  and dev says which of a group of servers of the same type is the one responsible for this file"  Code that relies on testing the server type is bad style anyway, its better if the server can supply a bunch of predicates so a client can test what operations a server supporta, a client should not mistake one type of device for another one. After all, an ethernet doesn't provide the same level of persistence a storage device does!
> 
>> And speaking of opaqueness, while Plan 9 might use portable data
>> representations, some interfaces are perhaps unnecessarily opaque in a
>> different way.  Consider the uart ctl file with its encoded string of
>> device attributes.
> 
> what kind of abstract manipulation of serial devices could
> you imagine would be useful?

I wasn't motivated about manipulating serial devices per se, I was thinking about reflective interfaces and programs that use metadata to learn about the behaviour of interfaces and provide user friendly UIs.
Having separate files for each attributes makes that a little easier. A separate hierarchy mirroring that of the ctl directory might return the allowable values/syntax for attributes/operations.

Serial devices are not that interesting for abstract manipulation but for the sake of the argument: we can classify the comms devices into virtual circuit based and datagram based. The VC based into permanent and switched VC (PVC and SVC). And we can classify VCs into byte oriented (stream) and record oriented. We probably should have QOS attributes too, A serial line can be considered a byte oriented PVC with indeterminate or perhaps configurable QOS. If we include serial lines as a type of PVC, we better know if PVCs are 8 bit clean - alternatively perhaps only 8 bit clean devices can be PVCs. There's nothing here about manipulation really, PVCs don't have much scope for that, but stay tuned, so far this is more about attributes that allow clients to select the comms medium they're interested in. Speaking of which, some optional attributes that make the implicit local and remote addresses/identifier explicit as well as the implied upper level protocol
 are useful too. Most of these are read only attributes too, but some will be read-write so they can have their values set by administrators.

So what can be manipulated? Flow control is something common to many virtual circuits including serial lines. Uart allow input and output queue lengths to be set which I assume affect flow control/blocking. TCP has windows which provide the same affect, *nixes allows them to be indirectly controlled by setting the send and receive buffer sizes via setsockopt or configured system wide, but Plan 9 doesn't seem to offer this facility.

However, if one adds a modem to a serial line or the end of any PVC for that matter we can treat it like a SVC, in much the same way we treat TCP. Apart from QOS the only real difference is that the number of SVCs has a much lower limit than TCP.
> 
>> Oh, and it would be better if the uart device was consistent with the
>> files in net with their separate data and ctl files, if one is going
>> to rely on conventions they should be consistent.  Despite the other
>> faults of ioctl there's no doubt where the control interface of a
>> device is.
> 
> but it does.  there's eia0status, eia0ctl and eia0 (the data file).
> ; ls '#t'
> '#t/eia0'
> '#t/eia0ctl'
> '#t/eia0status'
> '#t/eia1'
> '#t/eia1ctl'
> '#t/eia1status'
> 

Sorry, I meant structured like the files in net. So, the uart could instead by structured as:

eia/
    0/
       ctl
       data
       status
    1/
       ctl
       data
       status

there could also be clone entry directly under eia if we wished to have an allocation scheme if the various serial lines are equivalent, e.g. attached to modems and of course a stats interface too.

Regards,

Andrew
> - erik
>

Re: [9fans] RFS alternatives (Was: Living with Plan 9)

[erik quanstrom]

> Sorry, I meant structured like the files in net.  So, the uart could
> instead by structured as:
>
> eia/
>     0/
>        ctl data status
>     1/
>        ctl data status

usb serial does this, but unfortunately with a different naming
convention, and it's unfortunate since usb serial is then
not compatable with regular serial, or with the network style.

- erik