The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* Re: [TUHS] VFS prior to 1984
@ 2020-06-23 22:17 Norman Wilson
  2020-06-23 22:24 ` Dave Horsfall
  0 siblings, 1 reply; 56+ messages in thread
From: Norman Wilson @ 2020-06-23 22:17 UTC (permalink / raw)
  To: tuhs

Rob Pike:

  For my taste, the various Unix file system switches that I've seen are too
  firmly tied to the idea of blocks and disks and all that, making them less
  flexible than they should have been. That's why the Plan 9 version is about
  names and byte streams, to make it as general as possible.

=====

The only file-system switch I know well is that in the later
Research systems.  It has nothing to do with disks or blocks.
It is about names and inodes: turn name to struct inode, read
arbitrary chunk of data, write arbitrary chunk of data, create
file, unlink file, abominable ill-defined system call that just
wouldn't die erm I mean ioctl, and so on.

It was certainly a cheap hack (as its author cheerfully admitted)
but it really was about I/O operations, not about simulating
disks.  That's why it so easily supported /proc (the one that
was just about processes, not the misnamed Linux one that
is really about all things in the kernel so you don't need
/dev/kmem).

I don't know a lot about later VFSes like those in SunOS or
Linux or the BSDs.  Blockiness might well have crept in in
support of memory-mapped I/O or in collaboration with the
buffer-cache implementation.  The Research version was never
used directly to support different on-disk file system formats
(we did a different pjw cheap hack for the one case that
mattered in the kernel, and used the idea later developed
more by FUSE for other cases because speed didn't matter).

Norman Wilson
Toronto ON

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 22:17 [TUHS] VFS prior to 1984 Norman Wilson
@ 2020-06-23 22:24 ` Dave Horsfall
  2020-06-24  4:09   ` Warner Losh
  0 siblings, 1 reply; 56+ messages in thread
From: Dave Horsfall @ 2020-06-23 22:24 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 23 Jun 2020, Norman Wilson wrote:

> The only file-system switch I know well is that in the later Research 
> systems.  It has nothing to do with disks or blocks. It is about names 
> and inodes: turn name to struct inode, read arbitrary chunk of data, 
> write arbitrary chunk of data, create file, unlink file, abominable 
> ill-defined system call that just wouldn't die erm I mean ioctl, and so 
> on.

Ioctl: the Swiss Army knife of system calls.  I thought it was a neat idea 
when it arrived (much better then those primitive stty/gtty calls) but 
now...

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 22:24 ` Dave Horsfall
@ 2020-06-24  4:09   ` Warner Losh
  2020-06-24  5:10     ` Dave Horsfall
  0 siblings, 1 reply; 56+ messages in thread
From: Warner Losh @ 2020-06-24  4:09 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

On Tue, Jun 23, 2020, 4:25 PM Dave Horsfall <dave@horsfall.org> wrote:

> On Tue, 23 Jun 2020, Norman Wilson wrote:
>
> > The only file-system switch I know well is that in the later Research
> > systems.  It has nothing to do with disks or blocks. It is about names
> > and inodes: turn name to struct inode, read arbitrary chunk of data,
> > write arbitrary chunk of data, create file, unlink file, abominable
> > ill-defined system call that just wouldn't die erm I mean ioctl, and so
> > on.
>
> Ioctl: the Swiss Army knife of system calls.  I thought it was a neat idea
> when it arrived (much better then those primitive stty/gtty calls) but
> now...
>

More like the swiss army chainsaw with engine trouble and a dull blade....

Warner


-- Dave
>

[-- Attachment #2: Type: text/html, Size: 1411 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24  4:09   ` Warner Losh
@ 2020-06-24  5:10     ` Dave Horsfall
  2020-06-24  5:33       ` Bakul Shah
  0 siblings, 1 reply; 56+ messages in thread
From: Dave Horsfall @ 2020-06-24  5:10 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

On Tue, 23 Jun 2020, Warner Losh wrote:

> > Ioctl: the Swiss Army knife of system calls.  I thought it was a neat 
> > idea when it arrived (much better then those primitive stty/gtty 
> > calls) but now...
> 
> More like the swiss army chainsaw with engine trouble and a dull 
> blade....

(Slowly drifting off-topic (as all mailing lists do) so perhaps a new 
thread would be better.)

Love it!  On the other hand though, if an extra service is required of the 
OS and you don't want to extend ioctl(), then that means Yet Another 
System Call (tm).  I really like the idea of /proc, so if there's an 
equivalent for ioctl() then I'd love to see it; /sys would be an obvious 
choice, but it's already taken (Harris' Lament).

No doubt, someone will now chime in and say that there's an existing 
mechanism involving shared memory or something...  My Unix-fu is getting 
rather long in the tooth now (just like me).

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24  5:10     ` Dave Horsfall
@ 2020-06-24  5:33       ` Bakul Shah
  2020-06-24  9:10         ` Andrew Warkentin
  0 siblings, 1 reply; 56+ messages in thread
From: Bakul Shah @ 2020-06-24  5:33 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Jun 23, 2020, at 10:10 PM, Dave Horsfall <dave@horsfall.org> wrote:
> 
> On Tue, 23 Jun 2020, Warner Losh wrote:
> 
>> > Ioctl: the Swiss Army knife of system calls.  I thought it was a neat > idea when it arrived (much better then those primitive stty/gtty > calls) but now...
>> More like the swiss army chainsaw with engine trouble and a dull blade....
> 
> (Slowly drifting off-topic (as all mailing lists do) so perhaps a new thread would be better.)
> 
> Love it!  On the other hand though, if an extra service is required of the OS and you don't want to extend ioctl(), then that means Yet Another System Call (tm).  I really like the idea of /proc, so if there's an equivalent for ioctl() then I'd love to see it; /sys would be an obvious choice, but it's already taken (Harris' Lament).
> 
> No doubt, someone will now chime in and say that there's an existing mechanism involving shared memory or something...  My Unix-fu is getting rather long in the tooth now (just like me).

plan9 has ctl devices that accept commands as ascii strings and mostly
return results as ascii strings too. Very handy. For example if you have
a optical disk drive you can send commands via its ctl device. There is
no technical reason why this model can't be adapted in Unix.

The fact is that the Unix byte/block read/write model goes only so far.
Every device has its own peculiarities that need to be queried or
controlled. And then there are controllers or USB device treees where
you can hang useful devices at varius places. Bottom line: while dealing
with real devices you can push this complexity around. You can't make it
disappear. At one point a friend and I were looking at building a
product using plan9 and one of the things I wanted to do was to make
sure that the "ctl" file for every device responded to a "help" message
that yielded one line per command and "help <command>" for more details.
That way you can discover their capabilities at runtime.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24  5:33       ` Bakul Shah
@ 2020-06-24  9:10         ` Andrew Warkentin
  0 siblings, 0 replies; 56+ messages in thread
From: Andrew Warkentin @ 2020-06-24  9:10 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On 6/23/20, Bakul Shah <bakul@iitbombay.org> wrote:
>
> The fact is that the Unix byte/block read/write model goes only so far.
> Every device has its own peculiarities that need to be queried or
> controlled. And then there are controllers or USB device treees where
> you can hang useful devices at varius places. Bottom line: while dealing
> with real devices you can push this complexity around. You can't make it
> disappear. At one point a friend and I were looking at building a
> product using plan9 and one of the things I wanted to do was to make
> sure that the "ctl" file for every device responded to a "help" message
> that yielded one line per command and "help <command>" for more details.
> That way you can discover their capabilities at runtime.
>

Yes, complexity is unavoidable. IMO, the reason for going with a
purely file-oriented architecture isn't to make all user-visible APIs
use normal file calls, even though they are all going to be
implemented in terms of files (there's definitely a place for
libraries that provide a friendlier API on top of the file-based
transport). "Everything is a file" should not mean "all APIs are
visibly file-based".

The real advantages of a pure file-oriented architecture I can think
of off the top of my head are basically:

All security for all services in the system is enforced through a
single mechanism (rather than having many security primitives like
modern conventional OSes have)
On a microkernel, a purely stream-oriented IPC transport layer will
have better performance when dealing with bulk unstructured data than
a structured RPC transport layer with dynamic marshalling invoked on
every message (for servers that need RPC with dynamic marshalling, the
transport layer can easily support a type of file that preserves
message boundaries but otherwise has standard Unix semantics)
A purely stream-oriented IPC transport layer is simpler than one with
dynamic marshalling and has less attack surface
All services in the system can be extended, overridden, and exported
through the same mechanism (this includes making all parts of a purely
file-oriented OS network transparent for  basically free)
In some cases a higher-level service that extends another one doesn't
have to fully intercept all parts of the underlying interface (e.g. if
the underlying interface has multiple files and only some need to be
overridden, or possibly if some messages are treated as opaque)

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-06 16:51           ` Chris Torek
@ 2020-07-07  1:07             ` Bakul Shah
  0 siblings, 0 replies; 56+ messages in thread
From: Bakul Shah @ 2020-07-07  1:07 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Jul 6, 2020, at 9:51 AM, Chris Torek <torek@torek.net> wrote:
> 
>> On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
>> We found a bunch of those because computers used to be ECC or parity and
>> then 15 years ago or so, they just dumped the parity bit so single bit
>> errors go unreported (noice, computer industry).
> 
> To be fair, there are still some ECC systems.  Unfortunately most
> of the home-use Intel boxes aren't.  My own home-use box isn't and
> (now 6+ years ago) I had bad RAM in it that produced single-bit
> errors in an inode block that led to panics that led to me finding
> the single-bit errors, but I don't know if there are some damaged
> files.
> 
> I keep thinking I'll replace it with a new box that does have ECC,
> but haven't gotten around to it yet.  I see some consumer-priced
> AMD CPUs have at least theoretical ECC support but I haven't found
> anything that says the ECC actually works, and have seen a few
> articles that hint that it doesn't.  iX sells NAS boxes that do
> have ECC, though.

For my fileserver I am using a Gigabyte X470 Aorus Gaming board
with "qualified" ECC RAM (dmidecode confirms this) but I am not
sure if FreeBSD has ECC support for such boards. At least I don't
know how to tell!

What I really want is a laptop with ECC to use as the fileserver
to avoid having to deal with UPS that lasts a few minutes, cabling
and suboptimal support from NUT. SSDs are fast enough for me.
Laptop batteries last long and are easy to replace and easy to
test when to shutdown the laptop. And less heat and space!


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05  0:16         ` Larry McVoy
  2020-07-06  4:42           ` Dave Horsfall
@ 2020-07-06 16:51           ` Chris Torek
  2020-07-07  1:07             ` Bakul Shah
  1 sibling, 1 reply; 56+ messages in thread
From: Chris Torek @ 2020-07-06 16:51 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

>On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
>We found a bunch of those because computers used to be ECC or parity and
>then 15 years ago or so, they just dumped the parity bit so single bit
>errors go unreported (noice, computer industry).

To be fair, there are still some ECC systems.  Unfortunately most
of the home-use Intel boxes aren't.  My own home-use box isn't and
(now 6+ years ago) I had bad RAM in it that produced single-bit
errors in an inode block that led to panics that led to me finding
the single-bit errors, but I don't know if there are some damaged
files.

I keep thinking I'll replace it with a new box that does have ECC,
but haven't gotten around to it yet.  I see some consumer-priced
AMD CPUs have at least theoretical ECC support but I haven't found
anything that says the ECC actually works, and have seen a few
articles that hint that it doesn't.  iX sells NAS boxes that do
have ECC, though.

Chris

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05  0:16         ` Larry McVoy
@ 2020-07-06  4:42           ` Dave Horsfall
  2020-07-06 16:51           ` Chris Torek
  1 sibling, 0 replies; 56+ messages in thread
From: Dave Horsfall @ 2020-07-06  4:42 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, 4 Jul 2020, Larry McVoy wrote:

>> Aren't holes part of the file system semantics?
>
> I'm not talking about legit holes, I'm talking about where your data 
> used be served up as a list of zeros.

Ah; my mistake...  Too much blood in my coffee stream.

> The SCCS checksum is weak but kinda handy.  You could see single bit 
> errors with it (at least you could in BitKeeper).  [...]

I used to edit SCCS files if I stuffed up an update (and didn't want to 
make another update); I discovered that if I zeroed the checksum then it 
would be recalculated...  I have no idea what happened should the 
"genuine" checksum turn out to be zero.

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05 21:04                 ` Clem Cole
@ 2020-07-05 21:14                   ` Dan Cross
  0 siblings, 0 replies; 56+ messages in thread
From: Dan Cross @ 2020-07-05 21:14 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1735 bytes --]

On Sun, Jul 5, 2020 at 5:06 PM Clem Cole <clemc@ccc.com> wrote:

> On Sun, Jul 5, 2020 at 4:42 PM John Cowan <cowan@ccil.org> wrote:
>
>> I always used the design principle "Write locally, read over NFS".
>>
> This was the basic idea of AFS.  Originally, the CMU folks did whole file
> caching, but by AFS 4.0 time, they had a Locus token manager (think DLM)
> that scaled really well so partial caching was allowed.  It actually made a
> small disk system possible.  What tended to happen, on your first boot, of
> course, you had to fill /bin and lot of heavily used directories.   But
> what happened is that your system quickly had only the files you really
> needed on the local disk. - the ones you were writing, and the few you
> used over and over.
>
> FWIW: I know a couple of people that still run it.  I ran it until a few
> years ago when I switched NAS units just for cost reasons.
>

There was a neat paper out of CERN a few years ago about how they're
turning down their AFS (now OpenAFS) cells.
https://iopscience.iop.org/article/10.1088/1742-6596/898/6/062040/pdf

It seems that the idea of a big, shared, distributed file namespace is
sadly disappearing. I feel like most of the web-based replacements are not
as seamlessly integrated with my preferred toolset as what they're
replacing, but I've also become more and more acutely aware that I am not
the target audience for those things.

Certainly, real-time collaboration via e.g. Google Drive is pretty amazing
and very dynamic, particularly when paired with e.g. real-time video chat,
but it also forces one into a particular model of interaction that I've
spent most of the last three decades consciously avoiding but now find no
escape from.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2855 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05 20:42               ` John Cowan
@ 2020-07-05 21:04                 ` Clem Cole
  2020-07-05 21:14                   ` Dan Cross
  0 siblings, 1 reply; 56+ messages in thread
From: Clem Cole @ 2020-07-05 21:04 UTC (permalink / raw)
  To: John Cowan; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On Sun, Jul 5, 2020 at 4:42 PM John Cowan <cowan@ccil.org> wrote:

> I always used the design principle "Write locally, read over NFS".
>
This was the basic idea of AFS.  Originally, the CMU folks did whole file
caching, but by AFS 4.0 time, they had a Locus token manager (think DLM)
that scaled really well so partial caching was allowed.  It actually made a
small disk system possible.  What tended to happen, on your first boot, of
course, you had to fill /bin and lot of heavily used directories.   But
what happened is that your system quickly had only the files you really
needed on the local disk. - the ones you were writing, and the few you used
over and over.

FWIW: I know a couple of people that still run it.  I ran it until a few
years ago when I switched NAS units just for cost reasons.

[-- Attachment #2: Type: text/html, Size: 1585 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05 20:08             ` Clem Cole
@ 2020-07-05 20:42               ` John Cowan
  2020-07-05 21:04                 ` Clem Cole
  0 siblings, 1 reply; 56+ messages in thread
From: John Cowan @ 2020-07-05 20:42 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1802 bytes --]

I always used the design principle "Write locally, read over NFS".  This
obviated locking issues and fit in with the idea of fate-sharing: a write
would always succeed, even if reading would have to wait until R (the
machine doing the reading) was up.  The only additional thing I needed was
the ability for W (the machine doing the writing) to notify R that
something had changed, which I did by having R run a process that listened
on a port that would be opened and then closed by W: no data flowed over
this connection.  If this connection could not be made, the process on the
W side would loop in bounded exponential backoff.

On Sun, Jul 5, 2020 at 4:09 PM Clem Cole <clemc@ccc.com> wrote:

>
>
> On Sun, Jul 5, 2020 at 10:43 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> My guess is that other people didn't understand the "rules" and did
>> things that created problems.  Sun's clients did understand and did
>> not push NFS in ways that would break it.
>
> I >>believe<< that a difference was file I/O was based on mmap on SunOS
> and not on other systems (don't know about Solaris).   The error was
> handled by the OS memory system.  You tell me about how SGI handled I/O.
> Tru64 used mmap and I think macOS does also from the Mach heritage.
>  RTU/Ultrix was traditional BSD.  Stellix was SRV3.  Both had a file
> system cache with write-behind.
>
> I never knew for sure, but I always suspected that was crux of the
> difference in how/where the write failure were handled.  But as you
> pointed out, many production NFS sites not running Suns had huge problems
> with holes in files that were not discovered until it was too late to fix
> them.  SCCS/RCS repositories were particularly suspect and because people
> tried to use them for shared development areas, it could be a big issue.
>

[-- Attachment #2: Type: text/html, Size: 3290 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05 14:43           ` Larry McVoy
  2020-07-05 18:40             ` Arthur Krewat
@ 2020-07-05 20:08             ` Clem Cole
  2020-07-05 20:42               ` John Cowan
  1 sibling, 1 reply; 56+ messages in thread
From: Clem Cole @ 2020-07-05 20:08 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1067 bytes --]

On Sun, Jul 5, 2020 at 10:43 AM Larry McVoy <lm@mcvoy.com> wrote:

> My guess is that other people didn't understand the "rules" and did
> things that created problems.  Sun's clients did understand and did
> not push NFS in ways that would break it.

I >>believe<< that a difference was file I/O was based on mmap on SunOS and
not on other systems (don't know about Solaris).   The error was handled by
the OS memory system.  You tell me about how SGI handled I/O.  Tru64 used
mmap and I think macOS does also from the Mach heritage.   RTU/Ultrix was
traditional BSD.  Stellix was SRV3.  Both had a file system cache with
write-behind.

I never knew for sure, but I always suspected that was crux of the
difference in how/where the write failure were handled.  But as you pointed
out, many production NFS sites not running Suns had huge problems with
holes in files that were not discovered until it was too late to fix them.
SCCS/RCS repositories were particularly suspect and because people tried to
use them for shared development areas, it could be a big issue.

[-- Attachment #2: Type: text/html, Size: 2287 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05 14:43           ` Larry McVoy
@ 2020-07-05 18:40             ` Arthur Krewat
  2020-07-05 20:08             ` Clem Cole
  1 sibling, 0 replies; 56+ messages in thread
From: Arthur Krewat @ 2020-07-05 18:40 UTC (permalink / raw)
  To: tuhs

On 7/5/2020 10:43 AM, Larry McVoy wrote:
> So I've encountered lots of holes in NFS files where there shouldn't be
> any.  So it is/was a thing.  But that said, I can't remember a single
> case of encountering that on Sun's campus.  I don't know if my memory
> is failing me, but I do know that when I left Sun and started working
> with other NFS implementations, yeah, lots of problems.  Somehow Sun
> got it right where other people didn't.

I can say personally, since the early 90's on SunOS, I never ever saw 
this problem in a variety of environments. One being Nynex Science and 
Technology where I did a consulting stint. 800+ node Sun 3/4 SunOS 
workstation/server environment, basically everyone's desktop was a Sun 
workstation for email, documentation, whatever. Another being a defense 
contractor I was at for 7 years, they were all Sun for engineering 
workstations and servers.

There is one possibility I just thought of, and that's if the first 
write fails and then a context switch happens, if enough free space is 
made available before the next context switch back to the second write, 
I can see that being a problem ;)

As if you weren't already tired of my rambling... When it comes to 
non-Sun operating systems, all bets were off. They all (mostly) worked 
with their own kind. That usually wasn't the case when it came to 
cross-vendor support. Sun<->HP was not great, but that also may have 
been driver problems in the one instance I tried it for an extended 
period of time. Two instances at different customers of AIX<->Sun 
actually worked rather well. The YP integration was key.

The entire problem with "holey" files and NFS is certainly related to 
the usage type of the system in question. What was Sun doing? Email? 
Software development? Using a common NFS share for the compiler? And 
then copying their code up to a central location? Not a lot of 
sync/write/sync/write activity, unless object file generation is a lot 
of skipping around all over the place. ;)

art k.


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05  1:43         ` Clem Cole
@ 2020-07-05 14:43           ` Larry McVoy
  2020-07-05 18:40             ` Arthur Krewat
  2020-07-05 20:08             ` Clem Cole
  0 siblings, 2 replies; 56+ messages in thread
From: Larry McVoy @ 2020-07-05 14:43 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

On Sat, Jul 04, 2020 at 09:43:04PM -0400, Clem Cole wrote:
> On Sat, Jul 4, 2020 at 8:07 PM Dave Horsfall <dave@horsfall.org> wrote:
> 
> > Aren't holes part of the file system semantics?
> >
> Exactly - and that was the problem with NFS.   Consider two write
> operations.  Remember each op is a complete operation with a seek to where
> it's going.  If the first fails, but the error is not reported (NFS returns
> errors on close), the second operations seek over the failed write -- UNIX
> puts zeros in the file.   File closes later and the size is fine of
> course.  Oh yeah, whoever bothered to check for errors on close (like the
> traditional SCCS or RCS commands)?
> 
> Later to try to read your file back -- it will have a bunch of zeros.

So I've encountered lots of holes in NFS files where there shouldn't be
any.  So it is/was a thing.  But that said, I can't remember a single
case of encountering that on Sun's campus.  I don't know if my memory
is failing me, but I do know that when I left Sun and started working
with other NFS implementations, yeah, lots of problems.  Somehow Sun
got it right where other people didn't.  

The point I'm trying to make is that I don't think NFS was broken by
design, it worked when it was Sun servers and Sun clients.  Sun's 
entire campus, 10's of thousands of machines, used NFS.  Sun would
have screeched to a halt if NFS didn't work reliably all the time.
So it was possible to get it to work.

My guess is that other people didn't understand the "rules" and did
things that created problems.  Sun's clients did understand and did
not push NFS in ways that would break it.

My memory may not be the greatest but I can still remember being 
astonished when I first ran into people saying NFS didn't work.
It worked great for Sun and Sun's customers.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05  0:05       ` Dave Horsfall
  2020-07-05  0:16         ` Larry McVoy
@ 2020-07-05  1:43         ` Clem Cole
  2020-07-05 14:43           ` Larry McVoy
  1 sibling, 1 reply; 56+ messages in thread
From: Clem Cole @ 2020-07-05  1:43 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

On Sat, Jul 4, 2020 at 8:07 PM Dave Horsfall <dave@horsfall.org> wrote:

> Aren't holes part of the file system semantics?
>
Exactly - and that was the problem with NFS.   Consider two write
operations.  Remember each op is a complete operation with a seek to where
it's going.  If the first fails, but the error is not reported (NFS returns
errors on close), the second operations seek over the failed write -- UNIX
puts zeros in the file.   File closes later and the size is fine of
course.  Oh yeah, whoever bothered to check for errors on close (like the
traditional SCCS or RCS commands)?

Later to try to read your file back -- it will have a bunch of zeros.
As Larry says, running a simple checksum could catch a lot of these.

Anyway, I'm going to be good and lay off a diatribe on NFS.  It sort of
worked 'good enough.'   But I will say other systems (like AFS) were much
better, in practice, but it lost the war.
Clem

[-- Attachment #2: Type: text/html, Size: 2056 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-07-05  0:05       ` Dave Horsfall
@ 2020-07-05  0:16         ` Larry McVoy
  2020-07-06  4:42           ` Dave Horsfall
  2020-07-06 16:51           ` Chris Torek
  2020-07-05  1:43         ` Clem Cole
  1 sibling, 2 replies; 56+ messages in thread
From: Larry McVoy @ 2020-07-05  0:16 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
> On Wed, 24 Jun 2020, Larry McVoy wrote:
> 
> >>In the end, early NFS was notorious for putting 'holes' in the files
> >>because of the automatic seek in every operation and errors not coming
> >>until close(2) time.
> >
> >You have no idea how many of those holes that 16 bit SCCS checksum has
> >found (BitKeeper kept it).
> 
> Aren't holes part of the file system semantics?  

I'm not talking about legit holes, I'm talking about where your data used
be served up as a list of zeros.

The SCCS checksum is weak but kinda handy.  You could see single bit errors
with it (at least you could in BitKeeper).  It would say 

checksum error: wanted %d, got %d

and you could look at the numbers and see that it was one bit off.

We found a bunch of those because computers used to be ECC or parity and
then 15 years ago or so, they just dumped the parity bit so single bit
errors go unreported (noice, computer industry).

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 19:36     ` Larry McVoy
  2020-06-25  6:52       ` Rob Gingell
@ 2020-07-05  0:05       ` Dave Horsfall
  2020-07-05  0:16         ` Larry McVoy
  2020-07-05  1:43         ` Clem Cole
  1 sibling, 2 replies; 56+ messages in thread
From: Dave Horsfall @ 2020-07-05  0:05 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Wed, 24 Jun 2020, Larry McVoy wrote:

>> In the end, early NFS was notorious for putting 'holes' in the files 
>> because of the automatic seek in every operation and errors not coming 
>> until close(2) time.
>
> You have no idea how many of those holes that 16 bit SCCS checksum has 
> found (BitKeeper kept it).

Aren't holes part of the file system semantics?  Seek beyond EOF, write a 
block, and you've created unallocated holes.  Of course, they are filled 
with zeroes as soon as you copy the file...  I did fiddle with "cp" to 
detect those holes and preserve them, but gave up for some reason 
(performance hit in checking for all-zero blocks, I think, along with some 
weird problem with the last block being all zeroes).

I used NFS when it first appeared and got bitten quite badly; I've never 
really trusted it ever since, but I understand that it's vastly improved.

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-29  9:11 Paul Ruizendaal
  2020-06-29 14:45 ` Larry McVoy
@ 2020-06-29 16:34 ` Heinz Lycklama
  1 sibling, 0 replies; 56+ messages in thread
From: Heinz Lycklama @ 2020-06-29 16:34 UTC (permalink / raw)
  To: tuhs

We implemented filesystems as a supervisor process so that it
could support file access from both a UNIX supervisor process
and an RSX/11 supervisor process in the MERT operating system.
See Figure 1 in the MERT paper in the July/August 1978 BSTJ.

Heinz

On 6/29/2020 2:11 AM, Paul Ruizendaal wrote:
> Date: Wed, 24 Jun 2020 14:31:34 -0400 (EDT)
> From: norman@oclsc.org (Norman Wilson)
>> Reaching outside of UNIX, RSX/11 used external supervisor-mode processes called ACPs (ancillary control processes) to implement file systems. I don't know exactly how they were plugged in, but I do know they were pluggable, so their interface must have constituted a file-system switch of some sort. RSX dates back into the 1970s. At some point in the latter part of the 1980s, Ralph Stamerjohn (a name instantly recognizable in the 16-bit DEC software world) gave a DECUS talk about implementing a remote file system through ACPs: a stub ACP on the client exporting RPCs over the network, a real one at the server end. I remember chatting with him about how that did and didn't resemble the way pjw had done it; interesting architectural comparison.
>> Norman Wilson Toronto ON
> I am still digesting all the inputs (thanks, all!)
>
> The above post made me realise that the delineation of what is a FSS/VFS or not, is not so easy.
>
> I did a little bit of reading, and the concept of an ACP arrived with RSX11D in May 1973, but only matured in RSX11M in November 1974. As I understand it, originally in RSX11 file system code was closely tied to the low-level device driver for each device. ACP’s separated the file system code from the device driver itself, and became separate processes.
>
> In essence there were two switches: one switch into abstract devices, implemented in ACP code and one kernel switch to deal with hardware interfacing. The first is indeed like a file system switch (although still tied to specific devices).
>
> Looking at this stuff made me realise that my retro machine of choice (the TI990) went through a similar evolution. In the early seventies it had a sort of abstract device switch that linked to individual ‘device service routines’ (drivers). Initially, these modelled batch oriented ‘logical units’ that tied to files at the job control level. Later (late 70’s), the ‘open’ command would carry a file name and the file system was delegated to the device service routine. Still later (say 1983) this was used for networked disks.
>
> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch.
>
>
>
>


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-29 14:53   ` Paul Ruizendaal
@ 2020-06-29 15:14     ` Larry McVoy
  0 siblings, 0 replies; 56+ messages in thread
From: Larry McVoy @ 2020-06-29 15:14 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

On Mon, Jun 29, 2020 at 04:53:50PM +0200, Paul Ruizendaal wrote:
> 
> > On 29 Jun 2020, at 16:45, Larry McVoy <lm@mcvoy.com> wrote:
> > 
> > On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote:
> >> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch.
> > 
> > Yep.  If you think in objects, they are both objects with a set of methods.
> > Pretty much two instances of the same idea.
> 
> Phrased that way all OO programs have a close relationship between them. Maybe: "they are both rather similar objects with a rather similar set of methods"?

They are two different classes, similar, but not the same.  It's really
4 classes, vfs, vnode for FS and block/char for devices.  This is from
SunOS 4.1.1 that I happened to have an install image sitting around.
Took me forever to find the device definitions, I resorted to

find . -type f -name '*.h' -print | xargs grep strategy

It's in include/sys/conf.h because, well, that's obvious, right?

struct vfsops {
        int     (*vfs_mount)();         /* mount file system */
        int     (*vfs_unmount)();       /* unmount file system */
        int     (*vfs_root)();          /* get root vnode */
        int     (*vfs_statfs)();        /* get fs statistics */
        int     (*vfs_sync)();          /* flush fs buffers */
        int     (*vfs_vget)();          /* get vnode from fid */
        int     (*vfs_mountroot)();     /* mount the root filesystem */
        int     (*vfs_swapvp)();        /* return vnode for swap */
};

struct vnodeops {
        int     (*vn_open)();
        int     (*vn_close)();
        int     (*vn_rdwr)();
        int     (*vn_ioctl)();
        int     (*vn_select)();
        int     (*vn_getattr)();
        int     (*vn_setattr)();
        int     (*vn_access)();
        int     (*vn_lookup)();
        int     (*vn_create)();
        int     (*vn_remove)();
        int     (*vn_link)();
        int     (*vn_rename)();
        int     (*vn_mkdir)();
        int     (*vn_rmdir)();
        int     (*vn_readdir)();
        int     (*vn_symlink)();
        int     (*vn_readlink)();
        int     (*vn_fsync)();
        int     (*vn_inactive)();
        int     (*vn_lockctl)();
        int     (*vn_fid)();
        int     (*vn_getpage)();
        int     (*vn_putpage)();
        int     (*vn_map)();
        int     (*vn_dump)();
        int     (*vn_cmp)();
        int     (*vn_realvp)();
        int     (*vn_cntl)();
};

/* two different device classes, block and char */
struct bdevsw {
        int     (*d_open)();
        int     (*d_close)();
        int     (*d_strategy)();
        int     (*d_dump)();
        int     (*d_psize)();
        int     d_flags;
};

struct cdevsw {
        int     (*d_open)();
        int     (*d_close)();
        int     (*d_read)();
        int     (*d_write)();
        int     (*d_ioctl)();
        int     (*d_reset)();
        int     (*d_select)();
        int     (*d_mmap)();
        struct  streamtab *d_str;
        int     (*d_segmap)();
};

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-29 14:45 ` Larry McVoy
@ 2020-06-29 14:53   ` Paul Ruizendaal
  2020-06-29 15:14     ` Larry McVoy
  0 siblings, 1 reply; 56+ messages in thread
From: Paul Ruizendaal @ 2020-06-29 14:53 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list


> On 29 Jun 2020, at 16:45, Larry McVoy <lm@mcvoy.com> wrote:
> 
> On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote:
>> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch.
> 
> Yep.  If you think in objects, they are both objects with a set of methods.
> Pretty much two instances of the same idea.

Phrased that way all OO programs have a close relationship between them. Maybe: "they are both rather similar objects with a rather similar set of methods"?

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-29  9:11 Paul Ruizendaal
@ 2020-06-29 14:45 ` Larry McVoy
  2020-06-29 14:53   ` Paul Ruizendaal
  2020-06-29 16:34 ` Heinz Lycklama
  1 sibling, 1 reply; 56+ messages in thread
From: Larry McVoy @ 2020-06-29 14:45 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote:
> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch.

Yep.  If you think in objects, they are both objects with a set of methods.
Pretty much two instances of the same idea.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-29  9:11 Paul Ruizendaal
  2020-06-29 14:45 ` Larry McVoy
  2020-06-29 16:34 ` Heinz Lycklama
  0 siblings, 2 replies; 56+ messages in thread
From: Paul Ruizendaal @ 2020-06-29  9:11 UTC (permalink / raw)
  To: TUHS main list

Date: Wed, 24 Jun 2020 14:31:34 -0400 (EDT)
From: norman@oclsc.org (Norman Wilson)
> Reaching outside of UNIX, RSX/11 used external supervisor-mode processes called ACPs (ancillary control processes) to implement file systems. I don't know exactly how they were plugged in, but I do know they were pluggable, so their interface must have constituted a file-system switch of some sort. RSX dates back into the 1970s. At some point in the latter part of the 1980s, Ralph Stamerjohn (a name instantly recognizable in the 16-bit DEC software world) gave a DECUS talk about implementing a remote file system through ACPs: a stub ACP on the client exporting RPCs over the network, a real one at the server end. I remember chatting with him about how that did and didn't resemble the way pjw had done it; interesting architectural comparison.
> Norman Wilson Toronto ON

I am still digesting all the inputs (thanks, all!)

The above post made me realise that the delineation of what is a FSS/VFS or not, is not so easy.

I did a little bit of reading, and the concept of an ACP arrived with RSX11D in May 1973, but only matured in RSX11M in November 1974. As I understand it, originally in RSX11 file system code was closely tied to the low-level device driver for each device. ACP’s separated the file system code from the device driver itself, and became separate processes.

In essence there were two switches: one switch into abstract devices, implemented in ACP code and one kernel switch to deal with hardware interfacing. The first is indeed like a file system switch (although still tied to specific devices).

Looking at this stuff made me realise that my retro machine of choice (the TI990) went through a similar evolution. In the early seventies it had a sort of abstract device switch that linked to individual ‘device service routines’ (drivers). Initially, these modelled batch oriented ‘logical units’ that tied to files at the job control level. Later (late 70’s), the ‘open’ command would carry a file name and the file system was delegated to the device service routine. Still later (say 1983) this was used for networked disks.

As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch.





^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-26  8:34   ` Dave Horsfall
@ 2020-06-26 11:13     ` arnold
  0 siblings, 0 replies; 56+ messages in thread
From: arnold @ 2020-06-26 11:13 UTC (permalink / raw)
  To: tuhs, dave

Dave Horsfall <dave@horsfall.org> wrote:

> "talk"...  Well, at least it was a *bit* better than "write" :-)
>
> "mesg n" was your friend.

Unless you actually wanted to talk to the person on the other end.

Having it over the Internet was cool. In 1989 when my wife and I
were dating, we lived in different cities, and would occasionally
use talk to chat with each other in real time, w/o the expense of
a phone call.  It was fun. :-)

Arnold

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-26  5:49 ` Lars Brinkhoff
@ 2020-06-26  8:34   ` Dave Horsfall
  2020-06-26 11:13     ` arnold
  0 siblings, 1 reply; 56+ messages in thread
From: Dave Horsfall @ 2020-06-26  8:34 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Fri, 26 Jun 2020, Lars Brinkhoff wrote:

> RFC 441 specifies a protocol for TALK, and apparently the RSEXEC 
> protocol is involved somehow.  ITS has a server for this protocol.

"talk"...  Well, at least it was a *bit* better than "write" :-)

"mesg n" was your friend.

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-25 20:25 Noel Chiappa
@ 2020-06-26  5:49 ` Lars Brinkhoff
  2020-06-26  8:34   ` Dave Horsfall
  0 siblings, 1 reply; 56+ messages in thread
From: Lars Brinkhoff @ 2020-06-26  5:49 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

Noel Chiappa wrote:
> Neither was TENEX-specific; e.g. there were Multics implementations of
> RSEXEC and NSW.

RFC 441 specifies a protocol for TALK, and apparently the RSEXEC
protocol is involved somehow.  ITS has a server for this protocol.

https://tools.ietf.org/html/rfc441

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-25 20:25 Noel Chiappa
  2020-06-26  5:49 ` Lars Brinkhoff
  0 siblings, 1 reply; 56+ messages in thread
From: Noel Chiappa @ 2020-06-25 20:25 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Rob Gingell

    > RSEXEC (Resource Sharing EXEC) done on TENEX earlier in the 1970s.

That was associated with the National Software Works project:

  https://apps.dtic.mil/dtic/tr/fulltext/u2/a132320.pdf

I think? (It's been four decades since I looked at NSW, so I don't recall
much about it.) Neither was TENEX-specific; e.g. there were Multics
implementations of RSEXEC and NSW.

	Noel

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-25 20:23 Noel Chiappa
  0 siblings, 0 replies; 56+ messages in thread
From: Noel Chiappa @ 2020-06-25 20:23 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Richard Salz

    > A web search for "its mldev" finds several things (mostly by Lars
    > Brinkhoff it seems)

Yeah, other than the source:

  https://github.com/PDP-10/its/blob/master/src/sysen2/mldev.106

there's not a lot on it.


That's typical of a lot of the innovative work done at the AI Lab, LCS, etc;
they built some extraordinary tools, but to them they were just tools they
used to work on their research, their _real_ work. So why bother to write that
stuff up? So people who actually used them remember them, but other than
that...

Welcome to tomorrow, where everything gets re-invented, because everyone is too
busy to waste time learning about the past.

     Noel

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-25  0:45   ` Adam Thornton
@ 2020-06-25 19:40     ` Greg A. Woods
  0 siblings, 0 replies; 56+ messages in thread
From: Greg A. Woods @ 2020-06-25 19:40 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 3495 bytes --]

At Wed, 24 Jun 2020 17:45:38 -0700, Adam Thornton <athornton@gmail.com> wrote:
Subject: Re: [TUHS] VFS prior to 1984
>
> On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods <woods@robohack.ca> wrote:
> >
> > As far as I can remember Multics didn't really have the concept of a
> > "mount point".  All storage was single-level, i.e. segments (equivalent
> > in some respects to inodes, but they are also actually the value of the
> > segment register in the virtual memory hardware), and so files were
> > either physically in memory or paged out on physical disk devices or
> > similar, or even out on tape.  Where they actually resided was entirely
> > and permanently hidden from the user.  What was called the "filesystem"
> > was a form of database representing a hierarchical namespace which
> > pointed at all the known segments (files) regardless of where they were
> > actually stored.
> >
> >
> Coming to it from a Unix perspective, it's like all storage (core, disk,
> tape) is mmap()ed.
>
> The segment-name database then is just an index relating symbolic names to
> particular memory locations.

Yes, exactly.  A process is just a collection of segments of (virtual)
memory, some containing code, some containing data, and more can be
attached, some can be detached and/or swapped, as execution proceeds.
E.g. for shared code, a module can be swapped out for another equivalent
one, e.g. with a bug fix, without stopping or restarting the process
(and perhaps, if it is a system library, without the user or application
even knowing that has happened).

> It all feels very upside-down to me, but that's probably because I grew up
> in Unix and never actually used a Multics system until I emulated one with
> dps8m.

I kind of thought of it that way in my early years too, even though
Multics was one of the first systems I learned, at the same time as I
learned Unix.

In Multics I/O takes a far lesser role than it does in Unix (or most any
other OS then and since).

Modeling everything as if it were I/O has some advantages in terms of
implementation of the model (I guess it makes for far simpler operating
systems), but personally I now think it's completely upside down and
inside out from the user's perspective.

Having to read all data before you can actually access it, and/or even
just having to manage your allotted memory space and pointer offsets, is
quite an annoying addition to any algorithm -- i.e. adding I/O in front
of everything is like adding a significant problem on top of every other
problem, every time, all the time.  (Well using full-on only mmap() and
a good language and compiler could probably hide much of the nastiness,
but....  we don't really seem to do that even to this day, even in
languages that could make it easy.)

So, personally, I now see modeling everything as I/O as if it is forcing
you to access all your data through the eye of a needle, every time, all
the time.  No matter how big the hole you still have to coil all your
data up and push/pull it through if it is, or could be, bigger than the
hole.

I'm also quite sad that the extremely wide and near pervasive and
long-term availability of a CPU family with half-decent support for
segmented virtual memory has effectively been wasted for a generation or
two or more of possibilities in OS design.

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-25 19:31 Noel Chiappa
  0 siblings, 0 replies; 56+ messages in thread
From: Noel Chiappa @ 2020-06-25 19:31 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: "Greg A. Woods"

    > as Organick said in his 1972 book

A word of warning: i) Organick describes Multics as it was planned, not as it
was actually implemented; and ii) then it changed considerably during its
service life.

Examples of each: i) there was one linkage segment per processs, not per
segment (linkage info was copied across into the shared linkage segment when a
segment was dynamically added to a process' address space); ii) the New Storage
System.

The basic concepts (single level store, dynamic linking etc) are correct in
Organick, but be wary of anything past that. Not his fault, of course; things
just changed.


    > As far as I can remember Multics didn't really have the concept of a
    > "mount point".

In the original design, no. In 1973-75, with the New Storage System:

  https://multicians.org/nss.html

mountable volumes were added (see MTB 229, "Use of Demountable Logical
Volumes", linked from the page above). It was released to customers in
June, 1976.

	Noel

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 19:36     ` Larry McVoy
@ 2020-06-25  6:52       ` Rob Gingell
  2020-07-05  0:05       ` Dave Horsfall
  1 sibling, 0 replies; 56+ messages in thread
From: Rob Gingell @ 2020-06-25  6:52 UTC (permalink / raw)
  To: Larry McVoy, Clem Cole; +Cc: TUHS main list, Paul Ruizendaal

On 6/24/2020 12:36 PM, Larry McVoy wrote:
> On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote:

Larry's enumeration of NFS/VFS players was good.

I'd add Bob Lyon who led the NFS team (and who later founded Legato 
along with Rusty, Joe Moran and Jon Kepecs -- all except Bob being UW 
alumni.)

Bob came to Sun via PARC and brought with him a number of the ideas that 
Clem had referenced earlier in this thread about RPC, etc. Although he's 
on few papers his influence on defining and building NFS etc. was 
essential.

Another important contributor was David Goldberg, for building the YP 
services that, while functionally separate, enabled the administrative 
and deployment environment that helped make NFS useful. David also wrote 
the floating-point paper in the appendix to the Hennessy-Patterson 
architecture textbook.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 18:31 Norman Wilson
@ 2020-06-25  6:22 ` arnold
  0 siblings, 0 replies; 56+ messages in thread
From: arnold @ 2020-06-25  6:22 UTC (permalink / raw)
  To: tuhs

Pr1me had remote filesystems circa 1981 at least, maybe even earlier.
It was quite transparent, something equivalent to //node/path/to/file.

I only ever used Primos through the Georgia Tech Software Tools Subsytem
which made the filesystem look Unix-y. I don't know any more details.

HTH,

Arnold

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 18:31   ` Paul Ruizendaal
@ 2020-06-25  0:56     ` Rob Gingell via TUHS
  0 siblings, 0 replies; 56+ messages in thread
From: Rob Gingell via TUHS @ 2020-06-25  0:56 UTC (permalink / raw)
  To: Paul Ruizendaal, Anthony Martin, TUHS main list

On 6/24/2020 11:31 AM, Paul Ruizendaal wrote:
> I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
> https://ieeexplore.ieee.org/document/762533
> 
> The abstract says:
> 
> "RIDE (Resource-Sharing in a Distributed Environment) ...
RIDE seems very reminiscent of the RSEXEC (Resource Sharing EXEC) done 
on TENEX earlier in the 1970s. The implementation of RSEXEC involved 
encapsulation of a process fork by its parent rather than an 
object-oriented abstraction inserted behind an existing interface like 
VFS is. Both are functionally transparent to their users but are kind of 
going in opposite directions in implementation.

As with other mechanisms that have been referenced in this thread, these 
illustrate the claim that "all problems in computer science can be 
solved by [adding] another layer of indirection". I don't know that 
that's absolutely true but I've certainly applied it a lot, and like a 
lot of things, excessive application can create more problems.

(I never knew who was the first to utter the claim as I've heard it said 
by lots of people. However a quick check shows an attribution to David 
Wheeler in the form of the "Fundamental theorem of software engineering".)

https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering

RSEXEC was a pretty remarkable facility in action. It (approximately) 
was like having a command interpreter for the aggregated TENEX resources 
of the ARPAnet. It was at least partly intended to serve users of TIPs 
(the terminal interface processors used to access hosts) by letting them 
treat all TENEXes as a single resource and not worry about which one 
they got hosted onto "this time" with respect to their files, 
applications, etc., and importantly also not worry about the where their 
collaborators were hosted at the moment.

A description of RSEXEC is at:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.830.1403&rep=rep1&type=pdf

The encapsulation mechanism was JSYS Traps, JSYS being the system call 
instruction on PDP-10s modified to run TENEX. A description of that is at:

https://archive.org/details/bitsavers_bbntenexTh_1137399

In it's way, RSEXEC illustrated one characteristic of many excellent 
innovations in that when used as intended it was "deceptively 
unimpressive". As in: harder to demo.

NFS and other things share this trait, and indeed early trade show 
demonstrations of NFS were frustrating (in those days, the engineers did 
booth duty). At first, demos reduced to typing "ls". Which, 
unsurprisingly, produced a list of files. It's not until you know how it 
was done that it's impressive.

There can be a real utilitarian pleasure in things that, when shown to 
others, produce a "yeah, wasn't it supposed to do that?" reaction. Part 
of that aesthetic appreciation for me originated with RSEXEC. Which 
brings me to VFS and such.

Earlier in this thread Clem Cole observed:

On 6/23/2020 8:12 AM, Clem Cole wrote:
> That said, and to give the NFS team_a huge amount of credit _(and great 
> applause), the VFS layer was better thought out than the FSS and in fact 
> made it possible to add a lot of different file systems into UNIX 
> later.  FSS was much more ad hoc.

I think VFS and several related things are another instance of that 
aesthetic appreciation. [I'm not claiming any credit, the initial NFS 
work was just finishing up as I arrived at Sun. That work like that 
happened there was part of why I wanted to be there and so I share 
Clem's appreciation for the team.]

NFS could have been implemented by sprinkling "if (network)" statements 
around a lot of code. A user or application program would not know any 
differently had it been done that way. (Though they would over time as 
the code became less maintainable with that implementation approach.)

That VFS has the qualities leading to Clem's applause is I think due to 
a confluence of several things that likely hadn't occurred previously 
and thus entitle it to claim "first" in response to the thread-starting 
question.

One of the motivators was the desire to have a network filing 
abstraction, and to replace "nd" for diskless workstations among other 
things. (That took a while, "nd" really hung on for various reasons.) 
That's occurring in an environment of heterogeneity and "open systems" 
thinking (which is a different thing from "open source"), and the desire 
to be more "of the network" in the product offerings.

There was kind of a group-innate understanding that to be successful in 
introducing these capabilities that they had to occur largely "behind" 
the extant interfaces used by the nascent UNIX application market. To 
some extent Sun's market success was based on being deliberately "more 
vanilla than thou" with respect to APIs. The variations already extant 
were a market-fragmenting influence and thus didn't need an additional 
player doing more of it.

To be "of the network" made NFS a "file system for the Network (that 
happens to look like UNIX)". The similarity and the known semantic 
differences lead some to look at it as "it failed to do UNIX" but it's 
not. The intent was "do the network, the network is NOT 'a' computer 
(irony alert), the fallacies of networking apply and we resolve the 
irreconcilable in favor of the network".

(An intangible factor relates to the group that Sun formed, talented 
engineers, strikingly different people who shared good taste in design 
and the desire to do the work to find the "simple" in a problem. A 
different group of people with the same problem set wouldn't have 
automatically gotten the same or even similar answer, sometimes it's 
just in the air. Of course, a different group might have made an even 
better answer, "innovation happens elsewhere" is more true than the 
earlier-referenced layer of indirection principle.)

Sometimes (mostly even), good taste is expressed in framing the problem 
appropriately. The "N" in NFS is deliberately not a "D". That's obvious 
from the resulting definition but earlier in the effort it was a choice. 
The intended distinction was that we weren't doing operating system work 
in order to transcend the network by submerging it in the abstractions 
of an OS. Rather, we wanted to build a network environment populated by 
systems which were of a piece with it. While not codified until much 
later, Deutsch's "Fallacies of Networking" were the controlling notions 
in a lot of thinking and influenced things that at first wouldn't seem 
to have anything to do with networking (e.g., msync()).

However, it would be wrong to say we realized enough at the time to 
*know* that our goal was to build a system defined by a metric that it 
avoid all of the networking fallacies. We knew some of it, were 
incrementally hunting it, instincts were oriented that way. Had we been 
a research organization that might have given rise to an initiative that 
was more clearly defined. (But, I think we did a reasonable job of 
pursuing the ideas while making products that people who weren't 
thinking of such things bought. As it happens, a lot of FORTRAN users as 
has been noted here before.

Regarding the earlier irony alert: to the extent that the evocative 
marketing slogan of "The Network is the Computer" had any technical 
underpinnings, it was an expression of this aim to design in favor of 
the network. It isn't really irony so much as consequence.

The real irony is that when the slogan was first trotted out almost all 
of the company but particularly engineering barfed at it. I recall the 
marketing person(s) (individual contributors, not the executives later 
claiming credit) trying to push it receiving huge rations of abuse. But 
it stuck. And we came around once we looked beyond its technical 
deficiency as a literal statement and saw it as evocative of what we had 
been doing. They knew us better than we did. And, independent of any 
technical matter, it worked really well. 'Find the simple" is a useful 
admonition for marketing as well as engineering.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 21:33 ` Greg A. Woods
@ 2020-06-25  0:45   ` Adam Thornton
  2020-06-25 19:40     ` Greg A. Woods
  0 siblings, 1 reply; 56+ messages in thread
From: Adam Thornton @ 2020-06-25  0:45 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods <woods@robohack.ca> wrote:

>
> As far as I can remember Multics didn't really have the concept of a
> "mount point".  All storage was single-level, i.e. segments (equivalent
> in some respects to inodes, but they are also actually the value of the
> segment register in the virtual memory hardware), and so files were
> either physically in memory or paged out on physical disk devices or
> similar, or even out on tape.  Where they actually resided was entirely
> and permanently hidden from the user.  What was called the "filesystem"
> was a form of database representing a hierarchical namespace which
> pointed at all the known segments (files) regardless of where they were
> actually stored.
>
>
Coming to it from a Unix perspective, it's like all storage (core, disk,
tape) is mmap()ed.

The segment-name database then is just an index relating symbolic names to
particular memory locations.

It all feels very upside-down to me, but that's probably because I grew up
in Unix and never actually used a Multics system until I emulated one with
dps8m.

Adam

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23  9:09 Paul Ruizendaal
                   ` (2 preceding siblings ...)
  2020-06-24 20:27 ` Derek Fawcus
@ 2020-06-24 21:33 ` Greg A. Woods
  2020-06-25  0:45   ` Adam Thornton
  3 siblings, 1 reply; 56+ messages in thread
From: Greg A. Woods @ 2020-06-24 21:33 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 5575 bytes --]

At Tue, 23 Jun 2020 11:09:57 +0200, Paul Ruizendaal <pnr@planet.nl> wrote:
Subject: [TUHS] VFS prior to 1984
> 
> When googling for File System Switch or Virtual File System most
> sources mention Sun NFS and SysVr3 as the earliest
> implementations. Some sources mention 8th Edition.
> 
> I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex,
> Multics, CASS, etc.), but none of those seem to have had a comparable
> concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?

Well, with regard to Multics, it had what were called "Device Interface
Modules" (DIMs) for building device drivers to either hardware or
pseudo-devices (an example of a pseudo-DIM is the File System Interface
Module (FSIM) which associates a segment (file) in the filesystem (by
path name) with a stream, i.e. it implements I/O redirection).

DIMs had a well defined API, and as Organick said in his 1972 book:
"The DIM converts a device independent request into a device dependent
one." ... "The file-system interface DIM is used to make a segment look
like an I/O device."

A DIM also had a name, an identifier, and that's how you attached a
"stream" to it:

	call attach (stream_name, DIM_name, device_name);

All of those parameters are strings.

As an aside I haven't found any evidence that DIM identifiers were ever
listed in the filesystem, e.g. as Unix would have in "/dev", but that
would be an obvious thing to do these days.  Then they were just listed
in the programmer's manual, and presumably user-written ones were added
to the online documentation.  Today such an idea would be "obvious".

As far as I can remember Multics didn't really have the concept of a
"mount point".  All storage was single-level, i.e. segments (equivalent
in some respects to inodes, but they are also actually the value of the
segment register in the virtual memory hardware), and so files were
either physically in memory or paged out on physical disk devices or
similar, or even out on tape.  Where they actually resided was entirely
and permanently hidden from the user.  What was called the "filesystem"
was a form of database representing a hierarchical namespace which
pointed at all the known segments (files) regardless of where they were
actually stored.

In terms of virtual filesystems for Multics, I think it would be (have
been) relatively easy enough (i.e. with the addition of some APIs to the
filesystem itself) to implement a way of "mounting" a pseudo-DIM into
the filesystem namespace and thus being able to implement anything from
the likes of /kern and /proc to remote filesystems, network directories,
etc.  I.e. create an inverse twin of the filesystem DIM that takes
filesystem requests and turns them into back-end pseudo-DIM requests.
Effectively this would be implementing a VFS for Multics.  Indeed today,
and perhaps any time after Apollo Domain was designed, it would be
plainly obvious, and probably desirable, to do such a thing, at least
for some kinds of concepts, such as /proc.

Note though that I suspect the original Multics designers would have
preferred it if remote filesystems, in particular, were also effectively
invisible to the user, though I do not know how they might have
approached the issue of remote segments not always being available,
including indeed parts of the filesystem itself, i.e. directories and
metadata.  Perhaps in a similar way to how files migrated to backup
storage were handled -- if you referenced a file that had been migrated
offline then your process trapped to a retrieval process that involved
prompting a human tape operator to load the appropriate backup tape(s).

In fact Organick says the following about this idea of an inverse
relationship between the filesystem and I/O subsystem in his 1972 book
(p351,352):

    "We have just seen how the I/O system may behave as a customer of
    the file system, which supplies needed services.  One might wonder
    if the reverse of those roles is ever true.  That is, does the file
    system through its Page Control Module, when seeking to transfer a
    page of information to/from core [[memory]] and disk or tape
    storage, ever find itself to be a customer of the I/O system?
    Emphasis on objectives of modularity might cause one to guess the
    affirmative.  In actual fact however, considerations of efficiency
    have dictated that paging I/O in Multics be treated with
    special-purpose I/O software that greatly streamlines the
    processor's task in initiating and controlling such I/O."

So, at least early on, he appears to have denied the idea of the
filesystem routing requests through a DIM or pseudo-DIM, though he did
so on the basis that this would be done at the pager level and that
doing so could be too inefficient to consider.  However he doesn't seem
to have considered, or at least does not mention in the 1972 book, the
idea of a pseudo-DIM extending the filesystem namespace by implementing
a mount point, which as I say would today be considered a quite obvious,
desirable, and feasible task.

In any case the idea of pseudo device drivers interacting with the
filesystem, albeit the other way around, was firmly in place by 1965
(the file-system interface pseudo-DIM was described in the 1965 Multics
I/O system paper).

-- 
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24  5:14       ` arnold
@ 2020-06-24 21:08         ` Dave Horsfall
  0 siblings, 0 replies; 56+ messages in thread
From: Dave Horsfall @ 2020-06-24 21:08 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 23 Jun 2020, arnold@skeeve.com wrote:

> Well, it's a progression, isn't it?  As ideas are tried out, they are 
> used and then refined, and used and refined further. Multics did that 
> for its predecessors, Unix did that for Multics, and Plan 9 did that for 
> Unix.  The key difference in Unix and Plan 9 from others being the 
> continual goal towards refinement through simplification.

"Make things as simple as possible, but no simpler."

-- Dave

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23  9:09 Paul Ruizendaal
  2020-06-23 14:01 ` Larry McVoy
  2020-06-24 16:51 ` Anthony Martin
@ 2020-06-24 20:27 ` Derek Fawcus
  2020-06-24 21:33 ` Greg A. Woods
  3 siblings, 0 replies; 56+ messages in thread
From: Derek Fawcus @ 2020-06-24 20:27 UTC (permalink / raw)
  To: TUHS main list

On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?

Surely any OS, even non unix, which offered a remote FS in addition to a local FS implicitly implements something like a FSS/VFS?

So we could look to CP/M (and CP/Net, CP/NOS), where this[1] seems to suggest a date between 1980 and 1982.
An appendix to that paper has the CPM file system operations (such as they were) remotely available and
listed with their codes for the messages.

That said, another OS from DR (FlexOS) has a recognisable VFS, is written in C, but dates to early 1985.
I'm not sure when the design dates from[3]. If one looks at this[2] article in Byte,
each of the Resource Managers operates through a VFS layer, in terms of:

    #define I_GET           3 /* NET DISK CON MSC PIPE KERN */
    #define I_SET           4 /* NET DISK CON MSC PIPE KERN */
    #define I_LOOKUP        5 /* NET DISK CON MSC PIPE KERN */
    #define I_OPEN          6 /* NET DISK CON MSC PIPE */
    #define I_CLOSE         7 /* NET DISK CON MSC PIPE */
    #define I_READ          8 /* NET DISK CON MSC PIPE */
    #define I_WRITE         9 /* NET DISK CON MSC PIPE */
    #define I_SEEK          10 /* NET DISK CON MSC PIPE */
    #define I_CREATE        11 /* NET DISK CON PIPE */
    #define I_DELETE        12 /* NET DISK CON PIPE */

Which can be seen in the public code for CDOS-68k.

Not all RMs implement all calls, but the NET, DISK and PIPE sort of did.

DF

[1] https://www.autometer.de/unix4fun/z80pack/cpnet/cpnet.htm
[2] http://tech-insider.org/personal-computers/research/1985/05.html
    https://archive.org/stream/byte-magazine-1985-05/1985_05_BYTE_10-05_Multiprocessing#page/n380/mode/1up/search/concurrent
[3] The 80286 was released in '82, but I don't know when Intel started giving early info to customers.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 15:12   ` Clem Cole
  2020-06-23 15:55     ` Rich Morin
  2020-06-23 20:38     ` Rob Pike
@ 2020-06-24 19:36     ` Larry McVoy
  2020-06-25  6:52       ` Rob Gingell
  2020-07-05  0:05       ` Dave Horsfall
  2 siblings, 2 replies; 56+ messages in thread
From: Larry McVoy @ 2020-06-24 19:36 UTC (permalink / raw)
  To: Clem Cole; +Cc: TUHS main list, Paul Ruizendaal

On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote:
> Rusty and team did the VFS layer Sun @ (Larry will have to tell
> you who actually built it, I never knew).  

Steve Kleiman is the only author on the VFS paper so I'm guessing that 
was him.

Rusty was NFS.

Joe Moran (sun!mojo if I remember correctly) did the VM implementation,
it was co-designed by Joe, Shannon, and Gingell.

Side story: part of the reason I wanted to go to Sun was because when
I was a CS student at UW-Madison, Joe was a little older than me, and
I sat and watched him do a BSD port to a 68K (I think) in a little more
than 24 hours (sketchy on the time, I think that's right, almost positive
it was more than 20 hours, it was a chunk of time).  As a guy who wanted
to be next to the metal, it was impressive.

> In the end, early NFS was
> notorious for putting 'holes' in the files because of the automatic seek in
> every operation and errors not coming until close(2) time.

You have no idea how many of those holes that 16 bit SCCS checksum has
found (BitKeeper kept it).

> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> was not nearly as flexible as what Sun built [which had a crude interface
> generator], although until years later when Mike Leibensger built PIG (the
> Paceline Interface Generator) NFS RPC was always a PITA and not much better
> in practice than what we had at Masscomp.

Oh, I don't know if I agree.  I took the RPC mechanism and vectorized it.
I probably have the code somewhere.  It was pretty neat, 10Mbit was the
common networking and I did RPC Vector calls to the rstatd on literally
every machine in the Mountain View campus.  It was thousands of machines
and it worked.  The code split the RPC back into send and receive and
the code had to start gathering up answers before it was finished 
sending the questions or the packets would be dropped.

It worked surprisingly well.  Ron Minnich and Gene Kim made an NFS client
that worked as demo-ware.  There is a paper about below, note that I'm
co-author only because I wrote the RPC vector code, I believe Gene did
the in kernel code, it was Ron and Gene's project.  

http://mcvoy.com/lm/papers/bigfoot.pdf

> In fact, the point of the EFS paper was it's all about the recovery when
> there is a failure/error.  If you read his paper, Rusty's point was who
> cares if there is an error (I've always felt vindicated that while I lost
> the war, over time everyone came to our way of thinking and now NFS
> V4 looks a whole lot like EFS did).

Yeah, they were sort of cavalier about errors in public.  The weird
thing is Sun ran their entire network on Sun machines.  Everything was
NFSv2 when I got there, the routers were Sun workstations, the servers
were Sun servers, the clients were Sun workstations.  They ate their own
dogfood and NFS just worked, it was pretty unheard of to hear about NFS
issues other than performance tuning (stupid biods etc).

> Having a DFS as a feature was an incredible advance and proved we needed
> something (and it needed to be standard in all systems).   NFS really lead
> the market with that advance, although it sort of took a few years to make
> it really good.  The fact is that others had the same idea before or at
> least contemporary with it.

NFS was solid in v2 at Sun.  I went to SGI and NFS was complete garbage.
SGI's networking principle engineer told me, when I complained about
NFS not being reliable, "That's what rcp is for".  I was flabbergasted,
the networking group generated $18M/year in NFS sales.

It wasn't just SGI, every vender other Sun had a crappy NFS implementation.
I'm surprised NFS didn't have a worse rep because while NFS worked on Suns,
it really didn't work anywhere near as well elsewhere.

> That said, and to give the NFS team* a huge amount of credit *(and great
> applause), the VFS layer was better thought out than the FSS and in fact
> made it possible to add a lot of different file systems into UNIX later.
> FSS was much more ad hoc.

The VFS layer was Steve, NFS was Rusty.

> At LCC, when we built VPROC for TNC a few years later, we used some of the
> same ideas from VFS and of course used VFS for the file system layer since
> TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> the way VFS did).

VPROC was very cool and a requirement for any sort of distributed single
system image OS.  I agree it is a shame that it didn't catch on.

--lm

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 20:38     ` Rob Pike
  2020-06-23 20:57       ` Larry McVoy
  2020-06-24  5:14       ` arnold
@ 2020-06-24 19:30       ` Clem Cole
  2 siblings, 0 replies; 56+ messages in thread
From: Clem Cole @ 2020-06-24 19:30 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 8147 bytes --]

Paul -- slow cache refill  ... but I finally remembered some more of this
enough to reply...

I'm not sure if Larry remembers any of this and if he does, he might have
more details from the Sun side.

In the early/mid 90's, Dave Rosenthal was working on what become LOCKSS.  I
think he was at still at Sun at that point, but he would eventually move to
Stanford fulltime, but he could have left Sun already - I've forgotten.
 The UCLA folks (some of Popek's guys) had built a stackable file system
layer for BSD (which I think is in the contributed stuff in 4.4 - the
original BSD union FS and the UCLA replication FS for FICUS used it IIRC).
Some of the UMICH folks were messing with enhancements to NFS and it was an
active time.

Under the aegis of Unix International (UI), Gerry sponsored a pow-wow at
LCC (in the LA office by the airport) to talk about how stacking and VFS
layers.  IIRC Kleinman from then NetApp was there.   Besides' Dave, I've
forgotten now who else from Sun was there.  There must have been someone
from USL and maybe Mical Gien of Chorus.     But the whole discussion came
back to having FSS/VFS and N different  DFS's, PFS's and layers like
replication or union, *etc* in everyone's kernel.   Gerry saw there was
going to be a 'market' for support of different filing schemes, but at that
point, everything needed to be 'ported' because they were all different.

But also this idea of stacking was appealing.   If we could come up with
something that clean, then maybe we could be more independent of the
different physical file systems and we could 'push' behavior into the stack
more invisible.

But to do we needed to get some sort of agreement and all use the same
thing.

USL (Sun) was insisting that a Sun VFS++ -- one interface to rule them all
- NFS was the standard already of course.  This was not going over with the
Academic (UCLA/BSD/UMICH folks IIRC), particularly because of the lack of a
stacking property.

Dave proposed an idea of a general interposition layer on the inode that
would allow pushing and popping routines similar to the way STREAMS
worked.  Which we landed on -- everybody had to change a little.     I had
thought Dave implemented in an SVR4 kernel at some point and Chorus may
have had it.  There was talk of trying to get the spec out of UI and LCC
was acting either as an arms merchant or the swiss, was trying to get
IBM/DEC/HP/OSF to come along.   I went looking in my files and so far have
not been able to dig it up.  But you might find something hunting around
the LOCKSS work as Dave was the mastermind behind it.

Eventually, is all blew up and I'm not sure what happened to either the
specs or the code.

Clem

On Tue, Jun 23, 2020 at 4:39 PM Rob Pike <robpike@gmail.com> wrote:

> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
>
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
>
> -rob
>
>
> On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:
>
>> I agree with Larry with his observations.   The only thing I would add is
>> that Popek and Walker had a file system in the UCLA Locus distributed
>> system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
>> Gerry came up with the idea after a sabbatical @ PARC.   But it was not
>> formalized like NFS as a separate (layered) FS, it was just part of the
>> basic Locus FS.
>>
>> Peter Weinberger created the file system switch (FSS) for V8 and in the
>> UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
>> while Perry Flynn and I used it for EFS, both of us after talking
>> to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
>> you who actually built it, I never knew).  Rusty and I both published
>> papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
>> difference is that Sun gave away NFS and Masscomp was not smart enough to
>> do that).
>>
>> I talked to Steve Kleinman extensively about VFS at one point, and I'm
>> pretty sure he and the rest of the Sun guys had talked to the PARC folks
>> who after Gerry went back to UCLA started working on a DFS.   The idea of
>> the state-less (idempotent) file system RPC that NFS used based on stuff
>> PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
>> layer.  Peter, Dave, and I used a stateful scheme because we chose to have
>> full UNIX FS semantics, which NFS did not.  In the end, early NFS was
>> notorious for putting 'holes' in the files because of the automatic seek in
>> every operation and errors not coming until close(2) time.
>>
>> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
>> was not nearly as flexible as what Sun built [which had a crude interface
>> generator], although until years later when Mike Leibensger built PIG (the
>> Paceline Interface Generator) NFS RPC was always a PITA and not much better
>> in practice than what we had at Masscomp.
>>
>> In fact, the point of the EFS paper was it's all about the recovery when
>> there is a failure/error.  If you read his paper, Rusty's point was who
>> cares if there is an error (I've always felt vindicated that while I lost
>> the war, over time everyone came to our way of thinking and now NFS
>> V4 looks a whole lot like EFS did).
>>
>> Having a DFS as a feature was an incredible advance and proved we needed
>> something (and it needed to be standard in all systems).   NFS really lead
>> the market with that advance, although it sort of took a few years to make
>> it really good.  The fact is that others had the same idea before or at
>> least contemporary with it.
>>
>> That said, and to give the NFS team* a huge amount of credit *(and great
>> applause), the VFS layer was better thought out than the FSS and in fact
>> made it possible to add a lot of different file systems into UNIX later.
>> FSS was much more ad hoc.
>>
>> At LCC, when we built VPROC for TNC a few years later, we used some of
>> the same ideas from VFS and of course used VFS for the file system layer
>> since TNC had to have full POSIX semantics.  (It's a shame VPROC never
>> caught on the way VFS did).
>>
>> Clem
>>
>>
>> On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
>>
>>> I like to claim credit for creating the ChangeSet concept, the grouping
>>> of a set of deltas across a number of files.  But I wasn't the first.
>>> Back when dejanews was a thing and you could search usenet posts in a
>>> date range, if you searched before I started talking about ChangeSets,
>>> you could find 6, count 'em, 6 hits.  There was a really obscure system
>>> called Aide De Camp, that had a similar concept.
>>>
>>> But if you searched after I started talking about them you would see
>>> millions of hits.
>>>
>>> So I didn't invent the concept but I sure as heck made the world
>>> understand the concept.
>>>
>>> I suspect Sun could be in a similar position.  The VFS concept is
>>> pretty sweet so there might have been someone before Sun.  I'll
>>> long odds that if there was, it didn't gain traction until Sun
>>> did it.
>>>
>>> If there is nothing that predates it, then the inspiration was
>>> almost certainly the device driver interface.  One interface,
>>> many devices.  VFS is the same.
>>>
>>> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
>>> > When googling for File System Switch or Virtual File System most
>>> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
>>> sources mention 8th Edition.
>>> >
>>> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
>>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>>> concept.
>>> >
>>> > Does anybody recall prior art (prior to 1984) in this area?
>>> >
>>> > Paul
>>>
>>> --
>>> ---
>>> Larry McVoy                  lm at mcvoy.com
>>> http://www.mcvoy.com/lm
>>>
>>

[-- Attachment #2: Type: text/html, Size: 12341 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 16:51 ` Anthony Martin
  2020-06-24 17:31   ` Anthony Martin
  2020-06-24 18:31   ` Paul Ruizendaal
@ 2020-06-24 19:05   ` Paul Ruizendaal
  2 siblings, 0 replies; 56+ messages in thread
From: Paul Ruizendaal @ 2020-06-24 19:05 UTC (permalink / raw)
  To: Anthony Martin, TUHS main list


> I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
> 
Full paper is available here:
https://www.computer.org/csdl/pds/api/csdl/proceedings/download-article/12OmNzd7c1v/pdf

> I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach.

Indeed it appears to have been like the (later) Newcastle Connection in its approach.


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 18:13 ` Richard Salz
@ 2020-06-24 18:46   ` Lars Brinkhoff
  0 siblings, 0 replies; 56+ messages in thread
From: Lars Brinkhoff @ 2020-06-24 18:46 UTC (permalink / raw)
  To: coff, Richard Salz; +Cc: TUHS main list, Noel Chiappa

Redirected to COFF, since this isn't very TUHS related.

Richard Salz wrote:
> Noel Chiappa wote:
>> MLDEV on ITS would, I think, fit under that description.
>> I don't know if there's a paper on it; it's mid-70's.

I don't think there's anything like an academic paper.

The earliest evidence I found for the MLDEV facility, or a predecessor,
is from 1972:
https://retrocomputing.stackexchange.com/questions/13709/what-are-some-early-network-file-systems

> A web search for "its mldev" finds several things (mostly by Lars Brinkhoff
> it seems) , including
> https://github.com/larsbrinkhoff/mldev/blob/master/doc/mldev.protoc

That's just a copy I made.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-24 18:31 Norman Wilson
  2020-06-25  6:22 ` arnold
  0 siblings, 1 reply; 56+ messages in thread
From: Norman Wilson @ 2020-06-24 18:31 UTC (permalink / raw)
  To: tuhs

Reaching outside of UNIX, RSX/11 used external
supervisor-mode processes called ACPs (ancillary
control processes) to implement file systems.
I don't know exactly how they were plugged in,
but I do know they were pluggable, so their
interface must have constituted a file-system
switch of some sort.  RSX dates back into the
1970s.

At some point in the latter part of the 1980s,
Ralph Stamerjohn (a name instantly recognizable
in the 16-bit DEC software world) gave a DECUS
talk about implementing a remote file system
through ACPs: a stub ACP on the client exporting
RPCs over the network, a real one at the server
end.  I remember chatting with him about how
that did and didn't resemble the way pjw had
done it; interesting architectural comparison.

Norman Wilson
Toronto ON

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 16:51 ` Anthony Martin
  2020-06-24 17:31   ` Anthony Martin
@ 2020-06-24 18:31   ` Paul Ruizendaal
  2020-06-25  0:56     ` Rob Gingell via TUHS
  2020-06-24 19:05   ` Paul Ruizendaal
  2 siblings, 1 reply; 56+ messages in thread
From: Paul Ruizendaal @ 2020-06-24 18:31 UTC (permalink / raw)
  To: Anthony Martin, TUHS main list

On 24 Jun 2020, at 18:51, Anthony Martin <ality@pbrane.org> wrote:
> 
> Paul Ruizendaal <pnr@planet.nl> once said:
>> When googling for File System Switch or Virtual File System most sources
>> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
>> mention 8th Edition.
>> 
>> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex,
>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>> concept.
>> 
>> Does anybody recall prior art (prior to 1984) in this area?
> 
> The concept of the "remote inode" was introduced in:
> 
> Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall,
> W. T. (1981). A distributed UNIX system based on a virtual circuit
> switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168.


> The only other prior art I know of is
> 
>  Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE
>  Transactions on Communications, 28(4), 575-584.
> 
> wherein they state the following:
> 
>  A virtual file system, from the viewpoint of application programs on a
>  computer, models the file system functions of other computers. This is
>  done in the same way as with virtual terminals, a virtual file system
>  consists of a UPP having virtual files (VF), and a UCP which executes
>  virtual file system protocols.

Thanks for those pointers: I had not thought to include distributed systems in my initial search. Clem also pointed out LOCUS in this area. It would seem that S/F-Unix (the Luderer paper) and LOCUS happened at about the same time and both have the concept of a remote inode.

The Luderer paper has only a few details about implementation and besides describing the remote inode concept they mention:

“Of 27 system call types related to files, 18 result in message traffic if remote files are involved. Of these, 10 contain a path name as an argument, and the remaining 8 refer to already opened files. Path names or the data read or written can be up to 64K bytes long. The structure of each message is a type code followed by type-dependent data.”

In a way, the approach resembles the system call forwarding that Heinz Lycklama did for his 'satellite systems' work.

I asked Bill Marshall (one of the S/F co-authors) about it a few months ago and he recalls that they ran into problems with corner case semantics in their implementation, and that Peter Weinberger later approached the issue afresh 'thinking about the hard cases first.' Bill does not have S/F-Unix source code, and I have not been able to contact Gottfried Luderer yet.

I have not yet been able to look at the DCNA paper yet.

> I'd be interested if you find anything earlier.

I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
https://ieeexplore.ieee.org/document/762533

The abstract says:

"RIDE (Resource-Sharing in a Distributed Environment) has been designed and implemented for the UNIX operating system to provide sharing of remote files, remote process invocation and interprocess communication in a distributed computing environment. The system is designed to support a uniform interface for both local and remote access in a network. The user programs can be executed in a single or multiple machine environment without any program modification.”

I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach.

It does seem an interesting paper in any case, as its front page includes: 'This system was designed to replace the network interface programs that previously operated on the SPIDER system.'






^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 17:51 Noel Chiappa
@ 2020-06-24 18:13 ` Richard Salz
  2020-06-24 18:46   ` Lars Brinkhoff
  0 siblings, 1 reply; 56+ messages in thread
From: Richard Salz @ 2020-06-24 18:13 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

> MLDEV on ITS would, I think, fit under that description.
>
> I don't know if there's a paper on it; it's mid-70's.
>
>
A web search for "its mldev" finds several things (mostly by Lars Brinkhoff
it seems) , including
https://github.com/larsbrinkhoff/mldev/blob/master/doc/mldev.protoc

[-- Attachment #2: Type: text/html, Size: 651 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-24 17:51 Noel Chiappa
  2020-06-24 18:13 ` Richard Salz
  0 siblings, 1 reply; 56+ messages in thread
From: Noel Chiappa @ 2020-06-24 17:51 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Anthony Martin

    > wherein they state the following:
    >   A virtual file system, from the viewpoint of application programs on a
    >   computer, models the file system functions of other computers. This is
    >   done in the same way as with virtual terminals, a virtual file system
    >   consists of a UPP having virtual files (VF), and a UCP which executes
    >   virtual file system protocols.
    > I'd be interested if you find anything earlier.

MLDEV on ITS would, I think, fit under that description.

I don't know if there's a paper on it; it's mid-70's.

	Noel

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 16:51 ` Anthony Martin
@ 2020-06-24 17:31   ` Anthony Martin
  2020-06-24 18:31   ` Paul Ruizendaal
  2020-06-24 19:05   ` Paul Ruizendaal
  2 siblings, 0 replies; 56+ messages in thread
From: Anthony Martin @ 2020-06-24 17:31 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

The only other prior art I know of is

  Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE
  Transactions on Communications, 28(4), 575-584.

wherein they state the following:

  A virtual file system, from the viewpoint of application programs on a
  computer, models the file system functions of other computers. This is
  done in the same way as with virtual terminals, a virtual file system
  consists of a UPP having virtual files (VF), and a UCP which executes
  virtual file system protocols.

I'd be interested if you find anything earlier.

  Anthony

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23  9:09 Paul Ruizendaal
  2020-06-23 14:01 ` Larry McVoy
@ 2020-06-24 16:51 ` Anthony Martin
  2020-06-24 17:31   ` Anthony Martin
                     ` (2 more replies)
  2020-06-24 20:27 ` Derek Fawcus
  2020-06-24 21:33 ` Greg A. Woods
  3 siblings, 3 replies; 56+ messages in thread
From: Anthony Martin @ 2020-06-24 16:51 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

Paul Ruizendaal <pnr@planet.nl> once said:
> When googling for File System Switch or Virtual File System most sources
> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
> mention 8th Edition.
>
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex,
> Multics, CTSS, etc.), but none of those seem to have had a comparable
> concept.
>
> Does anybody recall prior art (prior to 1984) in this area?

The concept of the "remote inode" was introduced in:

Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall,
W. T. (1981). A distributed UNIX system based on a virtual circuit
switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168.

Cheers,
  Anthony

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-24 14:31 Noel Chiappa
@ 2020-06-24 15:21 ` Clem Cole
  0 siblings, 0 replies; 56+ messages in thread
From: Clem Cole @ 2020-06-24 15:21 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2280 bytes --]

On Wed, Jun 24, 2020 at 10:32 AM Noel Chiappa <jnc@mercury.lcs.mit.edu>
wrote:

>     > From: Dave Horsfall <dave@horsfall.org>
>
>     > Ioctl: the Swiss Army knife of system calls.  I thought it was a neat
>     > idea when it arrived (much better then those primitive stty/gtty
> calls)
>     > but now...
>
> Like they say, when the only tool you have is a hammer...
>
> Better syntax than stty/gtty, maybe but I'm not sure the semantics are that
> much better. The problem is that, especially with devices, what the I/O
> commands do is so widely varied that it's hard to fit them all under a
> unified umbrella. Maybe some (e.g. asynchronous I/O), but not all.
>
I remember the change from stty/gtty which was peculiar to tty's in v6 to
ioctl in v7 and thinking it was a nice clean generalization. Then I
realized it had at least two big issues -- no concept of direction nor the
size of the container being moved.   So, then I see it start to get abused
and I said -- oh my -- that was clearly not well thought out.   Plan 9's
solution of the CTL file with an ASCII interface, I thought was cleaner.
 IIRC TRIX had a controller interface too, but I've forgotten the details
now.

Here is the problem....   if you make everything a byte stream (which is a
great fundamental way of building up a small cooperating program with a
common interface), how do you handle the mechanisms to modify the stream in
a manner than make sense (control if you will) in an out-of-band manner,
but not end up with a fisherman's stew of different and special things
(access methods, frameworks, *et al*) that are peculiar to that specific
stream?

The problem with stty/getty was is was specific to TTY's.  It was a fine
OOB scheme, but did not make sense for say changing parameters on other
devices (tape as an example).   The tape solution was different names,
which worked there because you were not supposed to change things like
density mid tape write.  Other peripherals worked differently.   ioctl
became the only way out, as Noel points out.

So ... this begs the question?  Does the ctl file idea work better than
ioctl?   You still have to know what the ctl interface is for that device?
 It is ASCII which is nice and huge improvement over the binary command
encoding crap.

Clem

[-- Attachment #2: Type: text/html, Size: 3539 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
@ 2020-06-24 14:31 Noel Chiappa
  2020-06-24 15:21 ` Clem Cole
  0 siblings, 1 reply; 56+ messages in thread
From: Noel Chiappa @ 2020-06-24 14:31 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Dave Horsfall <dave@horsfall.org>

    > Ioctl: the Swiss Army knife of system calls.  I thought it was a neat
    > idea when it arrived (much better then those primitive stty/gtty calls)
    > but now...

Like they say, when the only tool you have is a hammer...

Better syntax than stty/gtty, maybe but I'm not sure the semantics are that
much better. The problem is that, especially with devices, what the I/O
commands do is so widely varied that it's hard to fit them all under a
unified umbrella. Maybe some (e.g. asynchronous I/O), but not all.

	Noel


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 20:38     ` Rob Pike
  2020-06-23 20:57       ` Larry McVoy
@ 2020-06-24  5:14       ` arnold
  2020-06-24 21:08         ` Dave Horsfall
  2020-06-24 19:30       ` Clem Cole
  2 siblings, 1 reply; 56+ messages in thread
From: arnold @ 2020-06-24  5:14 UTC (permalink / raw)
  To: robpike, clemc; +Cc: tuhs, pnr

Rob Pike <robpike@gmail.com> wrote:

> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
>
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
>
> -rob

Well, it's a progression, isn't it?  As ideas are tried out, they are
used and then refined, and used and refined further. Multics did that
for its predecessors,  Unix did that for Multics, and Plan 9 did that
for Unix.  The key difference in Unix and Plan 9 from others being
the continual goal towards refinement through simplification.

Arnold

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 20:38     ` Rob Pike
@ 2020-06-23 20:57       ` Larry McVoy
  2020-06-24  5:14       ` arnold
  2020-06-24 19:30       ` Clem Cole
  2 siblings, 0 replies; 56+ messages in thread
From: Larry McVoy @ 2020-06-23 20:57 UTC (permalink / raw)
  To: Rob Pike; +Cc: TUHS main list, Paul Ruizendaal

The SunOS one hid the block stuff pretty well.  It had mmap().  

Where it got hairy when I was working on it was block size 4k vs page
size 8k.  And frags, I hate frags.

On Wed, Jun 24, 2020 at 06:38:53AM +1000, Rob Pike wrote:
> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
> 
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
> 
> -rob
> 
> 
> On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:
> 
> > I agree with Larry with his observations.   The only thing I would add is
> > that Popek and Walker had a file system in the UCLA Locus distributed
> > system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
> > Gerry came up with the idea after a sabbatical @ PARC.   But it was not
> > formalized like NFS as a separate (layered) FS, it was just part of the
> > basic Locus FS.
> >
> > Peter Weinberger created the file system switch (FSS) for V8 and in the
> > UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
> > while Perry Flynn and I used it for EFS, both of us after talking
> > to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
> > you who actually built it, I never knew).  Rusty and I both published
> > papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
> > difference is that Sun gave away NFS and Masscomp was not smart enough to
> > do that).
> >
> > I talked to Steve Kleinman extensively about VFS at one point, and I'm
> > pretty sure he and the rest of the Sun guys had talked to the PARC folks
> > who after Gerry went back to UCLA started working on a DFS.   The idea of
> > the state-less (idempotent) file system RPC that NFS used based on stuff
> > PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
> > layer.  Peter, Dave, and I used a stateful scheme because we chose to have
> > full UNIX FS semantics, which NFS did not.  In the end, early NFS was
> > notorious for putting 'holes' in the files because of the automatic seek in
> > every operation and errors not coming until close(2) time.
> >
> > EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> > was not nearly as flexible as what Sun built [which had a crude interface
> > generator], although until years later when Mike Leibensger built PIG (the
> > Paceline Interface Generator) NFS RPC was always a PITA and not much better
> > in practice than what we had at Masscomp.
> >
> > In fact, the point of the EFS paper was it's all about the recovery when
> > there is a failure/error.  If you read his paper, Rusty's point was who
> > cares if there is an error (I've always felt vindicated that while I lost
> > the war, over time everyone came to our way of thinking and now NFS
> > V4 looks a whole lot like EFS did).
> >
> > Having a DFS as a feature was an incredible advance and proved we needed
> > something (and it needed to be standard in all systems).   NFS really lead
> > the market with that advance, although it sort of took a few years to make
> > it really good.  The fact is that others had the same idea before or at
> > least contemporary with it.
> >
> > That said, and to give the NFS team* a huge amount of credit *(and great
> > applause), the VFS layer was better thought out than the FSS and in fact
> > made it possible to add a lot of different file systems into UNIX later.
> > FSS was much more ad hoc.
> >
> > At LCC, when we built VPROC for TNC a few years later, we used some of the
> > same ideas from VFS and of course used VFS for the file system layer since
> > TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> > the way VFS did).
> >
> > Clem
> >
> >
> > On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
> >
> >> I like to claim credit for creating the ChangeSet concept, the grouping
> >> of a set of deltas across a number of files.  But I wasn't the first.
> >> Back when dejanews was a thing and you could search usenet posts in a
> >> date range, if you searched before I started talking about ChangeSets,
> >> you could find 6, count 'em, 6 hits.  There was a really obscure system
> >> called Aide De Camp, that had a similar concept.
> >>
> >> But if you searched after I started talking about them you would see
> >> millions of hits.
> >>
> >> So I didn't invent the concept but I sure as heck made the world
> >> understand the concept.
> >>
> >> I suspect Sun could be in a similar position.  The VFS concept is
> >> pretty sweet so there might have been someone before Sun.  I'll
> >> long odds that if there was, it didn't gain traction until Sun
> >> did it.
> >>
> >> If there is nothing that predates it, then the inspiration was
> >> almost certainly the device driver interface.  One interface,
> >> many devices.  VFS is the same.
> >>
> >> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> >> > When googling for File System Switch or Virtual File System most
> >> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
> >> sources mention 8th Edition.
> >> >
> >> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
> >> Multics, CTSS, etc.), but none of those seem to have had a comparable
> >> concept.
> >> >
> >> > Does anybody recall prior art (prior to 1984) in this area?
> >> >
> >> > Paul
> >>
> >> --
> >> ---
> >> Larry McVoy                  lm at mcvoy.com
> >> http://www.mcvoy.com/lm
> >>
> >

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 15:12   ` Clem Cole
  2020-06-23 15:55     ` Rich Morin
@ 2020-06-23 20:38     ` Rob Pike
  2020-06-23 20:57       ` Larry McVoy
                         ` (2 more replies)
  2020-06-24 19:36     ` Larry McVoy
  2 siblings, 3 replies; 56+ messages in thread
From: Rob Pike @ 2020-06-23 20:38 UTC (permalink / raw)
  To: Clem Cole; +Cc: TUHS main list, Paul Ruizendaal

[-- Attachment #1: Type: text/plain, Size: 5273 bytes --]

For my taste, the various Unix file system switches that I've seen are too
firmly tied to the idea of blocks and disks and all that, making them less
flexible than they should have been. That's why the Plan 9 version is about
names and byte streams, to make it as general as possible.

That's one of the reasons the possibilities of the file system approach to
data has not reached its potential in Unix.

-rob


On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:

> I agree with Larry with his observations.   The only thing I would add is
> that Popek and Walker had a file system in the UCLA Locus distributed
> system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
> Gerry came up with the idea after a sabbatical @ PARC.   But it was not
> formalized like NFS as a separate (layered) FS, it was just part of the
> basic Locus FS.
>
> Peter Weinberger created the file system switch (FSS) for V8 and in the
> UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
> while Perry Flynn and I used it for EFS, both of us after talking
> to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
> you who actually built it, I never knew).  Rusty and I both published
> papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
> difference is that Sun gave away NFS and Masscomp was not smart enough to
> do that).
>
> I talked to Steve Kleinman extensively about VFS at one point, and I'm
> pretty sure he and the rest of the Sun guys had talked to the PARC folks
> who after Gerry went back to UCLA started working on a DFS.   The idea of
> the state-less (idempotent) file system RPC that NFS used based on stuff
> PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
> layer.  Peter, Dave, and I used a stateful scheme because we chose to have
> full UNIX FS semantics, which NFS did not.  In the end, early NFS was
> notorious for putting 'holes' in the files because of the automatic seek in
> every operation and errors not coming until close(2) time.
>
> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> was not nearly as flexible as what Sun built [which had a crude interface
> generator], although until years later when Mike Leibensger built PIG (the
> Paceline Interface Generator) NFS RPC was always a PITA and not much better
> in practice than what we had at Masscomp.
>
> In fact, the point of the EFS paper was it's all about the recovery when
> there is a failure/error.  If you read his paper, Rusty's point was who
> cares if there is an error (I've always felt vindicated that while I lost
> the war, over time everyone came to our way of thinking and now NFS
> V4 looks a whole lot like EFS did).
>
> Having a DFS as a feature was an incredible advance and proved we needed
> something (and it needed to be standard in all systems).   NFS really lead
> the market with that advance, although it sort of took a few years to make
> it really good.  The fact is that others had the same idea before or at
> least contemporary with it.
>
> That said, and to give the NFS team* a huge amount of credit *(and great
> applause), the VFS layer was better thought out than the FSS and in fact
> made it possible to add a lot of different file systems into UNIX later.
> FSS was much more ad hoc.
>
> At LCC, when we built VPROC for TNC a few years later, we used some of the
> same ideas from VFS and of course used VFS for the file system layer since
> TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> the way VFS did).
>
> Clem
>
>
> On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> I like to claim credit for creating the ChangeSet concept, the grouping
>> of a set of deltas across a number of files.  But I wasn't the first.
>> Back when dejanews was a thing and you could search usenet posts in a
>> date range, if you searched before I started talking about ChangeSets,
>> you could find 6, count 'em, 6 hits.  There was a really obscure system
>> called Aide De Camp, that had a similar concept.
>>
>> But if you searched after I started talking about them you would see
>> millions of hits.
>>
>> So I didn't invent the concept but I sure as heck made the world
>> understand the concept.
>>
>> I suspect Sun could be in a similar position.  The VFS concept is
>> pretty sweet so there might have been someone before Sun.  I'll
>> long odds that if there was, it didn't gain traction until Sun
>> did it.
>>
>> If there is nothing that predates it, then the inspiration was
>> almost certainly the device driver interface.  One interface,
>> many devices.  VFS is the same.
>>
>> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
>> > When googling for File System Switch or Virtual File System most
>> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
>> sources mention 8th Edition.
>> >
>> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>> concept.
>> >
>> > Does anybody recall prior art (prior to 1984) in this area?
>> >
>> > Paul
>>
>> --
>> ---
>> Larry McVoy                  lm at mcvoy.com
>> http://www.mcvoy.com/lm
>>
>

[-- Attachment #2: Type: text/html, Size: 7637 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 15:12   ` Clem Cole
@ 2020-06-23 15:55     ` Rich Morin
  2020-06-23 20:38     ` Rob Pike
  2020-06-24 19:36     ` Larry McVoy
  2 siblings, 0 replies; 56+ messages in thread
From: Rich Morin @ 2020-06-23 15:55 UTC (permalink / raw)
  To: TUHS main list

I recall listening to an interesting survey talk on DFS (pronounced Doofus)
variants at a USENIX ATC conference in the 80's.  Someone here may be able
to recall the specifics...

-r


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23 14:01 ` Larry McVoy
@ 2020-06-23 15:12   ` Clem Cole
  2020-06-23 15:55     ` Rich Morin
                       ` (2 more replies)
  0 siblings, 3 replies; 56+ messages in thread
From: Clem Cole @ 2020-06-23 15:12 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list, Paul Ruizendaal

[-- Attachment #1: Type: text/plain, Size: 4645 bytes --]

I agree with Larry with his observations.   The only thing I would add is
that Popek and Walker had a file system in the UCLA Locus distributed
system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
Gerry came up with the idea after a sabbatical @ PARC.   But it was not
formalized like NFS as a separate (layered) FS, it was just part of the
basic Locus FS.

Peter Weinberger created the file system switch (FSS) for V8 and in the
UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
while Perry Flynn and I used it for EFS, both of us after talking
to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
you who actually built it, I never knew).  Rusty and I both published
papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
difference is that Sun gave away NFS and Masscomp was not smart enough to
do that).

I talked to Steve Kleinman extensively about VFS at one point, and I'm
pretty sure he and the rest of the Sun guys had talked to the PARC folks
who after Gerry went back to UCLA started working on a DFS.   The idea of
the state-less (idempotent) file system RPC that NFS used based on stuff
PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
layer.  Peter, Dave, and I used a stateful scheme because we chose to have
full UNIX FS semantics, which NFS did not.  In the end, early NFS was
notorious for putting 'holes' in the files because of the automatic seek in
every operation and errors not coming until close(2) time.

EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
was not nearly as flexible as what Sun built [which had a crude interface
generator], although until years later when Mike Leibensger built PIG (the
Paceline Interface Generator) NFS RPC was always a PITA and not much better
in practice than what we had at Masscomp.

In fact, the point of the EFS paper was it's all about the recovery when
there is a failure/error.  If you read his paper, Rusty's point was who
cares if there is an error (I've always felt vindicated that while I lost
the war, over time everyone came to our way of thinking and now NFS
V4 looks a whole lot like EFS did).

Having a DFS as a feature was an incredible advance and proved we needed
something (and it needed to be standard in all systems).   NFS really lead
the market with that advance, although it sort of took a few years to make
it really good.  The fact is that others had the same idea before or at
least contemporary with it.

That said, and to give the NFS team* a huge amount of credit *(and great
applause), the VFS layer was better thought out than the FSS and in fact
made it possible to add a lot of different file systems into UNIX later.
FSS was much more ad hoc.

At LCC, when we built VPROC for TNC a few years later, we used some of the
same ideas from VFS and of course used VFS for the file system layer since
TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
the way VFS did).

Clem


On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:

> I like to claim credit for creating the ChangeSet concept, the grouping
> of a set of deltas across a number of files.  But I wasn't the first.
> Back when dejanews was a thing and you could search usenet posts in a
> date range, if you searched before I started talking about ChangeSets,
> you could find 6, count 'em, 6 hits.  There was a really obscure system
> called Aide De Camp, that had a similar concept.
>
> But if you searched after I started talking about them you would see
> millions of hits.
>
> So I didn't invent the concept but I sure as heck made the world
> understand the concept.
>
> I suspect Sun could be in a similar position.  The VFS concept is
> pretty sweet so there might have been someone before Sun.  I'll
> long odds that if there was, it didn't gain traction until Sun
> did it.
>
> If there is nothing that predates it, then the inspiration was
> almost certainly the device driver interface.  One interface,
> many devices.  VFS is the same.
>
> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> > When googling for File System Switch or Virtual File System most sources
> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
> mention 8th Edition.
> >
> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
> Multics, CTSS, etc.), but none of those seem to have had a comparable
> concept.
> >
> > Does anybody recall prior art (prior to 1984) in this area?
> >
> > Paul
>
> --
> ---
> Larry McVoy                  lm at mcvoy.com
> http://www.mcvoy.com/lm
>

[-- Attachment #2: Type: text/html, Size: 6811 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [TUHS] VFS prior to 1984
  2020-06-23  9:09 Paul Ruizendaal
@ 2020-06-23 14:01 ` Larry McVoy
  2020-06-23 15:12   ` Clem Cole
  2020-06-24 16:51 ` Anthony Martin
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 56+ messages in thread
From: Larry McVoy @ 2020-06-23 14:01 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

I like to claim credit for creating the ChangeSet concept, the grouping
of a set of deltas across a number of files.  But I wasn't the first.
Back when dejanews was a thing and you could search usenet posts in a
date range, if you searched before I started talking about ChangeSets,
you could find 6, count 'em, 6 hits.  There was a really obscure system
called Aide De Camp, that had a similar concept.

But if you searched after I started talking about them you would see
millions of hits.

So I didn't invent the concept but I sure as heck made the world 
understand the concept.

I suspect Sun could be in a similar position.  The VFS concept is
pretty sweet so there might have been someone before Sun.  I'll
long odds that if there was, it didn't gain traction until Sun
did it.

If there is nothing that predates it, then the inspiration was
almost certainly the device driver interface.  One interface,
many devices.  VFS is the same.

On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition.
> 
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?
> 
> Paul

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [TUHS] VFS prior to 1984
@ 2020-06-23  9:09 Paul Ruizendaal
  2020-06-23 14:01 ` Larry McVoy
                   ` (3 more replies)
  0 siblings, 4 replies; 56+ messages in thread
From: Paul Ruizendaal @ 2020-06-23  9:09 UTC (permalink / raw)
  To: TUHS main list

When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition.

I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.

Does anybody recall prior art (prior to 1984) in this area?

Paul

^ permalink raw reply	[flat|nested] 56+ messages in thread

end of thread, other threads:[~2020-07-07  1:08 UTC | newest]

Thread overview: 56+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-23 22:17 [TUHS] VFS prior to 1984 Norman Wilson
2020-06-23 22:24 ` Dave Horsfall
2020-06-24  4:09   ` Warner Losh
2020-06-24  5:10     ` Dave Horsfall
2020-06-24  5:33       ` Bakul Shah
2020-06-24  9:10         ` Andrew Warkentin
  -- strict thread matches above, loose matches on Subject: below --
2020-06-29  9:11 Paul Ruizendaal
2020-06-29 14:45 ` Larry McVoy
2020-06-29 14:53   ` Paul Ruizendaal
2020-06-29 15:14     ` Larry McVoy
2020-06-29 16:34 ` Heinz Lycklama
2020-06-25 20:25 Noel Chiappa
2020-06-26  5:49 ` Lars Brinkhoff
2020-06-26  8:34   ` Dave Horsfall
2020-06-26 11:13     ` arnold
2020-06-25 20:23 Noel Chiappa
2020-06-25 19:31 Noel Chiappa
2020-06-24 18:31 Norman Wilson
2020-06-25  6:22 ` arnold
2020-06-24 17:51 Noel Chiappa
2020-06-24 18:13 ` Richard Salz
2020-06-24 18:46   ` Lars Brinkhoff
2020-06-24 14:31 Noel Chiappa
2020-06-24 15:21 ` Clem Cole
2020-06-23  9:09 Paul Ruizendaal
2020-06-23 14:01 ` Larry McVoy
2020-06-23 15:12   ` Clem Cole
2020-06-23 15:55     ` Rich Morin
2020-06-23 20:38     ` Rob Pike
2020-06-23 20:57       ` Larry McVoy
2020-06-24  5:14       ` arnold
2020-06-24 21:08         ` Dave Horsfall
2020-06-24 19:30       ` Clem Cole
2020-06-24 19:36     ` Larry McVoy
2020-06-25  6:52       ` Rob Gingell
2020-07-05  0:05       ` Dave Horsfall
2020-07-05  0:16         ` Larry McVoy
2020-07-06  4:42           ` Dave Horsfall
2020-07-06 16:51           ` Chris Torek
2020-07-07  1:07             ` Bakul Shah
2020-07-05  1:43         ` Clem Cole
2020-07-05 14:43           ` Larry McVoy
2020-07-05 18:40             ` Arthur Krewat
2020-07-05 20:08             ` Clem Cole
2020-07-05 20:42               ` John Cowan
2020-07-05 21:04                 ` Clem Cole
2020-07-05 21:14                   ` Dan Cross
2020-06-24 16:51 ` Anthony Martin
2020-06-24 17:31   ` Anthony Martin
2020-06-24 18:31   ` Paul Ruizendaal
2020-06-25  0:56     ` Rob Gingell via TUHS
2020-06-24 19:05   ` Paul Ruizendaal
2020-06-24 20:27 ` Derek Fawcus
2020-06-24 21:33 ` Greg A. Woods
2020-06-25  0:45   ` Adam Thornton
2020-06-25 19:40     ` Greg A. Woods

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).