[TUHS] VFS prior to 1984

[TUHS] VFS prior to 1984

[Paul Ruizendaal]

When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition.

I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.

Does anybody recall prior art (prior to 1984) in this area?

Paul

Re: [TUHS] VFS prior to 1984

[Larry McVoy]

I like to claim credit for creating the ChangeSet concept, the grouping
of a set of deltas across a number of files.  But I wasn't the first.
Back when dejanews was a thing and you could search usenet posts in a
date range, if you searched before I started talking about ChangeSets,
you could find 6, count 'em, 6 hits.  There was a really obscure system
called Aide De Camp, that had a similar concept.

But if you searched after I started talking about them you would see
millions of hits.

So I didn't invent the concept but I sure as heck made the world 
understand the concept.

I suspect Sun could be in a similar position.  The VFS concept is
pretty sweet so there might have been someone before Sun.  I'll
long odds that if there was, it didn't gain traction until Sun
did it.

If there is nothing that predates it, then the inspiration was
almost certainly the device driver interface.  One interface,
many devices.  VFS is the same.

On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition.
> 
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?
> 
> Paul

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

Re: [TUHS] VFS prior to 1984

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 4645 bytes --]

I agree with Larry with his observations.   The only thing I would add is
that Popek and Walker had a file system in the UCLA Locus distributed
system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
Gerry came up with the idea after a sabbatical @ PARC.   But it was not
formalized like NFS as a separate (layered) FS, it was just part of the
basic Locus FS.

Peter Weinberger created the file system switch (FSS) for V8 and in the
UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
while Perry Flynn and I used it for EFS, both of us after talking
to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
you who actually built it, I never knew).  Rusty and I both published
papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
difference is that Sun gave away NFS and Masscomp was not smart enough to
do that).

I talked to Steve Kleinman extensively about VFS at one point, and I'm
pretty sure he and the rest of the Sun guys had talked to the PARC folks
who after Gerry went back to UCLA started working on a DFS.   The idea of
the state-less (idempotent) file system RPC that NFS used based on stuff
PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
layer.  Peter, Dave, and I used a stateful scheme because we chose to have
full UNIX FS semantics, which NFS did not.  In the end, early NFS was
notorious for putting 'holes' in the files because of the automatic seek in
every operation and errors not coming until close(2) time.

EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
was not nearly as flexible as what Sun built [which had a crude interface
generator], although until years later when Mike Leibensger built PIG (the
Paceline Interface Generator) NFS RPC was always a PITA and not much better
in practice than what we had at Masscomp.

In fact, the point of the EFS paper was it's all about the recovery when
there is a failure/error.  If you read his paper, Rusty's point was who
cares if there is an error (I've always felt vindicated that while I lost
the war, over time everyone came to our way of thinking and now NFS
V4 looks a whole lot like EFS did).

Having a DFS as a feature was an incredible advance and proved we needed
something (and it needed to be standard in all systems).   NFS really lead
the market with that advance, although it sort of took a few years to make
it really good.  The fact is that others had the same idea before or at
least contemporary with it.

That said, and to give the NFS team* a huge amount of credit *(and great
applause), the VFS layer was better thought out than the FSS and in fact
made it possible to add a lot of different file systems into UNIX later.
FSS was much more ad hoc.

At LCC, when we built VPROC for TNC a few years later, we used some of the
same ideas from VFS and of course used VFS for the file system layer since
TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
the way VFS did).

Clem


On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:

> I like to claim credit for creating the ChangeSet concept, the grouping
> of a set of deltas across a number of files.  But I wasn't the first.
> Back when dejanews was a thing and you could search usenet posts in a
> date range, if you searched before I started talking about ChangeSets,
> you could find 6, count 'em, 6 hits.  There was a really obscure system
> called Aide De Camp, that had a similar concept.
>
> But if you searched after I started talking about them you would see
> millions of hits.
>
> So I didn't invent the concept but I sure as heck made the world
> understand the concept.
>
> I suspect Sun could be in a similar position.  The VFS concept is
> pretty sweet so there might have been someone before Sun.  I'll
> long odds that if there was, it didn't gain traction until Sun
> did it.
>
> If there is nothing that predates it, then the inspiration was
> almost certainly the device driver interface.  One interface,
> many devices.  VFS is the same.
>
> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> > When googling for File System Switch or Virtual File System most sources
> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
> mention 8th Edition.
> >
> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
> Multics, CTSS, etc.), but none of those seem to have had a comparable
> concept.
> >
> > Does anybody recall prior art (prior to 1984) in this area?
> >
> > Paul
>
> --
> ---
> Larry McVoy                  lm at mcvoy.com
> http://www.mcvoy.com/lm
>

[-- Attachment #2: Type: text/html, Size: 6811 bytes --]

Re: [TUHS] VFS prior to 1984

[Rich Morin]

I recall listening to an interesting survey talk on DFS (pronounced Doofus)
variants at a USENIX ATC conference in the 80's.  Someone here may be able
to recall the specifics...

-r

Re: [TUHS] VFS prior to 1984

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 5273 bytes --]

For my taste, the various Unix file system switches that I've seen are too
firmly tied to the idea of blocks and disks and all that, making them less
flexible than they should have been. That's why the Plan 9 version is about
names and byte streams, to make it as general as possible.

That's one of the reasons the possibilities of the file system approach to
data has not reached its potential in Unix.

-rob


On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:

> I agree with Larry with his observations.   The only thing I would add is
> that Popek and Walker had a file system in the UCLA Locus distributed
> system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
> Gerry came up with the idea after a sabbatical @ PARC.   But it was not
> formalized like NFS as a separate (layered) FS, it was just part of the
> basic Locus FS.
>
> Peter Weinberger created the file system switch (FSS) for V8 and in the
> UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
> while Perry Flynn and I used it for EFS, both of us after talking
> to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
> you who actually built it, I never knew).  Rusty and I both published
> papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
> difference is that Sun gave away NFS and Masscomp was not smart enough to
> do that).
>
> I talked to Steve Kleinman extensively about VFS at one point, and I'm
> pretty sure he and the rest of the Sun guys had talked to the PARC folks
> who after Gerry went back to UCLA started working on a DFS.   The idea of
> the state-less (idempotent) file system RPC that NFS used based on stuff
> PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
> layer.  Peter, Dave, and I used a stateful scheme because we chose to have
> full UNIX FS semantics, which NFS did not.  In the end, early NFS was
> notorious for putting 'holes' in the files because of the automatic seek in
> every operation and errors not coming until close(2) time.
>
> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> was not nearly as flexible as what Sun built [which had a crude interface
> generator], although until years later when Mike Leibensger built PIG (the
> Paceline Interface Generator) NFS RPC was always a PITA and not much better
> in practice than what we had at Masscomp.
>
> In fact, the point of the EFS paper was it's all about the recovery when
> there is a failure/error.  If you read his paper, Rusty's point was who
> cares if there is an error (I've always felt vindicated that while I lost
> the war, over time everyone came to our way of thinking and now NFS
> V4 looks a whole lot like EFS did).
>
> Having a DFS as a feature was an incredible advance and proved we needed
> something (and it needed to be standard in all systems).   NFS really lead
> the market with that advance, although it sort of took a few years to make
> it really good.  The fact is that others had the same idea before or at
> least contemporary with it.
>
> That said, and to give the NFS team* a huge amount of credit *(and great
> applause), the VFS layer was better thought out than the FSS and in fact
> made it possible to add a lot of different file systems into UNIX later.
> FSS was much more ad hoc.
>
> At LCC, when we built VPROC for TNC a few years later, we used some of the
> same ideas from VFS and of course used VFS for the file system layer since
> TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> the way VFS did).
>
> Clem
>
>
> On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> I like to claim credit for creating the ChangeSet concept, the grouping
>> of a set of deltas across a number of files.  But I wasn't the first.
>> Back when dejanews was a thing and you could search usenet posts in a
>> date range, if you searched before I started talking about ChangeSets,
>> you could find 6, count 'em, 6 hits.  There was a really obscure system
>> called Aide De Camp, that had a similar concept.
>>
>> But if you searched after I started talking about them you would see
>> millions of hits.
>>
>> So I didn't invent the concept but I sure as heck made the world
>> understand the concept.
>>
>> I suspect Sun could be in a similar position.  The VFS concept is
>> pretty sweet so there might have been someone before Sun.  I'll
>> long odds that if there was, it didn't gain traction until Sun
>> did it.
>>
>> If there is nothing that predates it, then the inspiration was
>> almost certainly the device driver interface.  One interface,
>> many devices.  VFS is the same.
>>
>> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
>> > When googling for File System Switch or Virtual File System most
>> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
>> sources mention 8th Edition.
>> >
>> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>> concept.
>> >
>> > Does anybody recall prior art (prior to 1984) in this area?
>> >
>> > Paul
>>
>> --
>> ---
>> Larry McVoy                  lm at mcvoy.com
>> http://www.mcvoy.com/lm
>>
>

[-- Attachment #2: Type: text/html, Size: 7637 bytes --]

Re: [TUHS] VFS prior to 1984

[Larry McVoy]

The SunOS one hid the block stuff pretty well.  It had mmap().  

Where it got hairy when I was working on it was block size 4k vs page
size 8k.  And frags, I hate frags.

On Wed, Jun 24, 2020 at 06:38:53AM +1000, Rob Pike wrote:
> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
> 
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
> 
> -rob
> 
> 
> On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:
> 
> > I agree with Larry with his observations.   The only thing I would add is
> > that Popek and Walker had a file system in the UCLA Locus distributed
> > system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
> > Gerry came up with the idea after a sabbatical @ PARC.   But it was not
> > formalized like NFS as a separate (layered) FS, it was just part of the
> > basic Locus FS.
> >
> > Peter Weinberger created the file system switch (FSS) for V8 and in the
> > UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
> > while Perry Flynn and I used it for EFS, both of us after talking
> > to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
> > you who actually built it, I never knew).  Rusty and I both published
> > papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
> > difference is that Sun gave away NFS and Masscomp was not smart enough to
> > do that).
> >
> > I talked to Steve Kleinman extensively about VFS at one point, and I'm
> > pretty sure he and the rest of the Sun guys had talked to the PARC folks
> > who after Gerry went back to UCLA started working on a DFS.   The idea of
> > the state-less (idempotent) file system RPC that NFS used based on stuff
> > PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
> > layer.  Peter, Dave, and I used a stateful scheme because we chose to have
> > full UNIX FS semantics, which NFS did not.  In the end, early NFS was
> > notorious for putting 'holes' in the files because of the automatic seek in
> > every operation and errors not coming until close(2) time.
> >
> > EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> > was not nearly as flexible as what Sun built [which had a crude interface
> > generator], although until years later when Mike Leibensger built PIG (the
> > Paceline Interface Generator) NFS RPC was always a PITA and not much better
> > in practice than what we had at Masscomp.
> >
> > In fact, the point of the EFS paper was it's all about the recovery when
> > there is a failure/error.  If you read his paper, Rusty's point was who
> > cares if there is an error (I've always felt vindicated that while I lost
> > the war, over time everyone came to our way of thinking and now NFS
> > V4 looks a whole lot like EFS did).
> >
> > Having a DFS as a feature was an incredible advance and proved we needed
> > something (and it needed to be standard in all systems).   NFS really lead
> > the market with that advance, although it sort of took a few years to make
> > it really good.  The fact is that others had the same idea before or at
> > least contemporary with it.
> >
> > That said, and to give the NFS team* a huge amount of credit *(and great
> > applause), the VFS layer was better thought out than the FSS and in fact
> > made it possible to add a lot of different file systems into UNIX later.
> > FSS was much more ad hoc.
> >
> > At LCC, when we built VPROC for TNC a few years later, we used some of the
> > same ideas from VFS and of course used VFS for the file system layer since
> > TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> > the way VFS did).
> >
> > Clem
> >
> >
> > On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
> >
> >> I like to claim credit for creating the ChangeSet concept, the grouping
> >> of a set of deltas across a number of files.  But I wasn't the first.
> >> Back when dejanews was a thing and you could search usenet posts in a
> >> date range, if you searched before I started talking about ChangeSets,
> >> you could find 6, count 'em, 6 hits.  There was a really obscure system
> >> called Aide De Camp, that had a similar concept.
> >>
> >> But if you searched after I started talking about them you would see
> >> millions of hits.
> >>
> >> So I didn't invent the concept but I sure as heck made the world
> >> understand the concept.
> >>
> >> I suspect Sun could be in a similar position.  The VFS concept is
> >> pretty sweet so there might have been someone before Sun.  I'll
> >> long odds that if there was, it didn't gain traction until Sun
> >> did it.
> >>
> >> If there is nothing that predates it, then the inspiration was
> >> almost certainly the device driver interface.  One interface,
> >> many devices.  VFS is the same.
> >>
> >> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> >> > When googling for File System Switch or Virtual File System most
> >> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
> >> sources mention 8th Edition.
> >> >
> >> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
> >> Multics, CTSS, etc.), but none of those seem to have had a comparable
> >> concept.
> >> >
> >> > Does anybody recall prior art (prior to 1984) in this area?
> >> >
> >> > Paul
> >>
> >> --
> >> ---
> >> Larry McVoy                  lm at mcvoy.com
> >> http://www.mcvoy.com/lm
> >>
> >

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

Re: [TUHS] VFS prior to 1984

[arnold]

Rob Pike <robpike@gmail.com> wrote:

> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
>
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
>
> -rob

Well, it's a progression, isn't it?  As ideas are tried out, they are
used and then refined, and used and refined further. Multics did that
for its predecessors,  Unix did that for Multics, and Plan 9 did that
for Unix.  The key difference in Unix and Plan 9 from others being
the continual goal towards refinement through simplification.

Arnold

Re: [TUHS] VFS prior to 1984

[Dave Horsfall]

On Tue, 23 Jun 2020, arnold@skeeve.com wrote:

> Well, it's a progression, isn't it?  As ideas are tried out, they are 
> used and then refined, and used and refined further. Multics did that 
> for its predecessors, Unix did that for Multics, and Plan 9 did that for 
> Unix.  The key difference in Unix and Plan 9 from others being the 
> continual goal towards refinement through simplification.

"Make things as simple as possible, but no simpler."

-- Dave

Re: [TUHS] VFS prior to 1984

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 8147 bytes --]

Paul -- slow cache refill  ... but I finally remembered some more of this
enough to reply...

I'm not sure if Larry remembers any of this and if he does, he might have
more details from the Sun side.

In the early/mid 90's, Dave Rosenthal was working on what become LOCKSS.  I
think he was at still at Sun at that point, but he would eventually move to
Stanford fulltime, but he could have left Sun already - I've forgotten.
 The UCLA folks (some of Popek's guys) had built a stackable file system
layer for BSD (which I think is in the contributed stuff in 4.4 - the
original BSD union FS and the UCLA replication FS for FICUS used it IIRC).
Some of the UMICH folks were messing with enhancements to NFS and it was an
active time.

Under the aegis of Unix International (UI), Gerry sponsored a pow-wow at
LCC (in the LA office by the airport) to talk about how stacking and VFS
layers.  IIRC Kleinman from then NetApp was there.   Besides' Dave, I've
forgotten now who else from Sun was there.  There must have been someone
from USL and maybe Mical Gien of Chorus.     But the whole discussion came
back to having FSS/VFS and N different  DFS's, PFS's and layers like
replication or union, *etc* in everyone's kernel.   Gerry saw there was
going to be a 'market' for support of different filing schemes, but at that
point, everything needed to be 'ported' because they were all different.

But also this idea of stacking was appealing.   If we could come up with
something that clean, then maybe we could be more independent of the
different physical file systems and we could 'push' behavior into the stack
more invisible.

But to do we needed to get some sort of agreement and all use the same
thing.

USL (Sun) was insisting that a Sun VFS++ -- one interface to rule them all
- NFS was the standard already of course.  This was not going over with the
Academic (UCLA/BSD/UMICH folks IIRC), particularly because of the lack of a
stacking property.

Dave proposed an idea of a general interposition layer on the inode that
would allow pushing and popping routines similar to the way STREAMS
worked.  Which we landed on -- everybody had to change a little.     I had
thought Dave implemented in an SVR4 kernel at some point and Chorus may
have had it.  There was talk of trying to get the spec out of UI and LCC
was acting either as an arms merchant or the swiss, was trying to get
IBM/DEC/HP/OSF to come along.   I went looking in my files and so far have
not been able to dig it up.  But you might find something hunting around
the LOCKSS work as Dave was the mastermind behind it.

Eventually, is all blew up and I'm not sure what happened to either the
specs or the code.

Clem

On Tue, Jun 23, 2020 at 4:39 PM Rob Pike <robpike@gmail.com> wrote:

> For my taste, the various Unix file system switches that I've seen are too
> firmly tied to the idea of blocks and disks and all that, making them less
> flexible than they should have been. That's why the Plan 9 version is about
> names and byte streams, to make it as general as possible.
>
> That's one of the reasons the possibilities of the file system approach to
> data has not reached its potential in Unix.
>
> -rob
>
>
> On Wed, Jun 24, 2020 at 1:14 AM Clem Cole <clemc@ccc.com> wrote:
>
>> I agree with Larry with his observations.   The only thing I would add is
>> that Popek and Walker had a file system in the UCLA Locus distributed
>> system for the 11/70 that was published in SIGOPS in late 1970s.  In fact,
>> Gerry came up with the idea after a sabbatical @ PARC.   But it was not
>> formalized like NFS as a separate (layered) FS, it was just part of the
>> basic Locus FS.
>>
>> Peter Weinberger created the file system switch (FSS) for V8 and in the
>> UNIX world, it was first.    Dave Arnovitz used it for RFS in System V,
>> while Perry Flynn and I used it for EFS, both of us after talking
>> to Peter.   Rusty and team did the VFS layer Sun @ (Larry will have to tell
>> you who actually built it, I never knew).  Rusty and I both published
>> papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the
>> difference is that Sun gave away NFS and Masscomp was not smart enough to
>> do that).
>>
>> I talked to Steve Kleinman extensively about VFS at one point, and I'm
>> pretty sure he and the rest of the Sun guys had talked to the PARC folks
>> who after Gerry went back to UCLA started working on a DFS.   The idea of
>> the state-less (idempotent) file system RPC that NFS used based on stuff
>> PARC did.   But I'm not sure PARC had anything like the FSS or the VFS
>> layer.  Peter, Dave, and I used a stateful scheme because we chose to have
>> full UNIX FS semantics, which NFS did not.  In the end, early NFS was
>> notorious for putting 'holes' in the files because of the automatic seek in
>> every operation and errors not coming until close(2) time.
>>
>> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
>> was not nearly as flexible as what Sun built [which had a crude interface
>> generator], although until years later when Mike Leibensger built PIG (the
>> Paceline Interface Generator) NFS RPC was always a PITA and not much better
>> in practice than what we had at Masscomp.
>>
>> In fact, the point of the EFS paper was it's all about the recovery when
>> there is a failure/error.  If you read his paper, Rusty's point was who
>> cares if there is an error (I've always felt vindicated that while I lost
>> the war, over time everyone came to our way of thinking and now NFS
>> V4 looks a whole lot like EFS did).
>>
>> Having a DFS as a feature was an incredible advance and proved we needed
>> something (and it needed to be standard in all systems).   NFS really lead
>> the market with that advance, although it sort of took a few years to make
>> it really good.  The fact is that others had the same idea before or at
>> least contemporary with it.
>>
>> That said, and to give the NFS team* a huge amount of credit *(and great
>> applause), the VFS layer was better thought out than the FSS and in fact
>> made it possible to add a lot of different file systems into UNIX later.
>> FSS was much more ad hoc.
>>
>> At LCC, when we built VPROC for TNC a few years later, we used some of
>> the same ideas from VFS and of course used VFS for the file system layer
>> since TNC had to have full POSIX semantics.  (It's a shame VPROC never
>> caught on the way VFS did).
>>
>> Clem
>>
>>
>> On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy <lm@mcvoy.com> wrote:
>>
>>> I like to claim credit for creating the ChangeSet concept, the grouping
>>> of a set of deltas across a number of files.  But I wasn't the first.
>>> Back when dejanews was a thing and you could search usenet posts in a
>>> date range, if you searched before I started talking about ChangeSets,
>>> you could find 6, count 'em, 6 hits.  There was a really obscure system
>>> called Aide De Camp, that had a similar concept.
>>>
>>> But if you searched after I started talking about them you would see
>>> millions of hits.
>>>
>>> So I didn't invent the concept but I sure as heck made the world
>>> understand the concept.
>>>
>>> I suspect Sun could be in a similar position.  The VFS concept is
>>> pretty sweet so there might have been someone before Sun.  I'll
>>> long odds that if there was, it didn't gain traction until Sun
>>> did it.
>>>
>>> If there is nothing that predates it, then the inspiration was
>>> almost certainly the device driver interface.  One interface,
>>> many devices.  VFS is the same.
>>>
>>> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
>>> > When googling for File System Switch or Virtual File System most
>>> sources mention Sun NFS and SysVr3 as the earliest implementations. Some
>>> sources mention 8th Edition.
>>> >
>>> > I did a (short) search  on FSS/VFS in earlier, non-Unix OS???s (Tenex,
>>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>>> concept.
>>> >
>>> > Does anybody recall prior art (prior to 1984) in this area?
>>> >
>>> > Paul
>>>
>>> --
>>> ---
>>> Larry McVoy                  lm at mcvoy.com
>>> http://www.mcvoy.com/lm
>>>
>>

[-- Attachment #2: Type: text/html, Size: 12341 bytes --]

Re: [TUHS] VFS prior to 1984

[Larry McVoy]

On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote:
> Rusty and team did the VFS layer Sun @ (Larry will have to tell
> you who actually built it, I never knew).  

Steve Kleiman is the only author on the VFS paper so I'm guessing that 
was him.

Rusty was NFS.

Joe Moran (sun!mojo if I remember correctly) did the VM implementation,
it was co-designed by Joe, Shannon, and Gingell.

Side story: part of the reason I wanted to go to Sun was because when
I was a CS student at UW-Madison, Joe was a little older than me, and
I sat and watched him do a BSD port to a 68K (I think) in a little more
than 24 hours (sketchy on the time, I think that's right, almost positive
it was more than 20 hours, it was a chunk of time).  As a guy who wanted
to be next to the metal, it was impressive.

> In the end, early NFS was
> notorious for putting 'holes' in the files because of the automatic seek in
> every operation and errors not coming until close(2) time.

You have no idea how many of those holes that 16 bit SCCS checksum has
found (BitKeeper kept it).

> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it
> was not nearly as flexible as what Sun built [which had a crude interface
> generator], although until years later when Mike Leibensger built PIG (the
> Paceline Interface Generator) NFS RPC was always a PITA and not much better
> in practice than what we had at Masscomp.

Oh, I don't know if I agree.  I took the RPC mechanism and vectorized it.
I probably have the code somewhere.  It was pretty neat, 10Mbit was the
common networking and I did RPC Vector calls to the rstatd on literally
every machine in the Mountain View campus.  It was thousands of machines
and it worked.  The code split the RPC back into send and receive and
the code had to start gathering up answers before it was finished 
sending the questions or the packets would be dropped.

It worked surprisingly well.  Ron Minnich and Gene Kim made an NFS client
that worked as demo-ware.  There is a paper about below, note that I'm
co-author only because I wrote the RPC vector code, I believe Gene did
the in kernel code, it was Ron and Gene's project.  

http://mcvoy.com/lm/papers/bigfoot.pdf

> In fact, the point of the EFS paper was it's all about the recovery when
> there is a failure/error.  If you read his paper, Rusty's point was who
> cares if there is an error (I've always felt vindicated that while I lost
> the war, over time everyone came to our way of thinking and now NFS
> V4 looks a whole lot like EFS did).

Yeah, they were sort of cavalier about errors in public.  The weird
thing is Sun ran their entire network on Sun machines.  Everything was
NFSv2 when I got there, the routers were Sun workstations, the servers
were Sun servers, the clients were Sun workstations.  They ate their own
dogfood and NFS just worked, it was pretty unheard of to hear about NFS
issues other than performance tuning (stupid biods etc).

> Having a DFS as a feature was an incredible advance and proved we needed
> something (and it needed to be standard in all systems).   NFS really lead
> the market with that advance, although it sort of took a few years to make
> it really good.  The fact is that others had the same idea before or at
> least contemporary with it.

NFS was solid in v2 at Sun.  I went to SGI and NFS was complete garbage.
SGI's networking principle engineer told me, when I complained about
NFS not being reliable, "That's what rcp is for".  I was flabbergasted,
the networking group generated $18M/year in NFS sales.

It wasn't just SGI, every vender other Sun had a crappy NFS implementation.
I'm surprised NFS didn't have a worse rep because while NFS worked on Suns,
it really didn't work anywhere near as well elsewhere.

> That said, and to give the NFS team* a huge amount of credit *(and great
> applause), the VFS layer was better thought out than the FSS and in fact
> made it possible to add a lot of different file systems into UNIX later.
> FSS was much more ad hoc.

The VFS layer was Steve, NFS was Rusty.

> At LCC, when we built VPROC for TNC a few years later, we used some of the
> same ideas from VFS and of course used VFS for the file system layer since
> TNC had to have full POSIX semantics.  (It's a shame VPROC never caught on
> the way VFS did).

VPROC was very cool and a requirement for any sort of distributed single
system image OS.  I agree it is a shame that it didn't catch on.

--lm

Re: [TUHS] VFS prior to 1984

[Rob Gingell]

On 6/24/2020 12:36 PM, Larry McVoy wrote:
> On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote:

Larry's enumeration of NFS/VFS players was good.

I'd add Bob Lyon who led the NFS team (and who later founded Legato 
along with Rusty, Joe Moran and Jon Kepecs -- all except Bob being UW 
alumni.)

Bob came to Sun via PARC and brought with him a number of the ideas that 
Clem had referenced earlier in this thread about RPC, etc. Although he's 
on few papers his influence on defining and building NFS etc. was 
essential.

Another important contributor was David Goldberg, for building the YP 
services that, while functionally separate, enabled the administrative 
and deployment environment that helped make NFS useful. David also wrote 
the floating-point paper in the appendix to the Hennessy-Patterson 
architecture textbook.

Re: [TUHS] VFS prior to 1984

[Dave Horsfall]

On Wed, 24 Jun 2020, Larry McVoy wrote:

>> In the end, early NFS was notorious for putting 'holes' in the files 
>> because of the automatic seek in every operation and errors not coming 
>> until close(2) time.
>
> You have no idea how many of those holes that 16 bit SCCS checksum has 
> found (BitKeeper kept it).

Aren't holes part of the file system semantics?  Seek beyond EOF, write a 
block, and you've created unallocated holes.  Of course, they are filled 
with zeroes as soon as you copy the file...  I did fiddle with "cp" to 
detect those holes and preserve them, but gave up for some reason 
(performance hit in checking for all-zero blocks, I think, along with some 
weird problem with the last block being all zeroes).

I used NFS when it first appeared and got bitten quite badly; I've never 
really trusted it ever since, but I understand that it's vastly improved.

-- Dave

Re: [TUHS] VFS prior to 1984

[Larry McVoy]

On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
> On Wed, 24 Jun 2020, Larry McVoy wrote:
> 
> >>In the end, early NFS was notorious for putting 'holes' in the files
> >>because of the automatic seek in every operation and errors not coming
> >>until close(2) time.
> >
> >You have no idea how many of those holes that 16 bit SCCS checksum has
> >found (BitKeeper kept it).
> 
> Aren't holes part of the file system semantics?  

I'm not talking about legit holes, I'm talking about where your data used
be served up as a list of zeros.

The SCCS checksum is weak but kinda handy.  You could see single bit errors
with it (at least you could in BitKeeper).  It would say 

checksum error: wanted %d, got %d

and you could look at the numbers and see that it was one bit off.

We found a bunch of those because computers used to be ECC or parity and
then 15 years ago or so, they just dumped the parity bit so single bit
errors go unreported (noice, computer industry).

Re: [TUHS] VFS prior to 1984

[Dave Horsfall]

On Sat, 4 Jul 2020, Larry McVoy wrote:

>> Aren't holes part of the file system semantics?
>
> I'm not talking about legit holes, I'm talking about where your data 
> used be served up as a list of zeros.

Ah; my mistake...  Too much blood in my coffee stream.

> The SCCS checksum is weak but kinda handy.  You could see single bit 
> errors with it (at least you could in BitKeeper).  [...]

I used to edit SCCS files if I stuffed up an update (and didn't want to 
make another update); I discovered that if I zeroed the checksum then it 
would be recalculated...  I have no idea what happened should the 
"genuine" checksum turn out to be zero.

-- Dave

Re: [TUHS] VFS prior to 1984

[Chris Torek]

>On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
>We found a bunch of those because computers used to be ECC or parity and
>then 15 years ago or so, they just dumped the parity bit so single bit
>errors go unreported (noice, computer industry).

To be fair, there are still some ECC systems.  Unfortunately most
of the home-use Intel boxes aren't.  My own home-use box isn't and
(now 6+ years ago) I had bad RAM in it that produced single-bit
errors in an inode block that led to panics that led to me finding
the single-bit errors, but I don't know if there are some damaged
files.

I keep thinking I'll replace it with a new box that does have ECC,
but haven't gotten around to it yet.  I see some consumer-priced
AMD CPUs have at least theoretical ECC support but I haven't found
anything that says the ECC actually works, and have seen a few
articles that hint that it doesn't.  iX sells NAS boxes that do
have ECC, though.

Chris

Re: [TUHS] ECC memory

[John Gilmore]

Chris Torek <torek@torek.net> wrote:
> I keep thinking I'll replace it with a new box that does have ECC,
> but haven't gotten around to it yet.  I see some consumer-priced
> AMD CPUs have at least theoretical ECC support but I haven't found
> anything that says the ECC actually works, and have seen a few
> articles that hint that it doesn't.

All the AMD Ryzen CPUs and chipsets have built-in ECC.  It's easy since
the CPU pins talk directly to main memory.  This is one among thousands
of reasons to avoid buying Intel CPUs.

The machine I'm typing on has ECC memory, and I bought it in 2019.  You
have to pick a motherboard that wasn't designed by dolts.  (I got the
Gigabyte Aorus "AX-370-GAMING5", which of course is no longer
manufactured.)  And you have to spend an extra $5 or $10 on your DIMMs.
Get the motherboard maker's "Qualified Vendor List", e.g.:

    http://download.gigabyte.us/FileList/Memory/mb_memory_ga-ax370-gaming5_pinnacle_v4.pdf

Be sure there *are* some approved ECC dimms on the list.  Buy those.
You may or may not have to fiddle something in the BIOS settings.

Check dmesg when you first boot the machine (booting the installer is
fine).  Make sure the Linux kernel sees the DIMMS, isn't subverted by
the BIOS, and understands the CPU/memory control registers.  When it
works, you'll see something like:

[    0.180161] EDAC MC: Ver: 3.0.0
[    9.389338] EDAC amd64: Node 0: DRAM ECC enabled.
[    9.389339] EDAC amd64: F17h detected (node 0).
[    9.389375] EDAC MC: UMC0 chip selects:
[    9.389376] EDAC amd64: MC: 0:     0MB 1:     0MB
[    9.389376] EDAC amd64: MC: 2:  4096MB 3:  4096MB
[    9.389378] EDAC MC: UMC1 chip selects:
[    9.389379] EDAC amd64: MC: 0:     0MB 1:     0MB
[    9.389379] EDAC amd64: MC: 2:  4096MB 3:  4096MB
[    9.389380] EDAC amd64: using x8 syndromes.
[    9.389380] EDAC amd64: MCT channel count: 2
[    9.389422] EDAC MC0: Giving out device to module amd64_edac controller F17h: DEV 0000:00:18.3 (INTERRUPT)
[    9.389428] EDAC PCI0: Giving out device to module amd64_edac controller EDAC PCI controller: DEV 0000:00:18.0 (POLLED)
[    9.389429] AMD64 EDAC driver v3.5.0

Success!  Non-flaky main memory in a PC clone!  Cost: about $50 plus
paying attention.

	John
	

Re: [TUHS] VFS prior to 1984

[Bakul Shah]

On Jul 6, 2020, at 9:51 AM, Chris Torek <torek@torek.net> wrote:
> 
>> On Sun, Jul 05, 2020 at 10:05:57AM +1000, Dave Horsfall wrote:
>> We found a bunch of those because computers used to be ECC or parity and
>> then 15 years ago or so, they just dumped the parity bit so single bit
>> errors go unreported (noice, computer industry).
> 
> To be fair, there are still some ECC systems.  Unfortunately most
> of the home-use Intel boxes aren't.  My own home-use box isn't and
> (now 6+ years ago) I had bad RAM in it that produced single-bit
> errors in an inode block that led to panics that led to me finding
> the single-bit errors, but I don't know if there are some damaged
> files.
> 
> I keep thinking I'll replace it with a new box that does have ECC,
> but haven't gotten around to it yet.  I see some consumer-priced
> AMD CPUs have at least theoretical ECC support but I haven't found
> anything that says the ECC actually works, and have seen a few
> articles that hint that it doesn't.  iX sells NAS boxes that do
> have ECC, though.

For my fileserver I am using a Gigabyte X470 Aorus Gaming board
with "qualified" ECC RAM (dmidecode confirms this) but I am not
sure if FreeBSD has ECC support for such boards. At least I don't
know how to tell!

What I really want is a laptop with ECC to use as the fileserver
to avoid having to deal with UPS that lasts a few minutes, cabling
and suboptimal support from NUT. SSDs are fast enough for me.
Laptop batteries last long and are easy to replace and easy to
test when to shutdown the laptop. And less heat and space!

Re: [TUHS] VFS prior to 1984

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

On Sat, Jul 4, 2020 at 8:07 PM Dave Horsfall <dave@horsfall.org> wrote:

> Aren't holes part of the file system semantics?
>
Exactly - and that was the problem with NFS.   Consider two write
operations.  Remember each op is a complete operation with a seek to where
it's going.  If the first fails, but the error is not reported (NFS returns
errors on close), the second operations seek over the failed write -- UNIX
puts zeros in the file.   File closes later and the size is fine of
course.  Oh yeah, whoever bothered to check for errors on close (like the
traditional SCCS or RCS commands)?

Later to try to read your file back -- it will have a bunch of zeros.
As Larry says, running a simple checksum could catch a lot of these.

Anyway, I'm going to be good and lay off a diatribe on NFS.  It sort of
worked 'good enough.'   But I will say other systems (like AFS) were much
better, in practice, but it lost the war.
Clem

[-- Attachment #2: Type: text/html, Size: 2056 bytes --]

Re: [TUHS] VFS prior to 1984

[Larry McVoy]

On Sat, Jul 04, 2020 at 09:43:04PM -0400, Clem Cole wrote:
> On Sat, Jul 4, 2020 at 8:07 PM Dave Horsfall <dave@horsfall.org> wrote:
> 
> > Aren't holes part of the file system semantics?
> >
> Exactly - and that was the problem with NFS.   Consider two write
> operations.  Remember each op is a complete operation with a seek to where
> it's going.  If the first fails, but the error is not reported (NFS returns
> errors on close), the second operations seek over the failed write -- UNIX
> puts zeros in the file.   File closes later and the size is fine of
> course.  Oh yeah, whoever bothered to check for errors on close (like the
> traditional SCCS or RCS commands)?
> 
> Later to try to read your file back -- it will have a bunch of zeros.

So I've encountered lots of holes in NFS files where there shouldn't be
any.  So it is/was a thing.  But that said, I can't remember a single
case of encountering that on Sun's campus.  I don't know if my memory
is failing me, but I do know that when I left Sun and started working
with other NFS implementations, yeah, lots of problems.  Somehow Sun
got it right where other people didn't.  

The point I'm trying to make is that I don't think NFS was broken by
design, it worked when it was Sun servers and Sun clients.  Sun's 
entire campus, 10's of thousands of machines, used NFS.  Sun would
have screeched to a halt if NFS didn't work reliably all the time.
So it was possible to get it to work.

My guess is that other people didn't understand the "rules" and did
things that created problems.  Sun's clients did understand and did
not push NFS in ways that would break it.

My memory may not be the greatest but I can still remember being 
astonished when I first ran into people saying NFS didn't work.
It worked great for Sun and Sun's customers.

Re: [TUHS] VFS prior to 1984

[Arthur Krewat]

On 7/5/2020 10:43 AM, Larry McVoy wrote:
> So I've encountered lots of holes in NFS files where there shouldn't be
> any.  So it is/was a thing.  But that said, I can't remember a single
> case of encountering that on Sun's campus.  I don't know if my memory
> is failing me, but I do know that when I left Sun and started working
> with other NFS implementations, yeah, lots of problems.  Somehow Sun
> got it right where other people didn't.

I can say personally, since the early 90's on SunOS, I never ever saw 
this problem in a variety of environments. One being Nynex Science and 
Technology where I did a consulting stint. 800+ node Sun 3/4 SunOS 
workstation/server environment, basically everyone's desktop was a Sun 
workstation for email, documentation, whatever. Another being a defense 
contractor I was at for 7 years, they were all Sun for engineering 
workstations and servers.

There is one possibility I just thought of, and that's if the first 
write fails and then a context switch happens, if enough free space is 
made available before the next context switch back to the second write, 
I can see that being a problem ;)

As if you weren't already tired of my rambling... When it comes to 
non-Sun operating systems, all bets were off. They all (mostly) worked 
with their own kind. That usually wasn't the case when it came to 
cross-vendor support. Sun<->HP was not great, but that also may have 
been driver problems in the one instance I tried it for an extended 
period of time. Two instances at different customers of AIX<->Sun 
actually worked rather well. The YP integration was key.

The entire problem with "holey" files and NFS is certainly related to 
the usage type of the system in question. What was Sun doing? Email? 
Software development? Using a common NFS share for the compiler? And 
then copying their code up to a central location? Not a lot of 
sync/write/sync/write activity, unless object file generation is a lot 
of skipping around all over the place. ;)

art k.

Re: [TUHS] VFS prior to 1984

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1067 bytes --]

On Sun, Jul 5, 2020 at 10:43 AM Larry McVoy <lm@mcvoy.com> wrote:

> My guess is that other people didn't understand the "rules" and did
> things that created problems.  Sun's clients did understand and did
> not push NFS in ways that would break it.

I >>believe<< that a difference was file I/O was based on mmap on SunOS and
not on other systems (don't know about Solaris).   The error was handled by
the OS memory system.  You tell me about how SGI handled I/O.  Tru64 used
mmap and I think macOS does also from the Mach heritage.   RTU/Ultrix was
traditional BSD.  Stellix was SRV3.  Both had a file system cache with
write-behind.

I never knew for sure, but I always suspected that was crux of the
difference in how/where the write failure were handled.  But as you pointed
out, many production NFS sites not running Suns had huge problems with
holes in files that were not discovered until it was too late to fix them.
SCCS/RCS repositories were particularly suspect and because people tried to
use them for shared development areas, it could be a big issue.

[-- Attachment #2: Type: text/html, Size: 2287 bytes --]

Re: [TUHS] VFS prior to 1984

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 1802 bytes --]

I always used the design principle "Write locally, read over NFS".  This
obviated locking issues and fit in with the idea of fate-sharing: a write
would always succeed, even if reading would have to wait until R (the
machine doing the reading) was up.  The only additional thing I needed was
the ability for W (the machine doing the writing) to notify R that
something had changed, which I did by having R run a process that listened
on a port that would be opened and then closed by W: no data flowed over
this connection.  If this connection could not be made, the process on the
W side would loop in bounded exponential backoff.

On Sun, Jul 5, 2020 at 4:09 PM Clem Cole <clemc@ccc.com> wrote:

>
>
> On Sun, Jul 5, 2020 at 10:43 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> My guess is that other people didn't understand the "rules" and did
>> things that created problems.  Sun's clients did understand and did
>> not push NFS in ways that would break it.
>
> I >>believe<< that a difference was file I/O was based on mmap on SunOS
> and not on other systems (don't know about Solaris).   The error was
> handled by the OS memory system.  You tell me about how SGI handled I/O.
> Tru64 used mmap and I think macOS does also from the Mach heritage.
>  RTU/Ultrix was traditional BSD.  Stellix was SRV3.  Both had a file
> system cache with write-behind.
>
> I never knew for sure, but I always suspected that was crux of the
> difference in how/where the write failure were handled.  But as you
> pointed out, many production NFS sites not running Suns had huge problems
> with holes in files that were not discovered until it was too late to fix
> them.  SCCS/RCS repositories were particularly suspect and because people
> tried to use them for shared development areas, it could be a big issue.
>

[-- Attachment #2: Type: text/html, Size: 3290 bytes --]

Re: [TUHS] VFS prior to 1984

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On Sun, Jul 5, 2020 at 4:42 PM John Cowan <cowan@ccil.org> wrote:

> I always used the design principle "Write locally, read over NFS".
>
This was the basic idea of AFS.  Originally, the CMU folks did whole file
caching, but by AFS 4.0 time, they had a Locus token manager (think DLM)
that scaled really well so partial caching was allowed.  It actually made a
small disk system possible.  What tended to happen, on your first boot, of
course, you had to fill /bin and lot of heavily used directories.   But
what happened is that your system quickly had only the files you really
needed on the local disk. - the ones you were writing, and the few you used
over and over.

FWIW: I know a couple of people that still run it.  I ran it until a few
years ago when I switched NAS units just for cost reasons.

[-- Attachment #2: Type: text/html, Size: 1585 bytes --]

Re: [TUHS] VFS prior to 1984

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 1735 bytes --]

On Sun, Jul 5, 2020 at 5:06 PM Clem Cole <clemc@ccc.com> wrote:

> On Sun, Jul 5, 2020 at 4:42 PM John Cowan <cowan@ccil.org> wrote:
>
>> I always used the design principle "Write locally, read over NFS".
>>
> This was the basic idea of AFS.  Originally, the CMU folks did whole file
> caching, but by AFS 4.0 time, they had a Locus token manager (think DLM)
> that scaled really well so partial caching was allowed.  It actually made a
> small disk system possible.  What tended to happen, on your first boot, of
> course, you had to fill /bin and lot of heavily used directories.   But
> what happened is that your system quickly had only the files you really
> needed on the local disk. - the ones you were writing, and the few you
> used over and over.
>
> FWIW: I know a couple of people that still run it.  I ran it until a few
> years ago when I switched NAS units just for cost reasons.
>

There was a neat paper out of CERN a few years ago about how they're
turning down their AFS (now OpenAFS) cells.
https://iopscience.iop.org/article/10.1088/1742-6596/898/6/062040/pdf

It seems that the idea of a big, shared, distributed file namespace is
sadly disappearing. I feel like most of the web-based replacements are not
as seamlessly integrated with my preferred toolset as what they're
replacing, but I've also become more and more acutely aware that I am not
the target audience for those things.

Certainly, real-time collaboration via e.g. Google Drive is pretty amazing
and very dynamic, particularly when paired with e.g. real-time video chat,
but it also forces one into a particular model of interaction that I've
spent most of the last three decades consciously avoiding but now find no
escape from.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2855 bytes --]

Re: [TUHS] VFS prior to 1984

[Anthony Martin]

Paul Ruizendaal <pnr@planet.nl> once said:
> When googling for File System Switch or Virtual File System most sources
> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
> mention 8th Edition.
>
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex,
> Multics, CTSS, etc.), but none of those seem to have had a comparable
> concept.
>
> Does anybody recall prior art (prior to 1984) in this area?

The concept of the "remote inode" was introduced in:

Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall,
W. T. (1981). A distributed UNIX system based on a virtual circuit
switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168.

Cheers,
  Anthony

Re: [TUHS] VFS prior to 1984

[Anthony Martin]

The only other prior art I know of is

  Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE
  Transactions on Communications, 28(4), 575-584.

wherein they state the following:

  A virtual file system, from the viewpoint of application programs on a
  computer, models the file system functions of other computers. This is
  done in the same way as with virtual terminals, a virtual file system
  consists of a UPP having virtual files (VF), and a UCP which executes
  virtual file system protocols.

I'd be interested if you find anything earlier.

  Anthony

Re: [TUHS] VFS prior to 1984

[Paul Ruizendaal]

On 24 Jun 2020, at 18:51, Anthony Martin <ality@pbrane.org> wrote:
> 
> Paul Ruizendaal <pnr@planet.nl> once said:
>> When googling for File System Switch or Virtual File System most sources
>> mention Sun NFS and SysVr3 as the earliest implementations. Some sources
>> mention 8th Edition.
>> 
>> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex,
>> Multics, CTSS, etc.), but none of those seem to have had a comparable
>> concept.
>> 
>> Does anybody recall prior art (prior to 1984) in this area?
> 
> The concept of the "remote inode" was introduced in:
> 
> Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall,
> W. T. (1981). A distributed UNIX system based on a virtual circuit
> switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168.


> The only other prior art I know of is
> 
>  Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE
>  Transactions on Communications, 28(4), 575-584.
> 
> wherein they state the following:
> 
>  A virtual file system, from the viewpoint of application programs on a
>  computer, models the file system functions of other computers. This is
>  done in the same way as with virtual terminals, a virtual file system
>  consists of a UPP having virtual files (VF), and a UCP which executes
>  virtual file system protocols.

Thanks for those pointers: I had not thought to include distributed systems in my initial search. Clem also pointed out LOCUS in this area. It would seem that S/F-Unix (the Luderer paper) and LOCUS happened at about the same time and both have the concept of a remote inode.

The Luderer paper has only a few details about implementation and besides describing the remote inode concept they mention:

“Of 27 system call types related to files, 18 result in message traffic if remote files are involved. Of these, 10 contain a path name as an argument, and the remaining 8 refer to already opened files. Path names or the data read or written can be up to 64K bytes long. The structure of each message is a type code followed by type-dependent data.”

In a way, the approach resembles the system call forwarding that Heinz Lycklama did for his 'satellite systems' work.

I asked Bill Marshall (one of the S/F co-authors) about it a few months ago and he recalls that they ran into problems with corner case semantics in their implementation, and that Peter Weinberger later approached the issue afresh 'thinking about the hard cases first.' Bill does not have S/F-Unix source code, and I have not been able to contact Gottfried Luderer yet.

I have not yet been able to look at the DCNA paper yet.

> I'd be interested if you find anything earlier.

I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
https://ieeexplore.ieee.org/document/762533

The abstract says:

"RIDE (Resource-Sharing in a Distributed Environment) has been designed and implemented for the UNIX operating system to provide sharing of remote files, remote process invocation and interprocess communication in a distributed computing environment. The system is designed to support a uniform interface for both local and remote access in a network. The user programs can be executed in a single or multiple machine environment without any program modification.”

I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach.

It does seem an interesting paper in any case, as its front page includes: 'This system was designed to replace the network interface programs that previously operated on the SPIDER system.'

Re: [TUHS] VFS prior to 1984

[Rob Gingell via TUHS]

On 6/24/2020 11:31 AM, Paul Ruizendaal wrote:
> I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
> https://ieeexplore.ieee.org/document/762533
> 
> The abstract says:
> 
> "RIDE (Resource-Sharing in a Distributed Environment) ...
RIDE seems very reminiscent of the RSEXEC (Resource Sharing EXEC) done 
on TENEX earlier in the 1970s. The implementation of RSEXEC involved 
encapsulation of a process fork by its parent rather than an 
object-oriented abstraction inserted behind an existing interface like 
VFS is. Both are functionally transparent to their users but are kind of 
going in opposite directions in implementation.

As with other mechanisms that have been referenced in this thread, these 
illustrate the claim that "all problems in computer science can be 
solved by [adding] another layer of indirection". I don't know that 
that's absolutely true but I've certainly applied it a lot, and like a 
lot of things, excessive application can create more problems.

(I never knew who was the first to utter the claim as I've heard it said 
by lots of people. However a quick check shows an attribution to David 
Wheeler in the form of the "Fundamental theorem of software engineering".)

https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering

RSEXEC was a pretty remarkable facility in action. It (approximately) 
was like having a command interpreter for the aggregated TENEX resources 
of the ARPAnet. It was at least partly intended to serve users of TIPs 
(the terminal interface processors used to access hosts) by letting them 
treat all TENEXes as a single resource and not worry about which one 
they got hosted onto "this time" with respect to their files, 
applications, etc., and importantly also not worry about the where their 
collaborators were hosted at the moment.

A description of RSEXEC is at:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.830.1403&rep=rep1&type=pdf

The encapsulation mechanism was JSYS Traps, JSYS being the system call 
instruction on PDP-10s modified to run TENEX. A description of that is at:

https://archive.org/details/bitsavers_bbntenexTh_1137399

In it's way, RSEXEC illustrated one characteristic of many excellent 
innovations in that when used as intended it was "deceptively 
unimpressive". As in: harder to demo.

NFS and other things share this trait, and indeed early trade show 
demonstrations of NFS were frustrating (in those days, the engineers did 
booth duty). At first, demos reduced to typing "ls". Which, 
unsurprisingly, produced a list of files. It's not until you know how it 
was done that it's impressive.

There can be a real utilitarian pleasure in things that, when shown to 
others, produce a "yeah, wasn't it supposed to do that?" reaction. Part 
of that aesthetic appreciation for me originated with RSEXEC. Which 
brings me to VFS and such.

Earlier in this thread Clem Cole observed:

On 6/23/2020 8:12 AM, Clem Cole wrote:
> That said, and to give the NFS team_a huge amount of credit _(and great 
> applause), the VFS layer was better thought out than the FSS and in fact 
> made it possible to add a lot of different file systems into UNIX 
> later.  FSS was much more ad hoc.

I think VFS and several related things are another instance of that 
aesthetic appreciation. [I'm not claiming any credit, the initial NFS 
work was just finishing up as I arrived at Sun. That work like that 
happened there was part of why I wanted to be there and so I share 
Clem's appreciation for the team.]

NFS could have been implemented by sprinkling "if (network)" statements 
around a lot of code. A user or application program would not know any 
differently had it been done that way. (Though they would over time as 
the code became less maintainable with that implementation approach.)

That VFS has the qualities leading to Clem's applause is I think due to 
a confluence of several things that likely hadn't occurred previously 
and thus entitle it to claim "first" in response to the thread-starting 
question.

One of the motivators was the desire to have a network filing 
abstraction, and to replace "nd" for diskless workstations among other 
things. (That took a while, "nd" really hung on for various reasons.) 
That's occurring in an environment of heterogeneity and "open systems" 
thinking (which is a different thing from "open source"), and the desire 
to be more "of the network" in the product offerings.

There was kind of a group-innate understanding that to be successful in 
introducing these capabilities that they had to occur largely "behind" 
the extant interfaces used by the nascent UNIX application market. To 
some extent Sun's market success was based on being deliberately "more 
vanilla than thou" with respect to APIs. The variations already extant 
were a market-fragmenting influence and thus didn't need an additional 
player doing more of it.

To be "of the network" made NFS a "file system for the Network (that 
happens to look like UNIX)". The similarity and the known semantic 
differences lead some to look at it as "it failed to do UNIX" but it's 
not. The intent was "do the network, the network is NOT 'a' computer 
(irony alert), the fallacies of networking apply and we resolve the 
irreconcilable in favor of the network".

(An intangible factor relates to the group that Sun formed, talented 
engineers, strikingly different people who shared good taste in design 
and the desire to do the work to find the "simple" in a problem. A 
different group of people with the same problem set wouldn't have 
automatically gotten the same or even similar answer, sometimes it's 
just in the air. Of course, a different group might have made an even 
better answer, "innovation happens elsewhere" is more true than the 
earlier-referenced layer of indirection principle.)

Sometimes (mostly even), good taste is expressed in framing the problem 
appropriately. The "N" in NFS is deliberately not a "D". That's obvious 
from the resulting definition but earlier in the effort it was a choice. 
The intended distinction was that we weren't doing operating system work 
in order to transcend the network by submerging it in the abstractions 
of an OS. Rather, we wanted to build a network environment populated by 
systems which were of a piece with it. While not codified until much 
later, Deutsch's "Fallacies of Networking" were the controlling notions 
in a lot of thinking and influenced things that at first wouldn't seem 
to have anything to do with networking (e.g., msync()).

However, it would be wrong to say we realized enough at the time to 
*know* that our goal was to build a system defined by a metric that it 
avoid all of the networking fallacies. We knew some of it, were 
incrementally hunting it, instincts were oriented that way. Had we been 
a research organization that might have given rise to an initiative that 
was more clearly defined. (But, I think we did a reasonable job of 
pursuing the ideas while making products that people who weren't 
thinking of such things bought. As it happens, a lot of FORTRAN users as 
has been noted here before.

Regarding the earlier irony alert: to the extent that the evocative 
marketing slogan of "The Network is the Computer" had any technical 
underpinnings, it was an expression of this aim to design in favor of 
the network. It isn't really irony so much as consequence.

The real irony is that when the slogan was first trotted out almost all 
of the company but particularly engineering barfed at it. I recall the 
marketing person(s) (individual contributors, not the executives later 
claiming credit) trying to push it receiving huge rations of abuse. But 
it stuck. And we came around once we looked beyond its technical 
deficiency as a literal statement and saw it as evocative of what we had 
been doing. They knew us better than we did. And, independent of any 
technical matter, it worked really well. 'Find the simple" is a useful 
admonition for marketing as well as engineering.

[TUHS] Oh, things were very different.

[Rich Morin]

https://imgs.xkcd.com/comics/old_days_2.png

-r

Re: [TUHS] Oh, things were very different.

[Lawrence Stewart]

> On 2020, Jun 25, at 12:15 AM, Rich Morin <rdm@cfcl.com> wrote:
> 
> https://imgs.xkcd.com/comics/old_days_2.png
> 
> -r
> 

The trick to these is to include one nugget of truth.  If anyone asks, I’m saying the sweater part is absolutely true.

After all, BITNET actually did work by connecting a virtual card punch to a remote virtual card reader.

-L

Re: [TUHS] Oh, things were very different.

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

On Thu, Jun 25, 2020 at 9:52 AM Lawrence Stewart <stewart@serissa.com>
wrote:

>
>
> > On 2020, Jun 25, at 12:15 AM, Rich Morin <rdm@cfcl.com> wrote:
> >
> > https://imgs.xkcd.com/comics/old_days_2.png
> >
> > -r
> >
>
> The trick to these is to include one nugget of truth.  If anyone asks, I’m
> saying the sweater part is absolutely true.
>
> After all, BITNET actually did work by connecting a virtual card punch to
> a remote virtual card reader.
>

I have thousands of virtual sweaters in a drawer upstairs. They were what
inspired virtual machines and the cloud after all...

Warner

[-- Attachment #2: Type: text/html, Size: 1149 bytes --]

Re: [TUHS] Oh, things were very different.

[Arthur Krewat]

On 6/25/2020 11:45 AM, Lawrence Stewart wrote:
> The trick to these is to include one nugget of truth.  If anyone asks, I’m saying the sweater part is absolutely true.
>
> After all, BITNET actually did work by connecting a virtual card punch to a remote virtual card reader.
>
> -L

I always knew Mr. Rogers was a deep state operative. All those 
sweaters... probably generated by years of anti-encryption efforts...

Re: [TUHS] Oh, things were very different.

[Dave Horsfall]

On Wed, 24 Jun 2020, Rich Morin wrote:

> https://imgs.xkcd.com/comics/old_days_2.png

Got the original cartoon number for that?  I like to see the comment when 
I hover over it (and the explanation site can be funny, too).

-- Dave

Re: [TUHS] Oh, things were very different.

[Michael Kjörling]

On 26 Jun 2020 06:22 +1000, from dave@horsfall.org (Dave Horsfall):
>> https://imgs.xkcd.com/comics/old_days_2.png
> 
> Got the original cartoon number for that?

It's brand new.

https://xkcd.com/2324/

The first one is https://xkcd.com/1755/

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
 “Remember when, on the Internet, nobody cared that you were a dog?”

Re: [TUHS] Oh, things were very different.

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

2324

On Thu, Jun 25, 2020 at 4:23 PM Dave Horsfall <dave@horsfall.org> wrote:

> On Wed, 24 Jun 2020, Rich Morin wrote:
>
> > https://imgs.xkcd.com/comics/old_days_2.png
>
> Got the original cartoon number for that?  I like to see the comment when
> I hover over it (and the explanation site can be funny, too).
>
> -- Dave
>

[-- Attachment #2: Type: text/html, Size: 805 bytes --]

Re: [TUHS] Oh, things were very different.

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

On Thu, 25 Jun 2020, Clem Cole wrote:

> > Got the original cartoon number for that?  I like to see the comment 
> > when I hover over it (and the explanation site can be funny, too).
> 
> 2324

Thanks!  The hover-over makes a bit more sense now, and explainxkcd.com is 
fascinating.

-- Dave

Re: [TUHS] VFS prior to 1984

[Paul Ruizendaal]

> I came across this 1979 paper by P.M. Lu from Bell Labs Naperville
> 
Full paper is available here:
https://www.computer.org/csdl/pds/api/csdl/proceedings/download-article/12OmNzd7c1v/pdf

> I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach.

Indeed it appears to have been like the (later) Newcastle Connection in its approach.

Re: [TUHS] VFS prior to 1984

[Derek Fawcus]

On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote:
> I did a (short) search  on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?

Surely any OS, even non unix, which offered a remote FS in addition to a local FS implicitly implements something like a FSS/VFS?

So we could look to CP/M (and CP/Net, CP/NOS), where this[1] seems to suggest a date between 1980 and 1982.
An appendix to that paper has the CPM file system operations (such as they were) remotely available and
listed with their codes for the messages.

That said, another OS from DR (FlexOS) has a recognisable VFS, is written in C, but dates to early 1985.
I'm not sure when the design dates from[3]. If one looks at this[2] article in Byte,
each of the Resource Managers operates through a VFS layer, in terms of:

    #define I_GET           3 /* NET DISK CON MSC PIPE KERN */
    #define I_SET           4 /* NET DISK CON MSC PIPE KERN */
    #define I_LOOKUP        5 /* NET DISK CON MSC PIPE KERN */
    #define I_OPEN          6 /* NET DISK CON MSC PIPE */
    #define I_CLOSE         7 /* NET DISK CON MSC PIPE */
    #define I_READ          8 /* NET DISK CON MSC PIPE */
    #define I_WRITE         9 /* NET DISK CON MSC PIPE */
    #define I_SEEK          10 /* NET DISK CON MSC PIPE */
    #define I_CREATE        11 /* NET DISK CON PIPE */
    #define I_DELETE        12 /* NET DISK CON PIPE */

Which can be seen in the public code for CDOS-68k.

Not all RMs implement all calls, but the NET, DISK and PIPE sort of did.

DF

[1] https://www.autometer.de/unix4fun/z80pack/cpnet/cpnet.htm
[2] http://tech-insider.org/personal-computers/research/1985/05.html
    https://archive.org/stream/byte-magazine-1985-05/1985_05_BYTE_10-05_Multiprocessing#page/n380/mode/1up/search/concurrent
[3] The 80286 was released in '82, but I don't know when Intel started giving early info to customers.

Re: [TUHS] VFS prior to 1984

[Greg A. Woods]

[-- Attachment #1: Type: text/plain, Size: 5575 bytes --]

At Tue, 23 Jun 2020 11:09:57 +0200, Paul Ruizendaal <pnr@planet.nl> wrote:
Subject: [TUHS] VFS prior to 1984
> 
> When googling for File System Switch or Virtual File System most
> sources mention Sun NFS and SysVr3 as the earliest
> implementations. Some sources mention 8th Edition.
> 
> I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex,
> Multics, CASS, etc.), but none of those seem to have had a comparable
> concept.
> 
> Does anybody recall prior art (prior to 1984) in this area?

Well, with regard to Multics, it had what were called "Device Interface
Modules" (DIMs) for building device drivers to either hardware or
pseudo-devices (an example of a pseudo-DIM is the File System Interface
Module (FSIM) which associates a segment (file) in the filesystem (by
path name) with a stream, i.e. it implements I/O redirection).

DIMs had a well defined API, and as Organick said in his 1972 book:
"The DIM converts a device independent request into a device dependent
one." ... "The file-system interface DIM is used to make a segment look
like an I/O device."

A DIM also had a name, an identifier, and that's how you attached a
"stream" to it:

	call attach (stream_name, DIM_name, device_name);

All of those parameters are strings.

As an aside I haven't found any evidence that DIM identifiers were ever
listed in the filesystem, e.g. as Unix would have in "/dev", but that
would be an obvious thing to do these days.  Then they were just listed
in the programmer's manual, and presumably user-written ones were added
to the online documentation.  Today such an idea would be "obvious".

As far as I can remember Multics didn't really have the concept of a
"mount point".  All storage was single-level, i.e. segments (equivalent
in some respects to inodes, but they are also actually the value of the
segment register in the virtual memory hardware), and so files were
either physically in memory or paged out on physical disk devices or
similar, or even out on tape.  Where they actually resided was entirely
and permanently hidden from the user.  What was called the "filesystem"
was a form of database representing a hierarchical namespace which
pointed at all the known segments (files) regardless of where they were
actually stored.

In terms of virtual filesystems for Multics, I think it would be (have
been) relatively easy enough (i.e. with the addition of some APIs to the
filesystem itself) to implement a way of "mounting" a pseudo-DIM into
the filesystem namespace and thus being able to implement anything from
the likes of /kern and /proc to remote filesystems, network directories,
etc.  I.e. create an inverse twin of the filesystem DIM that takes
filesystem requests and turns them into back-end pseudo-DIM requests.
Effectively this would be implementing a VFS for Multics.  Indeed today,
and perhaps any time after Apollo Domain was designed, it would be
plainly obvious, and probably desirable, to do such a thing, at least
for some kinds of concepts, such as /proc.

Note though that I suspect the original Multics designers would have
preferred it if remote filesystems, in particular, were also effectively
invisible to the user, though I do not know how they might have
approached the issue of remote segments not always being available,
including indeed parts of the filesystem itself, i.e. directories and
metadata.  Perhaps in a similar way to how files migrated to backup
storage were handled -- if you referenced a file that had been migrated
offline then your process trapped to a retrieval process that involved
prompting a human tape operator to load the appropriate backup tape(s).

In fact Organick says the following about this idea of an inverse
relationship between the filesystem and I/O subsystem in his 1972 book
(p351,352):

    "We have just seen how the I/O system may behave as a customer of
    the file system, which supplies needed services.  One might wonder
    if the reverse of those roles is ever true.  That is, does the file
    system through its Page Control Module, when seeking to transfer a
    page of information to/from core [[memory]] and disk or tape
    storage, ever find itself to be a customer of the I/O system?
    Emphasis on objectives of modularity might cause one to guess the
    affirmative.  In actual fact however, considerations of efficiency
    have dictated that paging I/O in Multics be treated with
    special-purpose I/O software that greatly streamlines the
    processor's task in initiating and controlling such I/O."

So, at least early on, he appears to have denied the idea of the
filesystem routing requests through a DIM or pseudo-DIM, though he did
so on the basis that this would be done at the pager level and that
doing so could be too inefficient to consider.  However he doesn't seem
to have considered, or at least does not mention in the 1972 book, the
idea of a pseudo-DIM extending the filesystem namespace by implementing
a mount point, which as I say would today be considered a quite obvious,
desirable, and feasible task.

In any case the idea of pseudo device drivers interacting with the
filesystem, albeit the other way around, was firmly in place by 1965
(the file-system interface pseudo-DIM was described in the 1965 Multics
I/O system paper).

-- 
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [TUHS] VFS prior to 1984

[Adam Thornton]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods <woods@robohack.ca> wrote:

>
> As far as I can remember Multics didn't really have the concept of a
> "mount point".  All storage was single-level, i.e. segments (equivalent
> in some respects to inodes, but they are also actually the value of the
> segment register in the virtual memory hardware), and so files were
> either physically in memory or paged out on physical disk devices or
> similar, or even out on tape.  Where they actually resided was entirely
> and permanently hidden from the user.  What was called the "filesystem"
> was a form of database representing a hierarchical namespace which
> pointed at all the known segments (files) regardless of where they were
> actually stored.
>
>
Coming to it from a Unix perspective, it's like all storage (core, disk,
tape) is mmap()ed.

The segment-name database then is just an index relating symbolic names to
particular memory locations.

It all feels very upside-down to me, but that's probably because I grew up
in Unix and never actually used a Multics system until I emulated one with
dps8m.

Adam

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

Re: [TUHS] VFS prior to 1984

[Greg A. Woods]

[-- Attachment #1: Type: text/plain, Size: 3495 bytes --]

At Wed, 24 Jun 2020 17:45:38 -0700, Adam Thornton <athornton@gmail.com> wrote:
Subject: Re: [TUHS] VFS prior to 1984
>
> On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods <woods@robohack.ca> wrote:
> >
> > As far as I can remember Multics didn't really have the concept of a
> > "mount point".  All storage was single-level, i.e. segments (equivalent
> > in some respects to inodes, but they are also actually the value of the
> > segment register in the virtual memory hardware), and so files were
> > either physically in memory or paged out on physical disk devices or
> > similar, or even out on tape.  Where they actually resided was entirely
> > and permanently hidden from the user.  What was called the "filesystem"
> > was a form of database representing a hierarchical namespace which
> > pointed at all the known segments (files) regardless of where they were
> > actually stored.
> >
> >
> Coming to it from a Unix perspective, it's like all storage (core, disk,
> tape) is mmap()ed.
>
> The segment-name database then is just an index relating symbolic names to
> particular memory locations.

Yes, exactly.  A process is just a collection of segments of (virtual)
memory, some containing code, some containing data, and more can be
attached, some can be detached and/or swapped, as execution proceeds.
E.g. for shared code, a module can be swapped out for another equivalent
one, e.g. with a bug fix, without stopping or restarting the process
(and perhaps, if it is a system library, without the user or application
even knowing that has happened).

> It all feels very upside-down to me, but that's probably because I grew up
> in Unix and never actually used a Multics system until I emulated one with
> dps8m.

I kind of thought of it that way in my early years too, even though
Multics was one of the first systems I learned, at the same time as I
learned Unix.

In Multics I/O takes a far lesser role than it does in Unix (or most any
other OS then and since).

Modeling everything as if it were I/O has some advantages in terms of
implementation of the model (I guess it makes for far simpler operating
systems), but personally I now think it's completely upside down and
inside out from the user's perspective.

Having to read all data before you can actually access it, and/or even
just having to manage your allotted memory space and pointer offsets, is
quite an annoying addition to any algorithm -- i.e. adding I/O in front
of everything is like adding a significant problem on top of every other
problem, every time, all the time.  (Well using full-on only mmap() and
a good language and compiler could probably hide much of the nastiness,
but....  we don't really seem to do that even to this day, even in
languages that could make it easy.)

So, personally, I now see modeling everything as I/O as if it is forcing
you to access all your data through the eye of a needle, every time, all
the time.  No matter how big the hole you still have to coil all your
data up and push/pull it through if it is, or could be, bigger than the
hole.

I'm also quite sad that the extremely wide and near pervasive and
long-term availability of a CPU family with half-decent support for
segmented virtual memory has effectively been wasted for a generation or
two or more of possibilities in OS design.

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]