[TUHS] Early non-Unix filesystems?

[TUHS] Early non-Unix filesystems?

[Warren Toomey]

It's a bit off-topic, but what were non-Unix filesystems like around 1969-1970?
The PDP-7 filesystem has i-nodes (file metadata) and filenames separate
from the i-nodes. This allows hard links and thus a non-tree structured
filesystem.

This has always struck me to be one of the most important features of
the Unix filesystem: names separated from the rest of the file metadata,
and arbitrary hard links so that there is no preferred filename.

Were these features in other contemporaneous filesystems?

As a side note, the PDP-7 kernel knows about the top-level directory ("dd")
but it is agnostic to the concept of "." and "..". What that means is
that you can build a filesystem with "." and ".." links and the kernel
will deal with them as per all other links. But you can also build a
filesystem without "." or ".." and the kernel doesn't care.

There's not enough evidence (source code, papers, anecdotes) to confirm
or deny the presence of "." in the PDP-7 code that Norman scanned for us.
".." does seem to exist as it is mentioned in one source file, ds.s.
It's an intruiging mystery.

Cheers, Warren

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Friday, 18 March 2016 at 10:48:32 +1000, Warren Toomey wrote:
> It's a bit off-topic, but what were non-Unix filesystems like around
> 1969-1970?

A year or two earlier, maybe: the Master File Directory of UNIVAC
EXEC-8.  It's an amazingly complex and quite versatile system that
makes the Unix file system look simple.  There's some documentation at
http://bitsavers.trailing-edge.com/pdf/univac/1100, though right now I
can't find the references I'm looking for.

> The PDP-7 filesystem has i-nodes (file metadata) and filenames
> separate from the i-nodes. This allows hard links and thus a
> non-tree structured filesystem.
>
> This has always struck me to be one of the most important features
> of the Unix filesystem: names separated from the rest of the file
> metadata,

Agreed.  In fact, I'd go as far as to say that it's one of the most
important features of Unix.  And I'm pretty sure that the MFD didn't
have anything as straightforward and powerful.  But I'll research and
see if I can comeup with something interesting.

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160318/b05142ad/attachment.sig>

[TUHS] Early non-Unix filesystems?

[John Cowan]

Warren Toomey scripsit:

> It's a bit off-topic, but what were non-Unix filesystems like around
> 1969-1970?  The PDP-7 filesystem has i-nodes (file metadata) and
> filenames separate from the i-nodes. This allows hard links and thus
> a non-tree structured filesystem.

Not only that, but a non-tree-structured directory system: the directories
themselves were a general directed graph, and it was possible to have any
number of arbitrary hard links to a directory.  This was simplified in
later Unices to the tree-structured directory system we have today, where
the only hard links to a directory are the named link from the parent,
the "." link from self, and the ".." links from the child directories.

There was also no concept of pathnames in PDP-7 Unix, neither relative
nor absolute.  The link syscall was link("foo", "bar", "baz"), meaning
to cause entry "baz" in directory "bar" in the working directory to
be the same file or directory as entry "foo" in the working directory.
So for example, each home directory had a hard link to the main binaries
directory, conventionally named "bin", and the shell could not execute
commands out of a directory that did not contain a "bin" link.  With no
reliable ".." entry, there was also no way to recover from a chdir, and
traversal had to be done as digraph traversal rather than tree traversal.
In particular, incautious use of unlink could cause the directory graph
to become partitioned, requiring low-level disk surgery to recover.

An interesting property of the ODS-1 (RSX) file system was that the
root directory contained the i-node table as a file named INDEXF.SYS.
This allowed ordinary programs to inspect the i-node table without messing
about with raw disks.  It also contained a self-entry named 000000.DIR,
but other directories did not contain self-links.

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
The internet is a web of tiny tyrannies giving an illusion of anarchy.
                --David Rush

[TUHS] Early non-Unix filesystems?

[Warren Toomey]

On Thu, Mar 17, 2016 at 11:59:36PM -0400, John Cowan wrote:
> There was also no concept of pathnames in PDP-7 Unix, neither relative
> nor absolute.  ...
> directory, conventionally named "bin", and the shell could not execute
> commands out of a directory that did not contain a "bin" link.

Are you sure it was called "bin"? The PDP-7 team are working on the
assumption that it was called "system" because init links the shell
from "system" into a user's home directory:

   sys setuid		" Set the user's user-id,
   sys chdir; dd	" change into the "dd" directory
   sys chdir; dir	" and then into the user's directory
   ...
   sys link; system; sh; sh	" Link sh in system dir to this dir
system:
   ...
   <sy>;<st>;<em>; 040040

https://github.com/DoctorWkt/pdp7-unix/blob/master/src/cmd/init.s

There's no mention of the directory's name in the kernel, but i-node 3
has to contain init (we assume "system" again) and i-node 4 is the
directory that holds users' home directories ("dd").

Cheers, Warren

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Fri, 18 Mar 2016, Warren Toomey wrote:

> It's a bit off-topic, but what were non-Unix filesystems like around 
> 1969-1970?

With my IBM/360 experience, "horrible".

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Peter Jeremy]

On 2016-Mar-18 19:00:00 +1100, Dave Horsfall <dave at horsfall.org> wrote:
>On Fri, 18 Mar 2016, Warren Toomey wrote:
>> It's a bit off-topic, but what were non-Unix filesystems like around 
>> 1969-1970?
>
>With my IBM/360 experience, "horrible".

That's being unfair to filesystems.  You can't call what you got with
OS/360 a "filesystem".  IMHO, a filesystem needs more than a way to
associate a name with a manually-allocated region of DASD.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160318/0d164d6f/attachment-0001.sig>

[TUHS] Early non-Unix filesystems?

[Clem Cole]

FYI:  To be fair to the IBM systems of the time, TSS and MTS had a naming
scheme that used dots (.) as the path separators in their pathnames and
they too supported a directory/subdirectory idea.   While ugly in my
opinion compared to UNIX, not really much worse than later schemes that DEC
would use for TOPS, RSX, VMS et al (I once got into a "discussion" with
Cutler about VMS's #$%%^ filenames but that's a different story).

The key point is that around this time the idea of "sub-directories" began
to become important.   The file cabinet metaphor was beginning to come into
age as people recognized the systems using "soft files" had the same issues
that "paper files" and they needed some way to build up similar "filing
mechanisms" with cabinets, drawers, file folder etc....


That said, I agree that  OS/360 was primitive but being batch oriented, I
think did not consider the "human factors" the way the time sharing system
required started to do.

Anyway back to Warren's questions -- the idea of a disk "catalog" for the
meta-data was not new to UNIX.   Tannenbaum's books (starting with the
Minux book) describes the history and the idea reasonably well.    What
UNIX did was create a user file type called  a directory (eventually user
create-able) and was able to separated the human name for the bag of bit
(i.e. the data itself) from the catalog or system name for those same bits.
  As AST puts it, UNIX has a two level file system, one for the OS which is
numerical in nature (the i-node) and then layered on top of it was
something humans liked better.

Again, by breaking the idea/metaphor of a files and file cabinets into two
pieces - higher level (human oriented) and lower level (system specific),
like the splitting the command system of the shell out of the OS
fork/exec/wait primitives, UNIX ended up with something more more flexible
and scalable.

Clem

On Fri, Mar 18, 2016 at 4:42 AM, Peter Jeremy <peter at rulingia.com> wrote:

> On 2016-Mar-18 19:00:00 +1100, Dave Horsfall <dave at horsfall.org> wrote:
> >On Fri, 18 Mar 2016, Warren Toomey wrote:
> >> It's a bit off-topic, but what were non-Unix filesystems like around
> >> 1969-1970?
> >
> >With my IBM/360 experience, "horrible".
>
> That's being unfair to filesystems.  You can't call what you got with
> OS/360 a "filesystem".  IMHO, a filesystem needs more than a way to
> associate a name with a manually-allocated region of DASD.
>
> --
> Peter Jeremy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160318/f0c1d989/attachment.html>

[TUHS] Early non-Unix filesystems?

[Pat Barron]

On 3/17/2016 8:48 PM, Warren Toomey wrote:
> As a side note, the PDP-7 kernel knows about the top-level directory ("dd")
> but it is agnostic to the concept of "." and "..". What that means is
> that you can build a filesystem with "." and ".." links and the kernel
> will deal with them as per all other links. But you can also build a
> filesystem without "." or ".." and the kernel doesn't care.
>
> There's not enough evidence (source code, papers, anecdotes) to confirm
> or deny the presence of "." in the PDP-7 code that Norman scanned for us.
> ".." does seem to exist as it is mentioned in one source file, ds.s.
> It's an intruiging mystery.

So, I'm trying to figure out - at what point did "." and ".." really 
become part of the filesystem?  Even as late as V7, "." and ".." were 
not automatically added to directories when created - the "mkdir" 
program used mknod() to create the directory, and then created the links 
for "." and ".." by itself.  I see code in the kernel that has special 
handling of "." in unlink(), and also some in "namei()" - but I don't 
see anywhere where ".." is obviously handled at all.  Though I figure it 
has to be, somehow, because when mkdir makes the directory and creates 
the links for "." and "..", it doesn't have to figure out the inode 
number or anything - that part just seems to happen behind the scenes.  
I just can't figure out where or how....

The first place I saw where mkdir() became a system call, and the system 
call handled "." and ".." by itself, was in 4.1BSD....

--Pat.

[TUHS] Early non-Unix filesystems?

[scj]

It may seem strange to us today, but in the context of the day, one of the
most radical ideas in Unix was the concept of a file as an array of bytes,
with lines separated by newline characters.  Most mainframes had file
systems that were more or less decks of cards on disk (at one point, the
Bell Labs computing centers estimated that 2/3 of the disc was taken up
with trailing blanks.   Layered on top of this was a bit of cruft that
also made the file look like a magnetic tape holding punch card images. 
So files had "blocking factors" (how many cards in a contiguous block of
disc).

In the GE (later Honeywell) time sharing system, to create a file, you
entered a subsystem which asked you nearly a dozen questions (name,
initial size, maximum size, device, permissions, etc.) and stored up your
answers and finally did a system call to create the file.  The usual
result was that something was wrong in your answer to question 5 and you
had to do the whole dance over again from the top.  If you managed to
create a file, the subsystem congratulated you by printing:
   Successful!

In this world, typing
    echo hello Joe >foo
in Unix, and thereby creating a file, seemed like a miracle -- typically a
first-timer's jaw would drop when I said we had created a file named foo
and showed them that it had contents.  Their response was typically about
30 seconds of "But...  but.... but... but..." followed by a blinding grin
when they "got it".

Steve
>> That's being unfair to filesystems.  You can't call what you got with
>> OS/360 a "filesystem".  IMHO, a filesystem needs more than a way to
>> associate a name with a manually-allocated region of DASD.
>>
>> --
>> Peter Jeremy
>>
>

[TUHS] Early non-Unix filesystems?

[John Cowan]

Warren Toomey scripsit:

> Are you sure it was called "bin"? The PDP-7 team are working on the
> assumption that it was called "system" because init links the shell
> from "system" into a user's home directory:

Probably an incorrect memory.  Of course, in an edge-labeled general
graph, directories don't have names any more than files do in modern
Unix filesystems, so it isn't "what's its name?", it is "what's its
name in some directory?"

I assume you know about
https://www.bell-labs.com/usr/dmr/www/hist.html , which is an essential
resource on Ancientest Unix.  Note especially the absence of any way to
create directories while the system is running.

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
Please leave your values at the front desk.
                --sign in Paris hotel
Check your assumptions.  In fact, check your assumptions at the door.
                --Cordelia Vorkosigan

[TUHS] Early non-Unix filesystems?

[Random832]

On Fri, Mar 18, 2016, at 13:12, scj at yaccman.com wrote:
> It may seem strange to us today, but in the context of the day, one of
> the most radical ideas in Unix was the concept of a file as an array of
> bytes, with lines separated by newline characters.  Most mainframes had file
> systems that were more or less decks of cards on disk

Of course, I assume the answer to the question of why everyone didn't do
that is that there's a trade-off: We take for granted today that you
can't change a line in the middle of a text file without moving
everything after it, either by reading the whole thing into memory and
writing back the changed version, or creating a copy of the file with
the changes and replacing the original with it afterwards, but I assume
these "deck of cards" style files had provisions for editing one in the
middle. You also can't seek to a given line number in a file.

[TUHS] Early non-Unix filesystems?

[scj]

> On Fri, Mar 18, 2016, at 13:12, scj at yaccman.com wrote:
>> It may seem strange to us today, but in the context of the day, one of
>> the most radical ideas in Unix was the concept of a file as an array of
>> bytes, with lines separated by newline characters.  Most mainframes had
>> file
>> systems that were more or less decks of cards on disk
>
> Of course, I assume the answer to the question of why everyone didn't do
> that is that there's a trade-off: We take for granted today that you
> can't change a line in the middle of a text file without moving
> everything after it, either by reading the whole thing into memory and
> writing back the changed version, or creating a copy of the file with
> the changes and replacing the original with it afterwards, but I assume
> these "deck of cards" style files had provisions for editing one in the
> middle. You also can't seek to a given line number in a file.
>

At least in my experience, editing the "deck of cards" (and certainly,
editing anything on magnetic tape) was really painful -- there was no way
to move blocks of text around -- you started at the beginning of the file
and had to edit lines in order (one shot per line) until you got to the
end.  You could add lines or delete them, but only when you came to them. 
The editor copied the edited file into an output file, and then you had to
do another step to copy the new version back over the original one.

Shortly after we had a user-writable disc in the Murray Hill computation
center, I (working there as a summer intern) was delighted to copy the
2000 card file I had been lugging around onto disc, and dump the cards. 
As it turns out, my boss also dumped his copy of the card deck.

The first time I tried to edit the deck on disc, I specified the output
file to be equal to the input file.  The program did not check this, and I
ended by nuking about 20% of the card images!  Luckily I had a listing... 
I punched out the trash on the disc and spent an entire weekend
rearranging and repunching the cards to get back to where I had been...

It just goes to show that I should have taken my mother's advice -- before
you throw out a deck of cards, put a rubber band around it!

[TUHS] Early non-Unix filesystems?

[Tony Finch]

Random832 <random832 at fastmail.com> wrote:
>
> You also can't seek to a given line number in a file.

I was slightly startled by the coolness of the idea when I found out that
nvi uses Berkeley DB as its storage layer; its recno access method
makes a text file look like a random-access array of strings.

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/  -  I xn--zr8h punycode
Dover: Variable 3 or 4. Slight. Fair. Moderate or good.

[TUHS] Early non-Unix filesystems?

[John Cowan]

Tony Finch scripsit:

> I was slightly startled by the coolness of the idea when I found out that
> nvi uses Berkeley DB as its storage layer; its recno access method
> makes a text file look like a random-access array of strings.

Classical sequential files, however, were simply random-access files
such that seeking to line n was just a matter of seeking to byte
n * MAXCHARSLINE.  The last time I actually used such a thing was
on an early Tandem system when I was implementing the Software Tools.
Editable source used a different format, so I set things up so that
the Tools could either read source format or sequential format and
then wrote sequential format.

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
At the end of the Metatarsal Age, the dinosaurs abruptly vanished.
The theory that a single catastrophic event may have been responsible
has been strengthened by the recent discovery of a worldwide layer of
whipped cream marking the Creosote-Tutelary boundary.  --Science Made Stupid

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Fri, 18 Mar 2016, scj at yaccman.com wrote:

> At least in my experience, editing the "deck of cards" (and certainly, 
> editing anything on magnetic tape) was really painful -- there was no 
> way to move blocks of text around -- you started at the beginning of the 
> file and had to edit lines in order (one shot per line) until you got to 
> the end.  You could add lines or delete them, but only when you came to 
> them. The editor copied the edited file into an output file, and then 
> you had to do another step to copy the new version back over the 
> original one.

Memories of IEBUPDTE come to me...  A very useful program, when it came to 
doing things, ahem, not allowed by the computer centre, by us Comp Sci 
kiddies...

Ah, the time I got SPITBOL to work beyond its use-by date, for example; 
the thing was riddled with date checks (the first one was obvious, but the 
rest not so much; its endearing habit was to jump to whatever was in R0 at 
the time).  I wrote something that searched for that particular date 
string, and after inspecting the surrounding binary code I patched it...

> The first time I tried to edit the deck on disc, I specified the output
> file to be equal to the input file.  The program did not check this, and I
> ended by nuking about 20% of the card images!  Luckily I had a listing... 
> I punched out the trash on the disc and spent an entire weekend
> rearranging and repunching the cards to get back to where I had been...

[...]

And who here hasn't done "cat file ... > file"?

> It just goes to show that I should have taken my mother's advice -- 
> before you throw out a deck of cards, put a rubber band around it!

Walking down the corridors of Comp Sci, a student in front of me dropped 
his entire deck of approx 2000 cards, all over the floor...  I have no 
idea whether he got them sorted, but I sure as hell used rubber bands 
after that!

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Tuesday, 22 March 2016 at  1:11:07 +1100, Dave Horsfall wrote:
>
> Walking down the corridors of Comp Sci, a student in front of me
> dropped his entire deck of approx 2000 cards, all over the floor...
> I have no idea whether he got them sorted, but I sure as hell used
> rubber bands after that!

But that's what the sequence numbers in columns 73 to 80 are for!

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/e66abd2a/attachment.sig>

[TUHS] Early non-Unix filesystems?

[scj]

> On Tuesday, 22 March 2016 at  1:11:07 +1100, Dave Horsfall wrote:
>>
>> Walking down the corridors of Comp Sci, a student in front of me
>> dropped his entire deck of approx 2000 cards, all over the floor...
>> I have no idea whether he got them sorted, but I sure as hell used
>> rubber bands after that!
>
> But that's what the sequence numbers in columns 73 to 80 are for!
>
> Greg

Actually, a Magic Marker drawing a diagonal stripe down the top of the
cards worked almost as well, and didn't require repunching the deck with
every change...

Hmm.  I recall walking into the computing center to see an amazing cascade
of cards arcing 20 feet across the room.  The operator hadn't set up the
card reader correctly....  Panic ensued...

In another case, I had written a FORTRAN program that produced a page
header on the printer with a page number.  The first time I tried to print
200 pages, when I hit 100 the printer decided that I wanted one line per
page!  The printer was not a happy camper.  Neither were the operators.

But the closest I came to being murdered was when I submitted a job before
lunch, and the sign said the turnaround time was an hour.  When I got
back, the job was not done, and the wait was 2 hours.  Each hour, the wait
grew by an hour.  At about 4:30, four extremely angry operators and
managers came into my office with murder in their eyes and accused me of
trying to "be funny" (sic) with the computer.  I had no idea what they
were talking about.  It turns out that the operating system had been
upgraded, so that it automatically started pending jobs when it came up
after a crash.  It also turned out that I had made a common FORTRAN
error--I had "called" a labeled common region instead of a similarly-named
function.  And it turned out that the first data value was a -.25
floating-point value.  And it also turned out that the computer's idea of
instruction decode was to pick up the first few bits of the word and use
it as an index into the microcode.  And it also turned out that when -.25
was so "executed", it took the CPU into the power down sequence, and it
turned the machine off!  And, thanks to the software upgrade, when they
rebooted, it turned itself off again.  And again...  I was happy to escape
with my skin...

Steve

Steve

[TUHS] Early non-Unix filesystems?

[Lyndon Nerenberg]

> Actually, a Magic Marker drawing a diagonal stripe down the top of the
> cards worked almost as well, and didn't require repunching the deck with
> every change...

That's why you increment your sequence numbers by 100 ;-)

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Monday, 21 March 2016 at  8:06:34 -0400, John Cowan wrote:
> Tony Finch scripsit:
>
>> I was slightly startled by the coolness of the idea when I found out that
>> nvi uses Berkeley DB as its storage layer; its recno access method
>> makes a text file look like a random-access array of strings.
>
> Classical sequential files, however, were simply random-access files
> such that seeking to line n was just a matter of seeking to byte
> n * MAXCHARSLINE.  The last time I actually used such a thing was
> on an early Tandem system when I was implementing the Software
> Tools.

You're presumably not talking about a Tandem format here.  Were you
working with Denis Winn?

> Editable source used a different format, ...

Presumably you *are* talking about Tandem's format here.  I always
looked at it as an indication of how badly optimization attempts could
go wrong.  The format limited line length to something less than 256
bytes (exact length was content dependent).  The lines were split into
groups of up to 30 bytes, 15 bytes content and up to 15 bytes trailing
spaces.  And this was the reason: it was a piss-poor attempt at
efficient storage.  The only way to access it was with a special
library with procedures like EDITREAD and EDITWRITE.  I still have a
number of these files, and there's no easy way to convert them.

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/e0fe3b54/attachment.sig>

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Monday, 21 March 2016 at 15:57:24 -0700, scj at yaccman.com wrote:
>
> But the closest I came to being murdered was ...  it also turned out
> that when -.25 was so "executed", it took the CPU into the power
> down sequence, and it turned the machine off!

Not that I haven't done that sort of thing myself, but my favourite
one was a Tandem customer whose programmers had come from a different
system, and they knew that the correct way to terminate a program was
to jump to absolute location 0.  And how about that, it worked!

One day we received a problem report that CPU 4 of their system would
reliably fail at 16:04 every day.  We spent a long time puzzling over
it: there was kernel data corruption of a kind we had never seen
before.  TL;DR: it seems that the jump to 0, in combination with a
kernel library bug, had interpreted the "instruction" at location 0 to
modify this particular location in kernel memory.  More details at
http://www.lemis.com/grog/warstories/CPU-4-at-16-04.php

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/76ae7c38/attachment-0001.sig>

[TUHS] Early non-Unix filesystems?

[John Cowan]

Greg 'groggy' Lehey scripsit:

> > Classical sequential files, however, were simply random-access files
> > such that seeking to line n was just a matter of seeking to byte
> > n * MAXCHARSLINE.  The last time I actually used such a thing was
> > on an early Tandem system when I was implementing the Software
> > Tools.
> 
> You're presumably not talking about a Tandem format here.

I assumed that was the byte structure, but perhaps it wasn't.  It was the
format used for Fortran formatted sequential files and Cobol ORGANIZATION
IS SEQUENTIAL files.  So yes, a Tandem format.

> Were you working with Denis Winn?

I wasn't at Tandem, but at a Tandem customer, Kidder Peabody.

> Presumably you *are* talking about Tandem's format here.  I always
> looked at it as an indication of how badly optimization attempts could
> go wrong.  The format limited line length to something less than 256
> bytes (exact length was content dependent).  The lines were split into
> groups of up to 30 bytes, 15 bytes content and up to 15 bytes trailing
> spaces.  And this was the reason: it was a piss-poor attempt at
> efficient storage.  The only way to access it was with a special
> library with procedures like EDITREAD and EDITWRITE.  I still have a
> number of these files, and there's no easy way to convert them.

Right.  Our implementation of getchar for the Software Tools (Ratfor
version) checked the file format and either did sequential reads or
EDITREADs.  As I remember, though, you could only update existing files
with the library, you couldn't create new ones from scratch, which was
why we needed to use both formats -- all output files were sequential.
Of course, you could edit files with the Software Tools editor rather
than the Tandem editor, and then they'd come out as sequential files.

We also wrote a shell to complement the Software Tools, and a event-driven
pipe server that kept a table of all the pipes in the system.  If you
opened $PIPE.#new, it would allocate a new pipe named $PIPE.Pnnnnn,
and then you could open that for reading or writing.  Each pipe had a
one-line buffer, reads would hang if the buffer was empty and writes
would hang if it was full.  The shell was more or less a Bourne shell,
though not 100% compatible: I remember that the standard prompt was ::,
pronounced "quadpoint".

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
The first thing you learn in a lawin' family is that there ain't
no definite answers to anything.  --Calpurnia in To Kill A Mockingbird

[TUHS] Early non-Unix filesystems?

[shawn wilson]

On Mar 18, 2016 1:02 PM, "Pat Barron" <patbarron at acm.org> wrote:
>

> Even as late as V7, "." and ".." were not automatically added to
directories when created - the "mkdir" program used mknod() to create the
directory, and then created the links for "." and ".." by itself.

Is that the last Unix that had a major number for file system objects? I
hadn't run into this but, thinking about it, it's kinda smart (probably
slower than specific execve kernel calls but still cool).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/99cc144a/attachment.html>

[TUHS] Early non-Unix filesystems?

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2729 bytes --]

On Tue, Mar 22, 2016 at 1:21 AM, shawn wilson <ag4ve.us at gmail.com> wrote:

> Is that the last Unix that had a major number for file system objects? I
> hadn't run into this but, thinking about it, it's kinda smart (probably
> slower than specific execve kernel calls but still cool).


​I'm trying to parse your question a little and I think you may not quite
understand how “pre-BSD 4.2” UNIX systems worked.



In V7 the file system call works such that mknod(2) is used to create
special files and directories, it does it a little different than you
realize.  Generally, there is no need to use it to create a regular file,
although you could.   To wit the man page says:



SYNOPSIS

mknod(name, mode, addr)

char *name;



DESCRIPTION

*Mknod* creates a new file whose name is the null-terminated string pointed
to by name.  The mode of the new file (including directory and special file
bits) is initialized from mode. (The protection part of the mode is
modified by the process's mode mask; see umask(2)
<http://www.unix.com/man-page/v7/2/umask/>). The first block pointer of the
i-node is initialized from add. For ordinary files and directories addr is
normally zero.  In the case of a special file, addr specifies which special
file.





If you look at the code in mkdir.c (V7 mkdir.c source
<http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mkdir.c>) you
see that what is being done is a mknod with addr=0 is being called and then
the two links.





. . .

if ((mknod(d, 040777, 0)) < 0) {        fprintf(stderr,"mkdir: cannot make
directory ­%s\n", d);

      ++Errors;

      return;

 }

chown(d, getuid(), getgid());

strcpy(dname, d);

strcat(dname, "/.");

if((link(d, dname)) < 0) {

      fprintf(stderr, "mkdir: cannot link %s\n", dname);

      unlink(d);

      ++Errors;

      return;

}

strcat(dname, ".");

if((link(pname, dname)) < 0) {         fprintf(stderr, "mkdir: cannot link
%s\n",dname);

      dname[strlen(dname)] = '\0';            unlink(dname);

      unlink(d);

      ++Errors;

}



Where as creating special files did mess with the “special file” bits in
the mode.
​ ​
If you look at the code in mknod.c (V7 mknod.c source
<http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mknod.c>):

 . . .

if(*argv[2] == 'b')

      m = 060666; else

if(*argv[2] == 'c')

      m = 020666; else

      goto usage;

a = number(argv[3]);

if(a < 0)

      goto usage;

b = number(argv[4]);

if(b < 0)

      goto usage;

if(mknod(argv[1], m, (a<<8)|b) < 0)

      perror("mknod");

​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/5fbb89cc/attachment-0001.html>

[TUHS] Early non-Unix filesystems?

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3698 bytes --]

Arrrgh hit send too soon...  sorry about that...

If you look at the two code snippets...look at the second parameter to
mknod.   Note how for mkdir.c they are setting the directory bits (040777) and
in the mknod command they are setting special type (060666 for blocked) and
( 020666 for character) and also then adding in the 3rd parameter the value
to be placed in the special file. That is to say, directories do not have
major/minor numbers. They only differ from regular files in having the
040000 bit turned on, but when that is turned on, the kernel can interprets
contents. However if you open and read them, they look like regular files,
and you are just not allowed to write them.

Clem

On Tue, Mar 22, 2016 at 8:54 PM, Clem Cole <clemc at ccc.com> wrote:

>
> On Tue, Mar 22, 2016 at 1:21 AM, shawn wilson <ag4ve.us at gmail.com> wrote:
>
>> Is that the last Unix that had a major number for file system objects? I
>> hadn't run into this but, thinking about it, it's kinda smart (probably
>> slower than specific execve kernel calls but still cool).
>
>
> ​I'm trying to parse your question a little and I think you may not quite
> understand how “pre-BSD 4.2” UNIX systems worked.
>
>
>
> In V7 the file system call works such that mknod(2) is used to create
> special files and directories, it does it a little different than you
> realize.  Generally, there is no need to use it to create a regular file,
> although you could.   To wit the man page says:
>
>
>
> SYNOPSIS
>
> mknod(name, mode, addr)
>
> char *name;
>
>
>
> DESCRIPTION
>
> *Mknod* creates a new file whose name is the null-terminated string
> pointed to by name.  The mode of the new file (including directory and
> special file bits) is initialized from mode. (The protection part of the
> mode is modified by the process's mode mask; see umask(2)
> <http://www.unix.com/man-page/v7/2/umask/>). The first block pointer of
> the i-node is initialized from add. For ordinary files and directories addr
> is normally zero.  In the case of a special file, addr specifies which
> special file.
>
>
>
>
>
> If you look at the code in mkdir.c (V7 mkdir.c source
> <http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mkdir.c>)
> you see that what is being done is a mknod with addr=0 is being called and
> then the two links.
>
>
>
>
>
> . . .
>
> if ((mknod(d, 040777, 0)) < 0) {        fprintf(stderr,"mkdir: cannot make
> directory ­%s\n", d);
>
>       ++Errors;
>
>       return;
>
>  }
>
> chown(d, getuid(), getgid());
>
> strcpy(dname, d);
>
> strcat(dname, "/.");
>
> if((link(d, dname)) < 0) {
>
>       fprintf(stderr, "mkdir: cannot link %s\n", dname);
>
>       unlink(d);
>
>       ++Errors;
>
>       return;
>
> }
>
> strcat(dname, ".");
>
> if((link(pname, dname)) < 0) {         fprintf(stderr, "mkdir: cannot link
> %s\n",dname);
>
>       dname[strlen(dname)] = '\0';            unlink(dname);
>
>       unlink(d);
>
>       ++Errors;
>
> }
>
>
>
> Where as creating special files did mess with the “special file” bits in
> the mode.
> ​ ​
> If you look at the code in mknod.c (V7 mknod.c source
> <http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mknod.c>):
>
>  . . .
>
> if(*argv[2] == 'b')
>
>       m = 060666; else
>
> if(*argv[2] == 'c')
>
>       m = 020666; else
>
>       goto usage;
>
> a = number(argv[3]);
>
> if(a < 0)
>
>       goto usage;
>
> b = number(argv[4]);
>
> if(b < 0)
>
>       goto usage;
>
> if(mknod(argv[1], m, (a<<8)|b) < 0)
>
>       perror("mknod");
>
> ​
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160322/7cc61b2b/attachment.html>

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Tue, 22 Mar 2016, Greg 'groggy' Lehey wrote:

> > Walking down the corridors of Comp Sci, a student in front of me 
> > dropped his entire deck of approx 2000 cards, all over the floor... I 
> > have no idea whether he got them sorted, but I sure as hell used 
> > rubber bands after that!
> 
> But that's what the sequence numbers in columns 73 to 80 are for!

Yeah, but it would have cost him too much beer to bribe the comp ops into 
sorting them...

I can show the exact place; it was just down the hall from Ken Robinson's 
old office, in Elec Eng (Comp Sci floor).

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Milo Velimirovic]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 594 bytes --]


> On Mar 22, 2016, at 8:28 PM, Dave Horsfall <dave at horsfall.org> wrote:
> 
> On Tue, 22 Mar 2016, Greg 'groggy' Lehey wrote:
> 
>>> Walking down the corridors of Comp Sci, a student in front of me 
>>> dropped his entire deck of approx 2000 cards, all over the floor... I 
>>> have no idea whether he got them sorted, but I sure as hell used 
>>> rubber bands after that!
>> 
>> But that's what the sequence numbers in columns 73 to 80 are for!
> 
> Yeah, but it would have cost him too much beer to bribe the comp ops into 
> sorting them…

What? No model 82 offline sorter available? 

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Wednesday, 23 March 2016 at 12:28:41 +1100, Dave Horsfall wrote:
> On Tue, 22 Mar 2016, Greg 'groggy' Lehey wrote:
>
>>> Walking down the corridors of Comp Sci, a student in front of me
>>> dropped his entire deck of approx 2000 cards, all over the floor... I
>>> have no idea whether he got them sorted, but I sure as hell used
>>> rubber bands after that!
>>
>> But that's what the sequence numbers in columns 73 to 80 are for!
>
> Yeah, but it would have cost him too much beer to bribe the comp ops
> into sorting them...

They used to have big, mechanical machines for sorting cards.  What
did they call them?  Collators?

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160323/28f7144e/attachment.sig>

[TUHS] Early non-Unix filesystems?

[Peter Jeremy]

On 2016-Mar-23 17:49:19 +1100, Greg 'groggy' Lehey <grog at lemis.com> wrote:
>They used to have big, mechanical machines for sorting cards.  What
>did they call them?  Collators?

Sorters:
http://www.columbia.edu/cu/computinghistory/sorter.html

You select a column and it sorts the cards into bins by the value of that
column.  You then stack the cards out of all the bins back into the feed
hopper and sort by the next column.  You'd need 8 passes to do a full sort
(given sequence numbers in 73-80).

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160323/5c89a0a4/attachment.sig>

[TUHS] Early non-Unix filesystems?

[Greg 'groggy' Lehey]

On Wednesday, 23 March 2016 at 18:02:29 +1100, Peter Jeremy wrote:
> On 2016-Mar-23 17:49:19 +1100, Greg 'groggy' Lehey <grog at lemis.com> wrote:
>> They used to have big, mechanical machines for sorting cards.  What
>> did they call them?  Collators?
>
> Sorters:
> http://www.columbia.edu/cu/computinghistory/sorter.html
>
> You select a column and it sorts the cards into bins by the value of that
> column.  You then stack the cards out of all the bins back into the feed
> hopper and sort by the next column.  You'd need 8 passes to do a full sort
> (given sequence numbers in 73-80).

Ah, that's what Milo was referring to.

Greg
--
Sent from my desktop computer.
Finger grog at FreeBSD.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft MUA reports
problems, please read http://tinyurl.com/broken-mua
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160323/71f8900d/attachment.sig>

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Wed, 23 Mar 2016, Greg 'groggy' Lehey wrote:

> >> But that's what the sequence numbers in columns 73 to 80 are for!
> >
> > Yeah, but it would have cost him too much beer to bribe the comp ops
> > into sorting them...
> 
> They used to have big, mechanical machines for sorting cards.  What did 
> they call them?  Collators?

Yeah, but in those days they cost departmental funny money; it was easier
to just bribe the ops (and I've been both a briber and a bribee).

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Ron Natalie]

Yes, the model 82 SORTER  could sort into 13 bins based on a single column
on the card (there are 12 rows in each column, the 13th is the reject bin).
A special selector deals with two holes punched in the column (letters and
symbols).     

The collator (Typically a model 85)  had fewer bins (4) but it could make
more intelligent decisions (based on the wiring of a plug board) on how to
distribute to those bins.   In addition, it had two card hoppers that
allowed you to merge two files of cards together.

The  514 Reproducing Punch allowed you to copy info from one deck onto other
cards in a programmed fashion.   It could also be connected via a mongo
cable to the 402 Accounting machine that could read cards, do some math, and
either print or punch (via the 514) the results.

Even at the time I learned all this, it was rapidly becoming obsolete.
Our high school still used these monsters to do the attendance records.
In college, our registration procedure used those.   I could tell by the way
they did it that they were using my old friend the "Interspersed master gang
punching).

Once out of college two of my subsequent jobs (one at BRL and one at
Rutgers) was to deep six all the card processing equipment.

[TUHS] Early non-Unix filesystems?

[John Cowan]

Ron Natalie scripsit:

> Once out of college two of my subsequent jobs (one at BRL and one at
> Rutgers) was to deep six all the card processing equipment.

Virtually, however, unit-record processing is still with us in the
form of flow-based programming, of which Unix pipelines constitute a
degenerate case.

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
Original line from The Warrior's Apprentice by Lois McMaster Bujold:
"Only on Barrayar would pulling a loaded needler start a stampede toward one."
English-to-Russian-to-English mangling thereof: "Only on Barrayar you risk to
lose support instead of finding it when you threat with the charged weapon."

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Mon, 21 Mar 2016, scj at yaccman.com wrote:

> Actually, a Magic Marker drawing a diagonal stripe down the top of the 
> cards worked almost as well, and didn't require repunching the deck with 
> every change...

Yep, an old trick.  Trouble is, whenever I rearranged the cards I had to
use another colour, and then had to remember which...

> Hmm.  I recall walking into the computing center to see an amazing 
> cascade of cards arcing 20 feet across the room.  The operator hadn't 
> set up the card reader correctly....  Panic ensued...

I've seen something like that; it involved a high-speed CDC reader/punch, 
as I recall.  Cards everywhere :-)  Seems that the deck wasn't 
straightened before inserting by the trainee operator.  Pity the poor 
user.

> In another case, I had written a FORTRAN program that produced a page 
> header on the printer with a page number.  The first time I tried to 
> print 200 pages, when I hit 100 the printer decided that I wanted one 
> line per page!  The printer was not a happy camper.  Neither were the 
> operators.

Have you any idea how fast the IBM 1403 printer can slew blank pages?  
Cough cough...  It chewed through a box in an instant, and I got to 
re-pack it afterwards.  You also learned to not place your coffee cup on 
it (I was doing the midnight shift at the time), because when it ran out 
of paper then the door would open automatically...

I think it was because of an undefined format control character, and the 
printer's control tape went hunting for it.

> But the closest I came to being murdered was when I submitted a job 
> before lunch, and the sign said the turnaround time was an hour.  When I 
> got back, the job was not done, and the wait was 2 hours.  Each hour, 
> the wait grew by an hour.  At about 4:30, four extremely angry operators 
> and managers came into my office with murder in their eyes and accused 
> me of trying to "be funny" (sic) with the computer.  I had no idea what 
> they were talking about.  It turns out that the operating system had 
> been upgraded, so that it automatically started pending jobs when it 
> came up after a crash.  It also turned out that I had made a common 
> FORTRAN error--I had "called" a labeled common region instead of a 
> similarly-named function.  And it turned out that the first data value 
> was a -.25 floating-point value.  And it also turned out that the 
> computer's idea of instruction decode was to pick up the first few bits 
> of the word and use it as an index into the microcode.  And it also 
> turned out that when -.25 was so "executed", it took the CPU into the 
> power down sequence, and it turned the machine off!  And, thanks to the 
> software upgrade, when they rebooted, it turned itself off again.  And 
> again...  I was happy to escape with my skin...

Closest I've ever been murdered was when I "accidentally" filled the local 
11/70 with an uninterruptible instruction sequence.  And who here hasn't 
"tried out" that self-reproducing program?  The only way to recover was to 
switch it off, as the HALT key was implemented as an interrupt, and guess 
what it wasn't doing at the time?

Try writing a self-reproducing program some time i.e. one that creates an 
exact copy of itself; it's an exercise in computer science, and is not as 
easy as it sounds.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Ron Natalie]

> Closest I've ever been murdered was when I "accidentally" filled the local
> 11/70 with an uninterruptible instruction sequence."

SPL instruction.   The PDP-11 was odd that while SPL was a "privileged"
instruction, rather that trapping if you did it in user mode, it just
"ignored" it.
Well, what it ignored was the actual change of the processor level.   What
it still implemented was the side effect was that interrupts were locked out
until the next instruction fetch.
If you filled your instruction space up with SPLs you could lock up the
computer so that even the HALT key didn't work (you had to do a bus RESET).

In an attempt to do this in user mode in our hybrid-V6 system I tripped
across another bug.   I did it during the day when there was a high load on
the system and my process got swapped out.
When it got swapped in, it rounded up the BRK to the next higher 64 bit
chunk.   Of course this rolled around to zero.   It allocated zero bytes of
memory getting zero and then did a DMA from our swap device to it.
That was a bit more flamboyant.

Fortunately , my mentor Mike Muuss tolerated these fiascos as long as I went
and fixed the bug I had just exploited afterwards.

[TUHS] Early non-Unix filesystems?

[Milo Velimirovic]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]


> On Mar 23, 2016, at 7:32 PM, Ron Natalie <ron at ronnatalie.com> wrote:
> 
> 
>> Closest I've ever been murdered was when I "accidentally" filled the local
>> 11/70 with an uninterruptible instruction sequence."
> 
> SPL instruction.   The PDP-11 was odd that while SPL was a "privileged"
> instruction, rather that trapping if you did it in user mode, it just
> "ignored" it.
> Well, what it ignored was the actual change of the processor level.   What
> it still implemented was the side effect was that interrupts were locked out
> until the next instruction fetch.
> If you filled your instruction space up with SPLs you could lock up the
> computer so that even the HALT key didn't work (you had to do a bus RESET).

This is of course a fairly well known bug in the pdp11. I wonder if this was fixable with a hardware mod, like the “there’s this NAND gate here” sort of fix to the MTPx/MFPx instructions or if it required a change to the microcode, that presumably DEC wasn’t interested in.

> [snip]

[TUHS] Early non-Unix filesystems?

[Ron Natalie]

The nand gate fix wasn't to fix a bug in MTPI but rather to allow access to the user-I from the program itself, primarily to make the *abomination* nargs() function work.

I don't believe DEC ever worried about "fixing" this as none of the DEC OS's and not even the stock UNIXes could even implement such a problem.   You could chew up some processor time, but you couldn't absolutely lock things out.

[TUHS] Early non-Unix filesystems?

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

On Wed, Mar 23, 2016 at 6:46 PM, Dave Horsfall <dave at horsfall.org> wrote:

> Have you any idea how fast the IBM 1403 printer can slew blank pages?
> Cough cough...
>
​yep​



> It chewed through a box in an instant, and I got to
> re-pack it afterwards.  You also learned to not place your coffee cup on
> it (I was doing the midnight shift at the time), because when it ran out
> of paper then the door would open automatically...
>
​What's coffee - try beer glass/plastic cup.

Once nice thing about raised floors is they kept your beer cold.   On
weekend if we were having a party, I would bring a gallon milk jug fresh
from the tap's at my fraternity and put it under the floor.

I could usually bribe another operator to stay over for an unpaid hour or
so when the backups first ran.   Then we would finish our beers and if I
was lucky, the "console bell" was a fire alarm.  So with a patch to the TSS
kernel you could get it ring the console bell when the system needed human
attention.  I could then go into the printer paper room where there was an
Tek 4010 graphics tube.  Thus I could either go off to hack for a while
 knowing that I would be called if the system needed me.​

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160323/81784565/attachment.html>

[TUHS] Early non-Unix filesystems?

[scj]

A Canadian friend of mine had a similar problem.  He had been working in
the US for several years, and was returning to his new job in Canada, his
car filled with dozens of boxes of punched cards.  The Canadian Customs
officers told him he would need to pay a duty (as I recall, it was fairly
stiff for the day--$5 a box or so) and my friend argued in vain for quite
a few minutes.  Finally, he opened one of the boxes and said "look, these
all have data on them.  They aren't useful for anything..."  and the
customs officer said "Oh.  These are USED cards.  That's OK.  No duty!"

>
> "Ahh. You don't understand. We only use the holes."
>
>
> -- Charles
>

[TUHS] Early non-Unix filesystems?

[Dave Horsfall]

On Fri, 25 Mar 2016, Johnny Billquist wrote:

> > Some instructions inhibit the "check for interrupts at the end of this 
> > instruction" check.  I'm most familiar with the 8080 EI instruction, 
> > which enabled interrupts after the following instruction (so things 
> > like EI;HLT didn't have a window).  It seems the PDP-11 SPL behaves 
> > the same.
> 
> I don't think it should on the PDP-11, and the documentation do not 
> mention any such thing.

It most certainly did, at least on the 11/70 that I used...  Do you have 
experience otherwise?

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early non-Unix filesystems?

[Ronald Natalie]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 900 bytes --]

> 
> "Ahh. You don't understand. We only use the holes.”

Well actually, that’s pretty accurate.   The holes are the significant part. 

I had some corporate guys come down to talk about doing interfacility collaboration on Engineering projects.   They’d identified this networked manufacturing configuration system and though we could use it to manage our 750,000 line code base in our products.    They seemed to thing a file was just a part.   I pointed out we were already using a distributed software configuration management system (Clearcase Multisite) and if they wanted to expand that to some more sites, I’d be glad to help.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2284 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160326/43459d3c/attachment.bin>

[TUHS] Early non-Unix filesystems?

[Charles Anthony]

On Sat, Mar 26, 2016 at 2:05 PM, Ronald Natalie <ron at ronnatalie.com> wrote:

> I had for 25 years on my desk the program drum (complete with a card
> wrapped around it) that I snagged as a souvenir from the last keypunch
> machine at Rutgers.   It was extremely rare that anybody ever understood
> what it was.   Sometimes, I even had to explain what a punched card was.
>
>
>
> An aerospace company gets a big military contract to produce a new fighter
aircraft. During the design process it was realized that aircraft weight
issues were going to be a major ongoing issue, so they appointed a manager
to track weight across all of the engineering groups so that problems could
be detected and corrected early. The newly appointed manager checks in with
all of the group managers and collects current estimates and trend data.
Eventually, he gets to the Avionics Software group and asks for their
weight data.

"Um. Zero."

"What? Aren't you putting software onto the fighter?"

"Yes"

"Well, how much will it weigh?"

"Zero."

Exasperated, the Weight Manager points to a deck of punch cards on the desk.

"There... That's software, right?"

"Yes..."

"How much do the cards weigh?"

"Ahh. You don't understand. We only use the holes."


-- Charles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160326/f0c18377/attachment-0001.html>

[TUHS] Early non-Unix filesystems?

[Ronald Natalie]

I had for 25 years on my desk the program drum (complete with a card wrapped around it) that I snagged as a souvenir from the last keypunch machine at Rutgers.   It was extremely rare that anybody ever understood what it was.   Sometimes, I even had to explain what a punched card was.

>> 
> 
> The joy of drum cards.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2284 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160326/748d611b/attachment.bin>

[TUHS] Early non-Unix filesystems?

[Johnny Billquist]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3365 bytes --]

On 2016-03-25 00:27, Milo Velimirovic wrote:
>
>> On Mar 24, 2016, at 6:06 PM, Johnny Billquist <bqt at update.uu.se> wrote:
>>
>> On 2016-03-24 23:50, Peter Jeremy wrote:
>>> On 2016-Mar-24 11:17:18 +0100, Johnny Billquist <bqt at update.uu.se> wrote:
>>>> It is the normal behavior of any instruction that interrupts are not
>>>> recognized until the next instruction fetch. This is how the microcode
>>>> works, and it is also pretty much the same in any processor today.
>>> ...
>>>> individual instructions. You still get a fetch between each instruction,
>>>> at which point, interrupts will be recognized.
>>>
>>> Some instructions inhibit the "check for interrupts at the end of this
>>> instruction" check.  I'm most familiar with the 8080 EI instruction,
>>> which enabled interrupts after the following instruction (so things like
>>> EI;HLT didn't have a window).  It seems the PDP-11 SPL behaves the same.
>>
>> I don't think it should on the PDP-11, and the documentation do not mention any such thing.
>> There is a good reason the 8080 (and Z80, and others) have this property. The RETI instruction on these machines do not enable itnerrupts themselves, so just as you note, you need to both enable interrupts and return from interrupt atomically, or else you get into a mess.
>>
>> The PDP-11 RETI instruction changes the processor priority as a part of the instruction. You do not use SPL (whatever) before a RETI.
>> Thus, it do not make sense that SPL on a PDP-11 would have this property. If if indeed do disable recognizing interrupts after an SPL, it sounds more like a bug. I guess I'll go and read the microcode so see if that mentions any of this, since I'm sortof into reading it anyway as I was trying to debug a problem on an 11/70 only a couple of months ago…
>
> The PDP-11 has no RETI instruction; it has a RTT (ReTurn from Trap) and a RTI (ReTurn from Interrupt) instructions, but you already knew that, right? In some cases it’s  problem that it’s not possible to determine which is appropriate or correct to use. According to the PDP11 Architecture Handbook the difference between them is in what happens when the RTx instruction loads a PSW that has the T bit set and when it forces a Trace trap. RTI - immediate trap, RTT traps after the instruction following the RTT.

Oops. Yes, it's RTI and RTT. But the names are beside the point, and so 
is the difference between these two. The point is that the 
instruction(s) do set the processor priority level, and you do not use 
SPL in combination with them, so it makes no sense to have SPL inhibit 
interrupts for any length at all. (And yes, I did know that.)

Oh, and as far as RTT vs. RTI goes, not it's not hard to know which one 
you want. You want RTT for your debugger and RTI for everything else.
The difference is about what happens after the return. With RTT, the 
T-bit trap will not trap until another instruction has executed. With 
RTI, you would never manage to step to the next instruction with the 
T-bit, since every time you returned, you'd get another trap.
But I bet you knew that as well... ;-)

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Early non-Unix filesystems?

[Johnny Billquist]

On 2016-03-24 23:50, Peter Jeremy wrote:
> On 2016-Mar-24 11:17:18 +0100, Johnny Billquist <bqt at update.uu.se> wrote:
>> It is the normal behavior of any instruction that interrupts are not
>> recognized until the next instruction fetch. This is how the microcode
>> works, and it is also pretty much the same in any processor today.
> ...
>> individual instructions. You still get a fetch between each instruction,
>> at which point, interrupts will be recognized.
>
> Some instructions inhibit the "check for interrupts at the end of this
> instruction" check.  I'm most familiar with the 8080 EI instruction,
> which enabled interrupts after the following instruction (so things like
> EI;HLT didn't have a window).  It seems the PDP-11 SPL behaves the same.

I don't think it should on the PDP-11, and the documentation do not 
mention any such thing.
There is a good reason the 8080 (and Z80, and others) have this 
property. The RETI instruction on these machines do not enable 
itnerrupts themselves, so just as you note, you need to both enable 
interrupts and return from interrupt atomically, or else you get into a 
mess.

The PDP-11 RETI instruction changes the processor priority as a part of 
the instruction. You do not use SPL (whatever) before a RETI.
Thus, it do not make sense that SPL on a PDP-11 would have this 
property. If if indeed do disable recognizing interrupts after an SPL, 
it sounds more like a bug. I guess I'll go and read the microcode so see 
if that mentions any of this, since I'm sortof into reading it anyway as 
I was trying to debug a problem on an 11/70 only a couple of months ago...

I'll try and remember to report back when I know something more.

>> memory access to get the actual content. The fun thing happens if you
>> set the indirect bit, and give your own address. This is then an
>> infinite memory reference. And the KA10 can not be broken out of that
>> lookup. The only solution is to pull the power plug.
>
> I can't think of any modern architectures that still support this sort
> of indefinite indirection but I know the ITT 3200 (custom CPU for
> controlling telephone exchanges) could do this.  In it's case, a normal
> front-panel reset would recover.

Yeah. The fact that the microcode didn't even recognize any signals from 
the front panel while in this loop was bad, they realized... :-)

	Johnny

[TUHS] Early non-Unix filesystems?

[Peter Jeremy]

On 2016-Mar-24 11:17:18 +0100, Johnny Billquist <bqt at update.uu.se> wrote:
>It is the normal behavior of any instruction that interrupts are not 
>recognized until the next instruction fetch. This is how the microcode 
>works, and it is also pretty much the same in any processor today. 
...
>individual instructions. You still get a fetch between each instruction, 
>at which point, interrupts will be recognized.

Some instructions inhibit the "check for interrupts at the end of this
instruction" check.  I'm most familiar with the 8080 EI instruction,
which enabled interrupts after the following instruction (so things like
EI;HLT didn't have a window).  It seems the PDP-11 SPL behaves the same.

>memory access to get the actual content. The fun thing happens if you 
>set the indirect bit, and give your own address. This is then an 
>infinite memory reference. And the KA10 can not be broken out of that 
>lookup. The only solution is to pull the power plug.

I can't think of any modern architectures that still support this sort
of indefinite indirection but I know the ITT 3200 (custom CPU for
controlling telephone exchanges) could do this.  In it's case, a normal
front-panel reset would recover.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160325/256258c0/attachment-0001.sig>

[TUHS] Early non-Unix filesystems?

[Milo Velimirović]

> On Mar 24, 2016, at 2:13 PM, Nemo <cym224 at gmail.com> wrote:
> 
> On 21 March 2016 at 17:43, Greg 'groggy' Lehey <grog at lemis.com> wrote:
>> On Tuesday, 22 March 2016 at  1:11:07 +1100, Dave Horsfall wrote:
>>> 
>>> Walking down the corridors of Comp Sci, a student in front of me
>>> dropped his entire deck of approx 2000 cards, all over the floor...
>>> I have no idea whether he got them sorted, but I sure as hell used
>>> rubber bands after that!
>> 
>> But that's what the sequence numbers in columns 73 to 80 are for!
> 
> I did that religiously, even with my small PL/C runs -- PL/C runs were
> free.  One day, they decided to extend the code area to the entire
> card.... and so I learned another feature of the card punch.

The joy of drum cards.

[TUHS] Early non-Unix filesystems?

[Nemo]

On 21 March 2016 at 17:43, Greg 'groggy' Lehey <grog at lemis.com> wrote:
> On Tuesday, 22 March 2016 at  1:11:07 +1100, Dave Horsfall wrote:
>>
>> Walking down the corridors of Comp Sci, a student in front of me
>> dropped his entire deck of approx 2000 cards, all over the floor...
>> I have no idea whether he got them sorted, but I sure as hell used
>> rubber bands after that!
>
> But that's what the sequence numbers in columns 73 to 80 are for!

I did that religiously, even with my small PL/C runs -- PL/C runs were
free.  One day, they decided to extend the code area to the entire
card.... and so I learned another feature of the card punch.

N.

>
> Greg
> --

[TUHS] Early non-Unix filesystems?

[Johnny Billquist]

On 2016-03-24 12:35, Ron Natalie wrote:
>
>> Ok. Color me stupid, but I don't get it. I totally do not understand how this locks anything out.
>
> I misstated that a bit.   The issue is if the next instruction is also an SPL instruction, then interrupts never get enabled.    So if you have a run of SPL instructions, interrupts are locked out for the duration.    If you can fill your address space with SPL instructions (provided the OS doesn't fail in the process), you can cause an uninterruptable loop.   Interupt includes the HALT switch on the front.   You have to use the BUS RESET to jab it out of its catatonic state.

Hmm. Ok. Gotta check this out. Thanks for the details.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Early non-Unix filesystems?

[Ron Natalie]

> Ok. Color me stupid, but I don't get it. I totally do not understand how this locks anything out.

I misstated that a bit.   The issue is if the next instruction is also an SPL instruction, then interrupts never get enabled.    So if you have a run of SPL instructions, interrupts are locked out for the duration.    If you can fill your address space with SPL instructions (provided the OS doesn't fail in the process), you can cause an uninterruptable loop.   Interupt includes the HALT switch on the front.   You have to use the BUS RESET to jab it out of its catatonic state.

[TUHS] Early non-Unix filesystems?

[Johnny Billquist]

On 2016-03-24 03:00, "Ron Natalie"<ron at ronnatalie.com> wrote:
>
>> >Closest I've ever been murdered was when I "accidentally" filled the local
>> >11/70 with an uninterruptible instruction sequence."
> SPL instruction.   The PDP-11 was odd that while SPL was a "privileged"
> instruction, rather that trapping if you did it in user mode, it just
> "ignored" it.
> Well, what it ignored was the actual change of the processor level.   What
> it still implemented was the side effect was that interrupts were locked out
> until the next instruction fetch.
> If you filled your instruction space up with SPLs you could lock up the
> computer so that even the HALT key didn't work (you had to do a bus RESET).

Ok. Color me stupid, but I don't get it. I totally do not understand how 
this locks anything out.

It is the normal behavior of any instruction that interrupts are not 
recognized until the next instruction fetch. This is how the microcode 
works, and it is also pretty much the same in any processor today. 
Except for instructions that take a long time, and which can be 
interrupted in the middle, the context preserved, and the instruction 
restarted and continued, instructions are normally atomic. You cannot 
get interrupts in the middle of an instruction.

Second, I cannot understand how filling the memory with SPL instructions 
(or any other instruction) can lock out the CPU. As noted, they are 
individual instructions. You still get a fetch between each instruction, 
at which point, interrupts will be recognized.

Now, if you instead talked about actually raising the CPU to SPL 7, then 
I agree that no interrupts will happen. But that is because you 
essentially disabled interrupts.

The front panel still works though. It is not handled like an interrupt, 
but it is true that it do interact with the processor states, and 
normally if you pull HALT, it will only halt when it's going to fetch 
the next instruction. You can, of course, also set the front panel 
switch for single microcode instruction, at which point the CPU will 
halt at the next microcode instruction instead, and you can single step 
the microcode as well.

The one CPU I know you can sunset is the KA10 (PDP-10). I'm sure there 
are others, but I have never seen how this could be done on a PDP-11, so 
I'm most curious about this, and if you can provide more details I would 
be most interested. As I also happen to know where a PDP-11/70 is 
standing, I intend to test this out next I get close to it.

As for the KA10 (I think it was the KA10, but it might have been the 
PDP-6), the problem is related to the indirect addressing feature. Since 
memory is 36 bits, but addresses only 18, you have plenty of bits to 
play with. And one of them is the indirect bit. And if you refer to a 
memory location that also have the indirect bit set, you get another 
memory access to get the actual content. The fun thing happens if you 
set the indirect bit, and give your own address. This is then an 
infinite memory reference. And the KA10 can not be broken out of that 
lookup. The only solution is to pull the power plug.
The CPU is essentially stuck in one state, just tightly reading memory, 
and then repeating reading memory. Later PDP-10 models have an explicit 
check in the microcode in this loop to be able to break out of this.

Sorry for the offtopic content. :-)

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Early non-Unix filesystems?

[Doug McIlroy]

> Those file structures are collected into a single, global table. The
> question is why this latter table? One could rather imagine an
> implementation where open() allocates (e.g., via malloc())

Depending on how malloc() is implemented, fragmentation can be
serious in a program that runs forever with as many frees as
allocs. Separate allocations for each item can also be costly in time.

One malloc() strategy is to divide the arena into regions,
each of which caters for blocks of a single size, so
fragmentation doesn't occur. In essence that's how the
system tables work, except that these tables have
hard limits. Now, if the tables could be reallocated
as needed ...

Another problem with per-item allocations is performance
monitoring and debugging. It's hard to see what's
going on in a well mixed dynamic storage heap.

Doug

[TUHS] Early non-Unix filesystems?

[Doug McIlroy]

Sorry to continue the detour from disk file systems to card trays, but this

> Walking down the corridors of Comp Sci, a student in front of me
> dropped his entire deck of approx 2000 cards, all over the floor...
> I have no idea whether he got them sorted, but I sure as hell used
> rubber bands after that!

reminded me that Vic Vyssotsky liked to say of his BLODI (block diagram)
language for simulating sample-data systems that it was the only card-safe
language. You could toss a program deck down the stairs, pick it up at the
bottom, submit it to the compiler, and it would work. That was 10 years
before the filing of the famous "natural order" patent on spreadsheets,
which ordered execution the same way.

Doug

[TUHS] Early non-Unix filesystems?

[Charles Anthony]

On Thu, Mar 17, 2016 at 7:13 PM, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2016-03-18 03:00, Warren Toomey <wkt at tuhs.org> wrote:
>
>
>> I would think that something like Multics had something similar, but I
> have no idea about that one...
>
> Mutlics doesn't separate the name and the data.

It did do something really interesting tho; segments could have multiple
names.

Given a segment "foo", you could "add_name foo bar". "foo" and "bar" now
refer to the same segment, much like a UNIX link. However, if you move the
segment to another directory, the names follow -- they are part of the
segment metadata, not entries in the segment's directory.

This was leveraged by the binder; when binding an executable segment with
multiple entry points, the binder will add the entry points to the names of
the segment, and the shell would select the correct entry point based on
the name.

    bound_dir_cmds_
       list
       rename
       delete

You can use 'bound_dir_cmds_$list' or 'list'.

-- Charles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20160317/6215ce68/attachment.html>

[TUHS] Early non-Unix filesystems?

[Noel Chiappa]

    > From: Johnny Billquist

    > I would think that something like Multics had something similar

No, as far as I know, Multics was always 'old-style' - the directory contained
not just the name of the file, but also all the file's meta-data. Multics had
only symbolic links, I'm pretty sure.

To answer the original question, I can't think offhand of another system that
separated naming and file meta-data before Unix did it. I've always assumed
that that was one of Unix' novel ideas.

	Noel

[TUHS] Early non-Unix filesystems?

[Johnny Billquist]

On 2016-03-18 03:00, Warren Toomey <wkt at tuhs.org> wrote:

> It's a bit off-topic, but what were non-Unix filesystems like around 1969-1970?
> The PDP-7 filesystem has i-nodes (file metadata) and filenames separate
> from the i-nodes. This allows hard links and thus a non-tree structured
> filesystem.
>
> This has always struck me to be one of the most important features of
> the Unix filesystem: names separated from the rest of the file metadata,
> and arbitrary hard links so that there is no preferred filename.
>
> Were these features in other contemporaneous filesystems?

I don't know exactly how contemporary ODS-1 is. It's the file system 
used on RSX-11, and I think it should atleast trace back to around 1972, 
but I can't say more for sure.

Anyway, ODS-1, just like the Unix file system, have the directory hold 
just the filename and a file identifier (pretty much the same as inode 
number). There are of course some differences in details, but I would 
say it is very similar to how Unix works. ODS-1 do not have reference 
counting, but instead allows dangling directory entries that do not 
point to valid files. Instead ODS-1 have a generation counter for each 
file identifier, so that when it is reused, links to the old file will 
not accidentally refer to the new file.

I would think that something like Multics had something similar, but I 
have no idea about that one...

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol