The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] 2bsd tarball
@ 2020-07-28 23:03 Will Senn
  2020-07-29  0:09 ` Warner Losh
  2020-07-29  0:21 ` Clem Cole
  0 siblings, 2 replies; 16+ messages in thread
From: Will Senn @ 2020-07-28 23:03 UTC (permalink / raw)
  To: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1367 bytes --]

OK, I was able to locate 2bsd.tar.gz and spencer_2bsd.tar.gz in the 
archive. Neither is an installation tape. It appears that they are just 
tarballs of their respective systems (there are very minor differences 
between the two).

In the TAPE file in the tarball, it talks about reading the tar program 
off of the tape using:
dd if=/dev/mt0 bs=1b skip=1 of=tar

Well, tar is definitely not located at that address, which implies that 
the tarball isn't a distro tape. This note in the archive used to read:

...

The remaining gzipped tar files are other 2BSD distributions supplied by
Keith Bostic, except for spencer_2bsd.tar.gz which came from Henry Spencer.
They do not contain installation tape images. The 2.9BSD-Patch directory
contains patches to 2.9BSD dated August 85, and again supplied by Keith Bostic.

...
now it reads:
...

2.11BSD 2.11BSD-pl195.tar is a copy of 2.11BSD at patch level 195, supplied
by Tom Ivar Helbekkmo. spencer_2bsd.tar.gz is a version of 2BSD which came
from Henry Spencer.
...


I recall having to do something with cont.a files, which are not present 
on these images. So, my questions is, does anyone know of or have an 
actual 2bsd tape/tape image?


Thanks,

Will

Here's where I found the tarballs:
https://www.tuhs.org/Archive/Distributions/UCB/


-- 
GPG Fingerprint: 68F4 B3BD 1730 555A 4462  7D45 3EAA 5B6D A982 BAAF


[-- Attachment #2: Type: text/html, Size: 2087 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball
  2020-07-28 23:03 [TUHS] 2bsd tarball Will Senn
@ 2020-07-29  0:09 ` Warner Losh
  2020-07-29  0:19   ` Clem Cole
                     ` (2 more replies)
  2020-07-29  0:21 ` Clem Cole
  1 sibling, 3 replies; 16+ messages in thread
From: Warner Losh @ 2020-07-29  0:09 UTC (permalink / raw)
  To: Will Senn; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 2481 bytes --]

On Tue, Jul 28, 2020 at 5:04 PM Will Senn <will.senn@gmail.com> wrote:

> OK, I was able to locate 2bsd.tar.gz and spencer_2bsd.tar.gz in the
> archive. Neither is an installation tape. It appears that they are just
> tarballs of their respective systems (there are very minor differences
> between the two).
>
> In the TAPE file in the tarball, it talks about reading the tar program
> off of the tape using:
> dd if=/dev/mt0 bs=1b skip=1 of=tar
>
> Well, tar is definitely not located at that address, which implies that
> the tarball isn't a distro tape. This note in the archive used to read:
>
> ...
>
> The remaining gzipped tar files are other 2BSD distributions supplied by
> Keith Bostic, except for spencer_2bsd.tar.gz which came from Henry Spencer.
> They do not contain installation tape images. The 2.9BSD-Patch directory
> contains patches to 2.9BSD dated August 85, and again supplied by Keith Bostic.
>
> ...
> now it reads:
> ...
>
> 2.11BSD 2.11BSD-pl195.tar is a copy of 2.11BSD at patch level 195, supplied
> by Tom Ivar Helbekkmo. spencer_2bsd.tar.gz is a version of 2BSD which came
> from Henry Spencer.
> ...
>
> I recall having to do something with cont.a files, which are not present on these images. So, my questions is, does anyone know of or have an actual 2bsd tape/tape image?
>
> Both of the 2bsd tapes you found are from the days when Berkeley just sent
patches to the 7th Edition out. The 2.8BSD tape was the first one to have a
kernel that was bootable from the tape. The 2BSD tapes originally had 2
files on them. The first one was a binary copy of tar that ran on V7. The
second was a tarball of all the rest. As you discovered, they shipped with
a label like:
        Second Berkeley Software Tape
        May 10, 1979    TAR 800BPI

        %dd if=/dev/mt0 bs=1b skip=1 of=tar
        %chmod 755 tar
        % tar x

        10000 blocks
but the 2bsd.tar.gz file has just the second file.

The spensor_2bsd.tar.gz has a tar binary in it:

tar tvf spencer_2bsd.tar.gz | head
-rw-r--r--  0 0      0       24688 Feb 17  1980 tar
-rw-r--r--  0 0      10       3687 Feb 17  1980 tar.1
-rw-r--r--  0 0      10        456 Feb 17  1980 tar.ms
-rw-r--r--  0 0      10      15216 Feb 17  1980 install.ms

if you are looking for that pre-built. If you are looking to create a tape
with tar on it to extract other tar tapes, you'd need to use a variation on
the maketape.pl with a block size of 1 so the above dd will work on the
target system...

Warner

[-- Attachment #2: Type: text/html, Size: 3694 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball
  2020-07-29  0:09 ` Warner Losh
@ 2020-07-29  0:19   ` Clem Cole
  2020-07-29  0:45   ` Will Senn
  2020-07-29  0:46   ` Will Senn
  2 siblings, 0 replies; 16+ messages in thread
From: Clem Cole @ 2020-07-29  0:19 UTC (permalink / raw)
  To: Warner Losh; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 2742 bytes --]

Note the UID/GID are numeric not ASCII ;-)
I suspect henry's tape is closer to the official tape!!

On Tue, Jul 28, 2020 at 8:11 PM Warner Losh <imp@bsdimp.com> wrote:

>
>
> On Tue, Jul 28, 2020 at 5:04 PM Will Senn <will.senn@gmail.com> wrote:
>
>> OK, I was able to locate 2bsd.tar.gz and spencer_2bsd.tar.gz in the
>> archive. Neither is an installation tape. It appears that they are just
>> tarballs of their respective systems (there are very minor differences
>> between the two).
>>
>> In the TAPE file in the tarball, it talks about reading the tar program
>> off of the tape using:
>> dd if=/dev/mt0 bs=1b skip=1 of=tar
>>
>> Well, tar is definitely not located at that address, which implies that
>> the tarball isn't a distro tape. This note in the archive used to read:
>>
>> ...
>>
>> The remaining gzipped tar files are other 2BSD distributions supplied by
>> Keith Bostic, except for spencer_2bsd.tar.gz which came from Henry Spencer.
>> They do not contain installation tape images. The 2.9BSD-Patch directory
>> contains patches to 2.9BSD dated August 85, and again supplied by Keith Bostic.
>>
>> ...
>> now it reads:
>> ...
>>
>> 2.11BSD 2.11BSD-pl195.tar is a copy of 2.11BSD at patch level 195, supplied
>> by Tom Ivar Helbekkmo. spencer_2bsd.tar.gz is a version of 2BSD which came
>> from Henry Spencer.
>> ...
>>
>> I recall having to do something with cont.a files, which are not present on these images. So, my questions is, does anyone know of or have an actual 2bsd tape/tape image?
>>
>> Both of the 2bsd tapes you found are from the days when Berkeley just
> sent patches to the 7th Edition out. The 2.8BSD tape was the first one to
> have a kernel that was bootable from the tape. The 2BSD tapes originally
> had 2 files on them. The first one was a binary copy of tar that ran on V7.
> The second was a tarball of all the rest. As you discovered, they shipped
> with a label like:
>         Second Berkeley Software Tape
>         May 10, 1979    TAR 800BPI
>
>         %dd if=/dev/mt0 bs=1b skip=1 of=tar
>         %chmod 755 tar
>         % tar x
>
>         10000 blocks
> but the 2bsd.tar.gz file has just the second file.
>
> The spensor_2bsd.tar.gz has a tar binary in it:
>
> tar tvf spencer_2bsd.tar.gz | head
> -rw-r--r--  0 0      0       24688 Feb 17  1980 tar
> -rw-r--r--  0 0      10       3687 Feb 17  1980 tar.1
> -rw-r--r--  0 0      10        456 Feb 17  1980 tar.ms
> -rw-r--r--  0 0      10      15216 Feb 17  1980 install.ms
>
> if you are looking for that pre-built. If you are looking to create a tape
> with tar on it to extract other tar tapes, you'd need to use a variation on
> the maketape.pl with a block size of 1 so the above dd will work on the
> target system...
>
> Warner
>

[-- Attachment #2: Type: text/html, Size: 4372 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball
  2020-07-28 23:03 [TUHS] 2bsd tarball Will Senn
  2020-07-29  0:09 ` Warner Losh
@ 2020-07-29  0:21 ` Clem Cole
  2020-07-29  9:50   ` [TUHS] [simh] " Johnny Billquist
  2020-07-29 13:42   ` [TUHS] 2bsd tarball -> pdtar, with a side of uuslave John Gilmore
  1 sibling, 2 replies; 16+ messages in thread
From: Clem Cole @ 2020-07-29  0:21 UTC (permalink / raw)
  To: Will Senn; +Cc: TUHS main list, simh

[-- Attachment #1: Type: text/plain, Size: 6857 bytes --]

Cross posting to simh - since much of this has been discussed in the last
few days there also....

in for penny, in for pound ... here is the history ...  man ... I lived
this and I'll need a strong drink later tonight after I write it all up.


On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn@gmail.com> wrote:

> I recall having to do something with cont.a files, which are not present on these images. So, my questions is, does anyone know of or have an actual 2bsd tape/tape image?
>
> cont.a is a tp-v6 and earlier ism.

DECtape had a directory at the front of the tape (think superblock/ilist),
but could do cool things and be treated more like a disk.
When tp was created for a very early version of Unix (I'm not sure which,
could be V2), Ken/Dennis et al had DECtape units and so the original scheme
followed the media.   This also meant that the program could write files
and go back and update the directory, which is a no-no with many tape
systems.  Then research got a 9-track unit.   So tp was changed to
calculate how much space was going to be needed, write the directory, then
the datablocks.  All good ... except...

9-track could write more files than the directory could take.   So for many
years, people would use the ar(1) program to take a number of files in a
directory, create a file called cont.a and then delete the files.  Then the
tree would be written with tp, when you read it, you reversed the ar(1)
process.  If you look at the USENIX/Harvard tape on the TUHS you'll see
this scheme in use.

BTW: tp was written in assembler and all the data structures were using
PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp) in C
(which is on the USENIX tape Warren has in the archives) that worked like
the original assembler tp but also put a redundant directory at the end of
the tape.  Another issue with tp was if the you had a bad block in the
first few blocks you could not decode the rest of the tape.  [There were
some other issues with the UNIX tree structure as disks got bigger but I'm
going to ignore all that - other than to say, tp had lived it life].

Enter Mashey and the PWB 1.0 folks (which is based on V6).  Someone in USG
created cpio (and volcpy) as part of the PWB 1.0.   Like tp it was a PDP-11
binary format, but unlike tp, the tape directory is threaded.  *i.e.* block
one describes the first file only and includes the size of the following
file, then file itself, then a new directory block for the next file and
again that file (rinse and repeat).  So it improved on tp in the directory
threading, but was still binary and for a reasons I'll leave out had a
different user interface.

As part of V7, Ken wrote a new program, tar [you can ask him why].  But
like cpio he used a threaded tape directory, but unlike cpio it was always
ASCII and not PDP-11 specific.  Furthermore, the user interface was made to
parrot tp.  So, certainly, it had the advantage that changing tp scripts to
use tar was pretty easy i.e. s/tp/tar/     not so for coil.  And it was
muscle memory compliant.

For PWB 2.0, cpio was updated to allow a -c option to write the header in
ascii and -s to byte swap the binary.   But the damage had been done ...

Thus began 'tar wars' which was a battle that raged officially over tape
archive formats, but really was an argument about user interfaces.  Since
tar was part of Research and the Universities and commercial people used
it, only USG and the folks inside the Bell System were using cpio, as
officially none of us had it since PWB was not released to us (although
thanks to many AT&T employees doing an OYOC year, many schools like UCB,
MIT and CMU all had the sources to cpio anyway -- for instance you'll see
it hidden away on Kirk's CD).

I personally had used both, preferred tar for easy of use and ASCII
directories.  But, note I had written car at Masscomp, but never tpio.
This was our trick to use the file scripting list that cpio could do, but
create tar format tapes - which was handy.  I never wrote tpio which would
have been cpio format using tp/tar user interface as I did not need it.

Roll forward to the /usr/group UNIX standard that Heinz chaired.  We ended
up not being able to agree on a distribution format, but the ISVs were PO
because now they could create UNIX programs that might actually work across
systems, but they had not standard way to distribution.
Roll forward again to IEEE.  Heinz's committee was officially disbanded
(story discussed elsewhere) and we were created as IEEE P1003 with Jim
Issack as Chair. This time the ISV's said we had to have a distribution
format.  Since *.1 was only an API we were allowed to avoid the user
interface issue but only examine the on tape format.

It turns out while it seems to have been unintended, Ken's original V7
implementation has an interesting coding feature/bug which turns out to be
what clinched the deal.   When Ken creates the directory block for each
file, he did bcopy of 0's to the buffer before he wrote that data that
fills it in.  Then when he calculated the checksum for the directory header
block, he summed the entire block (which because of the bcopy was zeros).
This means if you write beyond the end of Ken's original header and include
that extra data in the chksum, the original program will ignore the new
information but accept the directory block as valid.  i.e. he had built an
extension mechanism into the tar on-tape format.

cpio's ASCII on tape format was not able to do that as the checksum used a
sizeof(header struct) in the checksum routine.

USTAR was born ... Ken had written things like the UID/GID as ASCII
representations of the integer value in the original header.  USTAR added
the ASCII representation of the username and the group name since that was
more often portable between systems than the numbers.   There were other
additions like more room for the pathname new file types *etc*.  But the
key is that a USTAR tape can be read by the original V7 (and follow on)
tape formats, although may not recognize all the filetype or use all of the
new information.

A few years later during *.2 discussions, we finally got into the user
interface stuff and pax(1) was born.  Knowing my hack with car, Keith
Bostic, Jim McGuiness and I wrote up a description of a program that could
with both users interfaces scheme.  USENIX provided funding for a student
to implement it and put the sources out on comp.unix.sources at some
point.  That proposal was originally accepted at the first tape user
interface program in *.2 [a few years later after I stopped being part of
the committee, the USG folks did get an alternate CPIO format accepted and
cpio as an allowed program.   USENIX paid to have the program updated to
operate like cpio if it was called that, pure V7 tar if called that and if
pax, user USTAR].

'nuf said ... I hope.

Clem

[-- Attachment #2: Type: text/html, Size: 10421 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball
  2020-07-29  0:09 ` Warner Losh
  2020-07-29  0:19   ` Clem Cole
@ 2020-07-29  0:45   ` Will Senn
  2020-07-29  0:46   ` Will Senn
  2 siblings, 0 replies; 16+ messages in thread
From: Will Senn @ 2020-07-29  0:45 UTC (permalink / raw)
  To: Warner Losh; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

On 7/28/20 7:09 PM, Warner Losh wrote:
>
> Both of the 2bsd tapes you found are from the days when Berkeley just 
> sent patches to the 7th Edition out. The 2.8BSD tape was the first one 
> to have a kernel that was bootable from the tape. The 2BSD tapes 
> originally had 2 files on them. The first one was a binary copy of tar 
> that ran on V7. The second was a tarball of all the rest. As you 
> discovered, they shipped with a label like:
>         Second Berkeley Software Tape
>         May 10, 1979    TAR 800BPI
>
>         %dd if=/dev/mt0 bs=1b skip=1 of=tar
>         %chmod 755 tar
>         % tar x
>
>         10000 blocks
> but the 2bsd.tar.gz file has just the second file.
>
> The spensor_2bsd.tar.gz has a tar binary in it:
>
>     tar tvf spencer_2bsd.tar.gz | head
>     -rw-r--r--  0 0      0       24688 Feb 17  1980 tar
>     -rw-r--r--  0 0      10       3687 Feb 17  1980 tar.1
>     -rw-r--r--  0 0      10        456 Feb 17  1980 tar.ms <http://tar.ms>
>     -rw-r--r--  0 0      10      15216 Feb 17  1980 install.ms
>     <http://install.ms>
>
> if you are looking for that pre-built. If you are looking to create a 
> tape with tar on it to extract other tar tapes, you'd need to use a 
> variation on the maketape.pl <http://maketape.pl> with a block size of 
> 1 so the above dd will work on the target system...
>
> Warner
OK. That makes sense. I'll just work with it as is on my v7 system.

Will

-- 
GPG Fingerprint: 68F4 B3BD 1730 555A 4462  7D45 3EAA 5B6D A982 BAAF


[-- Attachment #2: Type: text/html, Size: 3099 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball
  2020-07-29  0:09 ` Warner Losh
  2020-07-29  0:19   ` Clem Cole
  2020-07-29  0:45   ` Will Senn
@ 2020-07-29  0:46   ` Will Senn
  2 siblings, 0 replies; 16+ messages in thread
From: Will Senn @ 2020-07-29  0:46 UTC (permalink / raw)
  To: Warner Losh; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]

On 7/28/20 7:09 PM, Warner Losh wrote:
> Both of the 2bsd tapes you found are from the days when Berkeley just 
> sent patches to the 7th Edition out. The 2.8BSD tape was the first one 
> to have a kernel that was bootable from the tape. The 2BSD tapes 
> originally had 2 files on them. The first one was a binary copy of tar 
> that ran on V7. The second was a tarball of all the rest. As you 
> discovered, they shipped with a label like:
>         Second Berkeley Software Tape
>         May 10, 1979    TAR 800BPI
>
>         %dd if=/dev/mt0 bs=1b skip=1 of=tar
>         %chmod 755 tar
>         % tar x
>
>         10000 blocks
> but the 2bsd.tar.gz file has just the second file.
>
> The spensor_2bsd.tar.gz has a tar binary in it:
>
>     tar tvf spencer_2bsd.tar.gz | head
>     -rw-r--r--  0 0      0       24688 Feb 17  1980 tar
>     -rw-r--r--  0 0      10       3687 Feb 17  1980 tar.1
>     -rw-r--r--  0 0      10        456 Feb 17  1980 tar.ms <http://tar.ms>
>     -rw-r--r--  0 0      10      15216 Feb 17  1980 install.ms
>     <http://install.ms>
>
> if you are looking for that pre-built. If you are looking to create a 
> tape with tar on it to extract other tar tapes, you'd need to use a 
> variation on the maketape.pl <http://maketape.pl> with a block size of 
> 1 so the above dd will work on the target system...

This definitely makes sense. Thanks. This


-- 
GPG Fingerprint: 68F4 B3BD 1730 555A 4462  7D45 3EAA 5B6D A982 BAAF


[-- Attachment #2: Type: text/html, Size: 3012 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh]  2bsd tarball
  2020-07-29  0:21 ` Clem Cole
@ 2020-07-29  9:50   ` Johnny Billquist
  2020-07-29 13:52     ` John Cowan
       [not found]     ` <5A12E0BB-4FFF-4C3E-B486-D4E852FAA97F@comcast.net>
  2020-07-29 13:42   ` [TUHS] 2bsd tarball -> pdtar, with a side of uuslave John Gilmore
  1 sibling, 2 replies; 16+ messages in thread
From: Johnny Billquist @ 2020-07-29  9:50 UTC (permalink / raw)
  To: simh, Will Senn; +Cc: TUHS main list

Just a small comment. Whoever it was that thought DECtape was a tape was 
making a serious mistake. DECtapes are very different from magtapes.

   Johnny

On 2020-07-29 02:21, Clement T Cole wrote:
> 
> Cross posting to simh - since much of this has been discussed in the 
> last few days there also....
> 
> in for penny, in for pound ... here is the history ...  man ... I lived 
> this and I'll need a strong drink later tonight after I write it all up.
> 
> 
> On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn@gmail.com 
> <mailto:will.senn@gmail.com>> wrote:
> 
>     I recall having to do something with cont.a files, which are not
>     present on these images. So, my questions is, does anyone know of or
>     have an actual 2bsd tape/tape image?
> 
> cont.a is a tp-v6 and earlier ism.
> 
> DECtape had a directory at the front of the tape (think 
> superblock/ilist), but could do cool things and be treated more like a disk.
> When tp was created for a very early version of Unix (I'm not sure 
> which, could be V2), Ken/Dennis et al had DECtape units and so the 
> original scheme followed the media.   This also meant that the program 
> could write files and go back and update the directory, which is a no-no 
> with many tape systems.  Then research got a 9-track unit.   So tp was 
> changed to calculate how much space was going to be needed, write the 
> directory, then the datablocks.  All good ... except...
> 
> 9-track could write more files than the directory could take.   So for 
> many years, people would use the ar(1) program to take a number of files 
> in a directory, create a file called cont.a and then delete the files.  
> Then the tree would be written with tp, when you read it, you reversed 
> the ar(1) process.  If you look at the USENIX/Harvard tape on the TUHS 
> you'll see this scheme in use.
> 
> BTW: tp was written in assembler and all the data structures were using 
> PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp) in C  
> (which is on the USENIX tape Warren has in the archives) that worked 
> like the original assembler tp but also put a redundant directory at the 
> end of the tape.  Another issue with tp was if the you had a bad block 
> in the first few blocks you could not decode the rest of the tape.  
> [There were some other issues with the UNIX tree structure as disks got 
> bigger but I'm going to ignore all that - other than to say, tp had 
> lived it life].
> 
> Enter Mashey and the PWB 1.0 folks (which is based on V6).  Someone in 
> USG created cpio (and volcpy) as part of the PWB 1.0.   Like tp it was a 
> PDP-11 binary format, but unlike tp, the tape directory is threaded. 
> /i.e./ block one describes the first file only and includes the size of 
> the following file, then file itself, then a new directory block for the 
> next file and again that file (rinse and repeat).  So it improved on tp 
> in the directory threading, but was still binary and for a reasons I'll 
> leave out had a different user interface.
> 
> As part of V7, Ken wrote a new program, tar [you can ask him why].  But 
> like cpio he used a threaded tape directory, but unlike cpio it was 
> always ASCII and not PDP-11 specific.  Furthermore, the user interface 
> was made to parrot tp.  So, certainly, it had the advantage that 
> changing tp scripts to use tar was pretty easy i.e. s/tp/tar/     not so 
> for coil.  And it was muscle memory compliant.
> 
> For PWB 2.0, cpio was updated to allow a -c option to write the header 
> in ascii and -s to byte swap the binary.   But the damage had been done ...
> 
> Thus began 'tar wars' which was a battle that raged officially over tape 
> archive formats, but really was an argument about user interfaces.  
> Since tar was part of Research and the Universities and commercial 
> people used it, only USG and the folks inside the Bell System were using 
> cpio, as officially none of us had it since PWB was not released to us 
> (although thanks to many AT&T employees doing an OYOC year, many schools 
> like UCB, MIT and CMU all had the sources to cpio anyway -- for instance 
> you'll see it hidden away on Kirk's CD).
> 
> I personally had used both, preferred tar for easy of use and ASCII 
> directories.  But, note I had written car at Masscomp, but never tpio.  
> This was our trick to use the file scripting list that cpio could do, 
> but create tar format tapes - which was handy.  I never wrote tpio which 
> would have been cpio format using tp/tar user interface as I did not 
> need it.
> 
> Roll forward to the /usr/group UNIX standard that Heinz chaired.  We 
> ended up not being able to agree on a distribution format, but the ISVs 
> were PO because now they could create UNIX programs that might actually 
> work across systems, but they had not standard way to distribution.
> Roll forward again to IEEE.  Heinz's committee was officially disbanded 
> (story discussed elsewhere) and we were created as IEEE P1003 with Jim 
> Issack as Chair. This time the ISV's said we had to have a distribution 
> format.  Since *.1 was only an API we were allowed to avoid the user 
> interface issue but only examine the on tape format.
> 
> It turns out while it seems to have been unintended, Ken's original V7 
> implementation has an interesting coding feature/bug which turns out to 
> be what clinched the deal.   When Ken creates the directory block for 
> each file, he did bcopy of 0's to the buffer before he wrote that data 
> that fills it in.  Then when he calculated the checksum for the 
> directory header block, he summed the entire block (which because of the 
> bcopy was zeros).  This means if you write beyond the end of Ken's 
> original header and include that extra data in the chksum, the original 
> program will ignore the new information but accept the directory block 
> as valid.  i.e. he had built an extension mechanism into the tar on-tape 
> format.
> 
> cpio's ASCII on tape format was not able to do that as the checksum used 
> a sizeof(header struct) in the checksum routine.
> 
> USTAR was born ... Ken had written things like the UID/GID as ASCII 
> representations of the integer value in the original header.  USTAR 
> added the ASCII representation of the username and the group name since 
> that was more often portable between systems than the numbers.   There 
> were other additions like more room for the pathname new file types 
> /etc/.  But the key is that a USTAR tape can be read by the original V7 
> (and follow on) tape formats, although may not recognize all the 
> filetype or use all of the new information.
> 
> A few years later during *.2 discussions, we finally got into the user 
> interface stuff and pax(1) was born.  Knowing my hack with car, Keith 
> Bostic, Jim McGuiness and I wrote up a description of a program that 
> could with both users interfaces scheme.  USENIX provided funding for a 
> student to implement it and put the sources out on comp.unix.sources at 
> some point.  That proposal was originally accepted at the first tape 
> user interface program in *.2 [a few years later after I stopped being 
> part of the committee, the USG folks did get an alternate CPIO format 
> accepted and cpio as an allowed program.   USENIX paid to have the 
> program updated to operate like cpio if it was called that, pure V7 tar 
> if called that and if pax, user USTAR].
> 
> 'nuf said ... I hope.
> 
> Clem
> 
> _._,_._,_
> ------------------------------------------------------------------------
> Groups.io Links:
> 
> You receive all messages sent to this group.
> 
> View/Reply Online (#62) <https://groups.io/g/simh/message/62> | Reply To 
> Group 
> <mailto:simh@groups.io?subject=Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball> 
> | Reply To Sender 
> <mailto:clemc@ccc.com?subject=Private:%20Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball> 
> | Mute This Topic <https://groups.io/mt/75856261/4814011> | New Topic 
> <https://groups.io/g/simh/post>
> 
> Your Subscription <https://groups.io/g/simh/editsub/4814011> | Contact 
> Group Owner <mailto:simh+owner@groups.io> | Unsubscribe 
> <https://groups.io/g/simh/leave/8625569/104597204/xyzzy> [bqt@softjar.se]
> 
> _._,_._,_

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball -> pdtar, with a side of uuslave
  2020-07-29  0:21 ` Clem Cole
  2020-07-29  9:50   ` [TUHS] [simh] " Johnny Billquist
@ 2020-07-29 13:42   ` John Gilmore
  2020-07-29 15:40     ` Clem Cole
  1 sibling, 1 reply; 16+ messages in thread
From: John Gilmore @ 2020-07-29 13:42 UTC (permalink / raw)
  To: Clem Cole; +Cc: TUHS main list, simh

Clem Cole <clemc@ccc.com> wrote:
> 'nuf said ... I hope.

Not quite, Clem!

There was another chapter to the "tar wars" after UNIX and after POSIX.

After I left Sun in about 1985, I worked on a project with GNU and the
BSD folks, to find or write freely available implementations of many
popular UNIX commands.  Since we didn't find a free "tar" program, I
wrote one from scratch, based on the SunOS man page and on running the
tar binary from SunOS 3.3.

This was eventually posted to Usenet's mod.sources as "pdtar" on
1986-12-10, as Volume 7 Issue 88.  A July 1987 version went on the Sun
User Group tape.  A later version was posted to comp.sources.unix on
1987-11-29 as Volume 12, issue 68 in 3 parts.  By then, it built and ran
on SunOS, Xenix, Unisoft, Vax 4.2BSD, V7 and USG systems, MSDOS, and
Minix.  And "Utzoonix", Henry Spencer's customized system.

In the process I discovered various quirks in the Unix tar program.  It
null-terminated some of the fixed-length octal fields in the tar headers
and space-terminated others.  I made my code produce bit-for-bit
identical tar files, except for the leftover buffer garbage at the ends
of files and in the last block after the second end-of-file all-zero
header, which I zeroed.

The UNIX tar program was also reading and writing the files in the file
system in 512-byte reads and writes.  Eg when extracting a file from a
tape with the default blocking factor of 20, it would do 20 writes to
the file rather than a single 10k write.  pdtar did better.

Oh, and by the way, the tar format DID use numerical uid and gid fields,
encoded in ascii numerals in octal.  Thus they were both numeric and
ascii!  What it didn't do was put in the NAMES of the user and their
group; that idea came in with POSIX drafts, and went into separate
fields that had been undefined & ignored in UNIX tar.

I put the pdtar code into the public domain, so it could be widely used.
This produced a variety of support headaches.  In particular, people who
used it on MSDOS kept contacting me over the years about bugs or
documentation or not working with certain hardware.  Those ports were
largely distributed as binaries by other people who never bothered to
include nor publish the matching source code.  So I couldn't support the
users who were having trouble, which was frustrating for both them and
me.  This eventually led me to understand more of the value in using the
GNU General Public License.

When the GNU Project later wanted someone else to enhance tar to satisfy
some of the user requirements from the absence of dump(8), like
incremental backups, they asked if I would mind if they put their
improved version under the GPL.  I had no problem with that, and the
pdtar code became the base code for GNU Tar, which (along with the
simple BusyBox tar) seems to have become the main implementation in the
wild today.

	John
	
PS: In that period, I also wrote the first reliable free implementation
of uucp, called gnuucp.  This code was based on "uuslave.c 1.7 08/12/85
12:04:20" which had been posted to the ACGNJ BBS system at +1 201 753
9758.  Original author unknown.  BSD and I went through a legal
interaction with AT&T to verify that the uuslave code had NOT come from
any proprietary UNIX code.  The uuslave code barely limped (and had CP/M
ifdef's), but it did point out the basics of the 'g' protocol.  Once I
got it working on UNIX, I ran it for a few weeks with my friends at
lll-crg in Livermore, CA, then posted it to net.sources on 1987-03-25,
article-id hoptoad.1925.  By this time it had been ported to MSDOS by
Tim Pozar.  Soon it became the backbone of Tim's FIDOnet <-> Usenet
gateway (UFGATE) software, which enabled thousands of (largely teenage)
FIDOnet nodes to join the Usenet and swap email with UNIX machines and
thus with the early Internet.  Many years later, Ian Taylor wrote a
complete free replacement for uucp, called Taylor UUCP, which seems to
be what people were using last time I saw anyone using uucp.  By
coincidence, Ian also later worked with me at Cygnus, the first company
doing commercial support for free software.

A modern web search turns up this about uuslave's BBS:

  http://bbslist.textfiles.com/201/oldschool.html

  "201-753-9758   ACGNJ BBS #2 	(1983-1990)	Kevin Tillbrook 	RBBS

  "I used to run an RBBS system for the Amateur Computer Group of NJ
  (ACGNJ) for a number of years. It was run on a Zenith PC w/20 meg HD
  and later using DesqView for multi-tasking (which was way too slow on
  that hardware). I had a BBS running before that, but this is what I am
  noted for." - Kevin Tillbrook

Does anybody here (from New Jersey perhaps) know where uuslave came
from?  It's always been a mystery to me.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh] 2bsd tarball
  2020-07-29  9:50   ` [TUHS] [simh] " Johnny Billquist
@ 2020-07-29 13:52     ` John Cowan
  2020-07-29 14:30       ` Johnny Billquist
  2020-08-11 23:41       ` Dave Horsfall
       [not found]     ` <5A12E0BB-4FFF-4C3E-B486-D4E852FAA97F@comcast.net>
  1 sibling, 2 replies; 16+ messages in thread
From: John Cowan @ 2020-07-29 13:52 UTC (permalink / raw)
  To: Johnny Billquist; +Cc: TUHS main list, simh

[-- Attachment #1: Type: text/plain, Size: 8918 bytes --]

When I talk about DECtape in my capacity as the local old fart, i describe
it as "a disk with one track and about 1500 small sectors that spins ve-ry
ve-ry slow-ly..

On Wed, Jul 29, 2020 at 5:58 AM Johnny Billquist <bqt@softjar.se> wrote:

> Just a small comment. Whoever it was that thought DECtape was a tape was
> making a serious mistake. DECtapes are very different from magtapes.
>
>    Johnny
>
> On 2020-07-29 02:21, Clement T Cole wrote:
> >
> > Cross posting to simh - since much of this has been discussed in the
> > last few days there also....
> >
> > in for penny, in for pound ... here is the history ...  man ... I lived
> > this and I'll need a strong drink later tonight after I write it all up.
> >
> >
> > On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn@gmail.com
> > <mailto:will.senn@gmail.com>> wrote:
> >
> >     I recall having to do something with cont.a files, which are not
> >     present on these images. So, my questions is, does anyone know of or
> >     have an actual 2bsd tape/tape image?
> >
> > cont.a is a tp-v6 and earlier ism.
> >
> > DECtape had a directory at the front of the tape (think
> > superblock/ilist), but could do cool things and be treated more like a
> disk.
> > When tp was created for a very early version of Unix (I'm not sure
> > which, could be V2), Ken/Dennis et al had DECtape units and so the
> > original scheme followed the media.   This also meant that the program
> > could write files and go back and update the directory, which is a no-no
> > with many tape systems.  Then research got a 9-track unit.   So tp was
> > changed to calculate how much space was going to be needed, write the
> > directory, then the datablocks.  All good ... except...
> >
> > 9-track could write more files than the directory could take.   So for
> > many years, people would use the ar(1) program to take a number of files
> > in a directory, create a file called cont.a and then delete the files.
> > Then the tree would be written with tp, when you read it, you reversed
> > the ar(1) process.  If you look at the USENIX/Harvard tape on the TUHS
> > you'll see this scheme in use.
> >
> > BTW: tp was written in assembler and all the data structures were using
> > PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp) in C
> > (which is on the USENIX tape Warren has in the archives) that worked
> > like the original assembler tp but also put a redundant directory at the
> > end of the tape.  Another issue with tp was if the you had a bad block
> > in the first few blocks you could not decode the rest of the tape.
> > [There were some other issues with the UNIX tree structure as disks got
> > bigger but I'm going to ignore all that - other than to say, tp had
> > lived it life].
> >
> > Enter Mashey and the PWB 1.0 folks (which is based on V6).  Someone in
> > USG created cpio (and volcpy) as part of the PWB 1.0.   Like tp it was a
> > PDP-11 binary format, but unlike tp, the tape directory is threaded.
> > /i.e./ block one describes the first file only and includes the size of
> > the following file, then file itself, then a new directory block for the
> > next file and again that file (rinse and repeat).  So it improved on tp
> > in the directory threading, but was still binary and for a reasons I'll
> > leave out had a different user interface.
> >
> > As part of V7, Ken wrote a new program, tar [you can ask him why].  But
> > like cpio he used a threaded tape directory, but unlike cpio it was
> > always ASCII and not PDP-11 specific.  Furthermore, the user interface
> > was made to parrot tp.  So, certainly, it had the advantage that
> > changing tp scripts to use tar was pretty easy i.e. s/tp/tar/     not so
> > for coil.  And it was muscle memory compliant.
> >
> > For PWB 2.0, cpio was updated to allow a -c option to write the header
> > in ascii and -s to byte swap the binary.   But the damage had been done
> ...
> >
> > Thus began 'tar wars' which was a battle that raged officially over tape
> > archive formats, but really was an argument about user interfaces.
> > Since tar was part of Research and the Universities and commercial
> > people used it, only USG and the folks inside the Bell System were using
> > cpio, as officially none of us had it since PWB was not released to us
> > (although thanks to many AT&T employees doing an OYOC year, many schools
> > like UCB, MIT and CMU all had the sources to cpio anyway -- for instance
> > you'll see it hidden away on Kirk's CD).
> >
> > I personally had used both, preferred tar for easy of use and ASCII
> > directories.  But, note I had written car at Masscomp, but never tpio.
> > This was our trick to use the file scripting list that cpio could do,
> > but create tar format tapes - which was handy.  I never wrote tpio which
> > would have been cpio format using tp/tar user interface as I did not
> > need it.
> >
> > Roll forward to the /usr/group UNIX standard that Heinz chaired.  We
> > ended up not being able to agree on a distribution format, but the ISVs
> > were PO because now they could create UNIX programs that might actually
> > work across systems, but they had not standard way to distribution.
> > Roll forward again to IEEE.  Heinz's committee was officially disbanded
> > (story discussed elsewhere) and we were created as IEEE P1003 with Jim
> > Issack as Chair. This time the ISV's said we had to have a distribution
> > format.  Since *.1 was only an API we were allowed to avoid the user
> > interface issue but only examine the on tape format.
> >
> > It turns out while it seems to have been unintended, Ken's original V7
> > implementation has an interesting coding feature/bug which turns out to
> > be what clinched the deal.   When Ken creates the directory block for
> > each file, he did bcopy of 0's to the buffer before he wrote that data
> > that fills it in.  Then when he calculated the checksum for the
> > directory header block, he summed the entire block (which because of the
> > bcopy was zeros).  This means if you write beyond the end of Ken's
> > original header and include that extra data in the chksum, the original
> > program will ignore the new information but accept the directory block
> > as valid.  i.e. he had built an extension mechanism into the tar on-tape
> > format.
> >
> > cpio's ASCII on tape format was not able to do that as the checksum used
> > a sizeof(header struct) in the checksum routine.
> >
> > USTAR was born ... Ken had written things like the UID/GID as ASCII
> > representations of the integer value in the original header.  USTAR
> > added the ASCII representation of the username and the group name since
> > that was more often portable between systems than the numbers.   There
> > were other additions like more room for the pathname new file types
> > /etc/.  But the key is that a USTAR tape can be read by the original V7
> > (and follow on) tape formats, although may not recognize all the
> > filetype or use all of the new information.
> >
> > A few years later during *.2 discussions, we finally got into the user
> > interface stuff and pax(1) was born.  Knowing my hack with car, Keith
> > Bostic, Jim McGuiness and I wrote up a description of a program that
> > could with both users interfaces scheme.  USENIX provided funding for a
> > student to implement it and put the sources out on comp.unix.sources at
> > some point.  That proposal was originally accepted at the first tape
> > user interface program in *.2 [a few years later after I stopped being
> > part of the committee, the USG folks did get an alternate CPIO format
> > accepted and cpio as an allowed program.   USENIX paid to have the
> > program updated to operate like cpio if it was called that, pure V7 tar
> > if called that and if pax, user USTAR].
> >
> > 'nuf said ... I hope.
> >
> > Clem
> >
> > _._,_._,_
> > ------------------------------------------------------------------------
> > Groups.io Links:
> >
> > You receive all messages sent to this group.
> >
> > View/Reply Online (#62) <https://groups.io/g/simh/message/62> | Reply
> To
> > Group
> > <mailto:simh@groups.io?subject=Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
>
> > | Reply To Sender
> > <mailto:clemc@ccc.com?subject=Private:%20Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
>
> > | Mute This Topic <https://groups.io/mt/75856261/4814011> | New Topic
> > <https://groups.io/g/simh/post>
> >
> > Your Subscription <https://groups.io/g/simh/editsub/4814011> | Contact
> > Group Owner <mailto:simh+owner@groups.io> | Unsubscribe
> > <https://groups.io/g/simh/leave/8625569/104597204/xyzzy> [bqt@softjar.se
> ]
> >
> > _._,_._,_
>
> --
> Johnny Billquist                  || "I'm on a bus
>                                    ||  on a psychedelic trip
> email: bqt@softjar.se             ||  Reading murder books
> pdp is alive!                     ||  tryin' to stay hip" - B. Idol
>

[-- Attachment #2: Type: text/html, Size: 11309 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh]  2bsd tarball
       [not found]     ` <5A12E0BB-4FFF-4C3E-B486-D4E852FAA97F@comcast.net>
@ 2020-07-29 14:29       ` Johnny Billquist
  2020-08-11 23:55         ` Dave Horsfall
  0 siblings, 1 reply; 16+ messages in thread
From: Johnny Billquist @ 2020-07-29 14:29 UTC (permalink / raw)
  To: simh; +Cc: TUHS main list

On 2020-07-29 15:17, Paul Koning wrote:
> 
> 
>> On Jul 29, 2020, at 5:50 AM, Johnny Billquist <bqt@softjar.se> wrote:
>>
>> Just a small comment. Whoever it was that thought DECtape was a tape was making a serious mistake. DECtapes are very different from magtapes.
>>
>>   Johnny
> 
> Depends on what you're focusing on.  Most tapes are not random-write.  DECtape and EL-X1 tape are exceptional in that respect.  But tapes, DECtape include, have access time proportional to delta block number (and that time is large) unlike disks.
> 
>  From the point of view of I/O semantics, the first point is significant and the second one not so much.

True. But seek times are in the end only relevant as an aspect of the 
speed of the thing, nothing else.

However, seek times on DECtape aren't really comparable to magtape 
either. Because DECtape deals with absolute block numbers. So you can 
always, no matter where you are, find out where you are, and how far you 
will need to move to get to the correct block.
With magtapes, this is pretty much impossible. You'll have to rewind, 
and then start seeking.

   Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh] 2bsd tarball
  2020-07-29 13:52     ` John Cowan
@ 2020-07-29 14:30       ` Johnny Billquist
  2020-08-11 23:41       ` Dave Horsfall
  1 sibling, 0 replies; 16+ messages in thread
From: Johnny Billquist @ 2020-07-29 14:30 UTC (permalink / raw)
  To: John Cowan; +Cc: TUHS main list, simh

Yes. That is a much better way to look at it.

   Johnny

On 2020-07-29 15:52, John Cowan wrote:
> When I talk about DECtape in my capacity as the local old fart, i 
> describe it as "a disk with one track and about 1500 small sectors that 
> spins ve-ry ve-ry slow-ly..
> 
> On Wed, Jul 29, 2020 at 5:58 AM Johnny Billquist <bqt@softjar.se 
> <mailto:bqt@softjar.se>> wrote:
> 
>     Just a small comment. Whoever it was that thought DECtape was a tape
>     was
>     making a serious mistake. DECtapes are very different from magtapes.
> 
>         Johnny
> 
>     On 2020-07-29 02:21, Clement T Cole wrote:
>      >
>      > Cross posting to simh - since much of this has been discussed in the
>      > last few days there also....
>      >
>      > in for penny, in for pound ... here is the history ...  man ... I
>     lived
>      > this and I'll need a strong drink later tonight after I write it
>     all up.
>      >
>      >
>      > On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn@gmail.com
>     <mailto:will.senn@gmail.com>
>      > <mailto:will.senn@gmail.com <mailto:will.senn@gmail.com>>> wrote:
>      >
>      >     I recall having to do something with cont.a files, which are not
>      >     present on these images. So, my questions is, does anyone
>     know of or
>      >     have an actual 2bsd tape/tape image?
>      >
>      > cont.a is a tp-v6 and earlier ism.
>      >
>      > DECtape had a directory at the front of the tape (think
>      > superblock/ilist), but could do cool things and be treated more
>     like a disk.
>      > When tp was created for a very early version of Unix (I'm not sure
>      > which, could be V2), Ken/Dennis et al had DECtape units and so the
>      > original scheme followed the media.   This also meant that the
>     program
>      > could write files and go back and update the directory, which is
>     a no-no
>      > with many tape systems.  Then research got a 9-track unit.   So
>     tp was
>      > changed to calculate how much space was going to be needed, write
>     the
>      > directory, then the datablocks.  All good ... except...
>      >
>      > 9-track could write more files than the directory could take. 
>       So for
>      > many years, people would use the ar(1) program to take a number
>     of files
>      > in a directory, create a file called cont.a and then delete the
>     files.
>      > Then the tree would be written with tp, when you read it, you
>     reversed
>      > the ar(1) process.  If you look at the USENIX/Harvard tape on the
>     TUHS
>      > you'll see this scheme in use.
>      >
>      > BTW: tp was written in assembler and all the data structures were
>     using
>      > PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp)
>     in C
>      > (which is on the USENIX tape Warren has in the archives) that worked
>      > like the original assembler tp but also put a redundant directory
>     at the
>      > end of the tape.  Another issue with tp was if the you had a bad
>     block
>      > in the first few blocks you could not decode the rest of the tape.
>      > [There were some other issues with the UNIX tree structure as
>     disks got
>      > bigger but I'm going to ignore all that - other than to say, tp had
>      > lived it life].
>      >
>      > Enter Mashey and the PWB 1.0 folks (which is based on V6). 
>     Someone in
>      > USG created cpio (and volcpy) as part of the PWB 1.0.   Like tp
>     it was a
>      > PDP-11 binary format, but unlike tp, the tape directory is threaded.
>      > /i.e./ block one describes the first file only and includes the
>     size of
>      > the following file, then file itself, then a new directory block
>     for the
>      > next file and again that file (rinse and repeat).  So it improved
>     on tp
>      > in the directory threading, but was still binary and for a
>     reasons I'll
>      > leave out had a different user interface.
>      >
>      > As part of V7, Ken wrote a new program, tar [you can ask him
>     why].  But
>      > like cpio he used a threaded tape directory, but unlike cpio it was
>      > always ASCII and not PDP-11 specific.  Furthermore, the user
>     interface
>      > was made to parrot tp.  So, certainly, it had the advantage that
>      > changing tp scripts to use tar was pretty easy i.e. s/tp/tar/   
>       not so
>      > for coil.  And it was muscle memory compliant.
>      >
>      > For PWB 2.0, cpio was updated to allow a -c option to write the
>     header
>      > in ascii and -s to byte swap the binary.   But the damage had
>     been done ...
>      >
>      > Thus began 'tar wars' which was a battle that raged officially
>     over tape
>      > archive formats, but really was an argument about user interfaces.
>      > Since tar was part of Research and the Universities and commercial
>      > people used it, only USG and the folks inside the Bell System
>     were using
>      > cpio, as officially none of us had it since PWB was not released
>     to us
>      > (although thanks to many AT&T employees doing an OYOC year, many
>     schools
>      > like UCB, MIT and CMU all had the sources to cpio anyway -- for
>     instance
>      > you'll see it hidden away on Kirk's CD).
>      >
>      > I personally had used both, preferred tar for easy of use and ASCII
>      > directories.  But, note I had written car at Masscomp, but never
>     tpio.
>      > This was our trick to use the file scripting list that cpio could
>     do,
>      > but create tar format tapes - which was handy.  I never wrote
>     tpio which
>      > would have been cpio format using tp/tar user interface as I did not
>      > need it.
>      >
>      > Roll forward to the /usr/group UNIX standard that Heinz chaired.  We
>      > ended up not being able to agree on a distribution format, but
>     the ISVs
>      > were PO because now they could create UNIX programs that might
>     actually
>      > work across systems, but they had not standard way to distribution.
>      > Roll forward again to IEEE.  Heinz's committee was officially
>     disbanded
>      > (story discussed elsewhere) and we were created as IEEE P1003
>     with Jim
>      > Issack as Chair. This time the ISV's said we had to have a
>     distribution
>      > format.  Since *.1 was only an API we were allowed to avoid the user
>      > interface issue but only examine the on tape format.
>      >
>      > It turns out while it seems to have been unintended, Ken's
>     original V7
>      > implementation has an interesting coding feature/bug which turns
>     out to
>      > be what clinched the deal.   When Ken creates the directory block
>     for
>      > each file, he did bcopy of 0's to the buffer before he wrote that
>     data
>      > that fills it in.  Then when he calculated the checksum for the
>      > directory header block, he summed the entire block (which because
>     of the
>      > bcopy was zeros).  This means if you write beyond the end of Ken's
>      > original header and include that extra data in the chksum, the
>     original
>      > program will ignore the new information but accept the directory
>     block
>      > as valid.  i.e. he had built an extension mechanism into the tar
>     on-tape
>      > format.
>      >
>      > cpio's ASCII on tape format was not able to do that as the
>     checksum used
>      > a sizeof(header struct) in the checksum routine.
>      >
>      > USTAR was born ... Ken had written things like the UID/GID as ASCII
>      > representations of the integer value in the original header.  USTAR
>      > added the ASCII representation of the username and the group name
>     since
>      > that was more often portable between systems than the numbers. 
>       There
>      > were other additions like more room for the pathname new file types
>      > /etc/.  But the key is that a USTAR tape can be read by the
>     original V7
>      > (and follow on) tape formats, although may not recognize all the
>      > filetype or use all of the new information.
>      >
>      > A few years later during *.2 discussions, we finally got into the
>     user
>      > interface stuff and pax(1) was born.  Knowing my hack with car,
>     Keith
>      > Bostic, Jim McGuiness and I wrote up a description of a program that
>      > could with both users interfaces scheme.  USENIX provided funding
>     for a
>      > student to implement it and put the sources out on
>     comp.unix.sources at
>      > some point.  That proposal was originally accepted at the first tape
>      > user interface program in *.2 [a few years later after I stopped
>     being
>      > part of the committee, the USG folks did get an alternate CPIO
>     format
>      > accepted and cpio as an allowed program.   USENIX paid to have the
>      > program updated to operate like cpio if it was called that, pure
>     V7 tar
>      > if called that and if pax, user USTAR].
>      >
>      > 'nuf said ... I hope.
>      >
>      > Clem
>      >
>      > _._,_._,_
>      >
>     ------------------------------------------------------------------------
>      > Groups.io Links:
>      >
>      > You receive all messages sent to this group.
>      >
>      > View/Reply Online (#62) <https://groups.io/g/simh/message/62> |
>     Reply To
>      > Group
>      > <mailto:simh@groups.io
>     <mailto:simh@groups.io>?subject=Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
> 
>      > | Reply To Sender
>      > <mailto:clemc@ccc.com
>     <mailto:clemc@ccc.com>?subject=Private:%20Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
> 
>      > | Mute This Topic <https://groups.io/mt/75856261/4814011> | New
>     Topic
>      > <https://groups.io/g/simh/post>
>      >
>      > Your Subscription <https://groups.io/g/simh/editsub/4814011> |
>     Contact
>      > Group Owner <mailto:simh+owner@groups.io
>     <mailto:simh%2Bowner@groups.io>> | Unsubscribe
>      > <https://groups.io/g/simh/leave/8625569/104597204/xyzzy>
>     [bqt@softjar.se <mailto:bqt@softjar.se>]
>      >
>      > _._,_._,_
> 
>     -- 
>     Johnny Billquist                  || "I'm on a bus
>                                         ||  on a psychedelic trip
>     email: bqt@softjar.se <mailto:bqt@softjar.se>             || 
>     Reading murder books
>     pdp is alive!                     ||  tryin' to stay hip" - B. Idol
> 

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball -> pdtar, with a side of uuslave
  2020-07-29 13:42   ` [TUHS] 2bsd tarball -> pdtar, with a side of uuslave John Gilmore
@ 2020-07-29 15:40     ` Clem Cole
  2020-07-29 19:34       ` Richard Salz
  0 siblings, 1 reply; 16+ messages in thread
From: Clem Cole @ 2020-07-29 15:40 UTC (permalink / raw)
  To: John Gilmore; +Cc: TUHS main list, simh

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

On Wed, Jul 29, 2020 at 9:42 AM John Gilmore <gnu@toad.com> wrote:

> There was another chapter to the "tar wars" after UNIX and after POSIX.
>
Ah ..  indeed - I left out the Gnu Tar story as I was not 100% sure of how
it came about as I had not taken part in it.  And pretty much for the
purposes of how we go to where we are today, other than it exists, works,
is a popular implementation and can read/write things when called upon ...
I did not think it would add to the (already) long story. The null vs space
filling is an interesting point which I had left out.  Thank you for that
detail - I do remember it.

If I missspoke/was confusing (I hope not) about the UIDs I thought I had
said the way you did.  The key was that USTAR added the names in ASCII
which was not there before in Ken's original version.

Again, thanks for the friendly addition/update.


After I left Sun in about 1985, I worked on a project with GNU and the
> BSD folks, to find or write freely available implementations of many
> popular UNIX commands.

Yep, I do remember all that...



> Since we didn't find a free "tar" program, I wrote one from scratch,
> based on the SunOS man page and on running the
> tar binary from SunOS 3.3.
>
I always found that strange the folks that wrote that that tar
implementation (i.e. you and your mates) had not found the pax code, as the
USENIX version had been previously posted/was in the wild by then.  Keith
certainly knew about it (he could have even been part of the finding a
student to write it, but I don't remember), but he also might have been off
at BSDi by that time.  I think by then that the USENIX FOSS implementation
even knew how to behave like cpio, tar, or pax depending on its name.

I'm fairly sure, that Apple and HP had picked it up soon after it's
release.  DEC had a different set of tar switches, so pax was put in the
Ultrix contributed library, and they left theirs alone.  That said, the
USENIX version did have an MIT/UCB/CMU style license, not the gpl, which
our common 'friend' in Cambridge often (??always??) found suspicious.  So,
I had always >>suspected<< the licensing style was driver for yet
another version, and
have always been a little curious.

But to me it was like C compilers, as long as they all worked, I didn't
care.  As you know, I have never been super religious about the different
license flavors as long as I could use it.  Probably a good beer
discussion/story behind it all when I see you next at a future conference
post CV-19.



Clem

[-- Attachment #2: Type: text/html, Size: 5279 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball -> pdtar, with a side of uuslave
  2020-07-29 15:40     ` Clem Cole
@ 2020-07-29 19:34       ` Richard Salz
  2020-07-29 19:42         ` Warner Losh
  0 siblings, 1 reply; 16+ messages in thread
From: Richard Salz @ 2020-07-29 19:34 UTC (permalink / raw)
  To: Clem Cole; +Cc: TUHS main list, simh

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

Wasn't the tour header Fields the reason for the strange strncpy semantics?
Which came first?

[-- Attachment #2: Type: text/html, Size: 120 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] 2bsd tarball -> pdtar, with a side of uuslave
  2020-07-29 19:34       ` Richard Salz
@ 2020-07-29 19:42         ` Warner Losh
  0 siblings, 0 replies; 16+ messages in thread
From: Warner Losh @ 2020-07-29 19:42 UTC (permalink / raw)
  To: Richard Salz; +Cc: TUHS main list, simh

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

On Wed, Jul 29, 2020 at 1:35 PM Richard Salz <rich.salz@gmail.com> wrote:

> Wasn't the tour header Fields the reason for the strange strncpy
> semantics? Which came first?
>

strncpy appeared in V7 as far as I can tell. I can't find it in v6 or
earlier. I can't find any of the str functions in fact...

Also, there's a new libarchive-based tar as well that the BSDs are using
since it understands many other formats. It's largely replaced the gnutar
that had previously been in BSD.

Warneer

[-- Attachment #2: Type: text/html, Size: 910 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh] 2bsd tarball
  2020-07-29 13:52     ` John Cowan
  2020-07-29 14:30       ` Johnny Billquist
@ 2020-08-11 23:41       ` Dave Horsfall
  1 sibling, 0 replies; 16+ messages in thread
From: Dave Horsfall @ 2020-08-11 23:41 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Wed, 29 Jul 2020, John Cowan wrote:

> When I talk about DECtape in my capacity as the local old fart, i 
> describe it as "a disk with one track and about 1500 small sectors that 
> spins ve-ry ve-ry slow-ly..

According to an ex-boss of mine, a demonstration of the DECtape's 
reliability was to blow cigarette smoke over the heads when the tape was 
moving (this was back when smoking was socially acceptable, if not 
actually encouraged).

-- Dave

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [TUHS] [simh]  2bsd tarball
  2020-07-29 14:29       ` Johnny Billquist
@ 2020-08-11 23:55         ` Dave Horsfall
  0 siblings, 0 replies; 16+ messages in thread
From: Dave Horsfall @ 2020-08-11 23:55 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Wed, 29 Jul 2020, Johnny Billquist wrote:

> However, seek times on DECtape aren't really comparable to magtape 
> either. Because DECtape deals with absolute block numbers. So you can 
> always, no matter where you are, find out where you are, and how far you 
> will need to move to get to the correct block.
> 
> With magtapes, this is pretty much impossible. You'll have to rewind, 
> and then start seeking.

Not if you used 512-byte blocks, and mounted it read-only; that way, you 
could use it as a file system (and I did, just for fun).  Oh, you also 
needed a clever tape driver, of course.  And the IRG was a bit of a 
problem, chewing up a lot of tape space.

I posted this here a while back, and some idiot (his name is not
important) practically called me a liar, until he was handed his arse
on a plate by someone else with better knowledge than him.

-- Dave

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2020-08-11 23:56 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-28 23:03 [TUHS] 2bsd tarball Will Senn
2020-07-29  0:09 ` Warner Losh
2020-07-29  0:19   ` Clem Cole
2020-07-29  0:45   ` Will Senn
2020-07-29  0:46   ` Will Senn
2020-07-29  0:21 ` Clem Cole
2020-07-29  9:50   ` [TUHS] [simh] " Johnny Billquist
2020-07-29 13:52     ` John Cowan
2020-07-29 14:30       ` Johnny Billquist
2020-08-11 23:41       ` Dave Horsfall
     [not found]     ` <5A12E0BB-4FFF-4C3E-B486-D4E852FAA97F@comcast.net>
2020-07-29 14:29       ` Johnny Billquist
2020-08-11 23:55         ` Dave Horsfall
2020-07-29 13:42   ` [TUHS] 2bsd tarball -> pdtar, with a side of uuslave John Gilmore
2020-07-29 15:40     ` Clem Cole
2020-07-29 19:34       ` Richard Salz
2020-07-29 19:42         ` Warner Losh

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).