From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HTML_MESSAGE,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 25678 invoked from network); 29 Jul 2020 13:54:35 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 29 Jul 2020 13:54:35 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 1C7309CAB2; Wed, 29 Jul 2020 23:54:28 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id AB0A09CAA3; Wed, 29 Jul 2020 23:53:14 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ccil-org.20150623.gappssmtp.com header.i=@ccil-org.20150623.gappssmtp.com header.b="Aqi7pB26"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 9351C9CAA3; Wed, 29 Jul 2020 23:53:11 +1000 (AEST) Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by minnie.tuhs.org (Postfix) with ESMTPS id 76DFB9C9E4 for ; Wed, 29 Jul 2020 23:53:10 +1000 (AEST) Received: by mail-qt1-f173.google.com with SMTP id 6so17665536qtt.0 for ; Wed, 29 Jul 2020 06:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XpNw6umPRi+86AlTtyjny8s1F/IkWucZNr1yfk6JXk8=; b=Aqi7pB262v61KLnd22xbBoeDWFp10mCeUiuMFk/CLn5ps4mgqZmAk0RWAOe8uIymNX kR52YqpMX+jHaIRPOiNE16NiaoTUjJk4uYcdHRxWEhllZaWQMVJP1GAxRfLp59kPlLls 3LOBHRnkYxZSrMFpxIyWDph2Q5uND7VYhudLdzyM7cHZ4+LOcsUJZUZgTvGGZqdNrFs/ Q06KFZKnDRpSadUdFEZZCn7V4/S9XzR26Rww7n8h9CYbymI/l7MrhXPT7X3H3GGWGL/j QD1sEAyJEOPoXO+dvCVIENbuKima2deB4FsrekuCA12tKZpzHFwiLf1wYklJfU24q3sz uMhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XpNw6umPRi+86AlTtyjny8s1F/IkWucZNr1yfk6JXk8=; b=nrdnNCjWdqdolpTC/2DNPVj9WSrEeZ0CDFOQtHtFLa43aX2P5gKAoOv2+Bj1ONfDYX nFb1MjklWFld4DbYPs+5AIL3om+yu5XNu48WpIreu6pdWoncGPMh57Mz22C7xFM44VEm d7joCMXYmdFYZLB+7AxTlBqxpfRo+Sidpk6VSh4svSGWlGL0L++uFjgKtk0C+660VXvd I+DiFiTqH/GuRFOAmZ2GJDCA54LBHAK+7PW0l18tiVjEDmmF6CYJJMHlUi+d7mYYeX20 htR2ZUaKR8E7MIGyNxLlWtGu/cUCr5DDcdhLjI+2fIIYiEPPgMihY0IwrgkFpJ/m7HYl cHDA== X-Gm-Message-State: AOAM5326oxQvkuvv8Z2KqHe21SzLaJJvKvkTwk8f82mnbEgVxZQrb5Cs HCI0qepF/3hNhmuG6Qj2wVEz9RVnoQ2col5oefkUYg== X-Google-Smtp-Source: ABdhPJwSWdwKOkBSBol6o3XjATUC2zNS7ecbNLOK33af8DFBOgalCoMA9TC4ojoxMnljGiVioB8O9sWy82HjtW39/bo= X-Received: by 2002:ac8:729a:: with SMTP id v26mr32158779qto.362.1596030789602; Wed, 29 Jul 2020 06:53:09 -0700 (PDT) MIME-Version: 1.0 References: <6a0063f8-128d-751d-114f-a0f811d02098@gmail.com> <2cb3ad2a-f8c5-a003-661c-e257f7cbe38c@softjar.se> In-Reply-To: <2cb3ad2a-f8c5-a003-661c-e257f7cbe38c@softjar.se> From: John Cowan Date: Wed, 29 Jul 2020 09:52:58 -0400 Message-ID: To: Johnny Billquist Content-Type: multipart/alternative; boundary="00000000000043eb0905ab94e0a2" Subject: Re: [TUHS] [simh] 2bsd tarball X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: TUHS main list , simh@groups.io Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --00000000000043eb0905ab94e0a2 Content-Type: text/plain; charset="UTF-8" When I talk about DECtape in my capacity as the local old fart, i describe it as "a disk with one track and about 1500 small sectors that spins ve-ry ve-ry slow-ly.. On Wed, Jul 29, 2020 at 5:58 AM Johnny Billquist wrote: > Just a small comment. Whoever it was that thought DECtape was a tape was > making a serious mistake. DECtapes are very different from magtapes. > > Johnny > > On 2020-07-29 02:21, Clement T Cole wrote: > > > > Cross posting to simh - since much of this has been discussed in the > > last few days there also.... > > > > in for penny, in for pound ... here is the history ... man ... I lived > > this and I'll need a strong drink later tonight after I write it all up. > > > > > > On Tue, Jul 28, 2020 at 7:04 PM Will Senn > > wrote: > > > > I recall having to do something with cont.a files, which are not > > present on these images. So, my questions is, does anyone know of or > > have an actual 2bsd tape/tape image? > > > > cont.a is a tp-v6 and earlier ism. > > > > DECtape had a directory at the front of the tape (think > > superblock/ilist), but could do cool things and be treated more like a > disk. > > When tp was created for a very early version of Unix (I'm not sure > > which, could be V2), Ken/Dennis et al had DECtape units and so the > > original scheme followed the media. This also meant that the program > > could write files and go back and update the directory, which is a no-no > > with many tape systems. Then research got a 9-track unit. So tp was > > changed to calculate how much space was going to be needed, write the > > directory, then the datablocks. All good ... except... > > > > 9-track could write more files than the directory could take. So for > > many years, people would use the ar(1) program to take a number of files > > in a directory, create a file called cont.a and then delete the files. > > Then the tree would be written with tp, when you read it, you reversed > > the ar(1) process. If you look at the USENIX/Harvard tape on the TUHS > > you'll see this scheme in use. > > > > BTW: tp was written in assembler and all the data structures were using > > PDP-11 binary formats. Eventually, Harvard wrote stp (super-tp) in C > > (which is on the USENIX tape Warren has in the archives) that worked > > like the original assembler tp but also put a redundant directory at the > > end of the tape. Another issue with tp was if the you had a bad block > > in the first few blocks you could not decode the rest of the tape. > > [There were some other issues with the UNIX tree structure as disks got > > bigger but I'm going to ignore all that - other than to say, tp had > > lived it life]. > > > > Enter Mashey and the PWB 1.0 folks (which is based on V6). Someone in > > USG created cpio (and volcpy) as part of the PWB 1.0. Like tp it was a > > PDP-11 binary format, but unlike tp, the tape directory is threaded. > > /i.e./ block one describes the first file only and includes the size of > > the following file, then file itself, then a new directory block for the > > next file and again that file (rinse and repeat). So it improved on tp > > in the directory threading, but was still binary and for a reasons I'll > > leave out had a different user interface. > > > > As part of V7, Ken wrote a new program, tar [you can ask him why]. But > > like cpio he used a threaded tape directory, but unlike cpio it was > > always ASCII and not PDP-11 specific. Furthermore, the user interface > > was made to parrot tp. So, certainly, it had the advantage that > > changing tp scripts to use tar was pretty easy i.e. s/tp/tar/ not so > > for coil. And it was muscle memory compliant. > > > > For PWB 2.0, cpio was updated to allow a -c option to write the header > > in ascii and -s to byte swap the binary. But the damage had been done > ... > > > > Thus began 'tar wars' which was a battle that raged officially over tape > > archive formats, but really was an argument about user interfaces. > > Since tar was part of Research and the Universities and commercial > > people used it, only USG and the folks inside the Bell System were using > > cpio, as officially none of us had it since PWB was not released to us > > (although thanks to many AT&T employees doing an OYOC year, many schools > > like UCB, MIT and CMU all had the sources to cpio anyway -- for instance > > you'll see it hidden away on Kirk's CD). > > > > I personally had used both, preferred tar for easy of use and ASCII > > directories. But, note I had written car at Masscomp, but never tpio. > > This was our trick to use the file scripting list that cpio could do, > > but create tar format tapes - which was handy. I never wrote tpio which > > would have been cpio format using tp/tar user interface as I did not > > need it. > > > > Roll forward to the /usr/group UNIX standard that Heinz chaired. We > > ended up not being able to agree on a distribution format, but the ISVs > > were PO because now they could create UNIX programs that might actually > > work across systems, but they had not standard way to distribution. > > Roll forward again to IEEE. Heinz's committee was officially disbanded > > (story discussed elsewhere) and we were created as IEEE P1003 with Jim > > Issack as Chair. This time the ISV's said we had to have a distribution > > format. Since *.1 was only an API we were allowed to avoid the user > > interface issue but only examine the on tape format. > > > > It turns out while it seems to have been unintended, Ken's original V7 > > implementation has an interesting coding feature/bug which turns out to > > be what clinched the deal. When Ken creates the directory block for > > each file, he did bcopy of 0's to the buffer before he wrote that data > > that fills it in. Then when he calculated the checksum for the > > directory header block, he summed the entire block (which because of the > > bcopy was zeros). This means if you write beyond the end of Ken's > > original header and include that extra data in the chksum, the original > > program will ignore the new information but accept the directory block > > as valid. i.e. he had built an extension mechanism into the tar on-tape > > format. > > > > cpio's ASCII on tape format was not able to do that as the checksum used > > a sizeof(header struct) in the checksum routine. > > > > USTAR was born ... Ken had written things like the UID/GID as ASCII > > representations of the integer value in the original header. USTAR > > added the ASCII representation of the username and the group name since > > that was more often portable between systems than the numbers. There > > were other additions like more room for the pathname new file types > > /etc/. But the key is that a USTAR tape can be read by the original V7 > > (and follow on) tape formats, although may not recognize all the > > filetype or use all of the new information. > > > > A few years later during *.2 discussions, we finally got into the user > > interface stuff and pax(1) was born. Knowing my hack with car, Keith > > Bostic, Jim McGuiness and I wrote up a description of a program that > > could with both users interfaces scheme. USENIX provided funding for a > > student to implement it and put the sources out on comp.unix.sources at > > some point. That proposal was originally accepted at the first tape > > user interface program in *.2 [a few years later after I stopped being > > part of the committee, the USG folks did get an alternate CPIO format > > accepted and cpio as an allowed program. USENIX paid to have the > > program updated to operate like cpio if it was called that, pure V7 tar > > if called that and if pax, user USTAR]. > > > > 'nuf said ... I hope. > > > > Clem > > > > _._,_._,_ > > ------------------------------------------------------------------------ > > Groups.io Links: > > > > You receive all messages sent to this group. > > > > View/Reply Online (#62) | Reply > To > > Group > > > > > | Reply To Sender > > > > > | Mute This Topic | New Topic > > > > > > Your Subscription | Contact > > Group Owner | Unsubscribe > > [bqt@softjar.se > ] > > > > _._,_._,_ > > -- > Johnny Billquist || "I'm on a bus > || on a psychedelic trip > email: bqt@softjar.se || Reading murder books > pdp is alive! || tryin' to stay hip" - B. Idol > --00000000000043eb0905ab94e0a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
When I talk about DECtape in my capacity as the local old = fart, i describe it as "a disk with one track and about 1500 small sec= tors that spins ve-ry ve-ry slow-ly..

<= div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jul 29, 2020 at 5:58 AM Johnny= Billquist <bqt@softjar.se> wro= te:
Just a small= comment. Whoever it was that thought DECtape was a tape was
making a serious mistake. DECtapes are very different from magtapes.

=C2=A0 =C2=A0Johnny

On 2020-07-29 02:21, Clement T Cole wrote:
>
> Cross posting to simh - since much of this has been discussed in the <= br> > last few days there also....
>
> in for penny, in for pound ... here is the history ...=C2=A0 man ... I= lived
> this and I'll need a strong drink later tonight after I write it a= ll up.
>
>
> On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn@gmail.com
> <mailto:wi= ll.senn@gmail.com>> wrote:
>
>=C2=A0 =C2=A0 =C2=A0I recall having to do something with cont.a files, = which are not
>=C2=A0 =C2=A0 =C2=A0present on these images. So, my questions is, does = anyone know of or
>=C2=A0 =C2=A0 =C2=A0have an actual 2bsd tape/tape image?
>
> cont.a is a tp-v6 and earlier ism.
>
> DECtape had a directory at the front of the tape (think
> superblock/ilist), but could do cool things and be treated more like a= disk.
> When tp was created for a very early version of Unix (I'm not sure=
> which, could be V2), Ken/Dennis et al had DECtape units and so the > original scheme followed the media.=C2=A0 =C2=A0This also meant that t= he program
> could write files and go back and update the directory, which is a no-= no
> with many tape systems.=C2=A0 Then research got a 9-track unit.=C2=A0 = =C2=A0So tp was
> changed to calculate how much space was going to be needed, write the =
> directory, then the datablocks.=C2=A0 All good ... except...
>
> 9-track could write more files than the directory could=C2=A0take.=C2= =A0 =C2=A0So for
> many years, people would use the ar(1) program to take a number of fil= es
> in a directory, create a file called cont.a and then delete the files.= =C2=A0
> Then the tree would be written with tp, when you read it, you reversed=
> the ar(1) process.=C2=A0 If you look at the USENIX/Harvard tape on the= TUHS
> you'll see this scheme in use.
>
> BTW: tp was written in assembler and all the data structures were usin= g
> PDP-11 binary formats.=C2=A0 Eventually, Harvard wrote stp (super-tp) = in C=C2=A0
> (which is on the USENIX tape Warren has in the archives) that worked <= br> > like the original assembler tp but also put a redundant directory at t= he
> end of the tape.=C2=A0 Another issue with tp was if the you had a bad = block
> in the first few blocks you=C2=A0could not decode the rest of the tape= .=C2=A0
> [There were some other issues with the UNIX tree structure as disks go= t
> bigger but I'm going to ignore all that - other than to say, tp ha= d
> lived it life].
>
> Enter Mashey and the PWB 1.0 folks (which is based on V6).=C2=A0 Someo= ne in
> USG created cpio (and volcpy) as part of the PWB 1.0.=C2=A0 =C2=A0Like= tp it was a
> PDP-11 binary format, but unlike tp, the tape directory is threaded. <= br> > /i.e./ block one describes the first file only and includes the size o= f
> the following file, then file itself, then a new directory block for t= he
> next file and again that file (rinse and repeat).=C2=A0 So it improved= on tp
> in the directory threading, but was still binary and for a reasons I&#= 39;ll
> leave out had a different user interface.
>
> As part of V7, Ken wrote a new program, tar [you can ask him why].=C2= =A0 But
> like cpio he used a threaded tape directory, but unlike cpio it was > always ASCII and not PDP-11 specific.=C2=A0 Furthermore, the user inte= rface
> was made to parrot=C2=A0tp.=C2=A0 So, certainly, it had the advantage = that
> changing tp scripts to use tar was pretty easy i.e. s/tp/tar/=C2=A0 = =C2=A0 =C2=A0not so
> for coil.=C2=A0 And it was muscle memory compliant.
>
> For PWB 2.0, cpio was updated to allow a -c option to write the header=
> in ascii and -s to byte swap the binary.=C2=A0 =C2=A0But the damage ha= d been done ...
>
> Thus began 'tar wars' which was a battle that raged officially= over tape
> archive formats, but really was an argument about user interfaces.=C2= =A0
> Since tar was part of Research and the Universities=C2=A0and commercia= l
> people used it, only USG and the folks inside the Bell System were usi= ng
> cpio, as officially none of us had it since PWB was not released to us=
> (although thanks to many AT&T employees doing an OYOC year, many s= chools
> like UCB, MIT and CMU all had the sources to cpio anyway -- for instan= ce
> you'll see it hidden away on Kirk's CD).
>
> I personally had used both, preferred tar for easy of use and ASCII > directories.=C2=A0 But, note I had written car at Masscomp, but never = tpio.=C2=A0
> This was our trick to use the file scripting list that cpio could do, =
> but create tar format tapes - which was handy.=C2=A0 I never wrote tpi= o which
> would have been cpio format using tp/tar user interface as I did not <= br> > need it.
>
> Roll forward to the /usr/group UNIX standard that Heinz=C2=A0chaired.= =C2=A0 We
> ended up not being able to agree on a distribution format, but the ISV= s
> were PO because now they could create UNIX programs that might actuall= y
> work across systems, but they had not standard way to distribution. > Roll forward again to IEEE.=C2=A0 Heinz's committee was officially= disbanded
> (story discussed elsewhere) and we were created as IEEE P1003 with Jim=
> Issack as Chair. This time the ISV's=C2=A0said we had to have a di= stribution
> format.=C2=A0 Since *.1 was only an API we were allowed to avoid the u= ser
> interface issue but only examine the on tape format.
>
> It turns out while it seems to have been unintended, Ken's origina= l V7
> implementation has an interesting coding feature/bug which turns out t= o
> be what clinched the deal.=C2=A0 =C2=A0When Ken creates the directory = block for
> each file, he did bcopy of 0's to the buffer before he wrote that = data
> that fills it in.=C2=A0 Then when he calculated the checksum for the <= br> > directory header block, he summed the entire block (which because of t= he
> bcopy was zeros).=C2=A0 This means if you write beyond the end of Ken&= #39;s
> original header and include that extra data in the chksum, the origina= l
> program will ignore the new information but accept the directory block=
> as valid.=C2=A0 i.e. he had built an extension mechanism into the tar = on-tape
> format.
>
> cpio's ASCII on tape format was not able to do that as the checksu= m used
> a sizeof(header struct) in the checksum routine.
>
> USTAR was born ... Ken had written things like the UID/GID as ASCII > representations of the integer value in the original header.=C2=A0 UST= AR
> added the ASCII representation of the username and the group name sinc= e
> that was more often portable between systems than the numbers.=C2=A0 = =C2=A0There
> were other additions like more room for the pathname new file types > /etc/.=C2=A0 But the key is that a USTAR tape can be read by the origi= nal V7
> (and follow on) tape formats, although may not recognize all the
> filetype or use all of the new information.
>
> A few years later during *.2 discussions, we finally got into the user=
> interface stuff and pax(1) was born.=C2=A0 Knowing my hack with car, K= eith
> Bostic, Jim McGuiness and I wrote up a description of a program that <= br> > could with both users interfaces scheme.=C2=A0 USENIX provided funding= for a
> student to implement it and put the sources out on comp.unix.sources a= t
> some point.=C2=A0 That proposal was originally accepted at the first t= ape
> user interface program in *.2 [a few years later after I stopped being=
> part of the committee, the USG folks did get an alternate CPIO format =
> accepted and cpio as an allowed program.=C2=A0 =C2=A0USENIX paid to ha= ve the
> program updated to operate like cpio if it was called that, pure V7 ta= r
> if called that and if pax, user USTAR].
>
> 'nuf said ... I hope.
>
> Clem
>
> _._,_._,_
> ----------------------------------------------------------------------= --
> Groups.io Links:
>
> You receive all messages sent to this group.
>
> View/Reply Online (#62) <https://groups.io/g/simh/message= /62> | Reply To
> Group
> <mailto:simh@gr= oups.io?subject=3DRe:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarbal= l>
> | Reply To Sender
> <mailto:clemc@cc= c.com?subject=3DPrivate:%20Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd= %20tarball>
> | Mute This Topic <https://groups.io/mt/75856261/481401= 1> | New Topic
> <https://groups.io/g/simh/post>
>
> Your Subscription <https://groups.io/g/simh/editsub/= 4814011> | Contact
> Group Owner <mailto:simh+owner@groups.io> | Unsubscribe
> <https://groups.io/g/simh/leave/862556= 9/104597204/xyzzy> [bqt@softjar.se]
>
> _._,_._,_

--
Johnny Billquist=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 || "I'm on a bus
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0||=C2=A0 on a psychedel= ic trip
email: bqt@softjar.se=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0||=C2=A0 Reading murder b= ooks
pdp is alive!=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0||=C2=A0 tryin' to stay hip" - B. Idol
--00000000000043eb0905ab94e0a2--