From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=MAILING_LIST_MULTI, NICE_REPLY_A,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 32044 invoked from network); 29 Jul 2020 09:58:47 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 29 Jul 2020 09:58:47 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id CC09B9CAB2; Wed, 29 Jul 2020 19:58:41 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 1CCD29CAA3; Wed, 29 Jul 2020 19:57:16 +1000 (AEST) Received: by minnie.tuhs.org (Postfix, from userid 112) id 661C89CAA3; Wed, 29 Jul 2020 19:57:12 +1000 (AEST) X-Greylist: delayed 442 seconds by postgrey-1.36 at minnie.tuhs.org; Wed, 29 Jul 2020 19:57:10 AEST Received: from GW.SoftJAR.SE (gw.softjar.se [213.115.73.154]) by minnie.tuhs.org (Postfix) with ESMTP id 9D2829C9E4 for ; Wed, 29 Jul 2020 19:57:10 +1000 (AEST) Received: from Trollet.BQTnet.SE (80-219-208-146.dclient.hispeed.ch [80.219.208.146]) by GW.SoftJAR.SE (Postfix) with ESMTPSA id 5C44638FCEF; Wed, 29 Jul 2020 11:49:46 +0200 (CEST) (envelope-from bqt@softjar.se) To: simh@groups.io, Will Senn References: <6a0063f8-128d-751d-114f-a0f811d02098@gmail.com> From: Johnny Billquist Organization: SoftJAR AB Message-ID: <2cb3ad2a-f8c5-a003-661c-e257f7cbe38c@softjar.se> Date: Wed, 29 Jul 2020 11:50:25 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [TUHS] [simh] 2bsd tarball X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: TUHS main list Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" Just a small comment. Whoever it was that thought DECtape was a tape was making a serious mistake. DECtapes are very different from magtapes. Johnny On 2020-07-29 02:21, Clement T Cole wrote: > > Cross posting to simh - since much of this has been discussed in the > last few days there also.... > > in for penny, in for pound ... here is the history ...  man ... I lived > this and I'll need a strong drink later tonight after I write it all up. > > > On Tue, Jul 28, 2020 at 7:04 PM Will Senn > wrote: > > I recall having to do something with cont.a files, which are not > present on these images. So, my questions is, does anyone know of or > have an actual 2bsd tape/tape image? > > cont.a is a tp-v6 and earlier ism. > > DECtape had a directory at the front of the tape (think > superblock/ilist), but could do cool things and be treated more like a disk. > When tp was created for a very early version of Unix (I'm not sure > which, could be V2), Ken/Dennis et al had DECtape units and so the > original scheme followed the media.   This also meant that the program > could write files and go back and update the directory, which is a no-no > with many tape systems.  Then research got a 9-track unit.   So tp was > changed to calculate how much space was going to be needed, write the > directory, then the datablocks.  All good ... except... > > 9-track could write more files than the directory could take.   So for > many years, people would use the ar(1) program to take a number of files > in a directory, create a file called cont.a and then delete the files. > Then the tree would be written with tp, when you read it, you reversed > the ar(1) process.  If you look at the USENIX/Harvard tape on the TUHS > you'll see this scheme in use. > > BTW: tp was written in assembler and all the data structures were using > PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp) in C > (which is on the USENIX tape Warren has in the archives) that worked > like the original assembler tp but also put a redundant directory at the > end of the tape.  Another issue with tp was if the you had a bad block > in the first few blocks you could not decode the rest of the tape. > [There were some other issues with the UNIX tree structure as disks got > bigger but I'm going to ignore all that - other than to say, tp had > lived it life]. > > Enter Mashey and the PWB 1.0 folks (which is based on V6).  Someone in > USG created cpio (and volcpy) as part of the PWB 1.0.   Like tp it was a > PDP-11 binary format, but unlike tp, the tape directory is threaded. > /i.e./ block one describes the first file only and includes the size of > the following file, then file itself, then a new directory block for the > next file and again that file (rinse and repeat).  So it improved on tp > in the directory threading, but was still binary and for a reasons I'll > leave out had a different user interface. > > As part of V7, Ken wrote a new program, tar [you can ask him why].  But > like cpio he used a threaded tape directory, but unlike cpio it was > always ASCII and not PDP-11 specific.  Furthermore, the user interface > was made to parrot tp.  So, certainly, it had the advantage that > changing tp scripts to use tar was pretty easy i.e. s/tp/tar/     not so > for coil.  And it was muscle memory compliant. > > For PWB 2.0, cpio was updated to allow a -c option to write the header > in ascii and -s to byte swap the binary.   But the damage had been done ... > > Thus began 'tar wars' which was a battle that raged officially over tape > archive formats, but really was an argument about user interfaces. > Since tar was part of Research and the Universities and commercial > people used it, only USG and the folks inside the Bell System were using > cpio, as officially none of us had it since PWB was not released to us > (although thanks to many AT&T employees doing an OYOC year, many schools > like UCB, MIT and CMU all had the sources to cpio anyway -- for instance > you'll see it hidden away on Kirk's CD). > > I personally had used both, preferred tar for easy of use and ASCII > directories.  But, note I had written car at Masscomp, but never tpio. > This was our trick to use the file scripting list that cpio could do, > but create tar format tapes - which was handy.  I never wrote tpio which > would have been cpio format using tp/tar user interface as I did not > need it. > > Roll forward to the /usr/group UNIX standard that Heinz chaired.  We > ended up not being able to agree on a distribution format, but the ISVs > were PO because now they could create UNIX programs that might actually > work across systems, but they had not standard way to distribution. > Roll forward again to IEEE.  Heinz's committee was officially disbanded > (story discussed elsewhere) and we were created as IEEE P1003 with Jim > Issack as Chair. This time the ISV's said we had to have a distribution > format.  Since *.1 was only an API we were allowed to avoid the user > interface issue but only examine the on tape format. > > It turns out while it seems to have been unintended, Ken's original V7 > implementation has an interesting coding feature/bug which turns out to > be what clinched the deal.   When Ken creates the directory block for > each file, he did bcopy of 0's to the buffer before he wrote that data > that fills it in.  Then when he calculated the checksum for the > directory header block, he summed the entire block (which because of the > bcopy was zeros).  This means if you write beyond the end of Ken's > original header and include that extra data in the chksum, the original > program will ignore the new information but accept the directory block > as valid.  i.e. he had built an extension mechanism into the tar on-tape > format. > > cpio's ASCII on tape format was not able to do that as the checksum used > a sizeof(header struct) in the checksum routine. > > USTAR was born ... Ken had written things like the UID/GID as ASCII > representations of the integer value in the original header.  USTAR > added the ASCII representation of the username and the group name since > that was more often portable between systems than the numbers.   There > were other additions like more room for the pathname new file types > /etc/.  But the key is that a USTAR tape can be read by the original V7 > (and follow on) tape formats, although may not recognize all the > filetype or use all of the new information. > > A few years later during *.2 discussions, we finally got into the user > interface stuff and pax(1) was born.  Knowing my hack with car, Keith > Bostic, Jim McGuiness and I wrote up a description of a program that > could with both users interfaces scheme.  USENIX provided funding for a > student to implement it and put the sources out on comp.unix.sources at > some point.  That proposal was originally accepted at the first tape > user interface program in *.2 [a few years later after I stopped being > part of the committee, the USG folks did get an alternate CPIO format > accepted and cpio as an allowed program.   USENIX paid to have the > program updated to operate like cpio if it was called that, pure V7 tar > if called that and if pax, user USTAR]. > > 'nuf said ... I hope. > > Clem > > _._,_._,_ > ------------------------------------------------------------------------ > Groups.io Links: > > You receive all messages sent to this group. > > View/Reply Online (#62) | Reply To > Group > > | Reply To Sender > > | Mute This Topic | New Topic > > > Your Subscription | Contact > Group Owner | Unsubscribe > [bqt@softjar.se] > > _._,_._,_ -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol