[TUHS] Extracting files from various old dump/restore tapes

[TUHS] Extracting files from various old dump/restore tapes

[Lars Brinkhoff]

Hello,

I have on my hands many images of tapes that seems to have been written
by various implementaions of dump.  I see the magic numbers 60011 and
60012 in little and big endian at offsets 18 (16-bit version?) and 24
(32-bit version?).  I don't know the dating of the tapes, but around
1980 would be a reasonable guess.

Are there some easy to use (ready to run on a modern Unix) tools to
extract files from such tape files?

I'm not looking to restore a file system on disk, just extract the
files.

[TUHS] Re: Extracting files from various old dump/restore tapes

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

FreeBSD deleted some compat code in the kernel some time ago, and it turns
out that restore used that to read old dump tapes, so we broke old dump
tapes.
So you can't use FreeBSD's unmodified.

So 60011 is OFS_MAGIC and 60012 is NFS_MAGIC. Both of these are variants
on UFS, but really old. And given they are at different offsets, you'll
likely need
to reverse engineer the offsets used for the platform's dinode.

dump from 4.4BSD had 'natural' types (so long was used which would affect
the
offsets). It seems to be able to read both NFS_MAGIC and OFS_MAGIC tapes,
but you'll likely have to hack together u_spcl that's specific to the
platform which
may take a little trial and error if you don't know what the typedefs for
daddr_t
etc are. There's also a 4.1 dump/restore you might need (available in the
4.4BSD
Alpha archive and likely all earlier versions in TUHS).

Without more specific data it's hard to know if there's an extant binary
that
can be run in emulation to read these tapes.

Warner

On Mon, Aug 22, 2022 at 9:35 AM Lars Brinkhoff <lars@nocrew.org> wrote:

> Hello,
>
> I have on my hands many images of tapes that seems to have been written
> by various implementaions of dump.  I see the magic numbers 60011 and
> 60012 in little and big endian at offsets 18 (16-bit version?) and 24
> (32-bit version?).  I don't know the dating of the tapes, but around
> 1980 would be a reasonable guess.
>
> Are there some easy to use (ready to run on a modern Unix) tools to
> extract files from such tape files?
>
> I'm not looking to restore a file system on disk, just extract the
> files.
>

[-- Attachment #2: Type: text/html, Size: 2175 bytes --]

[TUHS] Re: Extracting files from various old dump/restore tapes

[Bakul Shah]

On Aug 22, 2022, at 9:27 AM, Warner Losh <imp@bsdimp.com> wrote:
> 
> FreeBSD deleted some compat code in the kernel some time ago, and it turns
> out that restore used that to read old dump tapes, so we broke old dump tapes.
> So you can't use FreeBSD's unmodified.

May be run an older release that has this code?

[TUHS] Re: Extracting files from various old dump/restore tapes

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Mon, Aug 22, 2022 at 10:53 AM Bakul Shah <bakul@iitbombay.org> wrote:

> On Aug 22, 2022, at 9:27 AM, Warner Losh <imp@bsdimp.com> wrote:
> >
> > FreeBSD deleted some compat code in the kernel some time ago, and it
> turns
> > out that restore used that to read old dump tapes, so we broke old dump
> tapes.
> > So you can't use FreeBSD's unmodified.
>
> May be run an older release that has this code?
>

That only works if the dinode structures line up exactly to whatever these
tapes
are from. The 16-bit ones likely need to use the V7 restore (maybe running
in
emulation in simh), the 32-bit ones might be able to use V32 or 4BSD
running in
emulation, if that's what the tapes are from. But the multi-endian
suggests  that
maybe things are more complex than that.

FreeBSD's restore from older releases might work, but only for the 32-bit
stuff and
only if we're lucky...

Warner

[-- Attachment #2: Type: text/html, Size: 1421 bytes --]

[TUHS] Re: Extracting files from various old dump/restore tapes

[Lars Brinkhoff]

Warner Losh wrote:
> So 60011 is OFS_MAGIC and 60012 is NFS_MAGIC. Both of these are
> variants on UFS, but really old. And given they are at different
> offsets, you'll likely need to reverse engineer the offsets used for
> the platform's dinode.

So anyway, it seems my best bet would be getting an old "restore" and
hack it till it runs.

Part of the problem is that there are hundreds of these images, so it
would be a lot of work to examine them individually in emulated systems.
A good first start to examine the content would be to just list the file
names.

> Without more specific data it's hard to know if there's an extant
> binary that can be run in emulation to read these tapes.

The tapes are from MIT's "Tapes of Tech Square" collection.  Likely
candidates include PDP-11 V7, 4.x BSD on VAX, and Sun workstations.  I
suppose the latter would use the big endian format.

There are also many variations of the tar and cpio formats, but I'm on
firmer ground there.

[TUHS] Re: Extracting files from various old dump/restore tapes

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1742 bytes --]

On Mon, Aug 22, 2022 at 11:55 PM Lars Brinkhoff <lars@nocrew.org> wrote:

> Warner Losh wrote:
> > So 60011 is OFS_MAGIC and 60012 is NFS_MAGIC. Both of these are
> > variants on UFS, but really old. And given they are at different
> > offsets, you'll likely need to reverse engineer the offsets used for
> > the platform's dinode.
>
> So anyway, it seems my best bet would be getting an old "restore" and
> hack it till it runs.
>
> Part of the problem is that there are hundreds of these images, so it
> would be a lot of work to examine them individually in emulated systems.
> A good first start to examine the content would be to just list the file
> names.
>

For V7 tapes, you can run the V7 binaries using apout with very little
effort.
This is a user-level emulation of a pdp-11 with the system calls for v5,
v6, v7 and
some of the BSDs. It's in the tuhs archives under
Distributions/Research/Dennis_v1/unix72/tools/apout.
I used it to extract files from V7 automatically that I used in the 2.11
back-patching-to-the-original-tapes
script I wrote.


> > Without more specific data it's hard to know if there's an extant
> > binary that can be run in emulation to read these tapes.
>
> The tapes are from MIT's "Tapes of Tech Square" collection.  Likely
> candidates include PDP-11 V7, 4.x BSD on VAX, and Sun workstations.  I
> suppose the latter would use the big endian format.
>

Very cool. I suspect if you want one tool for them all, you'll need to take
restore
and teach it to cope with multiple endians and word sizes... It's likely
not a huge
effort, but the restor code from V7 makes use of a lot of type-punning...


> There are also many variations of the tar and cpio formats, but I'm on
> firmer ground there.
>

Warner

[-- Attachment #2: Type: text/html, Size: 2602 bytes --]

[TUHS] Re: Extracting files from various old dump/restore tapes

[Lars Brinkhoff]

In the end I decided to roll my own rather than port an old version of
restore forward and possibly merge in a few other versions.  I
documented what I found out about the dump/restore format here:

https://gunkies.org/wiki/Unix_dump/restore_tape_format

Corrections welcome.

[TUHS] Re: Extracting files from various old dump/restore tapes

[Lars Brinkhoff]

Lars Brinkhoff wrote:
> In the end I decided to roll my own rather than port an old version of
> restore forward and possibly merge in a few other versions.

Now available from here:

https://github.com/larsbrinkhoff/tools-for-unusual-tape-formats

[TUHS] Re: Extracting files from various old dump/restore tapes

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

On Wed, 2022-09-14 17:48:56 +0000, Lars Brinkhoff <lars@nocrew.org> wrote:
> Lars Brinkhoff wrote:
> > In the end I decided to roll my own rather than port an old version of
> > restore forward and possibly merge in a few other versions.
> 
> Now available from here:
> 
> https://github.com/larsbrinkhoff/tools-for-unusual-tape-formats

Here's some stuff to build it, at least on my side.

  * -Ilibword does not exist
  * Seems `classify-tape.c` got renamed to `classify.c`
  * read_32bits_l() is unused and produces a warning

Thanks! I'll include that into my autobuilder.

MfG, JBG



diff --git a/Makefile b/Makefile
index fca9d3c..ae61913 100644
--- a/Makefile
+++ b/Makefile
@@ -1,5 +1,5 @@
 
-CFLAGS = -g -W -Wall -Ilibword
+CFLAGS = -g -W -Wall
 
 TOOLS =	cpio classify restore
 
@@ -8,7 +8,7 @@ all: $(TOOLS)
 clean:
 	rm -f $(TOOLS) *.o
 
-classify: classify-tape.o tape-image.o
+classify: classify.o tape-image.o
 	$(CC) $(CFLAGS) $^ -o $@
 
 cpio: cpio.o mkdirs.o
diff --git a/classify.c b/classify.c
index 8f81213..2843164 100644
--- a/classify.c
+++ b/classify.c
@@ -43,12 +43,6 @@ read_16bits_l (uint8_t *start)
   return start[0] | (start[1] << 8);
 }
 
-static uint32_t
-read_32bits_l (uint8_t *start)
-{
-  return read_16bits_l (start) | (read_16bits_l (start + 2) << 16);
-}
-
 static word_t
 read_36bits (uint8_t *x)
 {

-- 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[TUHS] Re: Extracting files from various old dump/restore tapes

[Lars Brinkhoff]

Jan-Benedict Glaw wrote:
> Here's some stuff to build it, at least on my side.
>   * -Ilibword does not exist
>   * Seems `classify-tape.c` got renamed to `classify.c`
>   * read_32bits_l() is unused and produces a warning

Thanks!  I fumbled some last-minute changes.  And frankly, I didn't
expect anyone to pick it up so quickly!