From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 13374 invoked from network); 28 Jan 2023 02:51:24 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 28 Jan 2023 02:51:24 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 417FA42562; Sat, 28 Jan 2023 12:51:00 +1000 (AEST) Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com [209.85.160.43]) by minnie.tuhs.org (Postfix) with ESMTPS id 9A3594255B for ; Sat, 28 Jan 2023 12:50:55 +1000 (AEST) Received: by mail-oa1-f43.google.com with SMTP id 586e51a60fabf-16332831ed0so8888341fac.10 for ; Fri, 27 Jan 2023 18:50:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=uA14ixhOShZGNlHDl4rsfjruARLXJxlJMGjuIhsbXSA=; b=U48JNsBDqma8mj19vlzTY56fmuboLYXMvAIEM4TTMXjEswdjV7S9Zyag0b8sQJ0gIB 2MyR3EyMLD7Fa2CBcrJSEUA5XIdK8D8B+uOAi96jElz5aBpyfev4XEectgGiE7wR+tJ0 44DFp3pMRe/vyyYKv7DZXrtbwIIvKIlIU09U+a0T++8+tuEJWeOs0e8RNtwemYyPG865 oxQPYB/BdBFS8fNTzNO+Fo5qazuWKzNbT9BWoSaxCp6vwRy3IauJU61eTWV6QFuaNHva 5cRktTXrQExPU6opX+f92k1l8PlRhM+F49DYLKRsHtogSXiQo14EMe1oTOpc01vi/AjB gFLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uA14ixhOShZGNlHDl4rsfjruARLXJxlJMGjuIhsbXSA=; b=NeqS5YD8OCh8kNOJt3EhcAoGl7mORCCucX2Eg+msFNpdH6lfFT7aYB+KQEPHreQiUD Wz7/1GX+hMvjWvBxLFFaGJMroYt1WZ3dw2NKGJzjLktgJw3kbNc53zkwzal+fkWcIJcE pfAl8XtqdbtCVp3vVNMhvEd8U20oYPSDOHmvNMHDQVDMCpaiL8p3sR+JN/0Vw7ffAPLa 7AUCWf/mqHRJdv4kNi5fslEm5bv05nk2qdQAU249fM2DQ/3GMEsfY3PhNbpMVwGPqmgq FXu7x+gf+RTFm5nA7BExFrd0ITq0i3kkvSjwyHLn52N0HpAHFZhi/7tWL1qMVRcjTj/H QO/A== X-Gm-Message-State: AO0yUKUwF76eT08x8GnPSPBGqbj0NtqiS15uyFqufqdIBdb76ICy/RYE IynFVCvvJpu1cbSgNDoAzeR/PH74HTBA91A/gdehhSxb/IM= X-Google-Smtp-Source: AK7set/C2uqxTrk2dOmRAi8GUqKdU4P4P4ftBqHh9yCLg6l4M31I+OsIVrCJ8jhEeDVugSv+mt1Mwm6av7XorLDi2xE= X-Received: by 2002:a05:6870:2216:b0:163:79c9:d1f0 with SMTP id i22-20020a056870221600b0016379c9d1f0mr243341oaf.75.1674874194755; Fri, 27 Jan 2023 18:49:54 -0800 (PST) MIME-Version: 1.0 References: <20230125203805.4762218C083@mercury.lcs.mit.edu> <7w8rhpdczd.fsf@junk.nocrew.org> <20230126105626.72CD922168@orac.inputplus.co.uk> <20230127135651.A70482135B@orac.inputplus.co.uk> <20230128021841.GG15592@mcvoy.com> In-Reply-To: <20230128021841.GG15592@mcvoy.com> From: Tom Perrine Date: Fri, 27 Jan 2023 18:49:43 -0800 Message-ID: To: Larry McVoy Content-Type: multipart/alternative; boundary="0000000000006c04fe05f34a0953" Message-ID-Hash: 36LIH2ZHP77AK6OYNBJPOJC5FAJCCHCH X-Message-ID-Hash: 36LIH2ZHP77AK6OYNBJPOJC5FAJCCHCH X-MailFrom: tom.perrine@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tuhs.tuhs.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: tuhs@tuhs.org X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: Setting up an X Development Environment for Mac OS List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --0000000000006c04fe05f34a0953 Content-Type: text/plain; charset="UTF-8" At that time (up to 2003), every time we upgraded the tape density, or added new archival storage, there was a "packing" job in the background. Every time a file was retrieved and used, if it wasn't on the most dense (newer) tape format, then the file was re-saved onto the newest, higher-density tape. This meant that active files were constantly copied forward onto the newer tape formats. As all the files were copied off the lower density tapes, the older tapes were marked "retired" and removed from the tape silos. Then, in the background, idle tape drive time was used by a background job that retrieved the oldest tapes, and copied the files on that tape forward. This was enable by tape-level metadata that included the batch number and the date that every physical tape was put into service. Now.... Later, at Playstation we investigated the Sony ODA. If we had needed the deep archive, we would have definitely gone with the Sony Petascale archive. This is optical, but a disc technology that is denser and a different formulation than Blu-Ray. https://pro.sony/ue_US/products/optical-disc/petasite-solutions On Fri, Jan 27, 2023 at 6:18 PM Larry McVoy wrote: > Actually I like this fork. I'm curious, do you know what is best practice > for keeping bits around these days? > > On Fri, Jan 27, 2023 at 01:42:17PM -0800, Tom Perrine wrote: > > A tiny bit of a fork, but... > > > > When I was at SDSC.EDU we did a project for the National Archives. Gotta > > love an agency that's mission is "data for the lifetime of the > Republic"... > > > > They wanted to be sure that they could still access data at least 100 > years > > later, even assuming that no one had accessed it in that 100 year period. > > > > Anyway, we looked at all the options at the time (very early 2000s). > > > > While media lifetime was indeed understood to be critical, we > specifically > > called out needing to retain the software and the encryption keys. AND > the > > encryption algorithms! > > At that time, media encryption was still quite new, and they hadn't > > considered that issue. At all. > > > > Overall, the best, most practical approach (at that time) was to > > periodically copy the data forward, into new media, into new > > storage software, and decrypting with the old keys and algos, and > > re-encrypting with new. > > > > Only by doing this periodically, we argued, could they really be sure of > > being able to recover data 100+ years from now. > > > > Don't get me started on the degradation of early generation optical media > > that was guaranteed for 50 years, but rusted internally within 2 years. > > > > And of course now there are companies that specialize in providing > > mothballed obsolete tape and other readers. > > > > --tep > > > > > > > > On Fri, Jan 27, 2023 at 6:55 AM Ron Natalie wrote: > > > > > When I worked in the intelligience industry, the government spent a lot > > > of money tasking someone (I think it was Kodak) to determine the best > > > media for archival storage. It included traditional 6250 9 track > > > tapes and the then-popular exabyte 8mm (which was atrociously short > > > lived). I pointed out that magnetic storage was probably always going > > > to be problematic and things needed ???digital refresh??? if you really > > > wanted to keep them. > > > > > > > > > If you know the tape may be problematic when played back, there are > > > things you can do. I was gifted the master tapes of one of the radio > > > shows originated at WJHU in the 70???s. I had them sent out to a > company > > > who ???baked??? them, but then they also had to redo all the splices > on them > > > when they were played back. > > > > > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > --0000000000006c04fe05f34a0953 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
At that time (up to 2003), every time we upgraded the tape= density, or added new archival storage, there was a "packing" jo= b in the background.

Every time a file was retrieved=C2= =A0and used, if it wasn't on the most dense (newer) tape format, then t= he file was re-saved onto the newest, higher-density=C2=A0tape.
<= br>
This meant that active files were constantly copied forward o= nto the newer tape formats. As all the files were copied off the lower dens= ity tapes, the older tapes were marked "retired" and removed from= the tape silos.

Then, in the background, idle tap= e drive=C2=A0time was used by a background job that retrieved the oldest ta= pes, and copied the files on that tape forward. This was enable by tape-lev= el metadata that included=C2=A0the batch number and the date that every phy= sical tape was put into service.

Now....

Later, at Playstation we investigated the Sony ODA. If we h= ad needed the deep archive, we would have definitely gone with the Sony Pet= ascale archive. This is optical, but a disc technology that is denser and a= different formulation than Blu-Ray.


<= /div>


On Fri, Jan 27, 2023 at 6:18 PM Larry McVoy <lm@mcvoy.com> wrote:
Actually I like this fork.=C2=A0 I'= ;m curious, do you know what is best practice
for keeping bits around these days?

On Fri, Jan 27, 2023 at 01:42:17PM -0800, Tom Perrine wrote:
> A tiny bit of a fork, but...
>
> When I was at SDSC.EDU we did a project for the National Archives. Gotta
> love an agency that's mission is "data for the lifetime of th= e Republic"...
>
> They wanted to be sure that they could still access data at least 100 = years
> later, even assuming that no one had accessed it in that 100 year peri= od.
>
> Anyway, we looked at all the options at the time (very early 2000s). >
> While media lifetime was indeed understood to be critical, we specific= ally
> called out needing to retain the software and the encryption keys. AND= the
> encryption algorithms!
> At that time, media encryption was still quite new, and they hadn'= t
> considered that issue. At all.
>
> Overall, the best, most practical approach (at that time) was to
> periodically copy the data forward, into new media, into new
> storage software, and decrypting with the old keys and algos, and
> re-encrypting with new.
>
> Only by doing this periodically, we argued, could they really be sure = of
> being able to recover data 100+ years from now.
>
> Don't get me started on the degradation of early generation optica= l media
> that was guaranteed for 50 years, but rusted internally within 2 years= .
>
> And of course now there are companies that specialize in providing
> mothballed obsolete tape and other readers.
>
> --tep
>
>
>
> On Fri, Jan 27, 2023 at 6:55 AM Ron Natalie <ron@ronnatalie.com> wrote:
>
> > When I worked in the intelligience industry, the government spent= a lot
> > of money tasking someone (I think it was Kodak) to determine the = best
> > media for archival storage.=C2=A0 =C2=A0 It included traditional = 6250 9 track
> > tapes and the then-popular exabyte 8mm (which was atrociously sho= rt
> > lived).=C2=A0 =C2=A0I pointed out that magnetic storage was proba= bly always going
> > to be problematic and things needed ???digital refresh??? if you = really
> > wanted to keep them.
> >
> >
> > If you know the tape may be problematic when played back, there a= re
> > things you can do.=C2=A0 =C2=A0 I was gifted the master tapes of = one of the radio
> > shows originated at WJHU in the 70???s.=C2=A0 =C2=A0I had them se= nt out to a company
> > who ???baked??? them, but then they also had to redo all the spli= ces on them
> > when they were played back.
> >

--
---
Larry McVoy=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Retired to fishing=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 http://www.mcvoy.com/lm/boat
--0000000000006c04fe05f34a0953--