The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] 'Huge' file support removed from PWB1
@ 2023-03-08 13:10 Noel Chiappa
  2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Noel Chiappa @ 2023-03-08 13:10 UTC (permalink / raw)
  To: tuhs

In PWB1, support for 'huge' files appears to have been removed. If one
compares bmap() in PWB1'S subr.c with V6's, the "'huge' fetch of double
indirect block" code is gone. I guess PWB didn't need very large (> 8*256*512
= 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
were, though - unless PWB was generating lots of files of between 7*256*512
and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
double-indirect block? (The savings in code space are derisory - unlike in
LSX/MINI-UNIX.) Anyone know?

		Noel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-08 13:10 [TUHS] 'Huge' file support removed from PWB1 Noel Chiappa
@ 2023-03-09  8:21 ` Kenneth Goodwin
  2023-03-09 12:40   ` Jonathan Gray
  2023-03-09 15:07   ` Ron Natalie
  2023-03-09 17:01 ` Tom Ivar Helbekkmo via TUHS
  2023-03-09 23:25 ` Clem Cole
  2 siblings, 2 replies; 8+ messages in thread
From: Kenneth Goodwin @ 2023-03-09  8:21 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

I have not seen the UNIX kernel source code in quite a while, but as I
recall the double indirect block algorithm did not kick in until the file
exceeded a certain threshold. So it would not make sense to remove the code
for performance reasons.

Perhaps this is more likely due to the use of larger logical block sizes....

Is the code physically removed or IFDEF'd out for conditional compilation?

Perhaps someone decided that programmers would never need to test code on
large files..

On Wed, Mar 8, 2023, 8:10 AM Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

> In PWB1, support for 'huge' files appears to have been removed. If one
> compares bmap() in PWB1'S subr.c with V6's, the "'huge' fetch of double
> indirect block" code is gone. I guess PWB didn't need very large (>
> 8*256*512
> = 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
> were, though - unless PWB was generating lots of files of between 7*256*512
> and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
> double-indirect block? (The savings in code space are derisory - unlike in
> LSX/MINI-UNIX.) Anyone know?
>
>                 Noel
>

[-- Attachment #2: Type: text/html, Size: 1649 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
@ 2023-03-09 12:40   ` Jonathan Gray
  2023-03-09 15:07   ` Ron Natalie
  1 sibling, 0 replies; 8+ messages in thread
From: Jonathan Gray @ 2023-03-09 12:40 UTC (permalink / raw)
  To: Kenneth Goodwin; +Cc: Noel Chiappa, tuhs

removed in PWB/UNIX 1.0.

ifdef'd out in the Harvard/Radcliffe Student Time-sharing System (HRSTS)
parts found in tuhs/Applications/Usenix_77. h/distrib.note includes:

'4.	"Huge" files are not supported by our modifications.  This is not
necessarily a hard restriction, however we early on decided
we wanted no more than one level of indirection all the way up to 1 megabyte,
and had no need for larger files.  The incompatibilities may be minimal,
but we have not even bothered to seek them out.'

On Thu, Mar 09, 2023 at 03:21:19AM -0500, Kenneth Goodwin wrote:
> I have not seen the UNIX kernel source code in quite a while, but as I
> recall the double indirect block algorithm did not kick in until the file
> exceeded a certain threshold. So it would not make sense to remove the code
> for performance reasons.
> 
> Perhaps this is more likely due to the use of larger logical block sizes....
> 
> Is the code physically removed or IFDEF'd out for conditional compilation?
> 
> Perhaps someone decided that programmers would never need to test code on
> large files..
> 
> On Wed, Mar 8, 2023, 8:10 AM Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
> > In PWB1, support for 'huge' files appears to have been removed. If one
> > compares bmap() in PWB1'S subr.c with V6's, the "'huge' fetch of double
> > indirect block" code is gone. I guess PWB didn't need very large (>
> > 8*256*512
> > = 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
> > were, though - unless PWB was generating lots of files of between 7*256*512
> > and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
> > double-indirect block? (The savings in code space are derisory - unlike in
> > LSX/MINI-UNIX.) Anyone know?
> >
> >                 Noel
> >

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
  2023-03-09 12:40   ` Jonathan Gray
@ 2023-03-09 15:07   ` Ron Natalie
  2023-03-09 15:13     ` Ron Natalie
  2023-03-09 17:35     ` arnold
  1 sibling, 2 replies; 8+ messages in thread
From: Ron Natalie @ 2023-03-09 15:07 UTC (permalink / raw)
  To: Kenneth Goodwin, Noel Chiappa; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

In the version 6 file system, If the file was less than 4K, the four 
blocks in the inode refer directly to the disk blocks containing the 
file data.
After thef ile grows past that, the large bit is set in the inode and 
these blocks contain the single indirect blocks, referencing 256 data 
blocks each,  until you used up seven of these.
The last one was the double indirect block that pointed to other 
indirect blocks.   The file size was stored in 24 bits, and the block 
numbers were 16 bit.

As Noel points out, PWB uses the same file system as V6 but with the 
HUGE (last double indirect) code commented out.  I have no idea why and 
I never noticed it before.   While we used a lot of stuff from the PWB 
distribution (notably SCCS) we had no real interest in using the PWB’s 
kernel because our V6 kernel had diverged sharply from the Bell 
versions.

Version 7 came around with a different filesystem layout.    File sizes 
were 32 bits, and the disk block indexes were 24 bits.   The latter was 
stored as a series of bytes on the disk but expanded into longs in 
memeory.
There was no large bit anymore.   The first 8 blocks addresses in the 
inode were always single indirect, the next two were single indirect, 
and the last one was double indirect.   (If my memory serves me right)

At BRL, we modified our very-hacked V6 kernel to have the ability to 
mount either V6 or V7 filesystems.

We never ran V5, but I think V5 is essentially the V6 file system 
without the HUGE support again.

-Ron

[-- Attachment #2: Type: text/html, Size: 2497 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-09 15:07   ` Ron Natalie
@ 2023-03-09 15:13     ` Ron Natalie
  2023-03-09 17:35     ` arnold
  1 sibling, 0 replies; 8+ messages in thread
From: Ron Natalie @ 2023-03-09 15:13 UTC (permalink / raw)
  To: Ron Natalie, Kenneth Goodwin, Noel Chiappa; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

That should read eight blocks in the inode not four in the first 
sentence (and up to eight later).



------ Original Message ------
From "Ron Natalie" <ron@ronnatalie.com>
To "Kenneth Goodwin" <kennethgoodwin56@gmail.com>; "Noel Chiappa" 
<jnc@mercury.lcs.mit.edu>
Cc "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
Date 3/9/2023 10:07:17 AM
Subject [TUHS] Re: 'Huge' file support removed from PWB1

>In the version 6 file system, If the file was less than 4K, the four 
>blocks in the inode refer directly to the disk blocks containing the 
>file data.
>After thef ile grows past that, the large bit is set in the inode and 
>these blocks contain the single indirect blocks, referencing 256 data 
>blocks each,  until you used up seven of these.
>The last one was the double indirect block that pointed to other 
>indirect blocks.   The file size was stored in 24 bits, and the block 
>numbers were 16 bit.
>
>As Noel points out, PWB uses the same file system as V6 but with the 
>HUGE (last double indirect) code commented out.  I have no idea why and 
>I never noticed it before.   While we used a lot of stuff from the PWB 
>distribution (notably SCCS) we had no real interest in using the PWB’s 
>kernel because our V6 kernel had diverged sharply from the Bell 
>versions.
>
>Version 7 came around with a different filesystem layout.    File sizes 
>were 32 bits, and the disk block indexes were 24 bits.   The latter was 
>stored as a series of bytes on the disk but expanded into longs in 
>memeory.
>There was no large bit anymore.   The first 8 blocks addresses in the 
>inode were always single indirect, the next two were single indirect, 
>and the last one was double indirect.   (If my memory serves me right)
>
>At BRL, we modified our very-hacked V6 kernel to have the ability to 
>mount either V6 or V7 filesystems.
>
>We never ran V5, but I think V5 is essentially the V6 file system 
>without the HUGE support again.
>
>-Ron

[-- Attachment #2: Type: text/html, Size: 3853 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-08 13:10 [TUHS] 'Huge' file support removed from PWB1 Noel Chiappa
  2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
@ 2023-03-09 17:01 ` Tom Ivar Helbekkmo via TUHS
  2023-03-09 23:25 ` Clem Cole
  2 siblings, 0 replies; 8+ messages in thread
From: Tom Ivar Helbekkmo via TUHS @ 2023-03-09 17:01 UTC (permalink / raw)
  To: TUHS

Noel Chiappa <jnc@mercury.lcs.mit.edu> writes:

> In PWB1, support for 'huge' files appears to have been removed.

They did make various changes to the system to protect against resource
exhaustion in what was a genuine production multi-user facility, and
might possibly have made such a change to ensure that a runaway process
couldn't write a file more than a megabyte in size.

It's not mentioned in the PWB article in the special UNIX issue of the
Bell System Technical Journal, though, and that article does describe
various changes they'd made, explaining how and why.

However: PWB grew out of Research, side by side with it, evaluating and
adopting changes along the way.  They aimed to stay mostly in synch,
but chose to keep and maintain a number of differences.  Maybe they
started their work before the "huge" file support was introduced, and
then simply did not include that change?  It's a backward incompatible
change (at least if you have at least one file that already uses the
last indirect block of its inode), and they might have decided to cross
that bridge if and when they ever got to it.

-tih
-- 
Most people who graduate with CS degrees don't understand the significance
of Lisp.  Lisp is the most important idea in computer science.  --Alan Kay

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-09 15:07   ` Ron Natalie
  2023-03-09 15:13     ` Ron Natalie
@ 2023-03-09 17:35     ` arnold
  1 sibling, 0 replies; 8+ messages in thread
From: arnold @ 2023-03-09 17:35 UTC (permalink / raw)
  To: ron, kennethgoodwin56, jnc; +Cc: tuhs

"Ron Natalie" <ron@ronnatalie.com> wrote:

> Version 7 came around with a different filesystem layout.    File sizes 
> were 32 bits, and the disk block indexes were 24 bits.   The latter was 
> stored as a series of bytes on the disk but expanded into longs in 
> memeory.
> There was no large bit anymore.   The first 8 blocks addresses in the 
> inode were always single indirect, the next two were single indirect, 
> and the last one was double indirect.   (If my memory serves me right)

Wasn't there a triple-indirect block as the very last one?

Arnold

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [TUHS] Re: 'Huge' file support removed from PWB1
  2023-03-08 13:10 [TUHS] 'Huge' file support removed from PWB1 Noel Chiappa
  2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
  2023-03-09 17:01 ` Tom Ivar Helbekkmo via TUHS
@ 2023-03-09 23:25 ` Clem Cole
  2 siblings, 0 replies; 8+ messages in thread
From: Clem Cole @ 2023-03-09 23:25 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]

I talked with John Mashey this PM and asked him about the huge file stuff
and PWB1.

First, he did not remember that they had done that, but ...

He then emphasized that the PWB team was running the first (and for a long
time the largest) "computer center" using UNIX in the Bell System in
Piscataway - supporting over 1000 programmers as a front end to the
mainframes. His memory is that they mostly used 88 Mbyte RP04s [remember,
at the time, RK05s were only 2.5M].  Needing to support a megabyte or
larger file would have been relatively rare for them.  He also pointed out
they needed to run their system on systems as small as 48K byte 11/40s to
as large as 1M 11/70s.

As he said, his team tried to track Dennis and Ken's Research
system closely since the PWB team was not primarily in the system
development/enhancement business -* i.e.* they tied to keep what would
become PWB1 and whatever was running at Research as similar as they could
[he told some stories about running to MH to get Dennis' latest compiler
binary because the incremental development scheme dmr used would make
someone like PWB that took snapshots at different times, end up with a
compiler that could not compile itself --#1#]. But the PWB group >> was <<
interested in making the system they were running as reliable as possible
and, more importantly, being as graceful as possible when different
resources were exhausted.

He said it would have made sense for them to remove the "huge file" support
to help to defend against running out of space and the system having issues
[V6 was extremely ungrateful about that condition]. So limiting
file size became a simple quota, if you will. A runaway program was less
likely to take the system out. So the process would die because the file
got too large (or, to use the IBM shop term in those days -- ABEND).


#1# He also regaled a bit about Ken's infamous Turning Award hack and
discovering symbols in the compiler binary they could not understand ;-)

[-- Attachment #2: Type: text/html, Size: 3050 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2023-03-09 23:26 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-08 13:10 [TUHS] 'Huge' file support removed from PWB1 Noel Chiappa
2023-03-09  8:21 ` [TUHS] " Kenneth Goodwin
2023-03-09 12:40   ` Jonathan Gray
2023-03-09 15:07   ` Ron Natalie
2023-03-09 15:13     ` Ron Natalie
2023-03-09 17:35     ` arnold
2023-03-09 17:01 ` Tom Ivar Helbekkmo via TUHS
2023-03-09 23:25 ` Clem Cole

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).