The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] PDP-7 UNIX filesystem
@ 2019-10-21  6:22 Angelo Papenhoff
  2019-10-21 10:43 ` Abhinav Rajagopalan
                   ` (2 more replies)
  0 siblings, 3 replies; 24+ messages in thread
From: Angelo Papenhoff @ 2019-10-21  6:22 UTC (permalink / raw)
  To: tuhs

DMR explained how PDP-7 UNIX was used in "The Evolution of the Unix
Time-sharing System" but having played with it myself, I stumbled in a
couple of cases and found it a bit awkward to use.
Maybe someone (ken and doug?) can shed some light on "elaborate set of
conventions" that dmr mentioned.

My questions are these:


you cannot execute a program if you're in a directory you can't write into.

	I asked Warren about this when I first tried pdp7 unix and he
explained it to me: the shell creates a link to the binary and executes
it. If it can't write into the current directory, it fails to create the
link and hence can't execute the program.
	How was this handled in practice? did users have write
permissions on all directories? did you just stay in your directory all
the time?


. and ..

	Was this introduced first with PDP-11 unix or did the convention
start on the PDP-7 already? It certainly seems to be the case with .
but how about ..? the dd directory seems to take on the role of a sort
of root directory and the now discovered program pd actually creates a
file .. (haven't tried to understand what it does though yet)
What does dd stand for, dotdot? directory directory?


aap

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21  6:22 [TUHS] PDP-7 UNIX filesystem Angelo Papenhoff
@ 2019-10-21 10:43 ` Abhinav Rajagopalan
  2019-10-21 11:38   ` Abhinav Rajagopalan
                     ` (2 more replies)
  2019-10-21 16:54 ` Angelo Papenhoff
  2019-10-23  5:48 ` Lars Brinkhoff
  2 siblings, 3 replies; 24+ messages in thread
From: Abhinav Rajagopalan @ 2019-10-21 10:43 UTC (permalink / raw)
  To: Angelo Papenhoff, tuhs

[-- Attachment #1: Type: text/plain, Size: 3008 bytes --]

>
> . and ..
>
>         Was this introduced first with PDP-11 unix or did the convention
> start on the PDP-7 already? It certainly seems to be the case with .
> but how about ..? the dd directory seems to take on the role of a sort
> of root directory and the now discovered program pd actually creates a
> file .. (haven't tried to understand what it does though yet)
> What does dd stand for, dotdot? directory directory?
>
>
>
. and .. are quite intuitive the more I think about it, they're essentially
acting as symbolic representations of the 'current' and 'parent' dirs as
in, when we look at it from / :

                   /
       |--|--|--|--|--|--|--|--|
     /etc /bin /root /home /mnt /dev /usr
       |--|--|--|--|--|--|--|--|
     ../ ../ ../ ../ ../ ../ ../ ../

As we see, each of the subdirectories hold a / prefix to them as we're used
to seeing, and this translated to the subfolders having their ../
descriptor denoting their parent (I think of them as recursive pointers to
parent dirs), where the parent holds ../ and child dir has a name like /..
and invoking

I don't know if it wasn't until the mkdir() syscall was added this approach
took form. There was an earlier 'Aha, Unix!' thread where Ken had mentioned
a similar thing, where invoking pwd gave only the /usr/name directory even
if the directory had been changed with chdir, essentially stemming out of
only the forked child changing dirs, when the cd wasn't built into the
shell (this was in the PDP-11 when he was writing the initial set utilities
like cd, ls)

Also, depending on the implementation of 'ls' we were to try this out
today, these are bound to have their differences. The original syscalls on
the PDP-7 had chdir,

swp:			" system call dispatch table
   jmp .		" base instruction
   .save; .getuid; .open; .read; .write; .creat; .seek; .tell
   .close; .link; .unlink; .setuid; .rename; .exit; .time; .intrp
   .chdir; .chmod; .chown; badcal; .sysloc; badcal; .capt; .rele
   .status; badcal; .smes; .rmes; .fork


mkdir was probably in the works at this stage.

We should also touch upon the "cd -" as this one takes you back like a back
button into the directory you were last inside, whereas the cd ../.. takes
you up or down the hierarchy depending on the placement of the / and adding
placeholders i.e directory names in place of the .. (dots).

Of course, only the creators can embark upon the design details of the
hierarchical system and the reasons behind all the above and more.

Since you mentioned the word dd, the utility which serves me like none
other for my frequent rewrites of images, it too has an interesting past,
it seems like 'dd' was non-unixy in it's design approach, if I'm to believe
the lores around, which leads me to think that this might have been another
one of the many idiosyncratic naming conventions used back then. More on
the dd stuff: http://www.catb.org/jargon/html/D/dd.html

Personally, I just like to think of it in my head as disk-disk.

-- 

Abhinav Rajagopalan

[-- Attachment #2: Type: text/html, Size: 6816 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 10:43 ` Abhinav Rajagopalan
@ 2019-10-21 11:38   ` Abhinav Rajagopalan
  2019-10-21 12:00     ` Thomas Paulsen
  2019-10-22 18:07   ` Peter Jeremy
  2019-10-24  2:29   ` Christopher Browne
  2 siblings, 1 reply; 24+ messages in thread
From: Abhinav Rajagopalan @ 2019-10-21 11:38 UTC (permalink / raw)
  To: Angelo Papenhoff, tuhs

[-- Attachment #1: Type: text/plain, Size: 3531 bytes --]

Also, I only now realized that only mknod existed, up until a long time,
only later on with the GNU coreutils did mkdir as a command come into
existence. Running the PDP-11 v7 on SIMH showed that, gotten so accustomed
to the Linux env that thinking backwards seemed suddenly arcane.

On Mon, Oct 21, 2019 at 4:13 PM Abhinav Rajagopalan <
abhinavrajagopalan@gmail.com> wrote:

>
>
>> . and ..
>>
>>         Was this introduced first with PDP-11 unix or did the convention
>> start on the PDP-7 already? It certainly seems to be the case with .
>> but how about ..? the dd directory seems to take on the role of a sort
>> of root directory and the now discovered program pd actually creates a
>> file .. (haven't tried to understand what it does though yet)
>> What does dd stand for, dotdot? directory directory?
>>
>>
>>
> . and .. are quite intuitive the more I think about it, they're
> essentially acting as symbolic representations of the 'current' and
> 'parent' dirs as in, when we look at it from / :
>
>                    /
>        |--|--|--|--|--|--|--|--|
>      /etc /bin /root /home /mnt /dev /usr
>        |--|--|--|--|--|--|--|--|
>      ../ ../ ../ ../ ../ ../ ../ ../
>
> As we see, each of the subdirectories hold a / prefix to them as we're
> used to seeing, and this translated to the subfolders having their ../
> descriptor denoting their parent (I think of them as recursive pointers to
> parent dirs), where the parent holds ../ and child dir has a name like /..
> and invoking
>
> I don't know if it wasn't until the mkdir() syscall was added this
> approach took form. There was an earlier 'Aha, Unix!' thread where Ken had
> mentioned a similar thing, where invoking pwd gave only the /usr/name
> directory even if the directory had been changed with chdir, essentially
> stemming out of only the forked child changing dirs, when the cd wasn't
> built into the shell (this was in the PDP-11 when he was writing the
> initial set utilities like cd, ls)
>
> Also, depending on the implementation of 'ls' we were to try this out
> today, these are bound to have their differences. The original syscalls on
> the PDP-7 had chdir,
>
> swp:			" system call dispatch table
>    jmp .		" base instruction
>    .save; .getuid; .open; .read; .write; .creat; .seek; .tell
>    .close; .link; .unlink; .setuid; .rename; .exit; .time; .intrp
>    .chdir; .chmod; .chown; badcal; .sysloc; badcal; .capt; .rele
>    .status; badcal; .smes; .rmes; .fork
>
>
> mkdir was probably in the works at this stage.
>
> We should also touch upon the "cd -" as this one takes you back like a
> back button into the directory you were last inside, whereas the cd ../..
> takes you up or down the hierarchy depending on the placement of the / and
> adding placeholders i.e directory names in place of the .. (dots).
>
> Of course, only the creators can embark upon the design details of the
> hierarchical system and the reasons behind all the above and more.
>
> Since you mentioned the word dd, the utility which serves me like none
> other for my frequent rewrites of images, it too has an interesting past,
> it seems like 'dd' was non-unixy in it's design approach, if I'm to believe
> the lores around, which leads me to think that this might have been another
> one of the many idiosyncratic naming conventions used back then. More on
> the dd stuff: http://www.catb.org/jargon/html/D/dd.html
>
> Personally, I just like to think of it in my head as disk-disk.
>
> --
>
> Abhinav Rajagopalan
>
>
>

-- 

Abhinav Rajagopalan

[-- Attachment #2: Type: text/html, Size: 8473 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 11:38   ` Abhinav Rajagopalan
@ 2019-10-21 12:00     ` Thomas Paulsen
  2019-10-21 13:31       ` Abhinav Rajagopalan
  0 siblings, 1 reply; 24+ messages in thread
From: Thomas Paulsen @ 2019-10-21 12:00 UTC (permalink / raw)
  To: Abhinav Rajagopalan; +Cc: tuhs


> Also, I only now realized that only mknod existed, up until a long time,
>
> only later on with the GNU coreutils did mkdir as a command come into
> existence. Running the PDP-11 v7 on SIMH showed that, gotten so accustomed
>
> to the Linux env that thinking backwards seemed suddenly arcane.
>
that's not true. mkdir isn't a GNU invention. Its much older.

https://en.wikipedia.org/wiki/Mkdir#History
http://man7.org/linux/man-pages/man2/mknod.2.html



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 12:00     ` Thomas Paulsen
@ 2019-10-21 13:31       ` Abhinav Rajagopalan
  0 siblings, 0 replies; 24+ messages in thread
From: Abhinav Rajagopalan @ 2019-10-21 13:31 UTC (permalink / raw)
  To: Thomas Paulsen; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

Thanks, didn't mean it as a GNU thing at all. Sure, mkdir would've existed
back in research Unix too. My experience has only been limited to the Linux
envs, which is why I was led to attribute mkdir to the evolution in
mknod(), I only know that they all either descended down from SysV or BSD,
those who have used the real Unices would know better. Seeing as I haven't
had the privilege of using any except briefly on an emulator.

On Mon, Oct 21, 2019 at 5:30 PM Thomas Paulsen <thomas.paulsen@firemail.de>
wrote:

>
> > Also, I only now realized that only mknod existed, up until a long time,
> >
> > only later on with the GNU coreutils did mkdir as a command come into
> > existence. Running the PDP-11 v7 on SIMH showed that, gotten so
> accustomed
> >
> > to the Linux env that thinking backwards seemed suddenly arcane.
> >
> that's not true. mkdir isn't a GNU invention. Its much older.
>
> https://en.wikipedia.org/wiki/Mkdir#History
> http://man7.org/linux/man-pages/man2/mknod.2.html
>
>
>

-- 

Abhinav Rajagopalan

[-- Attachment #2: Type: text/html, Size: 2701 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21  6:22 [TUHS] PDP-7 UNIX filesystem Angelo Papenhoff
  2019-10-21 10:43 ` Abhinav Rajagopalan
@ 2019-10-21 16:54 ` Angelo Papenhoff
  2019-10-23  5:48 ` Lars Brinkhoff
  2 siblings, 0 replies; 24+ messages in thread
From: Angelo Papenhoff @ 2019-10-21 16:54 UTC (permalink / raw)
  To: tuhs

On 21/10/19, Angelo Papenhoff wrote:
> . and ..

To this I seem to have found an answer by checking how stat.s and ls.s
work. 'stat foo' will check whether '../foo' (using /-notation) exists
and will then print the status.
'ls foo' will check whether '../foo' is a directory and will then open
foo and list it. It will default to .. if no argument is given.
This to me suggests that .. (confusingly) refers to the current
directory.
. does not seem to have any conventional meaning.


To the first question I still have no answer.


aap

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 10:43 ` Abhinav Rajagopalan
  2019-10-21 11:38   ` Abhinav Rajagopalan
@ 2019-10-22 18:07   ` Peter Jeremy
  2019-10-23  0:07     ` Mary Ann Horton
  2019-10-24  2:29   ` Christopher Browne
  2 siblings, 1 reply; 24+ messages in thread
From: Peter Jeremy @ 2019-10-22 18:07 UTC (permalink / raw)
  To: Abhinav Rajagopalan; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 1911 bytes --]

On 2019-Oct-21 16:13:54 +0530, Abhinav Rajagopalan <abhinavrajagopalan@gmail.com> wrote:
>We should also touch upon the "cd -" as this one takes you back like a back
>button into the directory you were last inside, whereas the cd ../.. takes
>you up or down the hierarchy depending on the placement of the / and adding
>placeholders i.e directory names in place of the .. (dots).

There are two different mechanisms here.  "-" as a "cd" argument is a
relatively recent shell builtin: More modern shells keep track of the
previous working directory ($OLDPWD) and evaluate "cd -" as "cd $OLDPWD",
which will return to the previous working directory (modulo filesystem
changes).  Note that it's a "swap" operation, not a "back" operation:
Repeated "cd -" invocations will swap between two directories, not keep
going back through previous working directorie.  I'm not sure when this
feature was introduced but don't believe it was part of ancient Unix.

OTOH, "cd ../.." just passes "../.." to the kernel as a pathname and the
kernel evaluates it using its normal pathname lookup.  Having it move
up (towards the root of) the filesystem relies on the presence of a ".."
directory entry being the parent directory inode.

>Since you mentioned the word dd, the utility which serves me like none
>other for my frequent rewrites of images, it too has an interesting past,
>it seems like 'dd' was non-unixy in it's design approach, if I'm to believe
>the lores around, which leads me to think that this might have been another
>one of the many idiosyncratic naming conventions used back then. More on
>the dd stuff: http://www.catb.org/jargon/html/D/dd.html

As noted in the jargon file, the dd(1) syntax is deliberately reminiscent
of the DD statement in IBM JCL.  This was presumably a joke and one of the
BTL old-timers on the list may know more of the background.

-- 
Peter Jeremy

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-22 18:07   ` Peter Jeremy
@ 2019-10-23  0:07     ` Mary Ann Horton
  2019-10-23  2:02       ` Kurt H Maier
  2019-10-23  8:34       ` Thomas Paulsen
  0 siblings, 2 replies; 24+ messages in thread
From: Mary Ann Horton @ 2019-10-23  0:07 UTC (permalink / raw)
  To: tuhs

dd is deprecated?  I'm surprised. I use it all the time. It's great when 
a blocksize of 512 is too slow, like for large files onto slow backup 
disks or network transfers, I might use dd bs=10240 or some such thing.

I suppose the block size on Linux might be bigger these days, but the 
command is still standard.

     Mary Ann

On 10/22/19 11:07 AM, Peter Jeremy wrote:
> Since you mentioned the word dd, the utility which serves me like none
>> other for my frequent rewrites of images, it too has an interesting past,
>> it seems like 'dd' was non-unixy in it's design approach, if I'm to believe
>> the lores around, which leads me to think that this might have been another
>> one of the many idiosyncratic naming conventions used back then. More on
>> the dd stuff: http://www.catb.org/jargon/html/D/dd.html
> As noted in the jargon file, the dd(1) syntax is deliberately reminiscent
> of the DD statement in IBM JCL.  This was presumably a joke and one of the
> BTL old-timers on the list may know more of the background.
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-23  0:07     ` Mary Ann Horton
@ 2019-10-23  2:02       ` Kurt H Maier
  2019-10-23  2:19         ` Larry McVoy
  2019-10-23  8:34       ` Thomas Paulsen
  1 sibling, 1 reply; 24+ messages in thread
From: Kurt H Maier @ 2019-10-23  2:02 UTC (permalink / raw)
  To: Mary Ann Horton; +Cc: tuhs

On Tue, Oct 22, 2019 at 05:07:33PM -0700, Mary Ann Horton wrote:
> dd is deprecated?  I'm surprised. I use it all the time. 

dd is not deprecated.  The maintainer of the jargon file is not a
reliable narrator.  dd(1) is even specified in IEEE Std 1003.1-2017,
which mentions the JCL history and explains that it takes 'operands' and
not 'options' to retain compatibility with the original syntax.

dd(1) on Plan 9 changes to regular option flags, such as -if= and -of=,
and some folks get disproportionately annoyed by the change.  Can't
please everyone, I guess.

khm

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-23  2:02       ` Kurt H Maier
@ 2019-10-23  2:19         ` Larry McVoy
  0 siblings, 0 replies; 24+ messages in thread
From: Larry McVoy @ 2019-10-23  2:19 UTC (permalink / raw)
  To: Mary Ann Horton, tuhs

On Tue, Oct 22, 2019 at 07:02:03PM -0700, Kurt H Maier wrote:
> On Tue, Oct 22, 2019 at 05:07:33PM -0700, Mary Ann Horton wrote:
> > dd is deprecated?? I'm surprised. I use it all the time. 
> 
> dd is not deprecated.  The maintainer of the jargon file is not a
> reliable narrator.  dd(1) is even specified in IEEE Std 1003.1-2017,
> which mentions the JCL history and explains that it takes 'operands' and
> not 'options' to retain compatibility with the original syntax.
> 
> dd(1) on Plan 9 changes to regular option flags, such as -if= and -of=,
> and some folks get disproportionately annoyed by the change.  Can't
> please everyone, I guess.

I really need to move this into the distros, it's dd on steriods and the
docs don't list the latest options, I made it go backwards, I made it
use the crc/xor stuff that bk has, I made it use the compression stuff
that bk has:

LMDD(8)                             LMBENCH                            LMDD(8)



NAME
       lmdd - move io for performance and debugging tests

SYNOPSIS
       lmdd [ option=value ] ...

DESCRIPTION
       lmdd  copies a specified input file to a specified output with possible
       conversions.  This program is primarily useful for timing I/O since  it
       prints out the timing statistics after completing.

OPTIONS
       if=name        Input  file is taken from name; internal is the default.
                      internal  is  a  special  file  that  acts  like   Sun's
                      /dev/zero,  i.e.,  it provides a buffer of zeros without
                      doing a system call to get them.
                      The following file names are taken to mean the  standard
                      input: -, 0, or stdin.

       of=name        Output file is taken from name; internal is the default.
                      internal is a special file  that  acts  like  /dev/null,
                      without doing a system call to get rid of the data.
                      The  following file names are taken to mean the standard
                      output: -, 1, or stdout.
                      The following file names are taken to mean the  standard
                      error: 2, or stderr.

       bs=n           Input  and  output  block  size  n bytes (default 8192).
                      Note that this is different from dd(1),  it  has  a  512
                      byte  default.    Also  note  that the block size can be
                      followed by 'k' or 'm' to indicate kilo bytes (*1024) or
                      megabytes (*1024*1024), respectively.

       ipat=n         If  n  is  non  zero, expect a known pattern in the file
                      (see opat).  Mismatches will  be  displayed  as  "ERROR:
                      off=%d  want=%x got=%x".  The pattern is a sequence of 4
                      byte integers with the first 0, second  1,  and  so  on.
                      The default is not to check for the pattern.

       opat=n         If n is non zero, generate a known pattern on the output
                      stream.  Used for  debugging  file  system  correctness.
                      The default is not to generate the pattern.

       mismatch=n     If  n  is  non zero, stop at the first mismatched value.
                      Used with ipat.

       skip=n         Skip n input blocks before starting copy.

       fsync=n        If n is non-zero,  call  fsync(2)  on  the  output  file
                      before exiting or printing timing statistics.

       sync=n         If  n is non-zero, call sync(2) before exiting or print-
                      ing timing statistics.

       rand=n         This argument, by default off, turns on random behavior.
                      The  argument  is not a flag, it is a size, that size is
                      used as the upper bound for the seeks.  Also  note  that
                      the block size can be followed by 'k' or 'm' to indicate
                      kilo bytes (*1024) or megabytes (*1024*1024),

       flush=n        If n is non-zero and mmap(2) is available, call msync(2)
                      to invalidate the output file.  This flushes the file to
                      disk so that you don't have unmount/mount.  It is not as
                      good as mount/unmount because it just flushes file pages
                      - it misses the indirect blocks which are still  cached.
                      Not supported on all systems, compile time option.

       rusage=n       If  n  is  non-zero,  print rusage statistics as well as
                      timing statistics.  Not supported on all  systems,  com-
                      pile time option.

       count=n        Copy only n input records.

EXAMPLES
       This  is  the  most common usage, the intent is to measure disk perfor-
       mance.  The disk is a spare partition mounted on /spare.

           # mount /spare
           # lmdd if=internal of=/spare/XXX count=1000 fsync=1
           7.81 MB in 3.78 seconds (2.0676 MB/sec)

           : Flush cache
           # umount /spare
           # mount /spare

           # lmdd if=/spare/XXX of=internal
           7.81 MB in 2.83 seconds (2.7611 MB/sec)


AUTHOR
       Larry McVoy, lm@sun.com



(c)1994 Larry McVoy                 $Date$                             LMDD(8)


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21  6:22 [TUHS] PDP-7 UNIX filesystem Angelo Papenhoff
  2019-10-21 10:43 ` Abhinav Rajagopalan
  2019-10-21 16:54 ` Angelo Papenhoff
@ 2019-10-23  5:48 ` Lars Brinkhoff
  2 siblings, 0 replies; 24+ messages in thread
From: Lars Brinkhoff @ 2019-10-23  5:48 UTC (permalink / raw)
  To: Angelo Papenhoff; +Cc: tuhs

Does anyone have any input for Angelo?  I think the questions he asks
are interesting.  Please note he's asking about PDP-7 Unix, not the
unrelated "dd" command and how "." and ".." work in later Unix versions.


Angelo Papenhoff wrote:
> you cannot execute a program if you're in a directory you can't write into.
>
> 	I asked Warren about this when I first tried pdp7 unix and he
> explained it to me: the shell creates a link to the binary and executes
> it. If it can't write into the current directory, it fails to create the
> link and hence can't execute the program.
> 	How was this handled in practice? did users have write
> permissions on all directories? did you just stay in your directory all
> the time?


> . and ..

This part is apparently resolved.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-23  0:07     ` Mary Ann Horton
  2019-10-23  2:02       ` Kurt H Maier
@ 2019-10-23  8:34       ` Thomas Paulsen
  2019-10-24  0:06         ` Warner Losh
  1 sibling, 1 reply; 24+ messages in thread
From: Thomas Paulsen @ 2019-10-23  8:34 UTC (permalink / raw)
  To: Mary Ann Horton; +Cc: tuhs



> dd is deprecated?  I'm surprised. I use it all the time. It's great when

I don't think so. A quick google search doesn't support that.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-23  8:34       ` Thomas Paulsen
@ 2019-10-24  0:06         ` Warner Losh
  2019-10-24  2:23           ` Michael Parson
  0 siblings, 1 reply; 24+ messages in thread
From: Warner Losh @ 2019-10-24  0:06 UTC (permalink / raw)
  To: Thomas Paulsen; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 401 bytes --]

On Wed, Oct 23, 2019 at 3:04 AM Thomas Paulsen <thomas.paulsen@firemail.de>
wrote:

>
>
> > dd is deprecated?  I'm surprised. I use it all the time. It's great when
>
> I don't think so. A quick google search doesn't support that.
>

dd isn't deprecated. There's no heir apparent, it's in POSIX.1 and there's
no advantage to changing. There's not even been talk of that, at east in
*BSD land.

Warner

[-- Attachment #2: Type: text/html, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-24  0:06         ` Warner Losh
@ 2019-10-24  2:23           ` Michael Parson
  2019-10-25 21:08             ` Michael Kjörling
  0 siblings, 1 reply; 24+ messages in thread
From: Michael Parson @ 2019-10-24  2:23 UTC (permalink / raw)
  To: TUHS main list

On 2019-10-23 19:06, Warner Losh wrote:
> On Wed, Oct 23, 2019 at 3:04 AM Thomas Paulsen
> <thomas.paulsen@firemail.de> wrote:
> 
>>> dd is deprecated?  I'm surprised. I use it all the time. It's
>> great when
>> 
>> I don't think so. A quick google search doesn't support that.
> 
> dd isn't deprecated. There's no heir apparent, it's in POSIX.1 and
> there's no advantage to changing. There's not even been talk of that,
> at east in *BSD land.

I'm sure the systemd people are eyeballing it.

-- 
Michael Parson
Pflugerville, TX
KF5LGQ

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 10:43 ` Abhinav Rajagopalan
  2019-10-21 11:38   ` Abhinav Rajagopalan
  2019-10-22 18:07   ` Peter Jeremy
@ 2019-10-24  2:29   ` Christopher Browne
  2019-10-24  8:25     ` Thomas Paulsen
  2 siblings, 1 reply; 24+ messages in thread
From: Christopher Browne @ 2019-10-24  2:29 UTC (permalink / raw)
  To: Abhinav Rajagopalan; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2350 bytes --]

On Mon, 21 Oct 2019 at 06:45, Abhinav Rajagopalan <
abhinavrajagopalan@gmail.com> wrote:

>
>>
> Since you mentioned the word dd, the utility which serves me like none
> other for my frequent rewrites of images, it too has an interesting past,
> it seems like 'dd' was non-unixy in it's design approach, if I'm to believe
> the lores around, which leads me to think that this might have been another
> one of the many idiosyncratic naming conventions used back then. More on
> the dd stuff: http://www.catb.org/jargon/html/D/dd.html
>
> Personally, I just like to think of it in my head as disk-disk.
>

I am pretty sure that "dd" derives from  the "DD" statement in IBM JCL that
stands for "Data Definition"

Here's a link to practical-ish documentation about that:
https://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zjcl/zjclc_jclDDstmt.htm

On MVS, TSO, and such systems, compiled programs would be written in
languages like COBOL, PL/1, FORTRAN, and such, and the programs would
commonly reference input and output devices.  The programs are controlled
(we could say, "scripted") by JCL code that indicate the files (possibly
other devices) to be connected up.  Each file that is accessed gets its own
DD line in the JCL script that indicates the various metadata about the
file, such as its name, block size, storage class, how much space is
allocated, literally dozens of options.

Back in the '90s my "Y2K remediation" involvement was at American Airlines;
I was one of the Unix guys working alongside mainframe guys; as soon as I
started seeing the JCL for their TSO batch jobs, it was pretty obvious that
this was from whence derived the dd command on Unix.  The mainframe guys
enthused a lot about a sorting tool with similar syntax called SyncSort
that they'd use to do many of the things we'd do with cut and grep.

The Jargon File claim that dd is deprecated makes little sense; dd is THE
good tool for grabbing exact chunks of data out of binary files, and I
haven't seen a would-be successor.  It would be interesting to see some
alternative constructed; given that it's all about dealing with pretty
messy sorts of I/O work, an alternative is liable to have its messiness in
different places.
-- 
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"

[-- Attachment #2: Type: text/html, Size: 3591 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-24  2:29   ` Christopher Browne
@ 2019-10-24  8:25     ` Thomas Paulsen
  0 siblings, 0 replies; 24+ messages in thread
From: Thomas Paulsen @ 2019-10-24  8:25 UTC (permalink / raw)
  To: Christopher Browne; +Cc: tuhs

MVS(Z/OS).DD statement means Data definition,
//DD-name DD Parameters
whereas DD-NAME identifies the dataset (=file). The real file is referenced by this name within the program. Say DD hello then the program source says: open hello; read hello, and so on.

Thus MVS(Z/OS).DD  having nothing todo withUNIX.DD.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-24  2:23           ` Michael Parson
@ 2019-10-25 21:08             ` Michael Kjörling
  2019-10-25 21:34               ` Arthur Krewat
  0 siblings, 1 reply; 24+ messages in thread
From: Michael Kjörling @ 2019-10-25 21:08 UTC (permalink / raw)
  To: tuhs

On 23 Oct 2019 21:23 -0500, from mparson@bl.org (Michael Parson):
>>>> dd is deprecated?  I'm surprised. I use it all the time.
>>> 
>>> I don't think so. A quick google search doesn't support that.
>> 
>> dd isn't deprecated. There's no heir apparent, it's in POSIX.1 and
>> there's no advantage to changing. There's not even been talk of that,
>> at east in *BSD land.

The only thing resembling "deprecation" of dd that I'm aware of is the
recommendation I saw many years ago to not use it to copy potentially
problematic storage media (for data recovery), but to use ddrescue for
that purpose instead. The reason for this is that with conv=noerror
(without which it'd simply abort if it encounters an I/O error), dd
would simply skip past data in the input that it can't read, but not
adjust any offsets in the output, which wreaks havoc with anything
where offsets matter (such as in file system metadata).


> I'm sure the systemd people are eyeballing it.

That wouldn't surprise me in the slightest.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
  “The most dangerous thought that you can have as a creative person
              is to think you know what you’re doing.” (Bret Victor)


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-25 21:08             ` Michael Kjörling
@ 2019-10-25 21:34               ` Arthur Krewat
  2019-10-25 21:50                 ` reed
  2019-10-25 22:54                 ` Warner Losh
  0 siblings, 2 replies; 24+ messages in thread
From: Arthur Krewat @ 2019-10-25 21:34 UTC (permalink / raw)
  To: tuhs

On 10/25/2019 5:08 PM, Michael Kjörling wrote:
> The only thing resembling "deprecation" of dd that I'm aware of is the
> recommendation I saw many years ago to not use it to copy potentially
> problematic storage media (for data recovery), but to use ddrescue for
> that purpose instead. The reason for this is that with conv=noerror
> (without which it'd simply abort if it encounters an I/O error), dd
> would simply skip past data in the input that it can't read, but not
> adjust any offsets in the output, which wreaks havoc with anything
> where offsets matter (such as in file system metadata).

Which is where conv=sync,noerror comes in. Of course, I have no freakin' 
idea what version of UNIX that came into being. ;)

art k.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-25 21:34               ` Arthur Krewat
@ 2019-10-25 21:50                 ` reed
  2019-10-25 22:54                 ` Warner Losh
  1 sibling, 0 replies; 24+ messages in thread
From: reed @ 2019-10-25 21:50 UTC (permalink / raw)
  To: tuhs

On Fri, 25 Oct 2019, Arthur Krewat wrote:

> Which is where conv=sync,noerror comes in. Of course, I have no 
> freakin' idea what version of UNIX that came into being. ;)

Looks like dd was introduced in Fifth Edition (and had those conversion 
values).

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-25 21:34               ` Arthur Krewat
  2019-10-25 21:50                 ` reed
@ 2019-10-25 22:54                 ` Warner Losh
  1 sibling, 0 replies; 24+ messages in thread
From: Warner Losh @ 2019-10-25 22:54 UTC (permalink / raw)
  To: Arthur Krewat; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

On Fri, Oct 25, 2019, 3:35 PM Arthur Krewat <krewat@kilonet.net> wrote:

> On 10/25/2019 5:08 PM, Michael Kjörling wrote:
> > The only thing resembling "deprecation" of dd that I'm aware of is the
> > recommendation I saw many years ago to not use it to copy potentially
> > problematic storage media (for data recovery), but to use ddrescue for
> > that purpose instead. The reason for this is that with conv=noerror
> > (without which it'd simply abort if it encounters an I/O error), dd
> > would simply skip past data in the input that it can't read, but not
> > adjust any offsets in the output, which wreaks havoc with anything
> > where offsets matter (such as in file system metadata).
>
> Which is where conv=sync,noerror comes in. Of course, I have no freakin'
> idea what version of UNIX that came into being. ;)
>

Ddrescue tries multiple times with different sizes, which dd doesn't do.

Warner

art k.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1588 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-22  2:09 ` Jaap Akkerhuis
@ 2019-10-23  2:00   ` Christopher Browne
  0 siblings, 0 replies; 24+ messages in thread
From: Christopher Browne @ 2019-10-23  2:00 UTC (permalink / raw)
  To: Jaap Akkerhuis; +Cc: The Eunuchs Hysterical Society, Noel Chiappa

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

On Mon, Oct 21, 2019, 10:16 PM Jaap Akkerhuis <jaapna@xs4all.nl> wrote:

>
>
> On Oct 21, 2019, at 19:58, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
>
> Maybe that was a typo for 'mkdir as a system call'? (I recall having to do
> a
> fork() to execute 'mkdir', back when.) But 4.2 had mkdir().
>
>
> I believe, it was 4.2 (or 4.1c) according to David Tilbrook's paper on
> rename. The site qef.com seems to have disappeard, but the wayback
> machine has  copies[1]
>

Timing being everything, I happily have a copy of it all, as I used wget to
pull it into a git repo.

It's doubtless not in "native" form as he used one of his little languages
(that were part of qef) to generate it.  But HTML isn't too bad for this.

[-- Attachment #2: Type: text/html, Size: 2268 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 11:58 Noel Chiappa
  2019-10-21 15:44 ` Abhinav Rajagopalan
@ 2019-10-22  2:09 ` Jaap Akkerhuis
  2019-10-23  2:00   ` Christopher Browne
  1 sibling, 1 reply; 24+ messages in thread
From: Jaap Akkerhuis @ 2019-10-22  2:09 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs


[-- Attachment #1.1: Type: text/plain, Size: 607 bytes --]



> On Oct 21, 2019, at 19:58, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
> Maybe that was a typo for 'mkdir as a system call'? (I recall having to do a
> fork() to execute 'mkdir', back when.) But 4.2 had mkdir().

I believe, it was 4.2 (or 4.1c) according to David Tilbrook's paper on rename. The site qef.com <http://qef.com/> seems to have disappeard, but the wayback machine has  copies[1]

	jaap

[1] as example, https://web.archive.org/web/20070228190911/http://www.qef.com/html/docs/rename.ps <https://web.archive.org/web/20070228190911/http://www.qef.com/html/docs/rename.ps>



[-- Attachment #1.2: Type: text/html, Size: 2360 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 267 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
  2019-10-21 11:58 Noel Chiappa
@ 2019-10-21 15:44 ` Abhinav Rajagopalan
  2019-10-22  2:09 ` Jaap Akkerhuis
  1 sibling, 0 replies; 24+ messages in thread
From: Abhinav Rajagopalan @ 2019-10-21 15:44 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 4331 bytes --]

Thanks, mkdir actually seems to go back to V4.

Also, V4 was the first to have the kernel in a 'high level language' C,
from the V4 tree,

The files in nsys/ come from nsys.tar.gz
> <http://minnie.tuhs.org/Archive/Distributions/Research/Dennis_v3/nsys.tar.gz>,
> which was donated by Dennis Ritchie. This is a version of the kernel quite
> close to that released in Fourth Edition, but *without pipes*. Dennis
> Ritchie writes:



> This is a tar archive derived from a DECtape labelled "nsys". What is
> contains is just the kernel source, written in the pre-K&R dialect of C. It
> is intended only for PDP-11/45, and has setup and memory-handling code that
> will not work on other models (it's missing things special to the later,
> smaller models, and the larger physical address space of the still later
> 11/70.) It appears that it is intended to be loaded into memory at physical
> address 0, and transferred to at location 0.


The efforts behind the PDP-7 file system were quite tedious and was a
general directed graph, only word addressable which meant lack of path
names as it ignored null chars, the link call being used to link
directories together which is where dd arises as the precursor to .. today.
The required files for the user was just linked in together in their dirs.

Quoting from DMR's Evolution of the Unix Time-sharing system,

So that every user did not need to maintain a link to all directories of
> interest, there existed a directory called *dd* that contained entries
> for the directory of each user. Thus, to make a link to file *x* in
> directory *ken*, I might do

ln dd ken ken
> ln ken x x
> rm ken
>
This scheme rendered subdirectories sufficiently hard to use as to make
> them unused in practice. Another important barrier was that there was no
> way to create a directory while the system was running; all were made
> during recreation of the file system from paper tape, so that directories
> were in effect a nonrenewable resource.


No mkdir/mknode while the system was running, the whole file system had to
be recreated from the paper tape each time! Thank DEC for the PDP-11. And
of course, no pipes until '72. Earlier parent had pondered on the write
permissions required to execute programs, this below explanation from the
paper might help.

Another mismatch between the system as it had been and the new process
> control scheme took longer to become evident. Originally, the read/write
> pointer associated with each open file was stored within the process that
> opened the file. (This pointer indicates where in the file the next read or
> write will take place.) The problem with this organization became evident
> only when we tried to use command files. Suppose a simple command file
> contains

ls
> who

and it is executed as follows:

sh comfile >output

The sequence of events was

1) The main shell creates a new process, which opens *outfile* to receive
> the standard output and executes the shell recursively.

2) The new shell creates another process to execute *ls*, which correctly
> writes on file *output* and then terminates.

3) Another process is created to execute the next command. However, the IO
> pointer for the output is copied from that of the shell, and it is still 0,
> because the shell has never written on its output, and IO pointers are
> associated with processes. The effect is that the output of *who*
> overwrites and destroys the output of the preceding *ls* command.

Solution of this problem required creation of a new system table to contain
> the IO pointers of open files independently of the process in which they
> were opened.


Source: https://www.bell-labs.com/usr/dmr/www/hist.html



On Mon, Oct 21, 2019 at 5:28 PM Noel Chiappa <jnc@mercury.lcs.mit.edu>
wrote:

>     > From: Abhinav Rajagopalan
>
>     > I only now realized that only mknod existed, up until a long time,
> only
>     > later on with the GNU coreutils did mkdir as a command come into
>     > existence.
>
> Huh? See:
>
>     https://minnie.tuhs.org//cgi-bin/utree.pl?file=V6/usr/man/man1/mkdir.1
>
> (And probably before that, that was the quickest one to find?)
>
> Maybe that was a typo for 'mkdir as a system call'? (I recall having to do
> a
> fork() to execute 'mkdir', back when.) But 4.2 had mkdir().
>
>        Noel
>
>

-- 

Abhinav Rajagopalan

[-- Attachment #2: Type: text/html, Size: 13774 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] PDP-7 UNIX filesystem
@ 2019-10-21 11:58 Noel Chiappa
  2019-10-21 15:44 ` Abhinav Rajagopalan
  2019-10-22  2:09 ` Jaap Akkerhuis
  0 siblings, 2 replies; 24+ messages in thread
From: Noel Chiappa @ 2019-10-21 11:58 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Abhinav Rajagopalan

    > I only now realized that only mknod existed, up until a long time, only
    > later on with the GNU coreutils did mkdir as a command come into
    > existence.

Huh? See:

    https://minnie.tuhs.org//cgi-bin/utree.pl?file=V6/usr/man/man1/mkdir.1

(And probably before that, that was the quickest one to find?)

Maybe that was a typo for 'mkdir as a system call'? (I recall having to do a
fork() to execute 'mkdir', back when.) But 4.2 had mkdir().

       Noel


^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2019-10-25 22:55 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-21  6:22 [TUHS] PDP-7 UNIX filesystem Angelo Papenhoff
2019-10-21 10:43 ` Abhinav Rajagopalan
2019-10-21 11:38   ` Abhinav Rajagopalan
2019-10-21 12:00     ` Thomas Paulsen
2019-10-21 13:31       ` Abhinav Rajagopalan
2019-10-22 18:07   ` Peter Jeremy
2019-10-23  0:07     ` Mary Ann Horton
2019-10-23  2:02       ` Kurt H Maier
2019-10-23  2:19         ` Larry McVoy
2019-10-23  8:34       ` Thomas Paulsen
2019-10-24  0:06         ` Warner Losh
2019-10-24  2:23           ` Michael Parson
2019-10-25 21:08             ` Michael Kjörling
2019-10-25 21:34               ` Arthur Krewat
2019-10-25 21:50                 ` reed
2019-10-25 22:54                 ` Warner Losh
2019-10-24  2:29   ` Christopher Browne
2019-10-24  8:25     ` Thomas Paulsen
2019-10-21 16:54 ` Angelo Papenhoff
2019-10-23  5:48 ` Lars Brinkhoff
2019-10-21 11:58 Noel Chiappa
2019-10-21 15:44 ` Abhinav Rajagopalan
2019-10-22  2:09 ` Jaap Akkerhuis
2019-10-23  2:00   ` Christopher Browne

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).