[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Nemo]

On 02/09/2017, Dave Horsfall <dave at horsfall.org> wrote:
> On Thu, 31 Aug 2017, Nemo wrote:
>
>> (By the way, why all the ^M is the files?)
>
> Err, because CR/LF was the line terminator?

Hhhmmm... This begs the historical question:  When did LF replace CR/LF in UNIX?

N.

>
> --
> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will
> suffer."
>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Dave Horsfall]

On Sat, 2 Sep 2017, Nemo wrote:

> Hhhmmm... This begs the historical question:  When did LF replace CR/LF 
> in UNIX?

Unix has always used NL as the terminator :-)

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Warner Losh]

On Sat, Sep 2, 2017 at 8:54 PM, Dave Horsfall <dave at horsfall.org> wrote:

> On Sat, 2 Sep 2017, Nemo wrote:
>
> Hhhmmm... This begs the historical question:  When did LF replace CR/LF in
>> UNIX?
>>
>
> Unix has always used NL as the terminator :-)


<CR><LF> was the line terminator in DEC operating systems that grew up
around the same time as Unix. CP/M and MS-DOS inherited that from them
since those systems were developed, in part, using cross compilers running
on DEC gear with DEC OSes. Unix came from the Multics world where LF was
used as the line terminator... Thankfully, neither CP/M nor MS-DOS picked
up DEC's RMS...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170903/242a3052/attachment.html>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Steve Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2034 bytes --]

I think the reason for this is pretty obvious.  At the time -- for
many of the paper terminals, line feed simply rotated the platen, and
the print head stayed at the same column position.  Carriage return
returned the print head to the first column without advancing the
paper (remember overstriking?).  But there were also some terminals
that both advanced the paper and returned the print head (I'm hazy
about this, but I think the IBM Selectric was one...).

The Honeywell system in Murray Hill had a teletype driver that
attempted to paper over these differences.  If it saw a linefeed on
some tty's they would issue the line feed and then space out to the
previous column location.   If it saw a carriage return it would
backspace to the first column.  It did recognize CR/LF and did
something more reasonable.

I do remember one guy who wrote his program to output LF/CR instead of
CR/LF.   The teletype driver would do a LF and then space out to the
previous column and then do a carriage return.  Pretty painful at 300
baud!

Doug might remember better than I but I suspect some terminal did the
combined CR/LF only, so that's why Unix adopted it.  Certainly by the
time glass teletypes came along, making overstriking (and often
underlining) impossible, I don't think anybody objected.

Steve

----- Original Message -----
From: "Dave Horsfall" <dave@horsfall.org>
To:"The Eunuchs Hysterical Society" <tuhs at tuhs.org>
Cc:
Sent:Sun, 3 Sep 2017 12:54:02 +1000 (EST)
Subject:Re: [TUHS] Line Terminators in Text Files [Was: Re: Why Pascal
is Not My Favorite Programming Language - Unearthed!]

 On Sat, 2 Sep 2017, Nemo wrote:

 > Hhhmmm... This begs the historical question: When did LF replace
CR/LF 
 > in UNIX?

 Unix has always used NL as the terminator :-)

 -- 
 Dave Horsfall DTM (VK2KFU) "Those who don't understand security will
suffer."

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170903/a84e58a5/attachment.html>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Chris Torek]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2353 bytes --]

>I think the reason for this is pretty obvious.  At the time -- for
>many of the paper terminals, line feed simply rotated the platen, and
>the print head stayed at the same column position.  Carriage return
>returned the print head to the first column without advancing the
>paper (remember overstriking?).

In what was either a remarkable coincidence, or actual planning,
the old ASR 33 at I think 110 baud (maybe 300) took two "character
times" to do the carriage return, so that if you sent CR and LF
in that order, the LF occurred mid-CR and the print head was
ready right at the time the next character arrived.

(One of the Stupid Pet Tricks of the day was to print a line with
an unfilled character halfway through, then send CR, then print
the middle character and have it come out in the right position.)

>But there were also some terminals that both advanced the paper
>and returned the print head (I'm hazy about this, but I think
>the IBM Selectric was one...).

Possibly, since as I recall there was a keyboard key that did the
equivalent of pushing the return bar on a manual typewriter.  If
you look at photos of Selectrics, there is no external bar (though
of course there are knobs to rotate the platen for inserting and
removing pages).

(I used a modified Selectric that had been converted into a
printer, hooking it to a custom driver that I wrote, and custom
hardware that came with the modified Selectric, on my home TRS-80,
back in high school -- 1980-81 or so.  I remember seven solenoids,
which must have been divided into 1 for shift -- 180 degree rotate
-- and 2 for tilt plus 4 for rotate.  I had access to an actual
Selectric as well, and when you pushed the shift key, the ball
would do the 180 degree rotate at that point, before doing the
tilt/rotate for each character.)

>Doug might remember better than I but I suspect some terminal did the
>combined CR/LF only, so that's why Unix adopted it.

As I recall, the original definition of ASCII suggested that the
LF character was either "line feed" or "new line", and that if it
*was* new-line, it would be stand-alone.

Wikipedia (https://en.wikipedia.org/wiki/ASCII) implies that Unix
got this from Multics.

Of course, DEL was code 0177 because it punched all the holes in
the paper tape, which would always work to "erase" any partially
punched holes....

Chris

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Grant Taylor]

On 09/03/2017 05:14 PM, Chris Torek wrote:
> Of course, DEL was code 0177 because it punched all the holes in
> the paper tape, which would always work to "erase" any partially
> punched holes....

I remember having a face palm moment when I realized / understood this
for the first time.

I've had the fortune of sharing that epiphany with others a handful of
times.

I think the order of the carriage return followed by the line feed will
become another such epiphany inducing pieces of trivia.



-- 
Grant. . . .
unix || die

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3717 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170903/5f70b4c1/attachment.bin>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Pete Turnbull]

On 04/09/2017 00:14, Chris Torek wrote:

> In what was either a remarkable coincidence, or actual planning,
> the old ASR 33 at I think 110 baud (maybe 300) took two "character
> times" to do the carriage return, so that if you sent CR and LF
> in that order, the LF occurred mid-CR and the print head was
> ready right at the time the next character arrived.

That's not right.  Yes, the ASR33 runs at 110 baud (10cps), but all - 
including mine and those of my friends -  take more like half a dozen 
character times to perform a CR from the right hand side.  Moreover, the 
time it takes depends in a non-linear fashionon how far along the line 
the printhead has travelled; it's driven by a spring and decelerated and 
stopped by an air dashpot.  Even an LF takes more than two character 
times.  That's why ASR33 drivers send a string of nulls after a CR. 
Many are smart enough to send the LF instead of the first null, if they 
see that LF is the first character after CR.

>> But there were also some terminals that both advanced the paper
>> and returned the print head (I'm hazy about this, but I think
>> the IBM Selectric was one...).

The ASR33 friction feed models, but not the sprocket fed ones, can be 
configured to do that but AFAIR only at the 72nd character position at 
the end of a line.

> Possibly, since as I recall there was a keyboard key that did the
> equivalent of pushing the return bar on a manual typewriter.  If
> you look at photos of Selectrics, there is no external bar (though
> of course there are knobs to rotate the platen for inserting and
> removing pages).

That's correct.  Selectrics have a RETURN key, which performs carriage 
return and linefeed, and on mine there was a switch for single/double 
line spacing too.  I no longer have one, but I used to have two.

-- 
Pete
Pete Turnbull

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Chris Torek]

>That's not right.  Yes, the ASR33 runs at 110 baud (10cps), but all - 
>including mine and those of my friends -  take more like half a dozen 
>character times to perform a CR from the right hand side.

Hm, OK.  I recall actually seeing the trick performed on
*something* though.

(I hooked up an old ASR 33 to someone's system with a current loop
circuit at one point, plus a bit-bang driver, and I thought I
remembered it being this particular ASR 33 on which I saw the trick,
but it might have been something else.)

Chris

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Warner Losh]

On Sun, Sep 3, 2017 at 7:47 PM, Chris Torek <torek at torek.net> wrote:

> >That's not right.  Yes, the ASR33 runs at 110 baud (10cps), but all -
> >including mine and those of my friends -  take more like half a dozen
> >character times to perform a CR from the right hand side.
>
> Hm, OK.  I recall actually seeing the trick performed on
> *something* though.
>
> (I hooked up an old ASR 33 to someone's system with a current loop
> circuit at one point, plus a bit-bang driver, and I thought I
> remembered it being this particular ASR 33 on which I saw the trick,
> but it might have been something else.)


Time delay for physical movement of the print head is the main reason that
termcap has entries for the number of NUL characters to insert after
carriage return. Not sure it is for the ASR 33, but when it takes a while
for the carriage to return to column 1, you have to insert NULs to wait for
it to be ready to print characters. If it takes 100ms to do the return,
that's still 10 characters of NUL needed...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170903/3ad2245e/attachment.html>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Pete Turnbull]

On 04/09/2017 03:58, Warner Losh wrote:
> Time delay for physical movement of the print head is the main reason 
> that termcap has entries for the number of NUL characters to insert 
> after carriage return. Not sure it is for the ASR 33, but when it takes 
> a while for the carriage to return to column 1, you have to insert NULs 
> to wait for it to be ready to print characters. If it takes 100ms to do 
> the return, that's still 10 characters of NUL needed...

I'm going to assume that's a typo :-)  A single NUL at 110 baud,or 
10cps, is 100ms.

-- 
Pete
Pete Turnbull

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Chris Torek]

>Time delay for physical movement of the print head is the main
>reason that termcap has entries for the number of NUL characters
>to insert after carriage return.

This is not quite right.  For instance, there are delay values for
line and character insert and delete operations, which were
required back when terminals were slow and bit rates were inching
up ("stty 19200" finally going in as a name for EXTA, one of the
two externally-controlled bit rates in the old drivers...).

(The 4BSD tty driver had CR/LF delays built in, too, no doubt
inherited from 32V. It had to be done in the tty driver, not in
termcap, since command line utilities mostly did not use termcap
in the first place.)

As for termcap delays: some of the software on those terminals was
particularly bad.  The Heathkit H19, for instance, implemented the
repeat-counted line and character insert and delete ANSI escape
sequences ESC [ %d <code> with a loop that inserted or deleted one
line or character, N times.  (I had an H19, and kept it in this
mode instead of the other one that did not have the N-character
ops -- even with the required delays, if you got the editor to use
them, they were much faster over slow modems.)

(I hacked on the Gosling Emacs display driver quite a bit to tune
it up for our displays, including the Ann Arbor Ambassador displays
that did up to 60 lines in Squint Mode.)

Chris

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Pete Turnbull]

On 04/09/2017 02:47, Chris Torek wrote:
>> That's not right.  Yes, the ASR33 runs at 110 baud (10cps), but all -
>> including mine and those of my friends -  take more like half a dozen
>> character times to perform a CR from the right hand side.
> 
> Hm, OK.  I recall actually seeing the trick performed on
> *something* though.
> 
> (I hooked up an old ASR 33 to someone's system with a current loop
> circuit at one point, plus a bit-bang driver, and I thought I
> remembered it being this particular ASR 33 on which I saw the trick,
> but it might have been something else.)

I don't doubt it's possible, just not on an ASR33.  Apart from the 
timing issue, the carriage return mechanism on Teletype 33s inhibits 
printing while the carriage is moving back, until it stops at the left 
margin.

-- 
Pete
Pete Turnbull

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Paul Winalski]

On 9/4/17, Pete Turnbull <pete at dunnington.plus.com> wrote:
>
> I don't doubt it's possible, just not on an ASR33.  Apart from the
> timing issue, the carriage return mechanism on Teletype 33s inhibits
> printing while the carriage is moving back, until it stops at the left
> margin.

I could swear I saw our old ASR33 at college do this (print characters
in the middle of the line while the carriage is moving back).  But
maybe my memory is faulty and it was another printing terminal of that
era.  Or maybe our ASR33 was broken.

-Paul W.

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Clem Cole]

On Tue, Sep 5, 2017 at 11:23 AM, Paul Winalski <paul.winalski at gmail.com>
wrote:

> On 9/4/17, Pete Turnbull <pete at dunnington.plus.com> wrote:
> >
> > I don't doubt it's possible, just not on an ASR33.  Apart from the
> > timing issue, the carriage return mechanism on Teletype 33s inhibits
> > printing while the carriage is moving back, until it stops at the left
> > margin.
>
> I could swear I saw our old ASR33 at college do this (print characters
> in the middle of the line while the carriage is moving back).  But
> maybe my memory is faulty and it was another printing terminal of that
> era.  Or maybe our ASR33 was broken.

I agree with Paul, maybe later models worked this way, but the first ASR33
I used definitely did print at strange times.   I have the same memories
from 1967 on a GE Mark III with Dartmouth Basic using an acoustic coupler,
where line noise was definitely a problem.   It would print random
characters, particularly during carriage motion.   Since the only way to
get a copy of your program was with the LIST command, it was rare to get a
clean copy.   If the program was of any length, there was usually a random
character or two overprinted in the middle of listing.

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170905/70a68c01/attachment.html>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Paul Winalski]

On 9/3/17, Pete Turnbull <pete at dunnington.plus.com> wrote:
>
> That's not right.  Yes, the ASR33 runs at 110 baud (10cps), but all -
> including mine and those of my friends -  take more like half a dozen
> character times to perform a CR from the right hand side.  Moreover, the
> time it takes depends in a non-linear fashionon how far along the line
> the printhead has travelled; it's driven by a spring and decelerated and
> stopped by an air dashpot.  Even an LF takes more than two character
> times.  That's why ASR33 drivers send a string of nulls after a CR.
> Many are smart enough to send the LF instead of the first null, if they
> see that LF is the first character after CR.

I distinctly recall the ASR33 printing exactly two characters while
the print head is in motion, if a CR is issued while the print head is
at the right margin.  The ASR33 had a very fast and forceful print
head return mechanism--you definitely didn't want to get your fingers
in the way!  In my programs that sent data to an ASR33, I always
followed CR with two non-printing characters (typically LF/NUL) and I
never had a problem with characters printing during the print head
return.

I don't recall LF taking more than one character time to execute on
the ASR33.  If it did, I never noticed it.

The DEC LA36 dot matrix terminal had a rather leisurely carriage
return mechanism, but it also could buffer up characters received
while the CR was in progress.  It would then speed up over the normal
printing rate to catch up and empty the buffer.  There was a third
party mod to the LA36 that you could buy that would cause it to always
think it was in catch-up mode and to print at the higher speed all the
time.  This shortened the life of the print mechanism, but at our shop
we found the extra speed in printing worth the expense of repairing
early breakdown of the mechanism.

-Paul W.

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Pete Turnbull]

On 05/09/2017 19:55, Paul Winalski wrote:
> I distinctly recall the ASR33 printing exactly two characters while
> the print head is in motion, if a CR is issued while the print head is
> at the right margin.  The ASR33 had a very fast and forceful print
> head return mechanism--you definitely didn't want to get your fingers
> in the way!  In my programs that sent data to an ASR33, I always
> followed CR with two non-printing characters (typically LF/NUL) and I
> never had a problem with characters printing during the print head
> return.

Well, I may have exaggerated the time a CR takes slightly, in fact 
thinking of 5 or 6 perhaps I was remembering my other teleprinter: a 50 
baud Creed 7B which I no longer have.  I'm still sure it's normally more 
than two character times.  Yes, I know about keeping fingers away - and 
how to adjust various parts of an ASR33, and I have the tech manuals. 
I'm sitting next to a working ASR33 right now.

-- 
Pete
Pete Turnbull

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Paul Winalski]

On 9/3/17, Steve Johnson <scj at yaccman.com> wrote:
> I think the reason for this is pretty obvious.  At the time -- for
> many of the paper terminals, line feed simply rotated the platen, and
> the print head stayed at the same column position.  Carriage return
> returned the print head to the first column without advancing the
> paper (remember overstriking?).  But there were also some terminals
> that both advanced the paper and returned the print head (I'm hazy
> about this, but I think the IBM Selectric was one...).

IIRC, you're right about this.  I'm pretty sure I remember the IBM
2741 and friends (the Selectric terminals) operating this way.
>
> I do remember one guy who wrote his program to output LF/CR instead of
> CR/LF.   The teletype driver would do a LF and then space out to the
> previous column and then do a carriage return.  Pretty painful at 300
> baud!

At 110 baud, the model 33 Teletype took up to two character times to
return the typeball to the left margin when executing a CR.  You
needed to follow CR with two non-printing characters to avoid
erroneously striking a character in the middle of the line.  With
CR/LF, the LF served as one of these non-printables.  CR/LF/NUL was
the common sequence used to return the carriage and advance the
platen.  If you did LF/CR you'd need LF/CR/NUL/NUL, thus wasting one
character time.  So just about everyone implemented newline as CR/LF
rather than LF/CR.
>
> Doug might remember better than I but I suspect some terminal did the
> combined CR/LF only, so that's why Unix adopted it.

I suspect UNIX used LF as EOL for two reasons:  [1] as just mentioned,
nearly everyone did CR/LF vs LF/CR, and [2] breaking lines on the LF
allowed for overstriking.

-Paul W.

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[ron minnich]

using cr/lf as end of line also introduced all kinds of problems on the DEC
OSes I used, to wit, what do
CR alone
LF alone
and LF/CR

mean? It was just a pain in the neck. I was grateful to get to an OS that
was sensible and just used LF.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170904/0f782b60/attachment.html>

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Bakul Shah]

CR by itself could be used for overstrike (e.g. if you wanted to underline
words). Faster than char, backspace, overstrike-char if you wanted lots
of this. LF by itself, not so much. Could've been if there was an inverse
linefeed.

> On Sep 4, 2017, at 1:30 PM, ron minnich <rminnich at gmail.com> wrote:
> 
> using cr/lf as end of line also introduced all kinds of problems on the DEC OSes I used, to wit, what do
> CR alone
> LF alone
> and LF/CR
> 
> mean? It was just a pain in the neck. I was grateful to get to an OS that was sensible and just used LF.

[TUHS] Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]

[Noel Chiappa]

    > From: Chris Torek

    > termcap has entries for the number of NUL characters to insert after
    > carriage return.

Actually, the stock terminal driver in V6 Unix (here:

  http://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/sys/dmr/tty.c

if anyone wants to see the actual code; it's in ttyoutput()) had some pretty
complex code to do just the right amount of delay (in clock ticks) for a large
number of different motion contral characters (TAB, etc, in addition to LF and
CR), and then uses the system timer to delay that amount of real time after
sending such a character (see the very bottom of ttstart()).

E.g. for a NL, it used the fact that it 'knew' which column the print head was
in to calculate the exact return time.

Clever, but alas, it did this by sticking 'characters' in the buffered output
stream with the high bit set, and the delay required in the low 0177 bits,
which the output start routine interpreted; as the code drolly notes, "thus
(unfortunately) restricting the transmission path to 7 bits". Which was a real
PITA recently when I tried to download a binary file to an LSI-11 from a V6
running in Ersatz-11! I had to tweak the TTY driver to allow 8-bit output...

   Noel