Re: [TUHS] screen editors / machine load

Re: [TUHS] screen editors / machine load

[Noel Chiappa]

    > When Bernie Greenberg did EMACS for Multics, he had a similar issue. I
    > recall reading a document with an extensive discussion of how they dealt
    > with this ... If anyone's really interested in this, and can't find it
    > themselves, I can try looking for it.

I got a request for this; a Web search turned up:

  https://www.multicians.org/mepap.html

which covers the points I mentioned (and more besides, such as why LISP was
chosen). I don't think this is the thing I remembered (which was, IIRC, an
informal note), but it does seem to be a later version of that.

	 Noel

Re: [TUHS] screen editors / machine load

[Christopher Browne]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

On Fri, 10 Jan 2020 at 17:19, Adam Thornton <athornton@gmail.com> wrote:

> A) The original WD UART chip had very limited buffering.   The timing was
> such that as high rates it could not empty accept a second character
> without the first being overwritten.  This was a long-standing issue for
> many UARTs long in the 1990s.  The original chip NS built and IBM used on
> the PC (the NS8250) was notorious for the same problem.  By the time of
> Motorola's 6881, it had 8 characters of buffering IIRC.
>
> Great, now I'm having flashbacks to upgrading my 4-port serial card with
> 16450s and then 16550s in the early 90s.
>

Yep, same sorts of memories here.  I recall the 16450 upgrade being a big
deal for Internet connectivity in that a PC lacking the extra bytes of
buffering in the UART would find that the 80386 was having clock cycles
eaten nearly completely by PPP connections.
It was amazing to realize how a few bytes of memory lurking in a crucial
system interface could affect performance in such dramatic ways.
Tagged command queueing on SCSI controllers had a slightly less dramatic
effect on I/O performance, but again, a few hundred bytes of memory in the
right spot could nevertheless have dramatic effects.

[-- Attachment #2: Type: text/html, Size: 1699 bytes --]

Re: [TUHS] screen editors / machine load

[Adam Thornton]

[-- Attachment #1: Type: text/plain, Size: 3899 bytes --]

A) The original WD UART chip had very limited buffering.   The timing was
such that as high rates it could not empty accept a second character
without the first being overwritten.  This was a long-standing issue for
many UARTs long in the 1990s.  The original chip NS built and IBM used on
the PC (the NS8250) was notorious for the same problem.  By the time of
Motorola's 6881, it had 8 characters of buffering IIRC.

Great, now I'm having flashbacks to upgrading my 4-port serial card with
16450s and then 16550s in the early 90s.


On Fri, Jan 10, 2020 at 8:49 AM Clem Cole <clemc@ccc.com> wrote:

>
>
> On Fri, Jan 10, 2020 at 10:00 AM Mary Ann Horton <mah@mhorton.net> wrote:
>
>> Yes, it was a real concern. Physical memory on the shared PDP-11 was
>> limited, and if everyone had a separate copy of vi running the machine
>> would swap itself silly.
>>
>> This only mattered if everyone had their own separate copy of vi
>> installed. The fix was to put vi in a single system directory, such as
>> /usr/ucb or /exptools. The instruction part of its memory would be
>> shared among all the users, resulting in much less swapping.
>>
> Actually it was much worse than that...
>
> What Mary Ann points out was mostly true of your PDP-11 had DH11's
> installed; which had deeper hardware buffering and 16 character DMA on
> output.   But these were expensive from DEC and also took up a 'full system
> unit' in the CPU for 16 lines.   Until Able (much later) released the
> DMAX-11 (*a.k.a.* DH/DM) product of a DH11 clone on a single board, many
> university sites did not use them; but used multiple DL-11/KL-11's instead.
>
> If your system was configured with DL/KL11s or similar (CMU had it's own
> called 'ASLIs' - a synchronous line interfaces) each character took one
> interrupt for each either input or output.  Moreover, the UARTS that DEC
> used which were made by Western Digital had 2 >>or less<< characters of
> input buffering, so they could drop chars[1].  The ASLI's used a later chip
> with a slightly better buffer IIRC but I admit I've forgotten the details
> (Jim Tetter probably remembers them).
>
> So if you had a single line, the interrupt load was huge on a PDP-11.  For
> this reason, a lot of sites limited glass TTYs to speeds like 2400 or 4800
> baud, not the full 9600.
>
> DEC later released the DZ-11 which worked on units of 8 ports per board.
> Unfortunately, it was not DMA and the buffering was still pretty shallow.
>  Joy did a lot of work on 4.1BSD in the DZ driver to cut down the
> interrupts because 9600 baud DZ lines could swamp a vax and when running
> the BerkNet between systems (before UCB had ethernet), 9600 baud serial
> lines were standard.
>
>
> [1]  Two things
>  A) The original WD UART chip had very limited buffering.   The timing was
> such that as high rates it could not empty accept a second character
> without the first being overwritten.  This was a long-standing issue for
> many UARTs long in the 1990s.  The original chip NS built and IBM used on
> the PC (the NS8250) was notorious for the same problem.  By the time of
> Motorola's 6881, it had 8 characters of buffering IIRC.
>
> B) As I understand the history, Gordon developed the original idea of the
> UART at DEC for the PDP-1. But I'm not sure of the patent details. He does
> not list the UART patent on his web site although he does mention inventing
> it.   I have been under the impression CMU was somehow mixed up in the
> patent and licensing of it, *i.e.* WD got the license from CMU to make
> them not DEC; which was part of why we had the ASLI's.  Again, IIRC, we got
> the UART chips from WD at cost and could make the ALSI's locally much
> cheaper than DL-11s.  >>I think<< the story was that one of Gordon's
> student's designed a chip, which WD fabbed and licensed.  Before that DEC
> had built UARTs on boards from transistors and later logic gates.
>
>

[-- Attachment #2: Type: text/html, Size: 5996 bytes --]

Re: [TUHS] screen editors / machine load

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3191 bytes --]

On Fri, Jan 10, 2020 at 10:00 AM Mary Ann Horton <mah@mhorton.net> wrote:

> Yes, it was a real concern. Physical memory on the shared PDP-11 was
> limited, and if everyone had a separate copy of vi running the machine
> would swap itself silly.
>
> This only mattered if everyone had their own separate copy of vi
> installed. The fix was to put vi in a single system directory, such as
> /usr/ucb or /exptools. The instruction part of its memory would be
> shared among all the users, resulting in much less swapping.
>
Actually it was much worse than that...

What Mary Ann points out was mostly true of your PDP-11 had DH11's
installed; which had deeper hardware buffering and 16 character DMA on
output.   But these were expensive from DEC and also took up a 'full system
unit' in the CPU for 16 lines.   Until Able (much later) released the
DMAX-11 (*a.k.a.* DH/DM) product of a DH11 clone on a single board, many
university sites did not use them; but used multiple DL-11/KL-11's instead.

If your system was configured with DL/KL11s or similar (CMU had it's own
called 'ASLIs' - a synchronous line interfaces) each character took one
interrupt for each either input or output.  Moreover, the UARTS that DEC
used which were made by Western Digital had 2 >>or less<< characters of
input buffering, so they could drop chars[1].  The ASLI's used a later chip
with a slightly better buffer IIRC but I admit I've forgotten the details
(Jim Tetter probably remembers them).

So if you had a single line, the interrupt load was huge on a PDP-11.  For
this reason, a lot of sites limited glass TTYs to speeds like 2400 or 4800
baud, not the full 9600.

DEC later released the DZ-11 which worked on units of 8 ports per board.
Unfortunately, it was not DMA and the buffering was still pretty shallow.
 Joy did a lot of work on 4.1BSD in the DZ driver to cut down the
interrupts because 9600 baud DZ lines could swamp a vax and when running
the BerkNet between systems (before UCB had ethernet), 9600 baud serial
lines were standard.


[1]  Two things
 A) The original WD UART chip had very limited buffering.   The timing was
such that as high rates it could not empty accept a second character
without the first being overwritten.  This was a long-standing issue for
many UARTs long in the 1990s.  The original chip NS built and IBM used on
the PC (the NS8250) was notorious for the same problem.  By the time of
Motorola's 6881, it had 8 characters of buffering IIRC.

B) As I understand the history, Gordon developed the original idea of the
UART at DEC for the PDP-1. But I'm not sure of the patent details. He does
not list the UART patent on his web site although he does mention inventing
it.   I have been under the impression CMU was somehow mixed up in the
patent and licensing of it, *i.e.* WD got the license from CMU to make them
not DEC; which was part of why we had the ASLI's.  Again, IIRC, we got the
UART chips from WD at cost and could make the ALSI's locally much cheaper
than DL-11s.  >>I think<< the story was that one of Gordon's student's
designed a chip, which WD fabbed and licensed.  Before that DEC had built
UARTs on boards from transistors and later logic gates.

[-- Attachment #2: Type: text/html, Size: 5009 bytes --]

Re: [TUHS] screen editors / machine load

[Mary Ann Horton]

Yes, it was a real concern. Physical memory on the shared PDP-11 was 
limited, and if everyone had a separate copy of vi running the machine 
would swap itself silly.

This only mattered if everyone had their own separate copy of vi 
installed. The fix was to put vi in a single system directory, such as 
/usr/ucb or /exptools. The instruction part of its memory would be 
shared among all the users, resulting in much less swapping.

In the early days, people tended to have their own personal copy because 
the Berkeley tools did not come standard with UNIX, especially at Bell 
Labs. That was one of the main motivations for Exptools (the 
"experimental tools"), which were basically 2BSD's applications and some 
other tools like Warren Montgomery's emacs. Disk space and people's time 
spend installing were also good reasons.

     Mary Ann

On 1/10/20 5:41 AM, Mike Markowski wrote:
> seeing him fire up vi was practically sci-fi to me.  He showed me a 
> few commands and vowed me to secrecy for fear if all students started 
> using it, it would bring the 11/70 to its knees.  Were multiple vi 
> sessions really such a potential burden to the machine?  I wouldn't 
> think so with the slow nature of human i/o, yet there certainly were 
> times when the pdp-11/70 crashed as project due dates loomed closer 
> and closer!

Re: [TUHS] screen editors / machine load

[Noel Chiappa]

    > From: Otto Moerbeek <otto@drijf.net>

    > I believe it was not only vi itself that was causing the load, it was
    > also running many terminals in raw mode that killed performance.

I'm not familiar with the tty driver in late versions of Unix like 4.1 (sic),
but I'm very familiar with the one in V6, and it's not the raw mode _itself_
that caused the load (the code paths in the kernel for cooked and raw aren't
that different), but rather the need to wake up and run a process on every
character that was the real load.

When Bernie Greenberg did EMACS for Multics, he had a similar issue. I recall
reading a document with an extensive discussion of how they dealt with this,
especially when using the system over the ARPANET. IIRC, normal printing
characters were echoed without waking up the process; remotely, when using
the network. If anyone's really interested in this, and can't find it themselves,
I can try looking for it.

	Noel

Re: [TUHS] screen editors / machine load

[Otto Moerbeek]

On Fri, Jan 10, 2020 at 08:41:53AM -0500, Mike Markowski wrote:

> [2020-01-09 08:49] Dave Horsfall <dave@horsfall.org>
> > > I had a boss once who insisted that all his staff learn "ed",
> > > because one day it might be the only editor available; he was
> > > right...
> >
> 
> I first used Unix on a pdp-11/70 in 1981, first year at university.  My
> professor stopped by the computing center to see how his students were
> doing - super nice of him and a perk to pre-PC times! - and was showing me
> something or other regarding Unix.  I had only used ed to that point and
> seeing him fire up vi was practically sci-fi to me.  He showed me a few
> commands and vowed me to secrecy for fear if all students started using it,
> it would bring the 11/70 to its knees.  Were multiple vi sessions really
> such a potential burden to the machine?  I wouldn't think so with the slow
> nature of human i/o, yet there certainly were times when the pdp-11/70
> crashed as project due dates loomed closer and closer!
> 
> Also, I very much enjoy this list.  As an EE I use Unix-like OSes as a tool
> rather being a builder of the tool like many here.  So I don't have the
> deep background to contribute to the collective history, but I'm on the
> sidelines enjoying the show.  As a brief tie-in to the editor comparisons,
> I do a lot of DSP work for RF systems these days.  Python makes it quick
> and easy to try new math, but has a maddening requirement that indentation
> be strictly tabs or strictly spaces.  Text window pasting into a tab
> indented python file wreaks havoc.  vim yank/put between split windows
> retains the type of white space and lets me use my vi muscle memory.
> 
> Happy 2020,
> Mike Markowski

In my first year at university (1984) we had a VAX-11/750 running
4.1BSD with too many students. We had to switch to ex once in a while
to get any editing done. I believe it was not only vi itself that was
causing the load, it was also running many terminals in raw mode that
killed performance.

	-Otto

Re: [TUHS] screen editors / machine load

[Mike Markowski]

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

[2020-01-09 08:49] Dave Horsfall <dave@horsfall.org>
> > I had a boss once who insisted that all his staff learn "ed",
> > because one day it might be the only editor available; he was
> > right...
>

I first used Unix on a pdp-11/70 in 1981, first year at university.  My
professor stopped by the computing center to see how his students were
doing - super nice of him and a perk to pre-PC times! - and was showing me
something or other regarding Unix.  I had only used ed to that point and
seeing him fire up vi was practically sci-fi to me.  He showed me a few
commands and vowed me to secrecy for fear if all students started using it,
it would bring the 11/70 to its knees.  Were multiple vi sessions really
such a potential burden to the machine?  I wouldn't think so with the slow
nature of human i/o, yet there certainly were times when the pdp-11/70
crashed as project due dates loomed closer and closer!

Also, I very much enjoy this list.  As an EE I use Unix-like OSes as a tool
rather being a builder of the tool like many here.  So I don't have the
deep background to contribute to the collective history, but I'm on the
sidelines enjoying the show.  As a brief tie-in to the editor comparisons,
I do a lot of DSP work for RF systems these days.  Python makes it quick
and easy to try new math, but has a maddening requirement that indentation
be strictly tabs or strictly spaces.  Text window pasting into a tab
indented python file wreaks havoc.  vim yank/put between split windows
retains the type of white space and lets me use my vi muscle memory.

Happy 2020,
Mike Markowski

[-- Attachment #2: Type: text/html, Size: 2017 bytes --]