[TUHS] History of top

[TUHS] History of top

[Noel Chiappa]

    > From: Lars Brinkhoff

    > I'm surprised it appeared that late. Were there any other versions or
    > similar Unix programs before that?

The MIT ~PWB1 system had a thing called 'dpy', I think written at MIT based on
'ps' (and no doubt inspired by ITS' PEEK), which had similar functionality.
Seems like it never escaped, though. Man page and source here:

  http://ana-3.lcs.mit.edu/~jnc/tech/unix/man1/dpy.1
  http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/dpy.c

The top of my hard-copy man page says 'November 1977', but I suspect it dates
back further than that.

     Noel

[TUHS] History of top

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3634 bytes --]

On Thu, May 24, 2018 at 5:20 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>     > From: Lars Brinkhoff
>
>     > I'm surprised it appeared that late. Were there any other versions or
>     > similar Unix programs before that?
>
> The MIT ~PWB1 system had a thing called 'dpy', I think written at MIT
> based on
> 'ps' (and no doubt inspired by ITS' PEEK), which had similar functionality.
> Seems like it never escaped, though. Man page and source here:
>
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man1/dpy.1
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/dpy.c
>
> The top of my hard-copy man page says 'November 1977', but I suspect it
> dates
> back further than that.
>
>      Noel
>
> There were a bunch of these​ - originally mods off of ps(1) that kicked
around at different UNIX sites.   CMU's locally hacked ps was similar to
dpy and BSD's *stat programs (systatm vmstat, ...). The problem all of
these had was getting the data out of the OS.   The original scheme used by
ps being it ran setuid/gid, and then was able to opening /dev/kmem and
reading interesting data structures the kernel.  A big problem was that ps
and the OS had to be compiled with exactly the same headers and you really
had to know the layout and the obvious security concerns of using
setuid/gid to control things.

FreeBSD's sysctl and V8's /proc would finally clean that up by making a
formal interface to the OS and controlling what was available in a more
granular manner both from a data structure as well as security path.
 Although before /proc came to being (/proc was a UNIX innovation), the
sysctl(2) like interface was reinvented a few times, as it really was not
new with UNIX.   It's very much like a similar scheme that came from TOPS,
RSX and VMS (which the Masscomp system parroted being ex-DECies almost
exactly the same as VMS).  I believe that Sun had something also, but  I've
forgotten its name, Larry might remember.   But so did  DG-OS and many of
the other commercial UNIXs snd I would not be surprised if many University
hacks did it also.

My favorite of all of these was the one that George Gobble at Purdue built
when he did the Dual VAX.   He also spliced an PDP-11 on the UBA on the
their Dual Vax System and wrote a program that like top(1) refreshed itself
(at least as fast as once a sec IIRC).  It would read the VAX memory and
then display what the OS was doing in real time.   He did a neat talk in a
mid-1980's USENIX with a movie from it.   They found an interesting bug in
the BSD scheduler which had been there since 6th edition, which had never
really been noticed until people could visually watch what as happening and
actually get enough information about how processes were being handled.
[This was also around the time one of the first '0-day' issues with UNIX
was found by ghg and team which is now a cute story but at the time was
quite worrisome].


​The best I remember, the popularity of top(1) started because it was first
released as USENET came of age; and thus it got wide distribution.  By that
point, ​ *BSD was the 'standard' in most places, and this the *stat
commands were pretty well known. The top(1) program consolidated these into
a single one, plus it used the curses library so it could be left running
on a glass TTY.   As Vaxen were being used as large timesharing systems for
Universities to teach with, such a program was handy for the sys admins to
keep an eye on what was happening.

Clem

ᐧ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20180524/59f9e2a6/attachment.html>

[TUHS] History of top

[Ronald Natalie]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 587 bytes --]


I remember George’s Unibus monitoring hack.    It was noted at the time that he didn’t have any bus terminators.   The SBI had CPUs on both ends and the Unibus had the monitoring 11/23 at the far end.

We built one of the Dual vaxes at BRL (BRL-VGR).   It was pretty straight forward, just needed to obtain (well we had it fabbed) a new cardcage and redo a special set of SBI cables to be an mirror image.
It was amazing how cheap this was compared to the 782 abomination (not to even mention the VaxCluster and it’s horrendous HSC50 that I ended up administering later at RU).

[TUHS] Control-T (was top)

[Ronald Natalie]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]


Another related creeping featurism was the integration of the control-T feature that was prevalent in many of the DEC 10/20 OSs.
When Mike Muuss was doing some scheduling tweaks he decided that being able to list out all the processes on the terminal with a few parameters (he had the weighted load averages at the top of the list).
It became real handy for me when I set out to reverse engineer how the Csh used job control as well.

For those who aren’t old time kernel hacks, there are two structures associated with every process.   The “proc” structure has essential things like uid/gid and memory sizes and priority.   The proc structure always was resident making it ripe pickings for control-T.    More detailed information including the stack of the process (and hence the command line arguments), were in the user structure.    While the user structure of the currently running process was always resident (and on the PDP-11 at 140000, which meant you could use the display register to display the UID to figure out who the CPU hog was).    This made it harder for the “ps”-like things to get.    I finally relented and put a copy of the first few bytes of the arg list in the proc structure.

[TUHS] Control-T (was top)

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]

below...

On Thu, May 24, 2018 at 7:50 AM, Ronald Natalie <ron at ronnatalie.com> wrote:

>
> Another related creeping featurism was the integration of the control-T
> feature that was prevalent in many of the DEC 10/20 OSs.
>
​Yeah I still sometimes missed that hack.  Zimmerman (of EMACS fame) put it
into the Masscomp tty handler.  He may have had it in an MIT system.   That
was the best UNIX version I ever used.​

That said, with a window manager on my modern Mac, I just leave 'Activity
Monitor' running, so I don't need it like I did when we ran on glass ttys.

​Actually, my favorite Control-T story is not from UNIX, but from TwinEx.
 When you typed Control-T on that system, it would reply with '*running,
load average, ...mumble stats, more mumble and more stats*.'   In the late
1970s, Stanford had a DEC System-20 called 'LOTS' which they bought for the
undergrads for teaching.​   Somebody (who will be left nameless) modified
the OS so the 'running' part changed depending on value of the load
average.  * i.e.* running, jogging, walking, crawling.   But the
administration made them remove the hack when it came back as 'dying' most
of the time.

Clem

ᐧ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20180524/16c67570/attachment.html>

[TUHS] Control-T (was top)

[Lars Brinkhoff]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

Clem Cole wrote:
> Stanford had a DEC System-20 called 'LOTS' which they bought for the
> undergrads for teaching.​ Somebody (who will be left nameless) modified
> the OS so the 'running' part changed depending on value of the load
> average. i.e.  running, jogging, walking, crawling.

Another PDP-10 operating system (from MIT) had this:
Warp (fastest), zoom, fly, run, walk, tenex, multix (slowest).

[TUHS] Control-T (was top)

[Arthur Krewat]

On 5/24/2018 10:50 AM, Ronald Natalie wrote:
> Another related creeping featurism was the integration of the control-T feature that was prevalent in many of the DEC 10/20 OSs.
Control-T in TOPS-10 was done straight out of the monitor. If you were 
on a slow 300 baud or 110 baud terminal on say a DZ11*, and kept hitting 
Control-T really fast, you'd slow down everything else on the machine as 
there was no buffering of serial TTY I/O and the monitor would sit and 
wait for each character to be transmitted.

ak

* a DZ11 is UNIBUS and was only available on a KS10 - but I'd imagine 
any straight serial port on a KA/KI/KL10 would do the same. If you were, 
say, on a DCA mux, that had a 14.4K serial link straight into the 
UNIBUS, there was much less of an effect.

[TUHS] Control-T (was top)

[Paul Winalski]

On 5/24/18, Arthur Krewat <krewat at kilonet.net> wrote:
>
> * a DZ11 is UNIBUS and was only available on a KS10 - but I'd imagine
> any straight serial port on a KA/KI/KL10 would do the same. If you were,
> say, on a DCA mux, that had a 14.4K serial link straight into the
> UNIBUS, there was much less of an effect.

DZ11s were available on PDP-11s and VAXen as well as the KS10.
Because the controller had no buffer it was dirt cheap.  OK, perhaps,
for a realtime or lab application, but if you were doing timesharing
it could bring the CPU to its knees in short order.  Some idiot
decided to use the same idea (no buffer in the controller) to reduce
the cost of the TU58 (aka DECtape II),  Unlike the original DECtape,
which was extremely reliable, the TU58 was prone to frequent data
overrun and underrun errors even if you set things up such that it got
top scheduler priority.

-Paul W.

[TUHS] Control-T (was top)

[Clem cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

And the other issue was besides not enough buffering DZ11s and the TU58 assumes  software (^S/^Q) for the flow control (DH11s with the DM11 option has hardware (RTS/CTS) to control the I/O rate.  So this of course made 8bit data diificult too. And as Paul points out, with HW flow the priority could have been lower. But the HW folks assumed a so called 3 wire interface because they thought they were saving money.  

A few years later when people tried to put high speed modems like the Trailblazers on them, let’s say the Unix folks quickly abandoned any hope.   Ken O’Humundro at Able Computer has quite a business in his ‘DHDM’ board that was cheaper than the DZ, more ports, full DMA and of course full hardware flow control.  

Btw it was not just DEC.  The original hw version of Masscomp MC500 had the same issue in cpu board brcause Jeff Mitchell (who had been one the 11/34 designer) did not know any better. I fixed the MPU board when it was relaided out and our first 8 port serial controller had rewritten microcode from the original OEM to allow trailblazers to work right (fortunately then HW was there on the serial board but it was just being ignored in the original firmware).

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

> On May 28, 2018, at 6:32 PM, Paul Winalski <paul.winalski at gmail.com> wrote:
> 
>> On 5/24/18, Arthur Krewat <krewat at kilonet.net> wrote:
>> 
>> * a DZ11 is UNIBUS and was only available on a KS10 - but I'd imagine
>> any straight serial port on a KA/KI/KL10 would do the same. If you were,
>> say, on a DCA mux, that had a 14.4K serial link straight into the
>> UNIBUS, there was much less of an effect.
> 
> DZ11s were available on PDP-11s and VAXen as well as the KS10.
> Because the controller had no buffer it was dirt cheap.  OK, perhaps,
> for a realtime or lab application, but if you were doing timesharing
> it could bring the CPU to its knees in short order.  Some idiot
> decided to use the same idea (no buffer in the controller) to reduce
> the cost of the TU58 (aka DECtape II),  Unlike the original DECtape,
> which was extremely reliable, the TU58 was prone to frequent data
> overrun and underrun errors even if you set things up such that it got
> top scheduler priority.
> 
> -Paul W.

[TUHS] Control-T (was top)

[Arthur Krewat]

On 5/28/2018 7:11 PM, Clem cole wrote:
> A few years later when people tried to put high speed modems like the Trailblazers on them
Hence why, in the early 90's, I used an 80486DX2-66 PC with 16550 UARTs 
for modems instead of the Sun IPC to do USENET ;)

[TUHS] Control-T (was top)

[Dave Horsfall]

On Mon, 28 May 2018, Paul Winalski wrote:

> DZ11s were available on PDP-11s and VAXen as well as the KS10. Because 
> the controller had no buffer it was dirt cheap.  OK, perhaps, for a 
> realtime or lab application, but if you were doing timesharing it could 
> bring the CPU to its knees in short order.  Some idiot decided to use 
> the same idea (no buffer in the controller) to reduce the cost of the 
> TU58 (aka DECtape II), Unlike the original DECtape, which was extremely 
> reliable, the TU58 was prone to frequent data overrun and underrun 
> errors even if you set things up such that it got top scheduler 
> priority.

Odd; I have a clear recollection that UNSW's driver (or was it Basser?) 
did not use interrupts (too slow, as you point out) but used the clock 
interrupt to empty the silos every so often.  I'd check the source in the 
Unix Archive, but I don't remember which disk image it's in; what I do 
remember was that it went like a bat out of hell, and Elec Eng had 
something like eight of them on their 11/70.

Can anyone confirm or deny this?

We also had a DZ-11 on our 11/60 (and the 11/40 before that), and it 
worked OK, keeping the DJ-11 company.

-- Dave, possibly suffering from mental bit-rot

[TUHS] History of top

[Lars Brinkhoff]

> The MIT ~PWB1 system had a thing called 'dpy', I think written at MIT
> based on 'ps' (and no doubt inspired by ITS' PEEK)

Indeed my hidden agenda was to find a link between top and PEEK.
But apparently the inspiration came from VMS:

http://web.archive.org/web/20170527120123/http://www.unixtop.org:80/about.shtml

[TUHS] Control-T (was top)

[Noel Chiappa]

    > From: Paul Winalski

    > DZ11s ... the controller had no buffer

Huh? The DZ11 did have an input buffer. (See the 'terminals and communications
handbook', 1978-79 edition, page 2-238: "As each character is received ...
the data bits are placed ... in a .. 64-word deep first-in/first-out hardware
buffer, called a 'silo'.")

Or did you mean output:

    > if you were doing timesharing it could bring the CPU to its knees in
    > short order

The thing that killed an OS was the fact that output was programmed I/O, a
character at a time; using interrupt-driven operation, it took an interrupt
per character. So for a 9600 baud line, 9 bits/character (1 start + 7 data + 1
stop - depending on the line configuration), that's about 1000 characters per
second -> 1000 interrupts per second.

The DH11 used DMA for output, and was much easier on the machine.

	Noel

[TUHS] Control-T (was top)

[Paul Winalski]

On 5/28/18, Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:
>
> The thing that killed an OS was the fact that output was programmed I/O, a
> character at a time; using interrupt-driven operation, it took an interrupt
> per character.
>
> The DH11 used DMA for output, and was much easier on the machine.

Yes, I meant output.  Thanks for refreshing my hazy memory.  I found
the TU58 technical manual online.  It turns out that the TU58 did have
a data buffer, but it's only 128 bytes, so transferring each 512-byte
block required multiple messages and interrupts, and that introduces
the overrun/underrun problems.

Most large VAX timesharing systems used the DH11.

-Paul W.

[TUHS] Control-T (was top)

[Noel Chiappa]

    > From: Dave Horsfall

    > I have a clear recollection that UNSW's driver (or was it Basser?)  did
    > not use interrupts .. but used the clock interrupt to empty the silos
    > every so often. I'd check the source in the Unix Archive, but I don't
    > remember which disk image it's in ... Can anyone confirm or deny this?

I found this one:

  https://minnie.tuhs.org//cgi-bin/utree.pl?file=AUSAM/sys/dmr/dz.c

which seems to be the one you're rhinking of, or close to it.

It actually does use interrupts, on both sides - sort of. On the input side,
it uses the 'silo alarm', which interrupts when the input buffer has 16
characters in it. This has the same issue as the silo on the DH11 - if there
are less characters than that waiting, the host never gets an interrupt. Which
may be why it does the timer-based input check also?

The output side is entirely interrupt driven; it _does_ reduce the number of
interrupts by checking _every_ output line (on every DZ11 in the machine) to
see if that line's ready for a character when it gets any output interrupt,
which will definitely seriously reduce the number of output interrupts - but
even then, if _one_ line is going flat out, that's still 1000 interrupts per
second.

	Noel

[TUHS] Control-T (was top)

[Dave Horsfall]

On Tue, 29 May 2018, Noel Chiappa wrote:

> I found this one:
>
>  https://minnie.tuhs.org//cgi-bin/utree.pl?file=AUSAM/sys/dmr/dz.c
>
> which seems to be the one you're rhinking of, or close to it.

Not sure; I'm positive that there were two drivers: the one above, and,
aha!  One called dz.c-elec, which was the one that I meant.

Ah yes:

/*
  *	DZ-11 driver
  *	------------
  *
  *	OPTIMIZED dz driver to handle multiple dzs as efficiently
  *	as possible.  The efficiency is gained by disabling all
  *	dz transmitter interrupts and using a KW11-P to generate
  *	suitable interrupts.  Carrier is supported but not Ring.
  *
  *				Ian Johnstone	UNSW
  *				May 1979
  */

Note this bit:

     #define	NDZ		 7		/* no. of dz-11s */

As I said, it went like a bat out of hell :-)  You'd be nuts to use
the interrupt per xmit char...

-- Dave

[TUHS] Control-T (was top)

[Norman Wilson]

Back in 1980 or 1981, when I first started hacking
on UNIX but still had some TOPS-10 DNA lingering in
my blood, I put in a really simple control-T
implementation.  Control-T became a new signal-
generating character in the tty driver; it sent
signal 16.  Unlike interrupt and quit, it did not
flush input or output buffers.  Unlike any other
signal, SIG_DFL caused the signal to be silently
ignored.  (I don't remember why I didn't just teach
/etc/init and login to set that signal to SIG_IGN
by default; maybe I looked and found too many other
programs that monkeyed with every signal, maybe I
just didn't think of it.)

I then wrote a little program meant to be run in the
background from .profile, that dug around in /dev/kmem,
figured out what was likely nearest-to-foreground process
associated with the same terminal, and printed a little
status info for that process.

It didn't take long for the remaining TOPS-10 DNA to
leach away, and besides it is much easier to run some
program in another window now that that is almost always
possible, so I don't miss it.  But I like that idea
better than, in effect, hacking a mini-ps into the kernel,
even though the kernel doesn't have to do as much work
to get the data.

I also thought it made more sense to have a general
mechanism that could be used for other things.  That
even happened once.  The systems I ran were used, among
other things, for developing SMP, the symbolic-manipulation
interpreter worked on by Stephen Wolfram, Geoffrey Fox,
Chris Cole, and a host of graduate and undergraduate students.
(My memory of who deserves credit differs somewhat from
that of at least one person named.)  SMP, by its nature,
sometimes had to spend a substantial time sitting and
computing.  Someone (probably Wolfram, says my aging
memory) heard about the control-T stuff, asked me how
to use it, and added code to SMP so that during a long
computation control-T would tell you something about
what it was doing and how it was progressing.

Since the signal was, like interrupt and kill, sent
to the whole process group, there was no conflict if
you also had my little control-T monitor running in
the background.

I never tried to send my hacked-up UNIX to anyone else,
so if anyone else did the same sort of control-T hack,
they likely invented it independently.

Norman Wilson
Toronto ON

[TUHS] Control-T (was top)

[arnold]

There was a patch floating around circa 1984 that did Control-T
as a mini-ps inside the kernel. We put it into our Vax running
4.2 at Georgia Tech and then later I put it into the Vaxen
I ran at the Emory U. Computing Center running 4.3 + NFS fromm
Mt. Xinu.

It was a cute thing to have.  One of the faculty at Georgia Tech
who had come from MIT saw it and was totally surprised it was
in Unix. :-)

Lord only knows where we got it from. Probably Unix Wizards
or something ...

Arnold

Norman Wilson <norman at oclsc.org> wrote:

> Back in 1980 or 1981, when I first started hacking
> on UNIX but still had some TOPS-10 DNA lingering in
> my blood, I put in a really simple control-T
> implementation.  Control-T became a new signal-
> generating character in the tty driver; it sent
> signal 16.  Unlike interrupt and quit, it did not
> flush input or output buffers.  Unlike any other
> signal, SIG_DFL caused the signal to be silently
> ignored.  (I don't remember why I didn't just teach
> /etc/init and login to set that signal to SIG_IGN
> by default; maybe I looked and found too many other
> programs that monkeyed with every signal, maybe I
> just didn't think of it.)
>
> I then wrote a little program meant to be run in the
> background from .profile, that dug around in /dev/kmem,
> figured out what was likely nearest-to-foreground process
> associated with the same terminal, and printed a little
> status info for that process.
>
> It didn't take long for the remaining TOPS-10 DNA to
> leach away, and besides it is much easier to run some
> program in another window now that that is almost always
> possible, so I don't miss it.  But I like that idea
> better than, in effect, hacking a mini-ps into the kernel,
> even though the kernel doesn't have to do as much work
> to get the data.
>
> I also thought it made more sense to have a general
> mechanism that could be used for other things.  That
> even happened once.  The systems I ran were used, among
> other things, for developing SMP, the symbolic-manipulation
> interpreter worked on by Stephen Wolfram, Geoffrey Fox,
> Chris Cole, and a host of graduate and undergraduate students.
> (My memory of who deserves credit differs somewhat from
> that of at least one person named.)  SMP, by its nature,
> sometimes had to spend a substantial time sitting and
> computing.  Someone (probably Wolfram, says my aging
> memory) heard about the control-T stuff, asked me how
> to use it, and added code to SMP so that during a long
> computation control-T would tell you something about
> what it was doing and how it was progressing.
>
> Since the signal was, like interrupt and kill, sent
> to the whole process group, there was no conflict if
> you also had my little control-T monitor running in
> the background.
>
> I never tried to send my hacked-up UNIX to anyone else,
> so if anyone else did the same sort of control-T hack,
> they likely invented it independently.
>
> Norman Wilson
> Toronto ON

[TUHS] Control-T (was top)

[Johnny Billquist]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]

On 2018-05-29 04:00, Clem cole<clemc at ccc.com> wrote:
> 
> And the other issue was besides not enough buffering DZ11s and the TU58 assumes  software (^S/^Q) for the flow control (DH11s with the DM11 option has hardware (RTS/CTS) to control the I/O rate.  So this of course made 8bit data diificult too. And as Paul points out, with HW flow the priority could have been lower. But the HW folks assumed a so called 3 wire interface because they thought they were saving money.

Uh? Say what? What does XON/XOFF flow control have to do with 8bit data?

And no, DEC was not trying to save money. They were actually sticking to 
the standard. Something very few companies managed to do, especially 
when it came to the RS-232 standard.
The "hardware flow control" is actually an abuse of the half duplex 
signalling.
And it does not necessarily work as you might expect if modems are in 
the middle.
You could get by with just three wires on serial ports, but DEC usually 
did want people to have more wires connected, in order to detect when a 
device was connected or not. But no, they did not abuse the half duplex 
control to do hardware flow control. The DZ11 also have partial modem 
control. You cannot run them in half-duplex mode, and they do not 
support the secondary channel stuff, but they do support more than just 
the 3 wires.

> A few years later when people tried to put high speed modems like the Trailblazers on them, let’s say the Unix folks quickly abandoned any hope.   Ken O’Humundro at Able Computer has quite a business in his ‘DHDM’ board that was cheaper than the DZ, more ports, full DMA and of course full hardware flow control.

Yes. It's essentially a DH11 with a partial DM11 on just one board, 
while the original DH11/DM11 was a fully 9-slot backplane. So it was a 
really good alternative. The DH11 is much better than the DZ11, but did 
in fact cost more. The Able board gave the DH11 capabilities at a much 
lower cost, and taking much less space in the box. And it actually 
performed better than the DH11. But it did not support the full 
functionality of the DM11. But what was lacking was the half duplex 
stuff, which noone really cared about anymore anyway.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Control-T (was top)

[Clem cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4608 bytes --]

A few comments.  

1). If you grab ^s/^q from the alphabet it means you can send raw 8 bit data because they are valid characters (yes you can use escape chars but it gets very bad)

2) RTS/CTS flow signaling was very much part of the standard but it’s optional  and was used differently by the bell modem interface spec as you point out. I have the spec full somewhere.  It was originally in the ECMA standard for RJE stations actually.  IIRC it was the one if the European firms like ICL that created it.  Again IIRC MacNamera describes it also in his early19079s classic book on data com.  

3) DEC did not stick to the standard on the DZ11 neither ECMA interface nor the WE/Bell system standards. either Paul is absolutely right.  It drove MacNamera crazy as I believe he was the lead designer of the original DH which did (the topic came up at a party once).  WE212 modems worked on dz11 for input but just barely and could not use a WE dialer for output (who’s number I now forget). Remember the AT command dialer stuff was not originally a standard from the telcos.  The modems were independent of the dialer (which used the RS-4xx standard and I also have some where and I forget the number).  

15 WE212 Modems went into WE modem rack and the first slot was special and could support either a 16tg modem or a single WE dialer which was common the 15 modems in the rack.  

The logic to make it all work is well described and spelled out in the original UUCP code.

Btw Tom Truscott of UNC I believe built a home made dialer that snapped a relay over the pots line to simulate a dialer using a DR11C and did paper in the 1980 winter usenix in Boulder (aka the black hole) — at the time the AT modem stuff did not yet exist, but universities often did not use the WE standard modems of racks (Racal Vadic was popular) and the original dec dialer interface to the bell system stuff I think I remember was from DEC’s CSS group and not easy to get.  

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

> On May 29, 2018, at 6:45 PM, Johnny Billquist <bqt at update.uu.se> wrote:
> 
>> On 2018-05-29 04:00, Clem cole<clemc at ccc.com> wrote:
>> And the other issue was besides not enough buffering DZ11s and the TU58 assumes  software (^S/^Q) for the flow control (DH11s with the DM11 option has hardware (RTS/CTS) to control the I/O rate.  So this of course made 8bit data diificult too. And as Paul points out, with HW flow the priority could have been lower. But the HW folks assumed a so called 3 wire interface because they thought they were saving money.
> 
> Uh? Say what? What does XON/XOFF flow control have to do with 8bit data?
> 
> And no, DEC was not trying to save money. They were actually sticking to the standard. Something very few companies managed to do, especially when it came to the RS-232 standard.
> The "hardware flow control" is actually an abuse of the half duplex signalling.
> And it does not necessarily work as you might expect if modems are in the middle.
> You could get by with just three wires on serial ports, but DEC usually did want people to have more wires connected, in order to detect when a device was connected or not. But no, they did not abuse the half duplex control to do hardware flow control. The DZ11 also have partial modem control. You cannot run them in half-duplex mode, and they do not support the secondary channel stuff, but they do support more than just the 3 wires.
> 
>> A few years later when people tried to put high speed modems like the Trailblazers on them, let’s say the Unix folks quickly abandoned any hope.   Ken O’Humundro at Able Computer has quite a business in his ‘DHDM’ board that was cheaper than the DZ, more ports, full DMA and of course full hardware flow control.
> 
> Yes. It's essentially a DH11 with a partial DM11 on just one board, while the original DH11/DM11 was a fully 9-slot backplane. So it was a really good alternative. The DH11 is much better than the DZ11, but did in fact cost more. The Able board gave the DH11 capabilities at a much lower cost, and taking much less space in the box. And it actually performed better than the DH11. But it did not support the full functionality of the DM11. But what was lacking was the half duplex stuff, which noone really cared about anymore anyway.
> 
>    Johnny
> 
> -- 
> Johnny Billquist                  || "I'm on a bus
>                                  ||  on a psychedelic trip
> email: bqt at softjar.se             ||  Reading murder books
> pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Control-T (was top)

[Clem cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4849 bytes --]

Argh.  Can not send ... 8 bit data 

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

> On May 29, 2018, at 8:19 PM, Clem cole <clemc at ccc.com> wrote:
> 
> A few comments.  
> 
> 1). If you grab ^s/^q from the alphabet it means you can send raw 8 bit data because they are valid characters (yes you can use escape chars but it gets very bad)
> 
> 2) RTS/CTS flow signaling was very much part of the standard but it’s optional  and was used differently by the bell modem interface spec as you point out. I have the spec full somewhere.  It was originally in the ECMA standard for RJE stations actually.  IIRC it was the one if the European firms like ICL that created it.  Again IIRC MacNamera describes it also in his early19079s classic book on data com.  
> 
> 3) DEC did not stick to the standard on the DZ11 neither ECMA interface nor the WE/Bell system standards. either Paul is absolutely right.  It drove MacNamera crazy as I believe he was the lead designer of the original DH which did (the topic came up at a party once).  WE212 modems worked on dz11 for input but just barely and could not use a WE dialer for output (who’s number I now forget). Remember the AT command dialer stuff was not originally a standard from the telcos.  The modems were independent of the dialer (which used the RS-4xx standard and I also have some where and I forget the number).  
> 
> 15 WE212 Modems went into WE modem rack and the first slot was special and could support either a 16tg modem or a single WE dialer which was common the 15 modems in the rack.  
> 
> The logic to make it all work is well described and spelled out in the original UUCP code.
> 
> Btw Tom Truscott of UNC I believe built a home made dialer that snapped a relay over the pots line to simulate a dialer using a DR11C and did paper in the 1980 winter usenix in Boulder (aka the black hole) — at the time the AT modem stuff did not yet exist, but universities often did not use the WE standard modems of racks (Racal Vadic was popular) and the original dec dialer interface to the bell system stuff I think I remember was from DEC’s CSS group and not easy to get.  
> 
> Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 
> 
>>> On May 29, 2018, at 6:45 PM, Johnny Billquist <bqt at update.uu.se> wrote:
>>> 
>>> On 2018-05-29 04:00, Clem cole<clemc at ccc.com> wrote:
>>> And the other issue was besides not enough buffering DZ11s and the TU58 assumes  software (^S/^Q) for the flow control (DH11s with the DM11 option has hardware (RTS/CTS) to control the I/O rate.  So this of course made 8bit data diificult too. And as Paul points out, with HW flow the priority could have been lower. But the HW folks assumed a so called 3 wire interface because they thought they were saving money.
>> 
>> Uh? Say what? What does XON/XOFF flow control have to do with 8bit data?
>> 
>> And no, DEC was not trying to save money. They were actually sticking to the standard. Something very few companies managed to do, especially when it came to the RS-232 standard.
>> The "hardware flow control" is actually an abuse of the half duplex signalling.
>> And it does not necessarily work as you might expect if modems are in the middle.
>> You could get by with just three wires on serial ports, but DEC usually did want people to have more wires connected, in order to detect when a device was connected or not. But no, they did not abuse the half duplex control to do hardware flow control. The DZ11 also have partial modem control. You cannot run them in half-duplex mode, and they do not support the secondary channel stuff, but they do support more than just the 3 wires.
>> 
>>> A few years later when people tried to put high speed modems like the Trailblazers on them, let’s say the Unix folks quickly abandoned any hope.   Ken O’Humundro at Able Computer has quite a business in his ‘DHDM’ board that was cheaper than the DZ, more ports, full DMA and of course full hardware flow control.
>> 
>> Yes. It's essentially a DH11 with a partial DM11 on just one board, while the original DH11/DM11 was a fully 9-slot backplane. So it was a really good alternative. The DH11 is much better than the DZ11, but did in fact cost more. The Able board gave the DH11 capabilities at a much lower cost, and taking much less space in the box. And it actually performed better than the DH11. But it did not support the full functionality of the DM11. But what was lacking was the half duplex stuff, which noone really cared about anymore anyway.
>> 
>>   Johnny
>> 
>> -- 
>> Johnny Billquist                  || "I'm on a bus
>>                                 ||  on a psychedelic trip
>> email: bqt at softjar.se             ||  Reading murder books
>> pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Control-T (was top)

[Johnny Billquist]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 5318 bytes --]

Clem...

On 30.05.18 02:19, Clem cole wrote:
> A few comments.
> 
> 1). If you grab ^s/^q from the alphabet it means you can send raw 8 bit data because they are valid characters (yes you can use escape chars but it gets very bad)

But this has nothing to do with 8 bit data. It's equally true if you are 
using 7 bit data. It is totally disconnected from the 8th bit.
You sometimes also have problem with 8bit data because of parity 
fiddling and whatnot, and thus any mention of 8bit data usually actually 
means something related to the topic of getting the 8th bit though 
cleanly, which is not always the case. Heck, there are even internet 
protocols to this day that don't guarantee that they are 8bit clean.

What you are talking about is simply the problem when you have inband 
signalling, and is a totally different problem than 8bit data.

> 2) RTS/CTS flow signaling was very much part of the standard but it’s optional  and was used differently by the bell modem interface spec as you point out. I have the spec full somewhere.  It was originally in the ECMA standard for RJE stations actually.  IIRC it was the one if the European firms like ICL that created it.  Again IIRC MacNamera describes it also in his early19079s classic book on data com.

RTS/CTS signalling is certainly a part of the standard, but it is not 
meant for flow control. It is for half duplex and the way to signal when 
you want to change the direction of communication.

It is meant for *half duplex* communication. Not flow control.

> 3) DEC did not stick to the standard on the DZ11 neither ECMA interface nor the WE/Bell system standards. either Paul is absolutely right.  It drove MacNamera crazy as I believe he was the lead designer of the original DH which did (the topic came up at a party once).  WE212 modems worked on dz11 for input but just barely and could not use a WE dialer for output (who’s number I now forget). Remember the AT command dialer stuff was not originally a standard from the telcos.  The modems were independent of the dialer (which used the RS-4xx standard and I also have some where and I forget the number).

The DZ11 explicitly says it only have limited modem control. It does 
indeed not support half duplex communication.

The AT "standard" was never actually a standard. It's only a defacto 
standard, but is just something Hayes did, as I'm sure you know.

   Johnny

> 
> Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite.
> 
>> On May 29, 2018, at 6:45 PM, Johnny Billquist <bqt at update.uu.se> wrote:
>>
>>> On 2018-05-29 04:00, Clem cole<clemc at ccc.com> wrote:
>>> And the other issue was besides not enough buffering DZ11s and the TU58 assumes  software (^S/^Q) for the flow control (DH11s with the DM11 option has hardware (RTS/CTS) to control the I/O rate.  So this of course made 8bit data diificult too. And as Paul points out, with HW flow the priority could have been lower. But the HW folks assumed a so called 3 wire interface because they thought they were saving money.
>>
>> Uh? Say what? What does XON/XOFF flow control have to do with 8bit data?
>>
>> And no, DEC was not trying to save money. They were actually sticking to the standard. Something very few companies managed to do, especially when it came to the RS-232 standard.
>> The "hardware flow control" is actually an abuse of the half duplex signalling.
>> And it does not necessarily work as you might expect if modems are in the middle.
>> You could get by with just three wires on serial ports, but DEC usually did want people to have more wires connected, in order to detect when a device was connected or not. But no, they did not abuse the half duplex control to do hardware flow control. The DZ11 also have partial modem control. You cannot run them in half-duplex mode, and they do not support the secondary channel stuff, but they do support more than just the 3 wires.
>>
>>> A few years later when people tried to put high speed modems like the Trailblazers on them, let’s say the Unix folks quickly abandoned any hope.   Ken O’Humundro at Able Computer has quite a business in his ‘DHDM’ board that was cheaper than the DZ, more ports, full DMA and of course full hardware flow control.
>>
>> Yes. It's essentially a DH11 with a partial DM11 on just one board, while the original DH11/DM11 was a fully 9-slot backplane. So it was a really good alternative. The DH11 is much better than the DZ11, but did in fact cost more. The Able board gave the DH11 capabilities at a much lower cost, and taking much less space in the box. And it actually performed better than the DH11. But it did not support the full functionality of the DM11. But what was lacking was the half duplex stuff, which noone really cared about anymore anyway.
>>
>>     Johnny
>>
>> -- 
>> Johnny Billquist                  || "I'm on a bus
>>                                   ||  on a psychedelic trip
>> email: bqt at softjar.se             ||  Reading murder books
>> pdp is alive!                     ||  tryin' to stay hip" - B. Idol


-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Control-T (was top)

[Pete Turnbull]

On 30/05/2018 23:10, Johnny Billquist wrote:
> On 30.05.18 02:19, Clem cole wrote:

>> 1). If you grab ^s/^q from the alphabet it means you can send raw 8 
>> bit data because they are valid characters (yes you can use escape 
>> chars but it gets very bad)
> 
> But this has nothing to do with 8 bit data.
[...]
> What you are talking about is simply the problem when you have inband 
> signalling, and is a totally different problem than 8bit data.

True, but what I believe Clem meant was "binary data".  He did write 
"raw data".

> RTS/CTS signalling is certainly a part of the standard, but it is not 
> meant for flow control. It is for half duplex and the way to signal when 
> you want to change the direction of communication.
> 
> It is meant for *half duplex* communication. Not flow control.

Actually, for both, explicitly in the standard.  The standard (which I 
have in front of me, RS232C August 1969 reaffirmed June 1981) does 
indeed explain at length how to use it for half duplex turnaround but 
also indicates how to use it for flow control.  It states the use on 
one-way channels and full-duplex channels to stop transmission in a 
paragraph before the paragraph discussing half duplex.

> The AT "standard" was never actually a standard. It's only a defacto 
> standard, but is just something Hayes did, as I'm sure you know.

Except for ITU-T V.250, which admittedly is a Recommendation not a Standard.

-- 
Pete
Pete Turnbull

[TUHS] Control-T (was top)

[Dave Horsfall]

On Wed, 30 May 2018, Johnny Billquist wrote:

> Uh? Say what? What does XON/XOFF flow control have to do with 8bit data?

Err, how do you send data that happen to be XON/OFF?  By futzing around
with DLE (which I've never seen used)?

-- Dave

[TUHS] Control-T (was top)

[Johnny Billquist]

On 31.05.18 04:00, Pete Turnbull<pete at dunnington.plus.com> wrote:
> On 30/05/2018 23:10, Johnny Billquist wrote:
>> On 30.05.18 02:19, Clem cole wrote:
>>> 1). If you grab ^s/^q from the alphabet it means you can send raw 8
>>> bit data because they are valid characters (yes you can use escape
>>> chars but it gets very bad)
>> But this has nothing to do with 8 bit data.
> [...]
>> What you are talking about is simply the problem when you have inband
>> signalling, and is a totally different problem than 8bit data.
> True, but what I believe Clem meant was "binary data".  He did write
> "raw data".

Clem originally said:
"And the other issue was besides not enough buffering DZ11s and the TU58 
assumes  software (^S/^Q) for the flow control (DH11s with the DM11 
option has hardware (RTS/CTS) to control the I/O rate.  So this of 
course made 8bit data diificult too."

And that is what I objected to. If we're talking about just getting data 
through a communications channel cleanly, then obviously inband 
signalling like XON/XOFF is a problem, but that was not what Clem claimed.

By the way, while I'm on this topic. The TU58 (aka DECtape II - curse 
that name) initially did have overrun problems, and that was because the 
protocol did not have proper handshaking. There is flow control when the 
TU58 sends data to the host, but there is no handshaking, which 
amplifies the problem with flow control on that device. The protocol the 
TU58 uses is called RSP, for Radial Serial Protocol. DEC realized the 
problem, and modified the protocol, which were then called MRSP, or 
Modified Radial Serial Protocol, which addressed the specific problem of 
handshaking when sending data to the host.

But in addition to the lack of handshaking, the TU58 is normally always 
expected to be connected to a DL11, and the DL11 is the truly stupid, 
simple device that only have a 1 character buffer. So that interface can 
easily drop characters on reception. Flow control or not. And the TU58 
is not really to blame. And if you have an updated TU58 which uses MRSP, 
you are better off.

>> RTS/CTS signalling is certainly a part of the standard, but it is not
>> meant for flow control. It is for half duplex and the way to signal when
>> you want to change the direction of communication.
>>
>> It is meant for*half duplex*  communication. Not flow control.
> Actually, for both, explicitly in the standard.  The standard (which I
> have in front of me, RS232C August 1969 reaffirmed June 1981) does
> indeed explain at length how to use it for half duplex turnaround but
> also indicates how to use it for flow control.  It states the use on
> one-way channels and full-duplex channels to stop transmission in a
> paragraph before the paragraph discussing half duplex.

Using RTS/CTS for flow control have eventually made it into the 
standard, but this only happened around 1990 with RS-232E-E. See 
https://groups.google.com/forum/#!original/comp.dcom.modems/iOZRZkTKc-o/ZcdcMgUmBDkJ 
for some background story on that.

And that obviously is way after DEC was making any of these serial 
interfaces.

Wikipedia have a pretty decent explanation of how these signals were 
defined back in the day:
"The RTS and CTS signals were originally defined for use with 
half-duplex (one direction at a time) modems such as the Bell 202. These 
modems disable their transmitters when not required and must transmit a 
synchronization preamble to the receiver when they are re-enabled. The 
DTE asserts RTS to indicate a desire to transmit to the DCE, and in 
response the DCE asserts CTS to grant permission, once synchronization 
with the DCE at the far end is achieved. Such modems are no longer in 
common use. There is no corresponding signal that the DTE could use to 
temporarily halt incoming data from the DCE. Thus RS-232's use of the 
RTS and CTS signals, per the older versions of the standard, is asymmetric."

See Wikipedia itself if you want to get even more signals explained.

If you have a copy of the 1969 standards document, could you share it? I 
was trying to locate a copy now, but failed. I know I've read it in the 
past, but have no recollection on where I had it then, or where I got it 
from.

>> The AT "standard" was never actually a standard. It's only a defacto
>> standard, but is just something Hayes did, as I'm sure you know.
> Except for ITU-T V.250, which admittedly is a Recommendation not a Standard.

This also becomes a question of who you accept as an authority to 
establish standards. One could certainly in one sense say that the Hayes 
AT commands were a standard. But all of that also happened long after 
DEC made these interfaces, and as such are also pretty irrelevant. The 
link to the post in news back in 1990 is from the standards 
representative from Hayes, by the way. So they were certainly working 
with the standards bodies to establish how to do things.

   Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Control-T (was top)

[Nemo]

On 31/05/2018, Johnny Billquist <bqt at update.uu.se> wrote (in part):
[Wonderful stuff clipped out.]

This really brought back fond memories of writing software to
interface with modems.

> This also becomes a question of who you accept as an authority to
> establish standards. One could certainly in one sense say that the Hayes
> AT commands were a standard. But all of that also happened long after
> DEC made these interfaces, and as such are also pretty irrelevant. The
> link to the post in news back in 1990 is from the standards
> representative from Hayes, by the way. So they were certainly working
> with the standards bodies to establish how to do things.

Indeed, now we have the ETSI AT commands. #6-)

N.

>
>    Johnny