Re: [TUHS] non-blocking IO

Re: [TUHS] non-blocking IO

[Noel Chiappa]

    > From: Paul Winalski

    > I'm curious as to what the rationale was for Unix to have been designed
    > with basic I/O being blocking rather than asynchronous.

It's a combination of two factors, I reckon. One, which is better depends a
lot on the type of thing you're trying to do. For many typical thing (e.g.
'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is
more complicated, and Unix was (well, back then at least) all about getting
the most bang for the least bucks.

More complicated things do sometimes benefit from asynchronous I/O, but
complicated things weren't Unix's 'target market'. E.g. even though pipes
post-date the I/O decision, they too are a better match to blocking I/O.


    > From: Arnold Skeeve

    > the early Unixs were on smaller -11s, not the /45 or /70 with split I&D
    > space and the ability to address lost more RAM.

Ahem. Lots more _core_. People keeep forgetting that we're looking at
decicions made at a time when each bit in main memory was stored in a
physically separate storage device, and having tons of memory was a dream of
the future.

E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total!
And that had to hold the resident OS, plus the application! It's no
surprise that Unix was so focused on small size - and as a corollary, on
high bang/buck ratio.

But even in his age of lighting one's cigars with gigabytes of main memory
(literally), small is still beautiful, because it's easier to understand, and
complexity is bad. So it's too bad Unix has lost that extreme parsimony.


    > From: Dan Cross 

    > question whether asynchrony itself remains untamed, as Doug put it, or
    > if rather it has proved difficult to retrofit asynchrony onto a system
    > designed around fundamentally synchronous primitives?

I'm not sure it's 'either or'; I reckon they are both true.


	Noel

Re: [TUHS] non-blocking IO

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 2080 bytes --]

On Tue, Jun 2, 2020 at 4:14 PM Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

> Ahem. Lots more _core_. People keeep forgetting that we're looking at
> decicions made at a time when each bit in main memory was stored in a
> physically separate storage device, and having tons of memory was a dream
> of
> the future.
>
Yeah -- that is something that forgotten.  There's a kit/hackday project to
make 32-byte core for an Arduino I did with some of my boy scouts doing
electronic MB a while back just to try to give them a feel what a 'bit'
was.    Similarly, there was a update of in late 1960's children's book
originally called 'A Million' it's now called: A Million Dots
<https://www.amazon.com/Million-Dots-Andrew-Clements/dp/0689858248/ref=sr_1_1?crid=2AX8H8L2EM0HL&dchild=1&keywords=a+million+dots+by+andrew+clements&qid=1591129965&sprefix=a+million+dots%2Caps%2C155&sr=8-1>
Each page has 10K dots.  The idea is to help young readers get a real feel
for what 'a million' means visually.


>
> E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total!
> And that had to hold the resident OS, plus the application! It's no
> surprise that Unix was so focused on small size - and as a corollary, on
> high bang/buck ratio.'

Amen -- I ran an 11/34 with 64K under V6 for about 3-6 months while we were
awaiting the 256K memory upgrade.


>
>
> But even in his age of lighting one's cigars with gigabytes of main memory
> (literally), small is still beautiful, because it's easier to understand,
> and
> complexity is bad. So it's too bad Unix has lost that extreme parsimony.
>
Yep -- I think we were discussing this last week WRT to cat -v/fmt et al.

I fear some people confuse 'progress' with 'feature creep.'   Just because
we can do something, does not mean we should.

As I said, I'm a real fan of async I/O and like Paul, feel that it is a
'better' primitive.  But I fully understand and accept, that given the
tradeoffs of the time, UNIX did really well and I much prefer what we got
than the alternative.  I'm happy we ended up with simply and just works.

[-- Attachment #2: Type: text/html, Size: 4080 bytes --]

Re: [TUHS] non-blocking IO

[Rich Morin]

Case study (read, war story :-) of non-blocking I/O...

Back in the mid-70's, I was doing scientific support programming for an X-Ray sky survey,
using Fortran-63 under DRUM SCOPE on a CDC 3800 system which NRL's Space Sciences Division
had (sigh) acquired and cobbled together from the US Government "excess property" list.

Most of the time, our satellite was used in "scan mode", spinning on an axis which pointed
at the Sun.  This let us collect data on a two degree wide strip of sky.  So, every six
months we scanned the entire X-ray sky.  The track was basically a helix, bent to match the
shape of the Earth's orbit.  Every so often, in order to look at a particularly interesting
area, the spinning would be stopped, so the instrument could be put into "point mode".

The challenge was to "bin" this data for further analysis.  We did this by loading sets of
7-track tapes onto a CDC 813 drive, then dumping it out into 36 2x10 degree bins.  This put
a quarter of the data onto a set of tapes.  Rinse, repeat for the other 3/4 of the data
(after skipping past the already-written data).  The result was 36 tapes, each holding an
8x10 degree bin (plus one extra tape for any point mode data). 

IIRC, we had five tape drives; my challenge was to keep them all as busy as possible, so as
to dump the data set expeditiously.  Because I had asynchronous I/O (mostly in the form of
BUFFER IN and BUFFER OUT commands), I was able to implement a simple but quite effective
polling loop.  The machine room was a bit of a madhouse, but the tapes were written about
as quickly as the hardware allowed.  Asynchronous I/O FTW...

https://en.wikipedia.org/wiki/United_States_Naval_Research_Laboratory#Space_sciences
https://en.wikipedia.org/wiki/High_Energy_Astronomy_Observatory_1#A1:_Large-Area_Sky_Survey_instrument
https://ub.fnwi.uva.nl/computermuseum/cdcdisk.html

-r

Re: [TUHS] non-blocking IO

[Paul Winalski]

On 6/2/20, Rich Morin <rdm@cfcl.com> wrote:
>
> IIRC, we had five tape drives; my challenge was to keep them all as busy as
> possible, so as
> to dump the data set expeditiously.  Because I had asynchronous I/O (mostly
> in the form of
> BUFFER IN and BUFFER OUT commands), I was able to implement a simple but
> quite effective
> polling loop.  The machine room was a bit of a madhouse, but the tapes were
> written about
> as quickly as the hardware allowed.  Asynchronous I/O FTW...

With 9-track magnetic tape devices, reading and writing can't start
until the tape is up to speed.  Once up to speed the drive can read
and write records while keeping the tape moving at speed.  This is
called streaming.  If there's a pause in the read/write requests from
the CPU, time is lost as the drive stops and starts moving the tape.
It was essential that applications doing large amounts of tape I/O
keep up the I/O requests at a rate that allows streaming.
Asynchronous I/O with multi-buffering is a straightforward way to
accomplish this.  The IBM S/360 channel commands for tape devices
provided a mechanism for the tape control unit to send an interrupt to
the CPU when a read or write channel command completed.  This notified
the sequential access method (the user program I/O interface) when I/O
to each buffer had completed and the buffer was available for reuse.
OS/360's Sequential Access Method could read or write an entire tape
using a single SIO (start I/O) instruction, as long as no read or
write errors were encountered.

-Paul W.

Re: [TUHS] non-blocking IO

[John P. Linderman]

[-- Attachment #1: Type: text/plain, Size: 3062 bytes --]

On a distantly related note, when I worked part time for the MIT
administration in the 60's, we'd do processing on the 7094 in the main comp
center, then bring a tape of results back to our office to do the printing
and card punching. For reasons that were never explained to me, the blocks
of print and punch data were combined on a single tape, and were
distinguished by being written with different parities. If you attempted to
read a block with the correct parity setting, the tape could stream. If you
guessed wrong, you had to stop, back up a block, change the parity, and try
again. So I wrote a simple 360 assembly program to keep track how often
blocks of each parity followed the observed parities of the previous 8
blocks. Essentially, 256 pairs of parity observations, indexed by the
previous 8 parity observations. Blocks of print and punch data tended to
fall into patterns that depended on what job was being run on the 7094, so
detecting those patterns and correctly anticipating the upcoming parity
made the tapes move much more smoothly. It was fun to watch the tape at the
start of a run. It was mostly just a coin-toss, so the tape was jerking
around fitfully. As the patterns started to emerge, the predictions got
better, the jerking got less and less common, and the tapes were streaming
most of the time. My introduction to learning algorithms.

On Wed, Jun 3, 2020 at 12:33 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> On 6/2/20, Rich Morin <rdm@cfcl.com> wrote:
> >
> > IIRC, we had five tape drives; my challenge was to keep them all as busy
> as
> > possible, so as
> > to dump the data set expeditiously.  Because I had asynchronous I/O
> (mostly
> > in the form of
> > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but
> > quite effective
> > polling loop.  The machine room was a bit of a madhouse, but the tapes
> were
> > written about
> > as quickly as the hardware allowed.  Asynchronous I/O FTW...
>
> With 9-track magnetic tape devices, reading and writing can't start
> until the tape is up to speed.  Once up to speed the drive can read
> and write records while keeping the tape moving at speed.  This is
> called streaming.  If there's a pause in the read/write requests from
> the CPU, time is lost as the drive stops and starts moving the tape.
> It was essential that applications doing large amounts of tape I/O
> keep up the I/O requests at a rate that allows streaming.
> Asynchronous I/O with multi-buffering is a straightforward way to
> accomplish this.  The IBM S/360 channel commands for tape devices
> provided a mechanism for the tape control unit to send an interrupt to
> the CPU when a read or write channel command completed.  This notified
> the sequential access method (the user program I/O interface) when I/O
> to each buffer had completed and the buffer was available for reuse.
> OS/360's Sequential Access Method could read or write an entire tape
> using a single SIO (start I/O) instruction, as long as no read or
> write errors were encountered.
>
> -Paul W.
>

[-- Attachment #2: Type: text/html, Size: 3600 bytes --]

Re: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency

[John Gilmore]

Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

>     > I'm curious as to what the rationale was for Unix to have been designed
>     > with basic I/O being blocking rather than asynchronous.
> 
> It's a combination of two factors, I reckon. One, which is better depends a
> lot on the type of thing you're trying to do. For many typical thing (e.g.
> 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is
> more complicated, and Unix was (well, back then at least) all about getting
> the most bang for the least bucks.

I just happened to open a binder today of old papers about UNIX(tm),
including Ken Thompson's "Unix Implementation" paper, which says at the
bottom of the introduction:

  "What is or is not implemented in the kernel represents both a great
  responsibiity and a great power.  It is a soap-box platform on "the
  way things should be done."  Even so, if "the way" is too radical, no
  one will follow it.  Every important decision was weighted carefully.
  Throughout, simplicity has been substituted for efficiency.  Complex
  algorithms are used only if their complexity can be localized."

	John
	

Re: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency

[arnold]

John Gilmore <gnu@toad.com> wrote:

> I just happened to open a binder today of old papers about UNIX(tm),
> including Ken Thompson's "Unix Implementation" paper, which says at the
> bottom of the introduction:
>
>   "What is or is not implemented in the kernel represents both a great
>   responsibiity and a great power.  It is a soap-box platform on "the
>   way things should be done."  Even so, if "the way" is too radical, no
>   one will follow it.  Every important decision was weighted carefully.
>   Throughout, simplicity has been substituted for efficiency.  Complex
>   algorithms are used only if their complexity can be localized."

Words to live by.  I remember reading that paper early on in my
career and that this statement certainly influenced my subsequent
thinking about programming in general.

Arnold