[TUHS] non-blocking IO

[TUHS] non-blocking IO

[Paul Ruizendaal]

> At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well.

> I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too).

Sounds very interesting!

Looked around a bit, but I did not find a source for the “Internet Monthly Reports” for the late 70’s (rfc-editor.org/museum/ has them for the 1990’s).

In the 1970’s era, it seems that NCP Unix went in another direction, using newly built message and event facilities to prevent blocking. This is described in "CAC Technical Memorandum No. 84, Illinois Inter-Process Communication Facility for Unix.” - but that document appears lost as well.

Ah, well, topics for another day.

Re: [TUHS] non-blocking IO

[Paul Winalski]

The operating systems that I cut my teeth on (OS/360, DOS/360,
VAX/VMS) all had basic I/O system calls that were non-blocking.
Blocking I/O calls were all built on top of that framework.  I thus
found it curious that Unix took the opposite tack, and non-blocking
I/O was an afterthought.

So I'm curious as to what the rationale was for Unix to have been
designed with basic I/O being blocking rather than asynchronous.
Especially that non-blocking I/O primitives were the norm for OSes in
those days.

-Paul W.

Re: [TUHS] non-blocking IO

[arnold]

Paul Winalski <paul.winalski@gmail.com> wrote:

> So I'm curious as to what the rationale was for Unix to have been
> designed with basic I/O being blocking rather than asynchronous.

I don't doubt that it was "simplify, simplify, simplify". Async I/O
is much messier than Unix's read/write model. The Unix model was
simpler to design, simpler to code, simpler to get right, and undoubtedly
took much less OS code than an async model would have; on the PDP-11
that would have mattered.

Also, the early Unixs were on smaller -11s, not the /45 or /70 with
split I&D space and the ability to address lost more RAM.

My guess, anyway. :-)

Arnold

Re: [TUHS] non-blocking IO

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

On Tue, Jun 2, 2020 at 1:47 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> The operating systems that I cut my teeth on (OS/360, DOS/360,
> VAX/VMS) all had basic I/O system calls that were non-blocking.
> Blocking I/O calls were all built on top of that framework.  I thus
> found it curious that Unix took the opposite tack, and non-blocking
> I/O was an afterthought.
>
> So I'm curious as to what the rationale was for Unix to have been
> designed with basic I/O being blocking rather than asynchronous.
> Especially that non-blocking I/O primitives were the norm for OSes in
> those days.


Doug addressed this, albeit in an oblique manner, on this list back in
2015: https://minnie.tuhs.org/pipermail/tuhs/2015-September/007509.html

Quoting him:

"""
Unix was what the authors wanted for a productive computing environment,
not a bag of everything they thought somebody somewhere might want.
One objective, perhaps subliminal originally, was to make program
behavior easy to reason about. Thus pipes were accepted into research
Unix, but more general (and unruly) IPC mechanisms such as messages
and events never were.

The infrastructure had to be asynchronous. The whole point was to
surmount that difficult model and keep everyday programming simple.
User visibility of asynchrony was held to a minimum: fork(), signal(),
wait(). Signal() was there first and foremost to support SIGKILL; it
did not purport to provide a sound basis for asynchronous IPC.
The complexity of sigaction() is evidence that asynchrony remains
untamed 40 years on.
"""

My response at the time was to question whether asynchrony itself remains
untamed, as Doug put it, or if rather it has proved difficult to retrofit
asynchrony onto a system designed around fundamentally synchronous
primitives?

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2453 bytes --]

Re: [TUHS] non-blocking IO

[Paul Winalski]

On 6/2/20, arnold@skeeve.com <arnold@skeeve.com> wrote:
> Paul Winalski <paul.winalski@gmail.com> wrote:
>
>> So I'm curious as to what the rationale was for Unix to have been
>> designed with basic I/O being blocking rather than asynchronous.
>
> Also, the early Unixs were on smaller -11s, not the /45 or /70 with
> split I&D space and the ability to address lost more RAM.

I first encountered DOS/360 on a System/360 model 25 with 48K of
memory.  This was a one-job-at-a-time batch system, but the I/O
primitive (EXCP--execute channel program) was asynchronous.  So I
don't think the small memory rationale really applies.

-Paul W.

Re: [TUHS] non-blocking IO

[Paul Winalski]

On 6/2/20, Dan Cross <crossd@gmail.com> wrote:
>
> My response at the time was to question whether asynchrony itself remains
> untamed, as Doug put it, or if rather it has proved difficult to retrofit
> asynchrony onto a system designed around fundamentally synchronous
> primitives?

I think that's a very good question.  It's analogous to
record-oriented I/O vs. byte stream I/O.  It's easy to build
record-oriented I/O on top of a byte stream, but it's a real bear to
do it the other way around.  Similarly, it's easy to build synchronous
I/O on top of asynchronous I/O but the reverse ends up looking
contrived.

-Paul W.

Re: [TUHS] non-blocking IO

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 2062 bytes --]

On Tue, Jun 2, 2020 at 2:54 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> I first encountered DOS/360 on a System/360 model 25 with 48K of
> memory.  This was a one-job-at-a-time batch system, but the I/O
> primitive (EXCP--execute channel program) was asynchronous.  So I
> don't think the small memory rationale really applies.
>
Hrrmpt... it was single task.   Being asynchronous in the I/O and allowed
process asynchrony takes a lot more housekeeping.  Paul you know I agree
with you, it was always an issue with UNIX IMO.  The problem is how to do
it differently.

At Masscomp RRU, our solution was not to try to 'fix it' as much as add a
new scheme beside the synchronous one.  We added a general AST's mechanism
that anyone could use (very much like RSX and VMS in semantics), but left
signals alone.  We added new async calls, which were just implemented via
ioctl's (the universal hack) for the specific HW that supported it.   In
retrospect, that was an error, it should have been aread(2)/awrite(2) and
then added the completion routine/call back as the 4th & 5th parameters.
 Since Stellix was not Real-Time, we left them out.  [ tjt and I have
argued about that for years ].

So back to Doug/Dan's answer -- I think for a small system, like the
original PDP-7 and the PDP-11/20 putting the effort to make it multiprocess
and leaving the I/O synchronous definitely made it easier.   Particularly,
since what was created with the signal semantics.   To me, UNIX got way
more right than wrong.   I suspect if signals had been more like AST's, the
idea of being async might have gone farther.  But to use Dennis's like
about C being quirky, signals are very quirky.  wnj tried to 'fix' them and
frankly it just made matters worse.  And IMO, signaction(3) as Doug says,
is a nightmare.

I've generally been a fan of introducing a new idea separately as a
'stronger strain' and then seeing if people likely it.

FWIW: A couple of us did try to get AST's into the POSIX.4 (they were there
in a draft), but ultimately *.4 was rejected as 'not UNIX.'

[-- Attachment #2: Type: text/html, Size: 3388 bytes --]

Re: [TUHS] non-blocking IO

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

On Tue, Jun 2, 2020 at 2:58 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> I think that's a very good question.  It's analogous to
> record-oriented I/O vs. byte stream I/O.  It's easy to build
> record-oriented I/O on top of a byte stream, but it's a real bear to
> do it the other way around.  Similarly, it's easy to build synchronous
> I/O on top of asynchronous I/O but the reverse ends up looking contrived.
>
Which was exactly the point I tried to make in the POSIX.4 discussions, but
it does take more work in the basic housekeeping and you need a way to
handle events and completions that are priority based, queued, and a few
other details.   As Doug said, they stayed away from some features (like
messaging).   async I/O was one of them.

But as I said, Ken, Dennis and the rest of the crew did an amazing job with
very little.

[-- Attachment #2: Type: text/html, Size: 1575 bytes --]

Re: [TUHS] non-blocking IO

[Lawrence Stewart]

I remember working on getting Arpanet access on an 11/34 running V7 around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run NCP, using a variation of Rand’s code.  I wrote some sort of bisync driver for packet communications between the /23 and the /34, and I think added an IOCTL or some hack to ask if there was a message ready.  So a polling variety of non-blocking I/O :)

Meanwhile, on the Alto, surely an underpowered machine, the style for UI programming and asynch I/O was mostly event driven, much like libevent in more recent years.  I found that very easy to understand.

The main trajectory of PARC was lightweight threads calling synchronous I/O which really has to be counted as a Bad Idea not yet fully stamped out.

I’ve always thought select(2) was a pretty good thing.  In 1995 at Open Market we had a single-process web server that had no difficulties running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.

-L

Re: [TUHS] non-blocking IO - threads

[Jon Steinhart]

Lawrence Stewart writes:
> I’ve always thought select(2) was a pretty good thing.  In 1995 at Open
> Market we had a single-process web server that had no difficulties
> running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.

So select was a great thing until interactive graphics came along.  Just
got too messy.  I recall writing assembly language code to support threads
in the mid to late 1980s, and I remember reading some papers from Sun about
the same.  Everybody was doing it.  Anybody have enough of the history to
paint a picture of how we got from there to standardized threads?

Jon

Re: [TUHS] non-blocking IO - threads

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]

It's only messy if you make a messy job of it.

-rob


On Wed, Jun 3, 2020 at 8:36 AM Jon Steinhart <jon@fourwinds.com> wrote:

> Lawrence Stewart writes:
> > I’ve always thought select(2) was a pretty good thing.  In 1995 at Open
> > Market we had a single-process web server that had no difficulties
> > running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.
>
> So select was a great thing until interactive graphics came along.  Just
> got too messy.  I recall writing assembly language code to support threads
> in the mid to late 1980s, and I remember reading some papers from Sun about
> the same.  Everybody was doing it.  Anybody have enough of the history to
> paint a picture of how we got from there to standardized threads?
>
> Jon
>

[-- Attachment #2: Type: text/html, Size: 1138 bytes --]

Re: [TUHS] non-blocking IO - threads

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

To be more forthcoming, there was a long period when the received wisdom
was that threads were unmanageable. But that was a category error: The
programming model in POSIX and others was what made it hard. The
fundamental concepts are a tiny step harder than trivial, and it was not
hard to build a programming model that matched them.

The first mistake people made was to split "thread" from "process". What
those words mean today is not what they meant a generation ago.

For me, it's like the mess sockets made of networking. Our programming
models could be so much simpler than what we are required to deal with.

-rob

[-- Attachment #2: Type: text/html, Size: 779 bytes --]

Re: [TUHS] non-blocking IO - threads

[Jon Steinhart]

Rob Pike writes:
>
> It's only messy if you make a messy job of it.

Maybe I should explain what I meant by messy.  To me, the issue with select is
that in many cases one has to either keep a lot of state somewhere, or live with
long response times while handlers complete.  Threads allowed handlers to be
written in the "normal" way with state kept on the stack.

Re: [TUHS] non-blocking IO - threads

[Larry McVoy]

On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote:
> The first mistake people made was to split "thread" from "process". What
> those words mean today is not what they meant a generation ago.

So I'm a fan of your quote, Rob, "If you think you need threads, your
processes are too fat".

Years ago, I asked Linus if he could fix the one flaw with that point
of view.  Which is, processes can't share page tables.  So N processes
will have N page tables, need an N way bigger TBL, to get the same
performance.

What I asked Linus to try and do is allow a process to share all or
part of its page table with another process.  If that could be done,
we could pretty do what you said, just use processes and if they want
to act like threads, just share the page table.

And I just looked, he seems to have done it.  man 2 clone on a
Linux box.  It's the swiss army knife of features, feels messy.

I don't know if CLONE_VM will share the page tables, I'll ask
Linus.

--lm

Re: [TUHS] non-blocking IO - threads

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

See the description of rfork at http://man.cat-v.org/plan_9/2/fork.

-rob


On Wed, Jun 3, 2020 at 9:21 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote:
> > The first mistake people made was to split "thread" from "process". What
> > those words mean today is not what they meant a generation ago.
>
> So I'm a fan of your quote, Rob, "If you think you need threads, your
> processes are too fat".
>
> Years ago, I asked Linus if he could fix the one flaw with that point
> of view.  Which is, processes can't share page tables.  So N processes
> will have N page tables, need an N way bigger TBL, to get the same
> performance.
>
> What I asked Linus to try and do is allow a process to share all or
> part of its page table with another process.  If that could be done,
> we could pretty do what you said, just use processes and if they want
> to act like threads, just share the page table.
>
> And I just looked, he seems to have done it.  man 2 clone on a
> Linux box.  It's the swiss army knife of features, feels messy.
>
> I don't know if CLONE_VM will share the page tables, I'll ask
> Linus.
>
> --lm
>

[-- Attachment #2: Type: text/html, Size: 1659 bytes --]

[TUHS] Unix on the Arpanet

[Lars Brinkhoff]

Lawrence Stewart wrote:
> I remember working on getting Arpanet access on an 11/34 running V7
> around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run
> NCP, using a variation of Rand’s code.

Has this, or Rand's code, been preserved?

I'm only aware of one Arpanet NCP implementation for Unix that is
online, the one from University of Illinois.

Re: [TUHS] Unix on the Arpanet

[Lawrence Stewart]

> On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff <lars@nocrew.org> wrote:
> 
> Lawrence Stewart wrote:
>> I remember working on getting Arpanet access on an 11/34 running V7
>> around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run
>> NCP, using a variation of Rand’s code.
> 
> Has this, or Rand's code, been preserved?
> 
> I'm only aware of one Arpanet NCP implementation for Unix that is
> online, the one from University of Illinois.

Alas I do not know.  I may have some old emails from that era. I will check.

Re: [TUHS] non-blocking IO - threads

[Paul Winalski]

On 6/2/20, Jon Steinhart <jon@fourwinds.com> wrote:
>
> Maybe I should explain what I meant by messy.  To me, the issue with select is
> that in many cases one has to either keep a lot of state somewhere, or live with
> long response times while handlers complete.  Threads allowed handlers to be
> written in the "normal" way with state kept on the stack.
>
TOPS-10 had a nifty program called ptycon that would allow one to
control multiple login sessions, each on its own pseudo-terminal
device, from a single physical terminal.  In 1985 our software tools
group implemented a pseudo-terminal device driver for VMS, and I wrote
a program called ptycon-32.  It was multithreaded, with one thread of
control for each pseudo-terminal.  The problem is that VMS at the time
didn't have a threads package.  I had to roll my own threads by using
the asynchronous system traps (ASTs) delivered by completion of I/O to
the pseudo-terminals.  Talk about messy....

-Paul W.

Re: [TUHS] non-blocking IO - threads

[Jon Forrest]

On 6/3/2020 9:42 AM, Paul Winalski wrote:
> The problem is that VMS at the time
> didn't have a threads package.  I had to roll my own threads by using
> the asynchronous system traps (ASTs) delivered by completion of I/O to
> the pseudo-terminals.  Talk about messy....

That's also how the Sybase SQL Server worked on VMS at the same time.

Jon