[TUHS] Stdin Redirect in Cu History/Alternatives?

[TUHS] Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Good morning all.  I've been doing some historical research on the UUCP cu utility this morning and have come across a little discrepancy between the various UNIX streams I was wondering if someone could illuminate.

So cu as of V7 supported the ~$ escape, a means of calling a local procedure and emitting stdout over the TTY line to the remote machine, all fine and good for packaging a character stream to emit.  However, what I'm not finding in that age of documentation is any means of requesting std*in* from the TTY line as input to a local procedure (in essence running a text filter or handshake-driven protocols over cu).  The context in which I'm researching this is integrating cu into my bare-metal SBC programming using XMODEM so I can rest a little easier my process is based on tools I'll probably find in most places.

So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin.  I did some further digging and it looks like different UUCP implementations cracked this nut with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+.  Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up any command for this.  This is indicative of there never being an agreed-upon mechanism for doing this, although I could see this being a very useful mechanism.

What I'm curious about is if the lack of a bi-directional redirect in early cu is reflective of a lack of need for that sort of functionality at the time or that matters such as that were handled through a different mechanism.  One thought I did have is that there wasn't file locking at the time (right?) and so theoretically nothing would prevent one from using a cu session on one terminal to send interactive commands and then a second using fd redirects in the shell to run filters/protocols separately of the interactive stream.

- Matt G.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3731 bytes --]

On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS <tuhs@tuhs.org> wrote:

> Good morning all.  I've been doing some historical research on the UUCP cu
> utility this morning and have come across a little discrepancy between the
> various UNIX streams I was wondering if someone could illuminate.
>
Maybe. see below....

>
> So cu as of V7 supported the ~$ escape, a means of calling a local
> procedure and emitting stdout over the TTY line to the remote machine, all
> fine and good for packaging a character stream to emit.  However, what I'm
> not finding in that age of documentation is any means of requesting std*in*
> from the TTY line as input to a local procedure (in essence running a text
> filter or handshake-driven protocols over cu).  The context in which I'm
> researching this is integrating cu into my bare-metal SBC programming using
> XMODEM so I can rest a little easier my process is based on tools I'll
> probably find in most places.
>
> So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no
> stdin.  I did some further digging and it looks like different UUCP
> implementations cracked this nut with different escapes, with BSD
> eventually going with ~C and Taylor UUCP opting for ~+.  Checking the
> current illumos manual pages (for a SVR4-ish example) doesn't turn up any
> command for this.  This is indicative of there never being an agreed-upon
> mechanism for doing this, although I could see this being a very useful
> mechanism.
>
maybe -- need to see more of what your session was like.  I never remember
missing anything I needed.

>
> What I'm curious about is if the lack of a bi-directional redirect in
> early cu is reflective of a lack of need for that sort of functionality at
> the time or that matters such as that were handled through a different
> mechanism.

I'm not sure I get the question.  We did all sorts of redirection and
used/abused cu and its friends all the time.    I suspect I'm not
understanding what you are trying to do.

From a history standpoint, cu(1) is just one of many programs in that
family.  In the mid/late 1970s, we used a program called 'connect' for
Sixth Edition at CMU, IIRC the Purdue folks had a similar one which was
called attach(1) and there was tip(1) which was from Case/UCB [Sam
Leffler].  If you look in the USENIX archives, I bet you will find a 1/2
doz or so of programs in the ilk before V7.  With V7 uucp. was delivered,
so cu(1) began to make inroads as it had the advantage that it was set up
to work on concert to uucico(8). Simply, V7 came out, and UUCP started to
used and eventually the 'USENET' born, cu(1) sort of 'won' because most of
the other programs tended to conflict with uucico(8) -  plus since it was
already there, people did not need something else.   But if you had written
one before V7, you often find sites sticking with what they had.

There were a number of UNIX implementations of XMODEM and friends.  The C
version of Kermit  (ckermit) was quite popular plus has
connect(1)/tip(1)/cu(1) style functionality built into it, but ....  IIRC
does not obey the locks that uucico(8) wants so if you used it on TTYs that
had a modem that uucico was trying grab, bad things happened.   That said,
in a microprocessor lab where you often dedicated. serial port to 'target'
micro/pc, kermit worked well.   My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
since I had any use for them, I've forgotten. Look in the both USENIX and
the USENET source archives.

Frankly, the last time I think I was trying to do this sort of thing, I was
using Kermit.

YMMV
Clem
ᐧ

[-- Attachment #2: Type: text/html, Size: 6222 bytes --]

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

[-- Attachment #1: Type: text/plain, Size: 5942 bytes --]

Luckily I can toss you my exact use-case with code.

https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c

Above is a little utility I spit out to very specifically perform XMODEM-CRC for the VisionFive RISC-V single-board computer. Word on the street is their recovery bootloader has a bug that prevents sx and other utilities from working properly. As far as I could tell, they don't send the standard ACK after each packet receipt, but instead some other character. I just opted to check for NAK and proceed if not received.

In any case, the above should achieve XMODEM-CRC transmission to the JH7100 RISC-V chip over UART.

The phenomenon I'm experiencing (and sorry, I'm not trying to turn this into Taylor UUCP troubleshooting, I promise) is that when using the ~$ escape, my code never receives the 'C' from the device to initiate XMODEM-CRC. I know it's reading local stdin still because if I type and submit a 'C', it picks up and starts transferring, likewise requiring me to submit a non-NAK character every packet. Otherwise it does work, so the stdout is going to the remote machine, but at least with ~$, the stdin from the serial line does stop showing on my screen, which made me think the redirect was happening, but it just seems to disappear off into the void (/dev/null?) while the local program reads local stdin and puts data out on the remote stdout. This does hold consistent with the manual verbiage for ~$, it only mentions stdout to remote, doesn't mention any ferrying of remote stdin into the local application being run.

I know my utility works because I've used it in the place of sx in GNU screen and it works like a charm. Plus it does send​ to the device if I provide the expected acknowledgement characters from my local machine in cu.

So this doesn't just turn into troubleshooting, all I hope to highlight here is getting stdin from the remote machine into a program launched via a ~ escape in cu, be it like the ~C option in BSD cu or ~+ option in Taylor. My curiosity is whether such a thing was in historic cu, and, if not, how it might have been accommodated otherwise. As always, thankful for the insight and feedback folks here provide!

- Matt G.
------- Original Message -------
On Saturday, December 10th, 2022 at 4:22 PM, Clem Cole <clemc@ccc.com> wrote:

> On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS <tuhs@tuhs.org> wrote:
>
>> Good morning all. I've been doing some historical research on the UUCP cu utility this morning and have come across a little discrepancy between the various UNIX streams I was wondering if someone could illuminate.
>
> Maybe. see below....
>
>> So cu as of V7 supported the ~$ escape, a means of calling a local procedure and emitting stdout over the TTY line to the remote machine, all fine and good for packaging a character stream to emit. However, what I'm not finding in that age of documentation is any means of requesting std*in* from the TTY line as input to a local procedure (in essence running a text filter or handshake-driven protocols over cu). The context in which I'm researching this is integrating cu into my bare-metal SBC programming using XMODEM so I can rest a little easier my process is based on tools I'll probably find in most places.
>>
>> So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin. I did some further digging and it looks like different UUCP implementations cracked this nut with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+. Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up any command for this. This is indicative of there never being an agreed-upon mechanism for doing this, although I could see this being a very useful mechanism.
>
> maybe -- need to see more of what your session was like. I never remember missing anything I needed.
>
>> What I'm curious about is if the lack of a bi-directional redirect in early cu is reflective of a lack of need for that sort of functionality at the time or that matters such as that were handled through a different mechanism.
>
> I'm not sure I get the question. We did all sorts of redirection and used/abused cu and its friends all the time. I suspect I'm not understanding what you are trying to do.
>
> From a history standpoint, cu(1) is just one of many programs in that family. In the mid/late 1970s, we used a program called 'connect' for Sixth Edition at CMU, IIRC the Purdue folks had a similar one which was called attach(1) and there was tip(1) which was from Case/UCB [Sam Leffler]. If you look in the USENIX archives, I bet you will find a 1/2 doz or so of programs in the ilk before V7. With V7 uucp. was delivered, so cu(1) began to make inroads as it had the advantage that it was set up to work on concert to uucico(8). Simply, V7 came out, and UUCP started to used and eventually the 'USENET' born, cu(1) sort of 'won' because most of the other programs tended to conflict with uucico(8) - plus since it was already there, people did not need something else. But if you had written one before V7, you often find sites sticking with what they had.
>
> There were a number of UNIX implementations of XMODEM and friends. The C version of Kermit (ckermit) was quite popular plus has connect(1)/tip(1)/cu(1) style functionality built into it, but .... IIRC does not obey the locks that uucico(8) wants so if you used it on TTYs that had a modem that uucico was trying grab, bad things happened. That said, in a microprocessor lab where you often dedicated. serial port to 'target' micro/pc, kermit worked well. My memory is there were also a bunch of two letter programs, rx/sx and rz/sz and the like. Frankly its been so long since I had any use for them, I've forgotten. Look in the both USENIX and the USENET source archives.
>
> Frankly, the last time I think I was trying to do this sort of thing, I was using Kermit.
>
> YMMV
> Clem
> ᐧ

[-- Attachment #2: Type: text/html, Size: 10094 bytes --]

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Michael Kjörling]

On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
> My memory is there were also a bunch of two
> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
> since I had any use for them, I've forgotten.

I remember at least sz/rz from my early (for me) forays into UNIX,
back when ZModem was pretty much state of the art at least on micros
and I had files locally that I wanted remotely or vice versa. The
mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
course, being local to the remote host, so the opposite sense of what
one would do with the terminal emulator program that one interacted
with locally. So "sz <some file>" at the prompt, then activate the
"receive ZModem transfer" function locally; or "rz <some file>", then
activate the "send using ZModem" function locally.

I think the host I was on at the time (which appears to have been some
Solaris) also offered XModem and YModem variants as [rs][xy], but I
never used those because someone had at some point told me that ZModem
was better. :-)

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Steve Nickolas]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

On Sun, 11 Dec 2022, Michael Kjörling wrote:

> On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
>> My memory is there were also a bunch of two
>> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
>> since I had any use for them, I've forgotten.
>
> I remember at least sz/rz from my early (for me) forays into UNIX,
> back when ZModem was pretty much state of the art at least on micros
> and I had files locally that I wanted remotely or vice versa. The
> mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
> course, being local to the remote host, so the opposite sense of what
> one would do with the terminal emulator program that one interacted
> with locally. So "sz <some file>" at the prompt, then activate the
> "receive ZModem transfer" function locally; or "rz <some file>", then
> activate the "send using ZModem" function locally.
>
> I think the host I was on at the time (which appears to have been some
> Solaris) also offered XModem and YModem variants as [rs][xy], but I
> never used those because someone had at some point told me that ZModem
> was better. :-)

Kind-of a pain to type as I'm SSHing in from my phone, but they still 
exist and I have "lrzsz" installed on my Debian box.

-uso.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Sun, Dec 11, 2022 at 9:28 AM Steve Nickolas <usotsuki@buric.co> wrote:
> On Sun, 11 Dec 2022, Michael Kjörling wrote:
> > On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
> >> My memory is there were also a bunch of two
> >> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
> >> since I had any use for them, I've forgotten.
> >
> > I remember at least sz/rz from my early (for me) forays into UNIX,
> > back when ZModem was pretty much state of the art at least on micros
> > and I had files locally that I wanted remotely or vice versa. The
> > mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
> > course, being local to the remote host, so the opposite sense of what
> > one would do with the terminal emulator program that one interacted
> > with locally. So "sz <some file>" at the prompt, then activate the
> > "receive ZModem transfer" function locally; or "rz <some file>", then
> > activate the "send using ZModem" function locally.
> >
> > I think the host I was on at the time (which appears to have been some
> > Solaris) also offered XModem and YModem variants as [rs][xy], but I
> > never used those because someone had at some point told me that ZModem
> > was better. :-)
>
> Kind-of a pain to type as I'm SSHing in from my phone, but they still
> exist and I have "lrzsz" installed on my Debian box.

And as I mentioned earlier, they're still very much used.

I recently wrote the initial bootstrap program that runs when the x86
cores on our machine come out of reset
(https://github.com/oxidecomputer/phbl/). However, that's for
production use; for internal engineering use we have _another_ tool,
also written in Rust, that is an interactive standalone program. This
lets us do things like poke at physical memory, read/write MSRs, and
all of that fun stuff, before loading an operating system. So how does
one load the OS? Our machine has a part that includes a 3
MBAUD-capable UART, and we use xmodem to transfer the kernel (and a
ramdisk image!) over that, where the engineering tool can load and
jump into it. The standalone program implements xmodem internally,
whereas we use `sx` on the dev host side. Basically, there is a
built-in commands to perform the transfer, another that effectively
says "there's an ELF image in physical memory $here: load it and
return the entry point" and finally another command that can jump to
an arbitrary virtual address (yes, we load the host OS in 64-bit mode
and treat the kernel like any old "normal" ELF binary).

As the ramdisk image has grown (more and more debugging tools are
included as we move ever forward in the bringup and development
process), latency here is annoying. I added support for compressed
kernel and ramdisk images, which gave a 2.5x speedup (directly
proportional to the usual compression ratio we see on these images)
which ameliorated but did not eliminate the pain. We briefly tossed
around the idea of implementing zmodem, but decided it wasn't worth
it. We _could_, in theory, make use of the enhanced kermit with
sliding windows and so on, but again decided it wasn't worth the
effort for an engineering tool.

Incidentally, my colleague Matt Keeter ran into performance and
reliability issues with xmodem over the USB serial driver in macOS,
and wrote a small shim that side-stepped the host OS device. He did a
nice write-up of it here:
https://www.mattkeeter.com/blog/2022-05-31-xmodem/

I think this is in line with Larry's question about non-legacy
use-cases for these ancient serial transfer protocols in late 2022.

        - Dan C.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 10:04:54AM -0500, Dan Cross wrote:
> I think this is in line with Larry's question about non-legacy
> use-cases for these ancient serial transfer protocols in late 2022.

Indeed, this was just what I was looking for.  I had no idea that 
serial transfer protocols were still useful.  

It makes me wish I had kept the source for quicknet, I wrote my
own file transfer and remote terminal for CP/M in the middle 
1980s.  I did it just because I wanted things to work a certain
way and none of the other options did what I wanted.  quicknet was
really pleasant on a 128K Z80 (not really 128K because the screen
was mapped in there, but more than 64K).  I spent many a pleasant
hour logged in to uwvax in quicknet, got a lot of work done.

Thanks to you and others who had shown that serial transfer still
matters.  Who knew?  You did.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Adam Thornton]

In the late 1990s I did a brief consulting project to move files between sites.  Something to do with customer billing and record transmission, from some large system that was not Internet-connected.  It ended up being a terrible teetering tower of kermit, sz or sx, and expect.  But it got the job done!

The funny thing about that...the project spanned the July 4th weekend.  I didn't get it working before the 4th.  Then when I went in very hungover after the weekend, I couldn't figure out quite what I had been trying to do, because it was something that was attempting to be very clever.  So in my dulled state, I rewrote the bottom layers to be much more straightforward.  Worked the first time.  That taught me the virtue of not overthinking it.

Adam

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Michael Kjörling]

On 11 Dec 2022 10:18 -0700, from athornton@gmail.com (Adam Thornton):
> The funny thing about that...the project spanned the July 4th
> weekend. I didn't get it working before the 4th. Then when I went in
> very hungover after the weekend, I couldn't figure out quite what I
> had been trying to do, because it was something that was attempting
> to be very clever. So in my dulled state, I rewrote the bottom
> layers to be much more straightforward. Worked the first time. That
> taught me the virtue of not overthinking it.

By definition, if you write code as cleverly as you can, then you
aren't clever enough to debug it...

There's something to be said for simplicity unless for some reason you
really need to be clever. Let's save the clever for the, say, 1% where
it's actually needed; premature application of cleverness is the root
of much evil.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

On Sun, 11 Dec 2022, Michael Kjörling wrote:

> By definition, if you write code as cleverly as you can, then you aren't 
> clever enough to debug it...

Indeed...

I've always used the maxim "Write code as though the next person to 
maintain it is an axe-wielding psychopath who knows where you live".

-- Dave

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
> On Sun, 11 Dec 2022, Michael Kj??rling wrote:
> 
> > By definition, if you write code as cleverly as you can, then you aren't 
> > clever enough to debug it...
> 
> Indeed...
> 
> I've always used the maxim "Write code as though the next person to 
> maintain it is an axe-wielding psychopath who knows where you live".

My main job, for the 20 years until I retired, was to keep telling 
people that code that you wrote 6 months ago might as well have been
written by someone else.  Optimize for reading the code, not writing
the code.  It's read many.

99.9% of the time, I detest clever code.  .1% of the time, I need it.
The problem is that smart engineers adore writing clever code.  They
usually, eventually, wise up.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Readable code also contributes to easier documentation.  It's easier to maintain a manual when you can just translate the code into the local language.  My current team started small and I learned pretty soon that time invested in a habit of readable code meant less time spent providing the person assigned documentation the juicy details.

- Matt G.

------- Original Message -------
On Sunday, December 11th, 2022 at 12:03 PM, Larry McVoy <lm@mcvoy.com> wrote:


> On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
> 
> > On Sun, 11 Dec 2022, Michael Kj??rling wrote:
> > 
> > > By definition, if you write code as cleverly as you can, then you aren't
> > > clever enough to debug it...
> > 
> > Indeed...
> > 
> > I've always used the maxim "Write code as though the next person to
> > maintain it is an axe-wielding psychopath who knows where you live".
> 
> 
> My main job, for the 20 years until I retired, was to keep telling
> people that code that you wrote 6 months ago might as well have been
> written by someone else. Optimize for reading the code, not writing
> the code. It's read many.
> 
> 99.9% of the time, I detest clever code. .1% of the time, I need it.
> The problem is that smart engineers adore writing clever code. They
> usually, eventually, wise up.

[TUHS] Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

On Dec 11, 2022, at 12:04 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> 
> 
> On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
>> On Sun, 11 Dec 2022, Michael Kj??rling wrote:
>> 
>>> By definition, if you write code as cleverly as you can, then you aren't 
>>> clever enough to debug it...
>> 
>> Indeed...
>> 
>> I've always used the maxim "Write code as though the next person to 
>> maintain it is an axe-wielding psychopath who knows where you live".
> 
> My main job, for the 20 years until I retired, was to keep telling 
> people that code that you wrote 6 months ago might as well have been
> written by someone else.  Optimize for reading the code, not writing
> the code.  It's read many.
> 
> 99.9% of the time, I detest clever code.  .1% of the time, I need it.
> The problem is that smart engineers adore writing clever code.  They
> usually, eventually, wise up.

Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).

I do prefer clever/smart design to locally clever/smart code. For example,
using Schönhage-Strassen algorithm for multiplying very large numbers.
Or transforming a problem to use a much more efficient data structure
or making equivalent transforms which may be more efficient to compute.
Such code may not be immediately clear but with proper documentation
it is not difficult + you can solve much larger problems. But agreed
these come up much less often. 

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Steve Nickolas]

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

On Sun, 11 Dec 2022, Bakul Shah wrote:

> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time. As an example, building "Unix as a service" as
> user processes on top of a small microkernel could provide the same
> functionality using much clearer and much less code but it would be
> slower so we don't do it. Plan9 sort of went in that direction and it
> is much simpler (but that could also be because it is not hacked on so
> much).
>
> I do prefer clever/smart design to locally clever/smart code. For example,
> using Schönhage-Strassen algorithm for multiplying very large numbers.
> Or transforming a problem to use a much more efficient data structure
> or making equivalent transforms which may be more efficient to compute.
> Such code may not be immediately clear but with proper documentation
> it is not difficult + you can solve much larger problems. But agreed
> these come up much less often.

My attitude is: if I'm doing an ugly on nonobvious hack, I'll drop a 
comment saying what it does.

Most recently I wrote code that set a "magic flag" - and the comment 
explained why the flag was set (so that it would be immediately altered, 
forcing a redraw).

-uso.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
>
>
> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time. As an example, building "Unix as a service" as
> user processes on top of a small microkernel could provide the same
> functionality using much clearer and much less code but it would be
> slower so we don't do it. Plan9 sort of went in that direction and it
> is much simpler (but that could also be because it is not hacked on so
> much).
>

It's not necessarily true that microkernels are significantly slower.
They mostly got that reputation because of Mach and kernels like it
with their heavyweight IPC. Lightweight microkernels like QNX and the
L4 family generally have significantly better performance (in fact,
QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
a proof-of-concept network driver on a current version of seL4 is
significantly faster than a Linux network driver).

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
> On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
> >
> >
> > Agree that clear code is preferable to complicated code. But in practice
> > people sacrifice clarity for performance improvement all the time. Look
> > at the kernel code of any modern os. Everybody pays lip service to this
> > but most anything other than toy programs ends up getting needlessly
> > complicated over time. As an example, building "Unix as a service" as
> > user processes on top of a small microkernel could provide the same
> > functionality using much clearer and much less code but it would be
> > slower so we don't do it. Plan9 sort of went in that direction and it
> > is much simpler (but that could also be because it is not hacked on so
> > much).
> >
> 
> It's not necessarily true that microkernels are significantly slower.
> They mostly got that reputation because of Mach and kernels like it
> with their heavyweight IPC. Lightweight microkernels like QNX and the
> L4 family generally have significantly better performance (in fact,
> QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
> a proof-of-concept network driver on a current version of seL4 is
> significantly faster than a Linux network driver).

I was friends with one of the 3 people who were allowed to commit to QNX
kernel, Dan Hildebrandt (RIP 1998).  Those 3 people actually understood
the "micro" in "microkernel".   Early versions of QNX kernel fit in
the 4K instruction cache and left space for user programs.  They had
benchmarks that measured cache misses and refused commits that added to
the number of cache misses.

I met Dan because I had 10 terminals on a 80286 with I think 256KB
of memory and we were all editing and compiling and it worked way the
heck better than a 11/780 with 4MB with 30-40 students trying to do the
same thing.  I just wanted to talk to the people who made that possible,
it was after I had made a connection to Dennis.  The early QNX people
were special.  Dan and I had many conversations about the mono kernel
vs micro kernel, it was super fun because we weren't trying to convince
each other, we were comparing approaches.

I've never seen a team like Dan's team that cared as much as they did
about "micro" actually being micro.  That's the only way that that
approach works.  I can not state that enough, if you want to have a
micro kernel with message passing, if you don't fit in L1 cache,
you are a joke compared to a mono kernel.  You have to embrace
micro.

I had hoped that Mach would be like that but it was a giant mess.  I know
my opinion is not liked but I so wanted to like Mach, read the papers,
loved it, then read the code, hated it, a few years ago I was working
with the BSD team at Netflix, looked through the VM system from Mach
and it was a mess, it was the exact opposite of my experience at Sun
where I looked and looked and things came in focus and it made sense.
The Mach stuff has never made sense to me, I know Clem says it was a
research platform, OK, but the platform never got to a nice clean place.

If you think of microkernels and think Mach is it, nope, look at QNX.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Kevin Bowling]

On Sun, Dec 11, 2022 at 8:35 PM Larry McVoy <lm@mcvoy.com> wrote:
>
> On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
> > On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
> > >
> > >
> > > Agree that clear code is preferable to complicated code. But in practice
> > > people sacrifice clarity for performance improvement all the time. Look
> > > at the kernel code of any modern os. Everybody pays lip service to this
> > > but most anything other than toy programs ends up getting needlessly
> > > complicated over time. As an example, building "Unix as a service" as
> > > user processes on top of a small microkernel could provide the same
> > > functionality using much clearer and much less code but it would be
> > > slower so we don't do it. Plan9 sort of went in that direction and it
> > > is much simpler (but that could also be because it is not hacked on so
> > > much).
> > >
> >
> > It's not necessarily true that microkernels are significantly slower.
> > They mostly got that reputation because of Mach and kernels like it
> > with their heavyweight IPC. Lightweight microkernels like QNX and the
> > L4 family generally have significantly better performance (in fact,
> > QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
> > a proof-of-concept network driver on a current version of seL4 is
> > significantly faster than a Linux network driver).
>
> I was friends with one of the 3 people who were allowed to commit to QNX
> kernel, Dan Hildebrandt (RIP 1998).  Those 3 people actually understood
> the "micro" in "microkernel".   Early versions of QNX kernel fit in
> the 4K instruction cache and left space for user programs.  They had
> benchmarks that measured cache misses and refused commits that added to
> the number of cache misses.
>
> I met Dan because I had 10 terminals on a 80286 with I think 256KB
> of memory and we were all editing and compiling and it worked way the
> heck better than a 11/780 with 4MB with 30-40 students trying to do the
> same thing.  I just wanted to talk to the people who made that possible,
> it was after I had made a connection to Dennis.  The early QNX people
> were special.  Dan and I had many conversations about the mono kernel
> vs micro kernel, it was super fun because we weren't trying to convince
> each other, we were comparing approaches.
>
> I've never seen a team like Dan's team that cared as much as they did
> about "micro" actually being micro.  That's the only way that that
> approach works.  I can not state that enough, if you want to have a
> micro kernel with message passing, if you don't fit in L1 cache,
> you are a joke compared to a mono kernel.  You have to embrace
> micro.
>
> I had hoped that Mach would be like that but it was a giant mess.  I know
> my opinion is not liked but I so wanted to like Mach, read the papers,
> loved it, then read the code, hated it, a few years ago I was working
> with the BSD team at Netflix, looked through the VM system from Mach
> and it was a mess, it was the exact opposite of my experience at Sun
> where I looked and looked and things came in focus and it made sense.
> The Mach stuff has never made sense to me, I know Clem says it was a
> research platform, OK, but the platform never got to a nice clean place.

Mach seems not incompatible to me to Windows NT ideas of layering,
especially in modern macOS guise, more so than a microkernel I'd call
it something like a "macrokernel" with a variety of ways to do
things.. dealer's choice for better and worse.  It is interesting to
note the success, often overlooked by more hardline *nix people, that
macOS (and iOS and tvOS and watchOS etc) is indeed mach (more so than
it is BSD as some people claim) and wildly successful.

> If you think of microkernels and think Mach is it, nope, look at QNX.
> --
> ---
> Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/11/22, Larry McVoy <lm@mcvoy.com> wrote:
>
> If you think of microkernels and think Mach is it, nope, look at QNX.
>
And yet, for some reason, QNX has had almost no influence on anything
else besides a few hobby OSes (only one of which ever reached a
reasonably mature state) as far as I can tell. I really don't get why
that is. As far as I know, I'm the only one who's working on a
QNX-like OS that's not QNX itself.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 10:26:37PM -0700, Andrew Warkentin wrote:
> On 12/11/22, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > If you think of microkernels and think Mach is it, nope, look at QNX.
> >
> And yet, for some reason, QNX has had almost no influence on anything
> else besides a few hobby OSes (only one of which ever reached a
> reasonably mature state) as far as I can tell. I really don't get why
> that is. As far as I know, I'm the only one who's working on a
> QNX-like OS that's not QNX itself.

It influenced me.

Lots of things that were the right answer failed to achieve success.
Beta vs VHS.  Plan 9.

Does that mean they were worthless?  I don't think so.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3091 bytes --]

On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
wrote:

> And yet, for some reason, QNX has had almost no influence on anything
>
Be careful with a statement like that.  It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you.   It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.

When a small, very lightweight UNIX-style programming API needed to be
used, QNX was often a favorite.

I sometimes think QNX must have had a really good salesperson in the
'Rust-Belt.'  I know I talked to several fans in companies doing that
work.   I do know of a least one firm that still uses it.  An inexpensive
x86 can be designed into a custom controller, and the only 'development' is
the customer interface to private HW.   The development system is a PC or
Vmware on an engineer's desk.

After Blackberry bought the company, it's interesting that they seem to be
all that is left of BB.  But they are still going strong: QNX Neutrino RTOS
<https://streaklinks.com/BUDwcGeMTFsTSXcFxQzjvNX5/https%3A%2F%2Fwww.qnx.com%2Fproducts%2Fintl%2Fneutrino_rtos%2F>

Funny, I loved Mach (still do), but it tried to be all things to all
people.  The QNX guys did not.   I also think it help in their success --
by the fact that the QNX folks concentrated on RT, while Mach tried to be
the replacement for all of BSD.    They both have their place ...  I'm
typing this on my macOS 13.0.1 (Ventura) M1 system, which is just the
current flavor of Mach.  As Tru64 hacker, as well as one of the folks that
work on Intel Paragon, which was OSF/1, all three are Mach based.

I also did some work with QNX back in the day and, like Larry, was always
very impressed.   At one time, I did some consulting in the Rust-Belt, and
the executive (*i.e.* -- Havard Business. School types) asked me if  *"this QNX
thing their engineers were using -- after all it was not from Microsoft,
IBM or DEC, of course.''* [they had converted/were in the process of
converting from DEC PMAX-based controllers running Ultrix to PCs running
QNX].   My analysis at the time, for a bunch of ex-Fortran Mech E's, had
done extremely well.  I told the execs then that this is good stuff; it's
going to save them buckets of money as it 'just works' (that was the core
SW in the automatic 'sorter' that at the time was being done under contract
for Amazon -- I know the CEO of that firm and they sold the same basic
system to UPS/FedEx/USPS -- they used to have a very cool movie taken
during the testing at FedEx with glasses full of champagne moving at 45
miles through the sorter without spilling -- the PMAX would never have been
able to do that).

Frankly, for anyone learning either about microkernels or RT, I would
certainly tell them to look at QNX.   Neither topic are what we call
'research' projects as much these days, but both have extremely practical
applications.

ᐧ

[-- Attachment #2: Type: text/html, Size: 5329 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Mon, Dec 12, 2022 at 10:30 AM Clem Cole <clemc@ccc.com> wrote:
>
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com> wrote:
>>
>> And yet, for some reason, QNX has had almost no influence on anything
>
> Be careful with a statement like that.  It's likely running in something in your car. and very likely to be running in something in the last Boeing or Airbus-based flight you took, and it was used when Amazon made the last delivery to you.   It has long been popular in process control/materials handling/robotics/fly-by-wire systems.
>
> When a small, very lightweight UNIX-style programming API needed to be used, QNX was often a favorite.
>
> I sometimes think QNX must have had a really good salesperson in the 'Rust-Belt.'  I know I talked to several fans in companies doing that work.   I do know of a least one firm that still uses it.  An inexpensive x86 can be designed into a custom controller, and the only 'development' is the customer interface to private HW.   The development system is a PC or Vmware on an engineer's desk.
>
> After Blackberry bought the company, it's interesting that they seem to be all that is left of BB.  But they are still going strong: QNX Neutrino RTOS

For a while, Neutrino was open source, but that seemed to change
quietly at some point. I can't seem to find the source code online
anywhere, which surprises me a bit: I'd have assumed anyone who got a
copy of the code under the OSS license would still be able to treat it
as open source, but maybe I'm just wrong.

        - Dan C.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole wrote:
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> wrote:
> 
> > And yet, for some reason, QNX has had almost no influence on anything
> >
> Be careful with a statement like that.  It's likely running in something
> in your car. and very likely to be running in something in the last Boeing
> or Airbus-based flight you took, and it was used when Amazon made the last
> delivery to you.   It has long been popular in process control/materials
> handling/robotics/fly-by-wire systems.
> 
> When a small, very lightweight UNIX-style programming API needed to be
> used, QNX was often a favorite.


Thanks for that Clem.  One question though, all those companies want
support.  Is there anyone still providing support?  I see QNX.com is
a thing but is there an actual team of good people working on it?

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

I don't know.  I'll try to poke and ask.  The engineer I used to work with
at the Mill and later Intelligrated has retired (a lot of that with our
generation).   But I still know some of the execs at the former
Intelligrated (now part of Honeywell), so I'll see if I can find out who
owns that product these days.  If I learn anything interesting, I'll pass
it back.

FWIW: BB is offering support: i.e.:   https://blackberry.qnx.com/en/support
claims such, but what does that mean?

Clem
ᐧ

On Mon, Dec 12, 2022 at 11:04 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole wrote:
> > On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> > wrote:
> >
> > > And yet, for some reason, QNX has had almost no influence on anything
> > >
> > Be careful with a statement like that.  It's likely running in something
> > in your car. and very likely to be running in something in the last
> Boeing
> > or Airbus-based flight you took, and it was used when Amazon made the
> last
> > delivery to you.   It has long been popular in process control/materials
> > handling/robotics/fly-by-wire systems.
> >
> > When a small, very lightweight UNIX-style programming API needed to be
> > used, QNX was often a favorite.
>
>
> Thanks for that Clem.  One question though, all those companies want
> support.  Is there anyone still providing support?  I see QNX.com is
> a thing but is there an actual team of good people working on it?
>

[-- Attachment #2: Type: text/html, Size: 2710 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Mon, 12 Dec 2022 at 17:04, Larry McVoy <lm@mcvoy.com> wrote:
>
> Thanks for that Clem.  One question though, all those companies want
> support.  Is there anyone still providing support?  I see QNX.com is
> a thing but is there an actual team of good people working on it?

QNX is alive and well and owned by Blackberry Ltd, the former Research
in Motion. One big Canadian tech co bought another.

They aren't doing _great_ but they're not dead by any means.

https://en.wikipedia.org/wiki/BlackBerry_Limited

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Well this has certainly piqued my curiosity on QNX.  One of my more long-term projects is to write a kernel (micro or otherwise), initially for RISC-V that can, if nothing else, serve as a springboard for my own projects.  I've thus far just been using AT&T and BSD UNIX kernels for research and inspiration, with a peek at Darwin every now and then to see how Apple/Mach does things, but could certainly stand to broaden my horizons.  That said, if I was to straight up port something as an exercise, I'd probably go for V7 given its ubiquity and the fact that work done there would likely serve as a good template for doing the same with other AT&T and BSD variants.  Plus, I think someone's already gone and done a V6 port to RISC-V, so I'd have stuff to refer to.

- Matt G.

------- Original Message -------
On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven@gmail.com> wrote:


> On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm@mcvoy.com wrote:
> 
> > Thanks for that Clem. One question though, all those companies want
> > support. Is there anyone still providing support? I see QNX.com is
> > a thing but is there an actual team of good people working on it?
> 
> 
> QNX is alive and well and owned by Blackberry Ltd, the former Research
> in Motion. One big Canadian tech co bought another.
> 
> They aren't doing great but they're not dead by any means.
> 
> https://en.wikipedia.org/wiki/BlackBerry_Limited
> 
> --
> Liam Proven ~ Profile: https://about.me/liamproven
> Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
> Twitter/LinkedIn: lproven ~ Skype: liamproven
> UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

It's been decades since I've used it, my comments are based on the QNX that
predated their POSIX conformance.  It was extremely light weight, fast,
and used as little memory as possible.  I literally had 10 active logins
(tty) with people editing and compiling on a 256KB 80286.  Nothing else
came close.  But it wasn't POSIX so it might be more bloated today.

On Mon, Dec 12, 2022 at 11:10:08PM +0000, segaloco via TUHS wrote:
> Well this has certainly piqued my curiosity on QNX.  One of my more long-term projects is to write a kernel (micro or otherwise), initially for RISC-V that can, if nothing else, serve as a springboard for my own projects.  I've thus far just been using AT&T and BSD UNIX kernels for research and inspiration, with a peek at Darwin every now and then to see how Apple/Mach does things, but could certainly stand to broaden my horizons.  That said, if I was to straight up port something as an exercise, I'd probably go for V7 given its ubiquity and the fact that work done there would likely serve as a good template for doing the same with other AT&T and BSD variants.  Plus, I think someone's already gone and done a V6 port to RISC-V, so I'd have stuff to refer to.
> 
> - Matt G.
> 
> ------- Original Message -------
> On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven@gmail.com> wrote:
> 
> 
> > On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm@mcvoy.com wrote:
> > 
> > > Thanks for that Clem. One question though, all those companies want
> > > support. Is there anyone still providing support? I see QNX.com is
> > > a thing but is there an actual team of good people working on it?
> > 
> > 
> > QNX is alive and well and owned by Blackberry Ltd, the former Research
> > in Motion. One big Canadian tech co bought another.
> > 
> > They aren't doing great but they're not dead by any means.
> > 
> > https://en.wikipedia.org/wiki/BlackBerry_Limited
> > 
> > --
> > Liam Proven ~ Profile: https://about.me/liamproven
> > Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
> > Twitter/LinkedIn: lproven ~ Skype: liamproven
> > UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/12/22, Clem Cole <clemc@ccc.com> wrote:
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> wrote:
>
>> And yet, for some reason, QNX has had almost no influence on anything
>>
> Be careful with a statement like that.  It's likely running in something
> in your car. and very likely to be running in something in the last Boeing
> or Airbus-based flight you took, and it was used when Amazon made the last
> delivery to you.   It has long been popular in process control/materials
> handling/robotics/fly-by-wire systems.
>

I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.

The only OSes with significant QNX influence to reach anything
resembling a mature state of which I am aware are VSTa
<https://vsta.org/> and WiNGs <http://wingsos.org/>, both of which
have been abandoned for quite a while. Besides those two, there was
also OpenBLT, which was able to run a simple shell and a few basic
utilities from its boot image but not much more, and RadiOS, which was
abandoned at a point where it couldn't run much more than a hello
world. I'm also working on my own QNX-like OS
<https://gitlab.com/uxrt/uxrt-toplevel> like I said earlier, although
it doesn't run user programs yet (I'm working on the VFS and IPC
transport layer at the moment).

The extensive commercial success of QNX makes it even more surprising
to me that it has had so little architectural influence. Similarly, I
don't get why people who bash microkernels always seem to think that
all of them are like Mach despite QNX being quite successful.

On 12/12/22, Larry McVoy <lm@mcvoy.com> wrote:
> It's been decades since I've used it, my comments are based on the QNX that
> predated their POSIX conformance.  It was extremely light weight, fast,
> and used as little memory as possible.  I literally had 10 active logins
> (tty) with people editing and compiling on a 256KB 80286.  Nothing else
> came close.  But it wasn't POSIX so it might be more bloated today.
>

QNX 4 and Neutrino are heavier than QNX "Classic", but they're still
fairly lightweight. QNX 4 had the fairly well-known demo disk with a
Photon desktop and browser on a single 1.4M floppy. Neutrino never had
anything quite like that (the smallest official live CD images of the
early versions are something like 60M in size) but it's still possible
to build images with it that are pretty small.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 07:00:34PM -0700, Andrew Warkentin wrote:
> On 12/12/22, Clem Cole <clemc@ccc.com> wrote:
> > On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> > wrote:
> >
> >> And yet, for some reason, QNX has had almost no influence on anything
> >>
> > Be careful with a statement like that.  It's likely running in something
> > in your car. and very likely to be running in something in the last Boeing
> > or Airbus-based flight you took, and it was used when Amazon made the last
> > delivery to you.   It has long been popular in process control/materials
> > handling/robotics/fly-by-wire systems.
> >
> 
> I'm well aware that QNX has been extremely successful commercially and
> can be found in a wide range of embedded systems. I'm specifically
> talking about architectural influence on other OSes.

Minix?  QNX predated that.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
>
>>
>> I'm well aware that QNX has been extremely successful commercially and
>> can be found in a wide range of embedded systems. I'm specifically
>> talking about architectural influence on other OSes.
>
> Minix?  QNX predated that.
>
Yes, QNX predated Minix by several years, but Minix was completely
independent and there was no QNX influence on it at all AFAIK.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew Warkentin wrote:
> On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
> >
> >>
> >> I'm well aware that QNX has been extremely successful commercially and
> >> can be found in a wide range of embedded systems. I'm specifically
> >> talking about architectural influence on other OSes.
> >
> > Minix?  QNX predated that.
> >
> Yes, QNX predated Minix by several years, but Minix was completely
> independent and there was no QNX influence on it at all AFAIK.

Have you talked to Andy and confirmed that?  I'd be quite surprised if
he hadn't played with QNX but who knows.  I wouldn't assume he hadn't.

And forgive me for asking, do you have some axe to grind against QNX
or something?

To me, it's not that surprising that the rest of the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach.  Don't get me started on Mach, it has defenders but I absolutely
hate it.  Mach is more of a distributed research OS that advertised 
itself as a microkernel.  There is _nothing_ micro about Mach.  It's
a big bloated mess.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Has anything material come of later Mach work other than knowledge and influence on other systems?  I don't have a complete picture but it's my understanding that there were plenty of developments after NeXT and OSF had their way with it that aren't reflected in, say, macOS.  Something about said OSs being forked before Mach 3, which is listed in some references as being more true to the micro kernel architecture.

As an aside, I wish Apple would actually engage with their "open source" nature more.  I'd love to plink around in the Darwin kernel but even with Google-fu have never managed to actually get a kernel build all the way through.  Their lack of documentation is a painful matter.  I'd love to bootstrap it on RISC-V if I could get there...

- Matt G.

------- Original Message -------
On Tuesday, December 13th, 2022 at 5:05 PM, Larry McVoy <lm@mcvoy.com> wrote:


> On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew Warkentin wrote:
> 
> > On 12/13/22, Larry McVoy lm@mcvoy.com wrote:
> > 
> > > > I'm well aware that QNX has been extremely successful commercially and
> > > > can be found in a wide range of embedded systems. I'm specifically
> > > > talking about architectural influence on other OSes.
> > > 
> > > Minix? QNX predated that.
> > 
> > Yes, QNX predated Minix by several years, but Minix was completely
> > independent and there was no QNX influence on it at all AFAIK.
> 
> 
> Have you talked to Andy and confirmed that? I'd be quite surprised if
> he hadn't played with QNX but who knows. I wouldn't assume he hadn't.
> 
> And forgive me for asking, do you have some axe to grind against QNX
> or something?
> 
> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach. Don't get me started on Mach, it has defenders but I absolutely
> hate it. Mach is more of a distributed research OS that advertised
> itself as a microkernel. There is nothing micro about Mach. It's
> a big bloated mess.
> --
> ---
> Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Rich Morin]

> On Dec 13, 2022, at 17:40, segaloco via TUHS <tuhs@tuhs.org> wrote:
> 
> ... I wish Apple would actually engage with their "open source" nature more.  I'd love to plink around in the Darwin kernel but even with Google-fu have never managed to actually get a kernel build all the way through.  Their lack of documentation is a painful matter.  I'd love to bootstrap it on RISC-V if I could get there...

You might want to take a look at http://www.puredarwin.org/ ...

-r

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
>
> Have you talked to Andy and confirmed that?  I'd be quite surprised if
> he hadn't played with QNX but who knows.  I wouldn't assume he hadn't.
>

I haven't actually talked to him about it. He definitely is aware of
QNX since he's mentioned it on a few occasions, but I'm not sure if he
was aware of it when he wrote the first version of Minix.

Personally I don't see a lot of resemblance between the two, besides
both being single-personality Unix-like microkernel OSes with
lightweight IPC. Minix is more akin to a "serverized" conventional
Unix, whereas QNX seems to embrace its microkernel-ness more fully
with its focus on extensibility and its fairly tight integration of
IPC transport layer and filesystem. There may have been a little bit
of influence, but it's not all that obvious to me.

The pre-3.x versions seem especially un-QNX-like with their more or
less closed set of servers. Even in 3.x, the kernel still seems to
have quite a bit of knowledge about what servers are present and what
messages they accept. QNX does colocate the process server in the
kernel, but it makes very few assumptions about user-mode servers.

>
> And forgive me for asking, do you have some axe to grind against QNX
> or something?
>

Quite the opposite, hence why I'm writing my own OS with a similar architecture.

>
> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach.  Don't get me started on Mach, it has defenders but I absolutely
> hate it.  Mach is more of a distributed research OS that advertised
> itself as a microkernel.  There is _nothing_ micro about Mach.  It's
> a big bloated mess.
>

Yes, I agree 100% that Mach is a complete and utter failure as a
microkernel, and seems to have almost single-handedly destroyed the
reputation of microkernels. I don't get why everyone was so focused on
Mach-like kernels when there was a better alternative that had been
around in some form for almost a decade before Mach (QNX wasn't the
first of its kind; it seems to have had pretty significant influence
from Thoth).

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[arnold]

Andrew Warkentin <andreww591@gmail.com> wrote:

> Yes, I agree 100% that Mach is a complete and utter failure as a
> microkernel, and seems to have almost single-handedly destroyed the
> reputation of microkernels. I don't get why everyone was so focused on
> Mach-like kernels when there was a better alternative that had been
> around in some form for almost a decade before Mach (QNX wasn't the
> first of its kind; it seems to have had pretty significant influence
> from Thoth).

I suspect because Mach was available if you had the right Unix licenses
and because it was hot in the research world in the mid 80s. Researchy
types tend to look at what other researchers are doing / using, it seems
to me often without knowledge of or caring about what people are using
in industry.  (My two cents, from having worked at universities.)

Arnold

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Brad Spencer]

arnold@skeeve.com writes:

> Andrew Warkentin <andreww591@gmail.com> wrote:
>
>> Yes, I agree 100% that Mach is a complete and utter failure as a
>> microkernel, and seems to have almost single-handedly destroyed the
>> reputation of microkernels. I don't get why everyone was so focused on
>> Mach-like kernels when there was a better alternative that had been
>> around in some form for almost a decade before Mach (QNX wasn't the
>> first of its kind; it seems to have had pretty significant influence
>> from Thoth).
>
> I suspect because Mach was available if you had the right Unix licenses
> and because it was hot in the research world in the mid 80s. Researchy
> types tend to look at what other researchers are doing / using, it seems
> to me often without knowledge of or caring about what people are using
> in industry.  (My two cents, from having worked at universities.)
>
> Arnold

In that time frame there was a number of microkernel designs.  One that
has not been mentioned was OS-9 for the 6809/68000 processor.  I used it
pretty extensively.  OS-9 was very unix like from the userland POV, when
you consider something like V5 unix, however it didn't share any of the
same command names, just many of the same concepts.  It was close enough
that if you had the C compiler, a very basic K&R compiler, you could get
some of the unix command to compile without too much trouble.  I ported
sed from the DEC user's group source and the termcap library from BSD
and created a varargs library for it.  OS-9 was very microkernel and
nothing like Mach or even Minix.  It was also very much positioned to
real time OS needs of the time and was not really marketed generally and
unless you happened to have a Color Computer from Radio Shack or was a
part of the nitch they served you would probably have never come across
it and it would have seemed to be expensive to acquire.  It was very
clean, but you needed to know 6809 or 68000 assembly to create anything
new for the OS itself, although at least one person had figured out how
to use the C compiler, sort of, to produce assembly that could be
assembled into device managers and device drivers.




-- 
Brad Spencer - brad@anduin.eldar.org - KC8VKS - http://anduin.eldar.org

[TUHS] Re: (TUHS -> COFF?) Re: Clever code

[Michael Kjörling]

On 14 Dec 2022 06:54 -0500, from brad@anduin.eldar.org (Brad Spencer):
> [...] but you needed to know 6809 or 68000 assembly to create anything
> new for the OS itself,

Wasn't that the norm at the time, though? As I recall one of the
things that really set UNIX apart from other operating systems up
until about the early 1990s was precisely how machine-independent it
was by virtue of (with the exception of the early versions) having
been written in something other than assembler.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Microware's OS-9 (was: Clever code)

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 4881 bytes --]

At 2022-12-14T06:54:04-0500, Brad Spencer wrote:
> arnold@skeeve.com writes:
> > I suspect because Mach was available if you had the right Unix
> > licenses and because it was hot in the research world in the mid
> > 80s. Researchy types tend to look at what other researchers are
> > doing / using, it seems to me often without knowledge of or caring
> > about what people are using in industry.  (My two cents, from having
> > worked at universities.)

The UNSW CSE department seemed to be a bit more outward facing than
that, at least in my brief exposure to it, long after the 1980s.

> In that time frame there was a number of microkernel designs.  One
> that has not been mentioned was OS-9 for the 6809/68000 processor.  I
> used it pretty extensively.  OS-9 was very unix like from the userland
> POV, when you consider something like V5 unix, however it didn't share
> any of the same command names, just many of the same concepts.

This is emphatically true.  I used this system as a kid on a 64KiB
machine, and I don't remember even a mention of Unix in the doorstop of
a manual by Dale Puckett and Peter Dibble (who gave you something like 6
chapters of architectural background before introducing the shell
prompt).  Maybe they did mention Unix , but since it had no meaning to
me at the time, it didn't sink in.  I think it is also possible they
avoided any names that they thought might draw legal ire from AT&T.

> It was close enough that if you had the C compiler, a very basic K&R
> compiler, you could get some of the unix command to compile without
> too much trouble.

Years later I went to college, landed on Sun IPC workstations, and
quickly recognized OS-9's "T/S Edit" as a vi clone, and its "T/S Word"
as a version of nroff.  There was also a "T/S Spell" product but I don't
recall it clearly enough to venture whether it was a clone of ispell.

> OS-9 was very microkernel

In that deployment environment, it had to be.

> and nothing like Mach or even Minix.

With the source of all three available, a technical paper analyzing and
contrasting them would be a worthwhile thing to have.  (It's unclear to
me if even a historical version of QNX is available for study.)

> It was also very much positioned to real time OS needs of the time and
> was not really marketed generally and unless you happened to have a
> Color Computer from Radio Shack

Lucky me!  How I yearned for a 128KiB Color Computer 3 so I could
upgrade to OS-9 Level 2 and the windowing system.  (512KiB was
preferred, but there had been a spike in RAM prices right about the time
the machine was released.  Not that greater market success would have
kept Tandy from under-promoting and eventually killing the machine.[1])

> It was very clean, but you needed to know 6809 or 68000 assembly to
> create anything new for the OS itself,

The 6809 was my first exposure to a (relatively) clean ISA design,
having come from the Z80.  It probably helped that I was born with a
big-endian head and thus had an instinctive revulsion to Intel byte
order at an extremely young age.

In the late 1990s, Apple decided they wanted to rebrand their operating
system (still "MacOS [Classic]" at the time), looked at Microware's
name for its system, and said, "right, we'll be having that".
Microware, having apparently so carefully followed the letter of
trademark law with respect to AT&T Unix, sued Apple for peddling "OS/9"
in the operating system market, and promptly got their asses handed to
them by the federal district court, which dutifully honored the foremost
principle of law: big people get to stomp smaller people as often, and
as hard, as they would like.[2]

(Later, apparently, Apple pointed out this precedent to Cisco with a
shark-toothed grin when Apple decided they wanted the name "iOS" for yet
another revitalizing rebrand of familiar technology.  Cisco rolled over
and took some undisclosed amount of money, which they promptly spent on
acquisitions--they then were suddenly startled by the proportion of op
ex going to salaries, and initiated layoffs.)

Apple's never changed its stripes, but OS-9 lives on, as Free Software,
under the name NitrOS-9.[4]

Regards,
Branden

[1] Here's a story you may have to sit down for from Frank Durda IV (now
    deceased) about how the same company knifed their m68k-based
    line--which ran XENIX--in the gut repeatedly.  It's hard to find
    this story via Web search so I've made a Facebook post
    temporarily(?) public.  I'd simply include it, but it's pretty long.

    https://www.facebook.com/g.branden.robinson/posts/pfbid0F8MrvauQ6KPQ1tytme9uDiWGvprXft5dsxUzABYtdTKA9viZhB6Q2nadvtP1aDNQl

[2] https://www.cnn.com/2000/TECH/computing/03/21/os9.suit.idg/index.html
[3] https://appleinsider.com/articles/10/06/08/cisco_licenses_ios_name_to_apple_screenshot_shows_iwork_on_iphone
[4] https://sourceforge.net/projects/nitros9/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Microware's OS-9 (was: Clever code)

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 1673 bytes --]

On Wed, Dec 14, 2022 at 10:15 AM G. Branden Robinson <
g.branden.robinson@gmail.com> wrote:


Microware, having apparently so carefully followed the letter of
> trademark law with respect to AT&T Unix, sued Apple for peddling "OS/9"
> in the operating system market, and promptly got their asses handed to
> them by the federal district court, which dutifully honored the foremost
> principle of law: big people get to stomp smaller people as often, and
> as hard, as they would like.[2]
>


To be fair, the judge decided that there wasn't a whole lot of risk of
consumer confusion between a consumer OS that ran only on Apple Macintoshes
and an OEM OS that did not.  The only actual victims were Mac people who
stumbled into comp.os.os9 and got seriously confused.

In any case, Microsoft v. Lindows pretty much established that sometimes
the little guy wins, even in trademark cases.  Microsoft claimed that
Lindows was infringing their trademark for Windows (it was a Linux distro
that came with Wine and some glue) and lost the case on prior-use grounds
(both Xerox and Apple).  So rather than risking all on a retrial and maybe
losing the Windows trademark altogether, they settled for US$20M and
Lindows changed its corporate and distro names to Linspire.

That said, "Windows" is a descriptive trademark, and those are always shaky
legally.

> Cisco rolled over
> and took some undisclosed amount of money,
>

Money undoubtedly flowed from Apple Computers to Apple Records when the
first company started selling music, too.  This sort of thing is routine.
While I was working at Chase Bank, they paid a small fortune to Chase
Research for the rights to "chase.com".

[-- Attachment #2: Type: text/html, Size: 3165 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Harald Arnesen]

Larry McVoy [14/12/2022 02.05]:

> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach.  Don't get me started on Mach, it has defenders but I absolutely
> hate it.  Mach is more of a distributed research OS that advertised
> itself as a microkernel.  There is_nothing_  micro about Mach.  It's
> a big bloated mess.

There was a rumour back in the early 90s that a new version of AmigaOS 
would be based on QNX. This was just before Commodore went bust, so 
nothing came out of it. A shame, I rather liked the Amiga and its 
so-called Workbench.
-- 
Hilsen Harald

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Wed, 14 Dec 2022 at 10:47, Harald Arnesen <skogtun@gmail.com> wrote:
>
> There was a rumour back in the early 90s that a new version of AmigaOS
> would be based on QNX.

Oh, it wasn't a rumour. The deal got quite advanced.

My now-employers covered it at the time:
https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/


> This was just before Commodore went bust, so
> nothing came out of it.

It was after. By this point it was Gateway Inc paying for it. There's
an account of what happened here:

https://www.trollaxor.com/2005/06/how-qnx-failed-amiga.html

The next-gen Amiga project was pretty much why QNX gained its GUI, Neutrino:

https://guidebookgallery.org/screenshots/qnx621

This is also the basis of the famous QNX Demo Disk, build by the late
great Dan Hildebrandt:

http://qnx.puslapiai.lt/qnxdemo/qnx_demo_disk.htm

>  A shame, I rather liked the Amiga and its
> so-called Workbench.

It was a good OS in its time and its admirers often call it a
microkernel, but IMVHO if all the code is in the same memory space,
that's not really a true microkernel.

Amiga Inc then went on to nearly do a deal with Tao Group for the much
more advanced Taos, in its later v2 incarnation as Intent/Elate.

https://wiki.c2.com/?TaoIntentOs

https://www.osnews.com/story/157/tao-group-on-elateos-amigade-and-more/

Ars has some info on this nearly-forgotten OS:

https://arstechnica.com/gadgets/2018/03/a-history-of-the-amiga-part-12-red-vs-blue/

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Harald Arnesen]

Liam Proven [15/12/2022 19.33]:

> On Wed, 14 Dec 2022 at 10:47, Harald Arnesen<skogtun@gmail.com>  wrote:
>> There was a rumour back in the early 90s that a new version of AmigaOS
>> would be based on QNX.
> Oh, it wasn't a rumour. The deal got quite advanced.
> 
> My now-employers covered it at the time:
> https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/

Thanks! Very interesting, I didn't know much of this.
-- 
Hilsen Harald

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Fri, 16 Dec 2022 at 11:43, Harald Arnesen <skogtun@gmail.com> wrote:
>
> Thanks! Very interesting, I didn't know much of this.

My pleasure.

I am a daily Unix user but my main interests tend to lie elsewhere.

I don't have any citations for this, but it looked to me, watching in
the magazines at the time, that _before_ the Amiga deal, QNX did run
on PCs for testing purposes, but I am not totally sure if it had a GUI
at all, and little to no multimedia support.

My impression is that QNX implemented that for Amiga Inc and then were
left with it when Amiga turned its gaze on Tao and Elate.

(Some of the best remaining info about Intent and Elate is in HN comments, e.g.

https://news.ycombinator.com/item?id=16053726

https://news.ycombinator.com/item?id=9807269

Which leads to:

http://www.uruk.org/emu/Taos.html )

I am curious to know if the TAOS virtual-processor,
all-binaries-are-CPU-independent, model influenced or inspired
Inferno. I think that Inferno postdates TAOS.

But returning to QNX: my impression is, they implemented a GUI and
multimedia frameworks for Amiga, Amiga changed its mind, and QNX
offered it as a PC dev kit for a while.

E.g.
https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3

But the only mass-market end-user-facing graphical multimedia-capable
QNX devices I know of were the Blackberry X smartphones. (And
cancelled tablet and netbook.)

I owned a Blackberry Passport. A lovely device with a lovely OS... but
too late and it flopped.

So, oddly, and accidentally and unintentionally, Amiga Inc was
directly responsible for the Blackberry smartphone OS.

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Stuff Received]

On 2022-12-18 09:05, Liam Proven wrote (in part):
[...]
> But returning to QNX: my impression is, they implemented a GUI and
> multimedia frameworks for Amiga, Amiga changed its mind, and QNX
> offered it as a PC dev kit for a while.
> 
> E.g.
> https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3

QNX initially ran only on x86 kit. I do not recall a GUI at that time.

> But the only mass-market end-user-facing graphical multimedia-capable
> QNX devices I know of were the Blackberry X smartphones. (And
> cancelled tablet and netbook.)

The BB10 i/f was designed by TAT 
(https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of-becoming-awesome.html).

> I owned a Blackberry Passport. A lovely device with a lovely OS... but
> too late and it flopped.

Quite so.

N.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Sun, 18 Dec 2022 at 16:08, Stuff Received <stuff@riddermarkfarm.ca> wrote:
>
> The BB10 i/f was designed by TAT
> (https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of-becoming-awesome.html).

Oh, fascinating. I did not know that -- thank you!

> > I owned a Blackberry Passport. A lovely device with a lovely OS... but
> > too late and it flopped.
>
> Quite so.

One of the slightly stranger things in the smartphone OS space
recently, for me, has been watching Huawei's response to the licensing
fight with Google.

Huawei now promotes its devices as running "Harmony OS" with a lot of
hot air about microkernels, convergence, cross-platform, blah blah.

It's nothing of the kind. It's AOSP, the open-source core of Android,
with their own skin on top.

I don't think Huawei has the tech skills to build a new OS as it's
claimed it has...

But it could make some of those claims come true if it licensed BB10
and spent a bit on updating it.



-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/18/22, Liam Proven <lproven@gmail.com> wrote:
>
> I don't have any citations for this, but it looked to me, watching in
> the magazines at the time, that _before_ the Amiga deal, QNX did run
> on PCs for testing purposes, but I am not totally sure if it had a GUI
> at all, and little to no multimedia support.
>
> My impression is that QNX implemented that for Amiga Inc and then were
> left with it when Amiga turned its gaze on Tao and Elate.
>

QNX Classic and 4.x only ran on x86 machines, most of which were
either standard PCs or at least sort of PC-like (although AFAIK the
ability to run without a BIOS was present very early on). It was quite
common to run QNX on desktops as a development host for embedded
systems AFAIK.

GUIs for QNX predate the Amiga deal, and have existed since the late
80s. The original was QNX Windows, which was either a reimplementation
or port (not quite sure which) of Open Look on a custom non-X11 window
server, running on later versions of 2.x and all versions of 4.x.

Later versions of 4.x added Photon 1, which looks like a cross between
Motif and Windows 9x, again based on a custom window server (this time
with a rather unconventional multi-process architecture). The 90s-era
demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
disk back in the 80s but this didn't have a GUI).

6.0 came with Photon 2, which is still Win9x-ish in terms of
organization and is a fairly straightforward evolution of Photon 1,
although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
out slightly after the Amiga deal, so that might be the reason for the
Amiga-like theming.

Neutrino was not the name of a GUI, but rather of the entire OS that
succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
incapable of self-hosting and were developed alongside 4.x; 6.0 was
the first mainline QNX version to be Neutrino-based).

>
> But the only mass-market end-user-facing graphical multimedia-capable
> QNX devices I know of were the Blackberry X smartphones. (And
> cancelled tablet and netbook.)
>

There were also the i-Opener (running 4.25) and Audrey (running 6.0)
internet appliances of the late 90s.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Tue, 20 Dec 2022 at 09:31, Andrew Warkentin <andreww591@gmail.com> wrote:

> QNX Classic and 4.x only ran on x86 machines, most of which were
> either standard PCs or at least sort of PC-like (although AFAIK the
> ability to run without a BIOS was present very early on). It was quite
> common to run QNX on desktops as a development host for embedded
> systems AFAIK.
>
> GUIs for QNX predate the Amiga deal, and have existed since the late
> 80s. The original was QNX Windows, which was either a reimplementation
> or port (not quite sure which) of Open Look on a custom non-X11 window
> server, running on later versions of 2.x and all versions of 4.x.
>
> Later versions of 4.x added Photon 1, which looks like a cross between
> Motif and Windows 9x, again based on a custom window server (this time
> with a rather unconventional multi-process architecture). The 90s-era
> demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
> disk back in the 80s but this didn't have a GUI).

Fascinating. Thanks for setting the record straight, and I apologise
for my inaccurate speculation.
>
> 6.0 came with Photon 2, which is still Win9x-ish in terms of
> organization and is a fairly straightforward evolution of Photon 1,
> although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
> out slightly after the Amiga deal, so that might be the reason for the
> Amiga-like theming.
>
> Neutrino was not the name of a GUI, but rather of the entire OS that
> succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
> incapable of self-hosting and were developed alongside 4.x; 6.0 was
> the first mainline QNX version to be Neutrino-based).

Aha!

> There were also the i-Opener (running 4.25) and Audrey (running 6.0)
> internet appliances of the late 90s.

I did not realise they were QNX devices. I vaguely thought the 3Com
Audrey was based on BeIA, in fact, but I must be confusing it with
something else -- perhaps the Sony eVilla.


-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> 
> It's not necessarily true that microkernels are significantly slower.

uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:

  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc

or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.

At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

Wasn't there some statement that QNX dropped some of these?  Copy plus
context switch?

On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> > 
> > It's not necessarily true that microkernels are significantly slower.
> 
> uKernels are usually quite fast as they do so little. What can be slow
> is emulating a Unix like OS on top due to context switches. For instance,
> a user process doing read() will have the following context switches:
> 
>   userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> 
> or worse (I didn't account for a few things). Because of this even some
> uKernels run a few critical services + drivers in the supervisor mode.
> But overall slowdown of such a unix emulation will very much depend on the
> workload and also what kind of performance improvements you are willing to
> try in a complex kernel vs same services running in user mode.
> 
> At present the linux kernel has about 31+ Million lines (accounting for
> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> This LoC growth is entirely understandable but I wonder how things may
> have turned out in an alternate universe of uKernel based designs....

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

Don't see how unless they put multiple related services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.

> On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> Wasn't there some statement that QNX dropped some of these?  Copy plus
> context switch?
> 
> On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
>> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
>>> 
>>> It's not necessarily true that microkernels are significantly slower.
>> 
>> uKernels are usually quite fast as they do so little. What can be slow
>> is emulating a Unix like OS on top due to context switches. For instance,
>> a user process doing read() will have the following context switches:
>> 
>>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
>> 
>> or worse (I didn't account for a few things). Because of this even some
>> uKernels run a few critical services + drivers in the supervisor mode.
>> But overall slowdown of such a unix emulation will very much depend on the
>> workload and also what kind of performance improvements you are willing to
>> try in a complex kernel vs same services running in user mode.
>> 
>> At present the linux kernel has about 31+ Million lines (accounting for
>> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
>> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
>> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
>> This LoC growth is entirely understandable but I wonder how things may
>> have turned out in an alternate universe of uKernel based designs....
> 
> -- 
> ---
> Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> Don't see how unless they put multiple related services in the same
> address space, which reduces context switching but tends toward a
> monokernel (& increased coupling). Unless I am misunderstanding you.

I don't see why two services in a microkernel couldn't arrange to
share a region of memory and implement bidirectional queues
between themselves. With an appropriate signalling mechanism,
you'd still be context switching but avoiding a lot of copying.

        - Dan C.


> > On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > Wasn't there some statement that QNX dropped some of these?  Copy plus
> > context switch?
> >
> > On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> >> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> >>>
> >>> It's not necessarily true that microkernels are significantly slower.
> >>
> >> uKernels are usually quite fast as they do so little. What can be slow
> >> is emulating a Unix like OS on top due to context switches. For instance,
> >> a user process doing read() will have the following context switches:
> >>
> >>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> >>
> >> or worse (I didn't account for a few things). Because of this even some
> >> uKernels run a few critical services + drivers in the supervisor mode.
> >> But overall slowdown of such a unix emulation will very much depend on the
> >> workload and also what kind of performance improvements you are willing to
> >> try in a complex kernel vs same services running in user mode.
> >>
> >> At present the linux kernel has about 31+ Million lines (accounting for
> >> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> >> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> >> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> >> This LoC growth is entirely understandable but I wonder how things may
> >> have turned out in an alternate universe of uKernel based designs....
> >
> > --
> > ---
> > Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat
>

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross wrote:
> On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> > Don't see how unless they put multiple related services in the same
> > address space, which reduces context switching but tends toward a
> > monokernel (& increased coupling). Unless I am misunderstanding you.
> 
> I don't see why two services in a microkernel couldn't arrange to
> share a region of memory and implement bidirectional queues
> between themselves. With an appropriate signalling mechanism,
> you'd still be context switching but avoiding a lot of copying.

My mind went to similar thoughts.  How did QNX manage the page cache?
Did they have mmap?

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Thu, Dec 15, 2022 at 9:07 AM Larry McVoy <lm@mcvoy.com> wrote:
> On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross wrote:
> > On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> > > Don't see how unless they put multiple related services in the same
> > > address space, which reduces context switching but tends toward a
> > > monokernel (& increased coupling). Unless I am misunderstanding you.
> >
> > I don't see why two services in a microkernel couldn't arrange to
> > share a region of memory and implement bidirectional queues
> > between themselves. With an appropriate signalling mechanism,
> > you'd still be context switching but avoiding a lot of copying.
>
> My mind went to similar thoughts.  How did QNX manage the page cache?
> Did they have mmap?

I don't know if they did historically, but I kind of doubt it; QNX predates
4.2BSD, and I imagine there wasn't much influence from TENEX/PMAP).
But they seem to now:
https://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.getting_started/topic/s1_resmgr_io_mmap.html

No idea how they handle page caching. That's an interesting question,
given how they adopted POSIX and try to at least give the outward
appearance of being Unix-y.

Given the emphasis on the message-based architecture, where messages
can be sent between nodes, I doubt there's that much support for shared
memory as a basis for IPC; I think they favored explicit messaging copying
facilitated by the ukernel. Whether you could build something bespoke using
the provided primitives is another matter.

        - Dan C.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

I wasn't clear.  There was someone who said that the copy to another
address space included a context switch to the receiving process
(I think that was it) that resulted in better performance.  Perhaps
the person who said that can elaborate.

On Wed, Dec 14, 2022 at 09:36:12PM -0800, Bakul Shah wrote:
> Don't see how unless they put multiple related services in the same
> address space, which reduces context switching but tends toward a
> monokernel (& increased coupling). Unless I am misunderstanding you.
> 
> > On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > 
> > Wasn't there some statement that QNX dropped some of these?  Copy plus
> > context switch?
> > 
> > On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> >> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> >>> 
> >>> It's not necessarily true that microkernels are significantly slower.
> >> 
> >> uKernels are usually quite fast as they do so little. What can be slow
> >> is emulating a Unix like OS on top due to context switches. For instance,
> >> a user process doing read() will have the following context switches:
> >> 
> >>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> >> 
> >> or worse (I didn't account for a few things). Because of this even some
> >> uKernels run a few critical services + drivers in the supervisor mode.
> >> But overall slowdown of such a unix emulation will very much depend on the
> >> workload and also what kind of performance improvements you are willing to
> >> try in a complex kernel vs same services running in user mode.
> >> 
> >> At present the linux kernel has about 31+ Million lines (accounting for
> >> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> >> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> >> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> >> This LoC growth is entirely understandable but I wonder how things may
> >> have turned out in an alternate universe of uKernel based designs....
> > 
> > -- 
> > ---
> > Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/14/22, Larry McVoy <lm@mcvoy.com> wrote:
> Wasn't there some statement that QNX dropped some of these?  Copy plus
> context switch?
>

Yeah, QNX/L4-type IPC is usually just copying followed by a context
switch. Some such kernels (like seL4) don't even have full message
queues per endpoint (the scheduler queue sort of functions as an IPC
queue). Mach IPC is slow because of stuff like permission checking,
which I would assume involves iteration over a list of permitted
threads. QNX/L4-type kernels usually either use constant-time
permission checks (like the old "clans and chiefs" model used by L3
and early L4, or the more modern capability-oriented model used by
seL4 and some others), or lack kernel permission checking entirely
leaving it up to servers.

Another issue is that Mach-type kernels don't have what is known as
"direct process switching" AFAIK. When a synchronous message is sent
on a QNX/L4-type kernel, the kernel immediately switches to the
receiving process, bypassing the scheduler queue entirely, with the
remainder of the sender's timeslice being given to the receiver
(depending on the kernel, priorities may factor into this so it isn't
always quite that simple though). Mach-like kernels often require the
sender to wait for the kernel to decide to schedule the receiver based
on the queue, and then once the reply is sent there's another wait for
the kernel to again decide to schedule the sender again, which makes
for rather poor performance.

On 12/14/22, Bakul Shah <bakul@iitbombay.org> wrote:
> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
>>
>> It's not necessarily true that microkernels are significantly slower.
>
> uKernels are usually quite fast as they do so little. What can be slow
> is emulating a Unix like OS on top due to context switches. For instance,
> a user process doing read() will have the following context switches:
>
>   userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
>
> or worse (I didn't account for a few things). Because of this even some
> uKernels run a few critical services + drivers in the supervisor mode.
> But overall slowdown of such a unix emulation will very much depend on the
> workload and also what kind of performance improvements you are willing to
> try in a complex kernel vs same services running in user mode.

Yeah, excessive vertical layering can be bad for performance. QNX
normally follows a process-per-subsystem-instance architecture, so the
chain is just:

client -> (kernel) -> diskServer  -> (kernel) -> client

where the disk server includes the disk driver, partition table driver
(if applicable), and disk filesystem. The VFS layer isn't even
involved at all on reads, writes, and the like (it's pretty much only
there to deal with operations that involve paths and not those that
only involve FDs), whereas some other Unix-like microkernel OSes have
it act as an intermediary on all FS operations. A lot of the time,
protection domains correspond more to subsystem instances rather than
layer instances, so there really isn't much harm in merging all layers
of a subsystem into a single process for the sake of performance. When
there is a benefit to separation of layers into different processes,
it is possible to use tap-type drivers to allow running subsystem
instances that only contain some of the layers (QNX doesn't do this
AFAIK, but my OS will).

[TUHS] Re: Clever code

[Michael Kjörling]

On 11 Dec 2022 18:15 -0800, from bakul@iitbombay.org (Bakul Shah):
> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time.

Performant code does not need to stand in opposition to clear code.
And if you are writing performance-critical or size-critical code
(such as the QNX microkernel that someone brought up), then of course
you might end up needing to do things in non-obvious ways; that's not
the point here. Clever is fine IMO _where the cleverness provides an
actual benefit in a real-world scenario_, and an operating system
kernel (and a language standard library) is a rather special type of
program where sometimes you have to do things in somewhat non-obvious
ways because of the environment the code is meant to execute in. But a
significant portion of the time, where I see "clever" code there is
_no_ significant benefit to the cleverness; it's often more about
"showing off", or saving a few source code characters at the expense
of at-a-glance readability, than it is about actual usefulness and
necessity.

Necessary complexity in order to solve the problem, combined with
things like performance requirements, may sometimes require clever
code. At that point, there is a benefit to the cleverness. But one can
still aim to write the clever code _clearly_, with everything from
comments to good variable and function names to formatting that
enhances readability by for example grouping related operations
together in the source code to keeping related parts grouped together
and separate from other code. Just such a simple thing as that I often
add extra parenthesis over and beyond what's actually required based
on operator precedence, because doing so makes it clearer what goes
together and unless the compiler is severely braindead by modern
standards, doing so costs _nothing_ past the compilation stage.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dave Horsfall]

On Sun, 11 Dec 2022, Larry McVoy wrote:

> > I've always used the maxim "Write code as though the next person to 
> > maintain it is an axe-wielding psychopath who knows where you live".
> 
> My main job, for the 20 years until I retired, was to keep telling 
> people that code that you wrote 6 months ago might as well have been 
> written by someone else.  Optimize for reading the code, not writing the 
> code.  It's read many.

I should clarify that on the odd occasion that I write obscure code it's 
for efficiency, and I comment it well because *I* might be the one to 
maintain it six months hence :-)

-- Dave

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Chet Ramey]

On 12/12/22 4:34 PM, Dave Horsfall wrote:

>> My main job, for the 20 years until I retired, was to keep telling
>> people that code that you wrote 6 months ago might as well have been
>> written by someone else.  Optimize for reading the code, not writing the
>> code.  It's read many.
> 
> I should clarify that on the odd occasion that I write obscure code it's
> for efficiency, and I comment it well because *I* might be the one to
> maintain it six months hence :-)

Six months? Try ten years. :-)

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

[TUHS] Re: Clever code

[Rudi Blom]

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

I vaguely remember having read here about 'clever code' which took into
account the time a magnetic drum needed to rotate in order to optimise
access.

Similarly I can imagine that with resource restraints you sometimes need to
be clever in order to get your program to fit. Of course, any such
cleverness needs extra documentation.

I only ever programmed in user space but even then without lots of comment
in my code I may already start wondering what I did after only a few months
past.

Cheers,
uncle rubl
-- 
The more I learn the better I understand I know nothing.

[-- Attachment #2: Type: text/html, Size: 800 bytes --]

[TUHS] Re: Clever code

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

On Mon, Dec 12, 2022, 8:32 PM Rudi Blom <rudi.j.blom@gmail.com> wrote:

>
> I vaguely remember having read here about 'clever code' which took into
> account the time a magnetic drum needed to rotate in order to optimise
> access.
>

Yes. Many ways this was done. Biggest ones were interleaving and striding.
Interleaving allowed one a little processing time for each sector while the
disk fpu. So the next logical sector isn't the next physical... and the
sectors are numbered in adjacent tracks to take into account rotation and
seek times.... there is a lot of research here...

Warner

Similarly I can imagine that with resource restraints you sometimes need to
> be clever in order to get your program to fit. Of course, any such
> cleverness needs extra documentation.
>
> I only ever programmed in user space but even then without lots of comment
> in my code I may already start wondering what I did after only a few months
> past.
>
> Cheers,
> uncle rubl
> --
> The more I learn the better I understand I know nothing.
>
>

[-- Attachment #2: Type: text/html, Size: 1780 bytes --]

[TUHS] Re: Clever code

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

On Tue, 13 Dec 2022, Rudi Blom wrote:

> I vaguely remember having read here about 'clever code' which took into
> account the time a magnetic drum needed to rotate in order to optimise
> access.

Sounds like you're referring to SOAP (Symbolic Optimal Assembly Program) 
on the IBM 650; the programmer wrote the code "straight down" and SOAP 
reordered it for rotational latency.

> Similarly I can imagine that with resource restraints you sometimes need to
> be clever in order to get your program to fit. Of course, any such
> cleverness needs extra documentation.

Try writing a bootstrap program in 512 bytes :-)  Self-modifying code was
the order of the day...

> I only ever programmed in user space but even then without lots of comment
> in my code I may already start wondering what I did after only a few months
> past.

You could be clever in kernel space too, such as taking advantage of
the DATIP/DATO cycles on DEC's Unibus when updating a memory word i.e. 
read/modify/write.

-- Dave

[TUHS] Re: Clever code

[George Michaelson]

The "sticky" bit was quite clever. tell the OS to keep something
memory resident, so you can binary patch the back end without
worrying.

Copy on Write was enormously clever.  Keeping FD open across fork/exec
was fantastically clever.

making a null pointer be a valid address was probably not clever
(PDP11) but I expect somebody will explain to me why it was clever.

On Tue, Dec 13, 2022 at 1:53 PM Dave Horsfall <dave@horsfall.org> wrote:
>
> On Tue, 13 Dec 2022, Rudi Blom wrote:
>
> > I vaguely remember having read here about 'clever code' which took into
> > account the time a magnetic drum needed to rotate in order to optimise
> > access.
>
> Sounds like you're referring to SOAP (Symbolic Optimal Assembly Program)
> on the IBM 650; the programmer wrote the code "straight down" and SOAP
> reordered it for rotational latency.
>
> > Similarly I can imagine that with resource restraints you sometimes need to
> > be clever in order to get your program to fit. Of course, any such
> > cleverness needs extra documentation.
>
> Try writing a bootstrap program in 512 bytes :-)  Self-modifying code was
> the order of the day...
>
> > I only ever programmed in user space but even then without lots of comment
> > in my code I may already start wondering what I did after only a few months
> > past.
>
> You could be clever in kernel space too, such as taking advantage of
> the DATIP/DATO cycles on DEC's Unibus when updating a memory word i.e.
> read/modify/write.
>
> -- Dave

[TUHS] Re: Clever code

[Ralph Corderoy]

Hi George,

> The "sticky" bit was quite clever. tell the OS to keep something
> memory resident

Was it ever more than a hint?  I've not heard of it locking or pinning
an executable to memory.

> so you can binary patch the back end without worrying.

I think it altered the worrying.  :-)

If not sticky, ‘chmod -x’ the file, ensure no process is running it from
before the chmod, patch, ‘chmod +x’.

If sticky then ‘chmod -tx’, ensure no process is running it from before
the chmod, ‘chmod +x’, run it and exit to remove any stuck copy,
‘chmod -x’, patch, chmod ‘+tx’.

I may have got the detail wrong but avoiding the unpatched copy
lingering in memory is the aim.

-- 
Cheers, Ralph.

[TUHS] Re: Clever code

[Dagobert Michelsen]

Hi Ralph,

> Am 13.12.2022 um 09:05 schrieb Ralph Corderoy <ralph@inputplus.co.uk>:
> 
> Hi George,
> 
>> The "sticky" bit was quite clever. tell the OS to keep something
>> memory resident
> 
> Was it ever more than a hint?  I've not heard of it locking or pinning
> an executable to memory.

Funny thing is that it meant the opposite on Solaris: files with the
sticky bit set avoided the file to go through the page cache and therefore
would never be held in memory. This was e.g. set for swapfiles.
Obviously it is counterproductive to cache stuff in memory which is going
to be swapped out...


Best regards

  — Dago

-- 
"You don't become great by trying to be great, you become great by wanting to do something,
and then doing it so hard that you become great in the process." - xkcd #896

[TUHS] Re: Clever code

[Ralph Corderoy]

Hi Dave,

> Rudi Blom wrote:
> > I vaguely remember having read here about 'clever code' which took
> > into account the time a magnetic drum needed to rotate in order to
> > optimise access.
>
> Sounds like you're referring to SOAP (Symbolic Optimal Assembly
> Program) 

I'd have thought the most widespread tale of drum-rotation time is the
wonderful prose version of ‘The Story of Mel’.

-- 
Cheers, Ralph.

[TUHS] Re: Clever code

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

On Tue, 13 Dec 2022, Ralph Corderoy wrote:

> I'd have thought the most widespread tale of drum-rotation time is the 
> wonderful prose version of ‘The Story of Mel’.

Ah yes, a classic!

-- Dave

[TUHS] Re: Clever code

[John P. Linderman]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

There was a story that old hands would torment newcomers to the IBM 650
by tinkering with the optimizer to make it as slow as possible (and, with
rotating drums, that could be VERY slow). Then they'd look at the
newcomer's code, make a trivial change, run it with the real optimizer,
and get dazzling improvements.

I also recall punched card bootstrap programs for the IBM 7094 that
would load column binary when run column binary, and load row binary
when run row binary. -- jpl

On Mon, Dec 12, 2022 at 10:53 PM Dave Horsfall <dave@horsfall.org> wrote:

> On Tue, 13 Dec 2022, Rudi Blom wrote:
>
> > I vaguely remember having read here about 'clever code' which took into
> > account the time a magnetic drum needed to rotate in order to optimise
> > access.
>
> Sounds like you're referring to SOAP (Symbolic Optimal Assembly Program)
> on the IBM 650; the programmer wrote the code "straight down" and SOAP
> reordered it for rotational latency.
>
> > Similarly I can imagine that with resource restraints you sometimes need
> to
> > be clever in order to get your program to fit. Of course, any such
> > cleverness needs extra documentation.
>
> Try writing a bootstrap program in 512 bytes :-)  Self-modifying code was
> the order of the day...
>
> > I only ever programmed in user space but even then without lots of
> comment
> > in my code I may already start wondering what I did after only a few
> months
> > past.
>
> You could be clever in kernel space too, such as taking advantage of
> the DATIP/DATO cycles on DEC's Unibus when updating a memory word i.e.
> read/modify/write.
>
> -- Dave

[-- Attachment #2: Type: text/html, Size: 2531 bytes --]

[TUHS] Re: Clever code

[Douglas McIlroy]

Apropos of accessing rotating storage, John Kelly used to describe the
Packard-Bell 250, which had a delay-line memory, as a machine where
addresses refer to time rather than space.

The PB 250 had two instruction-sequencing modes. In one mode, each
instruction included the address of its successor. In the other mode,
whatever popped out the delay line when the current instruction
completed would be executed next.

Doug

On Tue, Dec 13, 2022 at 6:48 AM John P. Linderman <jpl.jpl@gmail.com> wrote:
>
> There was a story that old hands would torment newcomers to the IBM 650
> by tinkering with the optimizer to make it as slow as possible (and, with
> rotating drums, that could be VERY slow). Then they'd look at the
> newcomer's code, make a trivial change, run it with the real optimizer,
> and get dazzling improvements.
>
> I also recall punched card bootstrap programs for the IBM 7094 that
> would load column binary when run column binary, and load row binary
> when run row binary. -- jpl
>
> On Mon, Dec 12, 2022 at 10:53 PM Dave Horsfall <dave@horsfall.org> wrote:
>>
>> On Tue, 13 Dec 2022, Rudi Blom wrote:
>>
>> > I vaguely remember having read here about 'clever code' which took into
>> > account the time a magnetic drum needed to rotate in order to optimise
>> > access.
>>
>> Sounds like you're referring to SOAP (Symbolic Optimal Assembly Program)
>> on the IBM 650; the programmer wrote the code "straight down" and SOAP
>> reordered it for rotational latency.
>>
>> > Similarly I can imagine that with resource restraints you sometimes need to
>> > be clever in order to get your program to fit. Of course, any such
>> > cleverness needs extra documentation.
>>
>> Try writing a bootstrap program in 512 bytes :-)  Self-modifying code was
>> the order of the day...
>>
>> > I only ever programmed in user space but even then without lots of comment
>> > in my code I may already start wondering what I did after only a few months
>> > past.
>>
>> You could be clever in kernel space too, such as taking advantage of
>> the DATIP/DATO cycles on DEC's Unibus when updating a memory word i.e.
>> read/modify/write.
>>
>> -- Dave

[TUHS] Re: Clever code

[arnold]

Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:

> Apropos of accessing rotating storage, John Kelly used to describe the
> Packard-Bell 250, which had a delay-line memory, as a machine where
> addresses refer to time rather than space.
>
> The PB 250 had two instruction-sequencing modes. In one mode, each
> instruction included the address of its successor. In the other mode,
> whatever popped out the delay line when the current instruction
> completed would be executed next.
>
> Doug

For us (relative) youngsters, can you explain some more how delay
line memory worked? The second mode you describe sounds like it
would be impossible to use if you wanted repeatable, reproducible
runs of your program.

Thanks,

Arnold

[TUHS] Re: Clever code

[Ralph Corderoy]

Hi Arnold,

> > The PB 250 had two instruction-sequencing modes.  In one mode, each
> > instruction included the address of its successor.  In the other
> > mode, whatever popped out the delay line when the current
> > instruction completed would be executed next.
...
> The second mode you describe sounds like it would be impossible to use
> if you wanted repeatable, reproducible runs of your program.

How so?  As long as the time taken by the current instruction was
constant then it would be known what's popping out of the delay line
when it's done.  And if the time varied, say because of an operand or
the need to carry, then that could be used to choose between addresses.
Either way, it's repeatable.

It is as if the PC register on today's CPU was steadily trundling
through program memory in parallel to the execution of the current
instruction and when the fetch cycle started, it got whatever PC was
pointing at.

BTW, there's https://en.wikipedia.org/wiki/Delay-line_memory if you
don't know much about them, though I don't think it covers how the
line's content was modified other than a simple block diagram showing
taps for input and output.
https://en.wikipedia.org/wiki/Delay-line_memory#/media/File:SEACComputer_010.png

-- 
Cheers, Ralph.

[TUHS] Re: Clever code

[Douglas McIlroy]

A delay line is logically like a drum, with circulating data that is
accessible only at one point on the circle. A delay line was
effectively a linear channel along which a train of data pulses was
sent. Pulses received at the far end were reshaped electronically. and
reinjected at the sending end. One kind of delay line was a mercury
column that carried acoustic pulses.. The PB 250 delay line was
magnetostrictive (a technology I know nothing about).

If instruction timing is known, then the next instruction to appear is
predictable. The only caveat is that instruction times should not be
data-dependent. You can lay out sequential code along the circle as
long as no instruction steps on one already placed. When that happens
you must switch modes to jump to an open spot, or perhaps insert nops
to jiggle the layout.

Doug

On Tue, Dec 13, 2022 at 9:31 AM <arnold@skeeve.com> wrote:
>
> Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:
>
> > Apropos of accessing rotating storage, John Kelly used to describe the
> > Packard-Bell 250, which had a delay-line memory, as a machine where
> > addresses refer to time rather than space.
> >
> > The PB 250 had two instruction-sequencing modes. In one mode, each
> > instruction included the address of its successor. In the other mode,
> > whatever popped out the delay line when the current instruction
> > completed would be executed next.
> >
> > Doug
>
> For us (relative) youngsters, can you explain some more how delay
> line memory worked? The second mode you describe sounds like it
> would be impossible to use if you wanted repeatable, reproducible
> runs of your program.
>
> Thanks,
>
> Arnold

[TUHS] Re: Clever code

[Stuff Received]

On 2022-12-13 10:10, Douglas McIlroy wrote:
> A delay line is logically like a drum, with circulating data that is
> accessible only at one point on the circle. A delay line was
> effectively a linear channel along which a train of data pulses was
> sent. Pulses received at the far end were reshaped electronically. and
> reinjected at the sending end.

I had always thought of a delay line as a precursor to a register (or 
stack) for storing intermediate results.  Is this not an accurate way of 
thinking about it?

N.


> One kind of delay line was a mercury
> column that carried acoustic pulses.. The PB 250 delay line was
> magnetostrictive (a technology I know nothing about).
> 
> If instruction timing is known, then the next instruction to appear is
> predictable. The only caveat is that instruction times should not be
> data-dependent. You can lay out sequential code along the circle as
> long as no instruction steps on one already placed. When that happens
> you must switch modes to jump to an open spot, or perhaps insert nops
> to jiggle the layout.
> 
> Doug
> 
> On Tue, Dec 13, 2022 at 9:31 AM <arnold@skeeve.com> wrote:
>>
>> Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:
>>
>>> Apropos of accessing rotating storage, John Kelly used to describe the
>>> Packard-Bell 250, which had a delay-line memory, as a machine where
>>> addresses refer to time rather than space.
>>>
>>> The PB 250 had two instruction-sequencing modes. In one mode, each
>>> instruction included the address of its successor. In the other mode,
>>> whatever popped out the delay line when the current instruction
>>> completed would be executed next.
>>>
>>> Doug
>>
>> For us (relative) youngsters, can you explain some more how delay
>> line memory worked? The second mode you describe sounds like it
>> would be impossible to use if you wanted repeatable, reproducible
>> runs of your program.
>>
>> Thanks,
>>
>> Arnold

[TUHS] Re: Clever code

[Ralph Corderoy]

Hi N.,

> I had always thought of a delay line as a precursor to a register (or
> stack) for storing intermediate results.  Is this not an accurate way
> of thinking about it?

As an example, https://en.wikipedia.org/wiki/EDVAC#Technical_description
says

    Physically, the computer comprised the following components:

        - a magnetic tape reader-recorder (Wilkes 1956:36 describes this
          as a wire recorder.)
        ...
        - a dual memory unit consisting of two sets of 64 mercury
          acoustic delay lines of eight words capacity on each line
          [1 Ki words]
        - three temporary delay-line tanks each holding a single word

It looks like the three temporaries were more akin to a stack or
registers with the main delay lines providing working memory distinct
from tape storage.

Another analogy for a delay line might be a steadily turning Rolodex
where the card on display can be read and then written, perhaps with a
different value, before it disappears.

-- 
Cheers, Ralph.

[TUHS] Re: Clever code

[Harald Arnesen]

Douglas McIlroy [13/12/2022 16.10]:

> A delay line is logically like a drum, with circulating data that is
> accessible only at one point on the circle. A delay line was
> effectively a linear channel along which a train of data pulses was
> sent. Pulses received at the far end were reshaped electronically. and
> reinjected at the sending end. One kind of delay line was a mercury
> column that carried acoustic pulses.

And according to Alan Turing, it might have been implemented with Gin 
and Tonic.

[TUHS] Re: Clever code

[arnold]

Thank you for the explanation.  The skill of the programmers who had to
write code for such machines amazes me. I might could have held all
that kind of info in my head when I was younger, but certainly not today...

Thanks,

Arnold

Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:

> A delay line is logically like a drum, with circulating data that is
> accessible only at one point on the circle. A delay line was
> effectively a linear channel along which a train of data pulses was
> sent. Pulses received at the far end were reshaped electronically. and
> reinjected at the sending end. One kind of delay line was a mercury
> column that carried acoustic pulses.. The PB 250 delay line was
> magnetostrictive (a technology I know nothing about).
>
> If instruction timing is known, then the next instruction to appear is
> predictable. The only caveat is that instruction times should not be
> data-dependent. You can lay out sequential code along the circle as
> long as no instruction steps on one already placed. When that happens
> you must switch modes to jump to an open spot, or perhaps insert nops
> to jiggle the layout.
>
> Doug
>
> On Tue, Dec 13, 2022 at 9:31 AM <arnold@skeeve.com> wrote:
> >
> > Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:
> >
> > > Apropos of accessing rotating storage, John Kelly used to describe the
> > > Packard-Bell 250, which had a delay-line memory, as a machine where
> > > addresses refer to time rather than space.
> > >
> > > The PB 250 had two instruction-sequencing modes. In one mode, each
> > > instruction included the address of its successor. In the other mode,
> > > whatever popped out the delay line when the current instruction
> > > completed would be executed next.
> > >
> > > Doug
> >
> > For us (relative) youngsters, can you explain some more how delay
> > line memory worked? The second mode you describe sounds like it
> > would be impossible to use if you wanted repeatable, reproducible
> > runs of your program.
> >
> > Thanks,
> >
> > Arnold

[TUHS] Re: Clever code

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 2329 bytes --]

Further on delay line storage:

Physically one of the most common ones was a cylinder of liquid mercury.
There was a device at one end for introducing pressure waves into the
mercury (think loudspeaker) and a device at the other end for measuring the
pressure waves arriving (think microphone).  The pulses that came off the
microphone end were then fed back to the loudspeaker end, after being
cleaned up.
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>


On Tue, Dec 13, 2022 at 10:12 AM Douglas McIlroy <
douglas.mcilroy@dartmouth.edu> wrote:

> A delay line is logically like a drum, with circulating data that is
> accessible only at one point on the circle. A delay line was
> effectively a linear channel along which a train of data pulses was
> sent. Pulses received at the far end were reshaped electronically. and
> reinjected at the sending end. One kind of delay line was a mercury
> column that carried acoustic pulses.. The PB 250 delay line was
> magnetostrictive (a technology I know nothing about).
>
> If instruction timing is known, then the next instruction to appear is
> predictable. The only caveat is that instruction times should not be
> data-dependent. You can lay out sequential code along the circle as
> long as no instruction steps on one already placed. When that happens
> you must switch modes to jump to an open spot, or perhaps insert nops
> to jiggle the layout.
>
> Doug
>
> On Tue, Dec 13, 2022 at 9:31 AM <arnold@skeeve.com> wrote:
> >
> > Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:
> >
> > > Apropos of accessing rotating storage, John Kelly used to describe the
> > > Packard-Bell 250, which had a delay-line memory, as a machine where
> > > addresses refer to time rather than space.
> > >
> > > The PB 250 had two instruction-sequencing modes. In one mode, each
> > > instruction included the address of its successor. In the other mode,
> > > whatever popped out the delay line when the current instruction
> > > completed would be executed next.
> > >
> > > Doug
> >
> > For us (relative) youngsters, can you explain some more how delay
> > line memory worked? The second mode you describe sounds like it
> > would be impossible to use if you wanted repeatable, reproducible
> > runs of your program.
> >
> > Thanks,
> >
> > Arnold
>

[-- Attachment #2: Type: text/html, Size: 3453 bytes --]

[TUHS] Re: Clever code

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 2739 bytes --]

Here is a page from the Computer History Museum on the topic:
https://www.computerhistory.org/revolution/memory-storage/8/309

about halfway down the page is a nice schematic.
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>


On Thu, Dec 15, 2022 at 1:06 PM Marc Donner <marc.donner@gmail.com> wrote:

> Further on delay line storage:
>
> Physically one of the most common ones was a cylinder of liquid mercury.
> There was a device at one end for introducing pressure waves into the
> mercury (think loudspeaker) and a device at the other end for measuring the
> pressure waves arriving (think microphone).  The pulses that came off the
> microphone end were then fed back to the loudspeaker end, after being
> cleaned up.
> =====
> nygeek.net
> mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>
>
>
> On Tue, Dec 13, 2022 at 10:12 AM Douglas McIlroy <
> douglas.mcilroy@dartmouth.edu> wrote:
>
>> A delay line is logically like a drum, with circulating data that is
>> accessible only at one point on the circle. A delay line was
>> effectively a linear channel along which a train of data pulses was
>> sent. Pulses received at the far end were reshaped electronically. and
>> reinjected at the sending end. One kind of delay line was a mercury
>> column that carried acoustic pulses.. The PB 250 delay line was
>> magnetostrictive (a technology I know nothing about).
>>
>> If instruction timing is known, then the next instruction to appear is
>> predictable. The only caveat is that instruction times should not be
>> data-dependent. You can lay out sequential code along the circle as
>> long as no instruction steps on one already placed. When that happens
>> you must switch modes to jump to an open spot, or perhaps insert nops
>> to jiggle the layout.
>>
>> Doug
>>
>> On Tue, Dec 13, 2022 at 9:31 AM <arnold@skeeve.com> wrote:
>> >
>> > Douglas McIlroy <douglas.mcilroy@dartmouth.edu> wrote:
>> >
>> > > Apropos of accessing rotating storage, John Kelly used to describe the
>> > > Packard-Bell 250, which had a delay-line memory, as a machine where
>> > > addresses refer to time rather than space.
>> > >
>> > > The PB 250 had two instruction-sequencing modes. In one mode, each
>> > > instruction included the address of its successor. In the other mode,
>> > > whatever popped out the delay line when the current instruction
>> > > completed would be executed next.
>> > >
>> > > Doug
>> >
>> > For us (relative) youngsters, can you explain some more how delay
>> > line memory worked? The second mode you describe sounds like it
>> > would be impossible to use if you wanted repeatable, reproducible
>> > runs of your program.
>> >
>> > Thanks,
>> >
>> > Arnold
>>
>

[-- Attachment #2: Type: text/html, Size: 4593 bytes --]

[TUHS] Re: Clever code

[Bakul Shah]

On Dec 12, 2022, at 7:30 PM, Rudi Blom <rudi.j.blom@gmail.com> wrote:
> 
> I vaguely remember having read here about 'clever code' which took into account the time a magnetic drum needed to rotate in order to optimise access.

Similar consideration applied in the early days of unix workstations.
Fortune 32:16 was a 5.6Mhz machine and couldn't process 1020KB/sec
(17 sectors/track of early ST412/ST506 disks) fast enough. As Warner
said, one dealt with it by formatting the disk so that the logical
blocks N & N+1 (from the OS PoV) were physically more than 1 sector
apart. No clever coding needed!

The "clever" coding we did was to use all 17 sectors rather than 16
+ 1 spare as was intended. Since our first disks were only 5MB in
size, we wanted to use all the space and typical error rate is much
less than 6%. This complicated handling bad blocks and slowed things
down as blocks with soft read errors were copied to blocks at the
very end of the disk. I don't recall whose idea it was but I was the
one who implemented it. I had an especially bad disk for testing on
which I used to build V7 kernels....

> Similarly I can imagine that with resource restraints you sometimes need to be clever in order to get your program to fit. Of course, any such cleverness needs extra documentation.

One has to be careful here as resource constraints change over time.
An optimization for rev N h/w can be a *pessimization* for later revs.
Even if you document code, people tend to leave "working code" alone.

> I only ever programmed in user space but even then without lots of comment in my code I may already start wondering what I did after only a few months past.

Over time comments tend to turn into fake news! Always go to the
primary sources!

[TUHS] Re: Clever code

[Ralph Corderoy]

Hi Bakul,

> Fortune 32:16 was a 5.6Mhz machine and couldn't process 1020KB/sec
> (17 sectors/track of early ST412/ST506 disks) fast enough.  As Warner
> said, one dealt with it by formatting the disk so that the logical
> blocks N & N+1 (from the OS PoV) were physically more than 1 sector
> apart.

Sticking with ST506 hard drives, by the time the 8 MHz ARM2 from Acorn
was reading a 56 MB Rodime, it was the drive which couldn't keep up so
executables were stored compressed on disk so the CPU had something to
do, uncompressing the sector's content, while it waited for the next
sector to arrive.  :-)

-- 
Cheers, Ralph.

[TUHS] Re: Clever code

[Bakul Shah]

On Dec 13, 2022, at 8:14 AM, Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> 
> Hi Bakul,
> 
>> Fortune 32:16 was a 5.6Mhz machine and couldn't process 1020KB/sec
>> (17 sectors/track of early ST412/ST506 disks) fast enough.  As Warner
>> said, one dealt with it by formatting the disk so that the logical
>> blocks N & N+1 (from the OS PoV) were physically more than 1 sector
>> apart.
> 
> Sticking with ST506 hard drives, by the time the 8 MHz ARM2 from Acorn
> was reading a 56 MB Rodime, it was the drive which couldn't keep up so
> executables were stored compressed on disk so the CPU had something to
> do, uncompressing the sector's content, while it waited for the next
> sector to arrive.  :-)

IIRC the slow part was due to running some common apps! Not much buffering
allowed on a 256KB machine so by the time the app asks for the next block,
on a 1:1 interleave the block would be past the read head and you had to
spend an extra revolution to grab it!

[TUHS] Re: Clever code

[Noel Chiappa]

    > From: Stuff Received

    > I had always thought of a delay line as a precursor to a register (or
    > stack) for storing intermediate results. Is this not an accurate way of
    > thinking about it?

No, not at all.

First: delay lines were a memory _technology_ (one that was inherently
serial, not random-access). They preceded all others.

Second: registers used to have two aspects - one now gone (and maybe the
second too). The first was that the _technology_ used to implement them
(latches built out of tubes, then transistors) was faster than main memory -
a distinction now mostly gone, especially since caches blur the speed
distinction between today's main memory and registers. The second was that
registers, being smaller in numbers, could be named with a few bits, allowing
them to be named with a small share of the bits in an instruction. (This one
still remains, although instructions are now so long it's probably less
important.)

Some delay-line machines had two different delay line sizes (since size is
equivalent to average access time) - what one might consider 'registers' were
kept in the small ones, for fast access at all times, whereas main memory
used the longer ones.

	Noel

[TUHS] Re: Clever code

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

At 2022-12-13T12:58:11-0500, Noel Chiappa wrote:
> ... registers used to have two aspects - one now gone (and maybe
> the second too). The first was that the _technology_ used to implement
> them (latches built out of tubes, then transistors) was faster than
> main memory - a distinction now mostly gone, especially since caches
> blur the speed distinction between today's main memory and registers.
> The second was that registers, being smaller in numbers, could be
> named with a few bits, allowing them to be named with a small share of
> the bits in an instruction. (This one still remains, although
> instructions are now so long it's probably less important.)

Maybe less important on x86, but the amount of space in the instruction
for encoding registers seems to me to have played a major role in the
design of the RV32I/E and C (compressed) extension instruction formats
of RISC-V.

https://riscv.org/wp-content/uploads/2017/05/riscv-spec-v2.2.pdf

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Clever code

[segaloco via TUHS]

Where RISC-V is very intentional on this, my reading has lead me to understand that many previous CPU architectures simply passed pieces of the opcode to further hardware in the microarchitecture, so it wasn't so much of a design a register system to fit in a specific bit width but rather a matter of bits 3-5 and 7-9 are connected directly to the two inputs of the ALU internally or something to that effect.  Hearsay of course, I wasn't there, but that's the explanation I've heard in the past.

Now how much settling on a bit width for the register field of opcodes influences the number of registers or vice versa, hard to say.  Did Motorola want a 3 bit register field in opcodes or a resolution of 8 registers per addressing mode in the 68k first for instance, and which decision then followed?  I don't know, maybe someone does?  In fact, that makes me now wonder if there are CPUs with non-power-of-two register counts outside of the early days.  Anything else would waste values in a bitfield.

- Matt G.

------- Original Message -------
On Tuesday, December 13th, 2022 at 10:51 AM, G. Branden Robinson <g.branden.robinson@gmail.com> wrote:


> At 2022-12-13T12:58:11-0500, Noel Chiappa wrote:
> 
> > ... registers used to have two aspects - one now gone (and maybe
> > the second too). The first was that the technology used to implement
> > them (latches built out of tubes, then transistors) was faster than
> > main memory - a distinction now mostly gone, especially since caches
> > blur the speed distinction between today's main memory and registers.
> > The second was that registers, being smaller in numbers, could be
> > named with a few bits, allowing them to be named with a small share of
> > the bits in an instruction. (This one still remains, although
> > instructions are now so long it's probably less important.)
> 
> 
> Maybe less important on x86, but the amount of space in the instruction
> for encoding registers seems to me to have played a major role in the
> design of the RV32I/E and C (compressed) extension instruction formats
> of RISC-V.
> 
> https://riscv.org/wp-content/uploads/2017/05/riscv-spec-v2.2.pdf
> 
> Regards,
> Branden

[TUHS] Re: Clever code

[Warren Toomey via TUHS]

I think we might move the discussion on memory technologies, CPU 
architectures etc. to COFF 😁

Thanks, Warren

[TUHS] Re: Clever code

[Luther Johnson]

[-- Attachment #1: Type: text/plain, Size: 3913 bytes --]

The CPU I have designed and (implemented, it's running in a Lattice FPGA 
right now)  has three general purpose registers, a frame pointer, and a 
stack pointer. But the encoding problem you mention is real. So instead 
of designing a scheme where the instruction word is split up into 
fields, I have the first byte as the instruction type, and then however 
many immediate data bytes (in this instruction set, 1, or 3) are 
necessary following. The first byte, after it is fetched, is simply fed 
to a lookup table, which then results in a 12 bit value, 6 bits for 
operation, and two 3 bit fields for the a and b registers - these 12 
bits go to the execution stage. This is a two register address, 24 bit 
machine - I designed it as a replacement for the Zilog eZ80, which has 
become hard to get. Anyhow, I get good code density and I've got lots of 
spare codes left. I've attached the ISA description.

I did go through lots of design alternatives to reach the parameters of 
this ISA - they key one was if I wanted to have the operations I needed, 
available across all the general purpose registers, that limited how 
many general purpose registers I could have and keep all the 
enumerations in less than 256 codes, with some to spare. Another set of 
of choices relates to how I wanted to implement C on this machine, and 
that I did not intend for it to support all possible styles of assembly 
language programming - it is meant to support code generated by the C 
compiler, with a minimum of assembly required.

This machine is called "COR24". I can describe the machine in further 
detail, or show you some sample code,  if you're interested.

Luther

On 12/13/2022 01:14 PM, segaloco via TUHS wrote:
> Where RISC-V is very intentional on this, my reading has lead me to understand that many previous CPU architectures simply passed pieces of the opcode to further hardware in the microarchitecture, so it wasn't so much of a design a register system to fit in a specific bit width but rather a matter of bits 3-5 and 7-9 are connected directly to the two inputs of the ALU internally or something to that effect.  Hearsay of course, I wasn't there, but that's the explanation I've heard in the past.
>
> Now how much settling on a bit width for the register field of opcodes influences the number of registers or vice versa, hard to say.  Did Motorola want a 3 bit register field in opcodes or a resolution of 8 registers per addressing mode in the 68k first for instance, and which decision then followed?  I don't know, maybe someone does?  In fact, that makes me now wonder if there are CPUs with non-power-of-two register counts outside of the early days.  Anything else would waste values in a bitfield.
>
> - Matt G.
>
> ------- Original Message -------
> On Tuesday, December 13th, 2022 at 10:51 AM, G. Branden Robinson <g.branden.robinson@gmail.com> wrote:
>
>
>> At 2022-12-13T12:58:11-0500, Noel Chiappa wrote:
>>
>>> ... registers used to have two aspects - one now gone (and maybe
>>> the second too). The first was that the technology used to implement
>>> them (latches built out of tubes, then transistors) was faster than
>>> main memory - a distinction now mostly gone, especially since caches
>>> blur the speed distinction between today's main memory and registers.
>>> The second was that registers, being smaller in numbers, could be
>>> named with a few bits, allowing them to be named with a small share of
>>> the bits in an instruction. (This one still remains, although
>>> instructions are now so long it's probably less important.)
>>
>> Maybe less important on x86, but the amount of space in the instruction
>> for encoding registers seems to me to have played a major role in the
>> design of the RV32I/E and C (compressed) extension instruction formats
>> of RISC-V.
>>
>> https://riscv.org/wp-content/uploads/2017/05/riscv-spec-v2.2.pdf
>>
>> Regards,
>> Branden


[-- Attachment #2: isa.txt --]
[-- Type: text/plain, Size: 14196 bytes --]

Inst    Description             Op      a,b     6+3+3 binary      Hex
----    -----------             --      ---     ---------------   ---
00      add   r0,r0             00      0,0     00 0000 000 000 | 000
01      add   r0,r1             00      0,1     00 0000 000 001 | 001
02      add   r0,r2             00      0,2     00 0000 000 010 | 002
03      add   r1,r0             00      1,0     00 0000 001 000 | 008
04      add   r1,r1             00      1,1     00 0000 001 001 | 009
05      add   r1,r2             00      1,2     00 0000 001 010 | 00a
06      add   r2,r0             00      2,0     00 0000 010 000 | 010
07      add   r2,r1             00      2,1     00 0000 010 001 | 011
08      add   r2,r2             00      2,2     00 0000 010 010 | 012

09      add   r0,dd             01      0,7     00 0001 000 111 | 047
0a      add   r1,dd             01      1,7     00 0001 001 111 | 04f
0b      add   r2,dd             01      2,7     00 0001 010 111 | 057
0c      add   sp,dd             01      4,7     00 0001 100 111 | 067

0d      and   r0,r1             02      0,1     00 0010 000 001 | 081
0e      and   r0,r2             02      0,2     00 0010 000 010 | 082
0f      and   r1,r0             02      1,0     00 0010 001 000 | 088
10      and   r1,r2             02      1,2     00 0010 001 010 | 08a
11      and   r2,r0             02      2,0     00 0010 010 000 | 090
12      and   r2,r1             02      2,1     00 0010 010 001 | 091

13      bra   dd                03      7,7     00 0011 111 111 | 0ff

14      brf   dd                04      7,7     00 0100 111 111 | 13f

15      brt   dd                05      7,7     00 0101 111 111 | 17f

16      ceq   r0,r1             06      0,1     00 0110 000 001 | 181
17      ceq   r0,r2             06      0,2     00 0110 000 010 | 182
18      ceq   r1,r2             06      1,2     00 0110 001 010 | 18a

19      cls   r0,r1             07      0,1     00 0111 000 001 | 1c1
1a      cls   r0,r2             07      0,2     00 0111 000 010 | 1c2
1b      cls   r1,r0             07      1,0     00 0111 001 000 | 1c8
1c      cls   r1,r2             07      1,2     00 0111 001 010 | 1ca
1d      cls   r2,r0             07      2,0     00 0111 010 000 | 1d0
1e      cls   r2,r1             07      2,1     00 0111 010 001 | 1d1

1f      clu   r0,r1             08      0,1     00 1000 000 001 | 201
20      clu   r0,r2             08      0,2     00 1000 000 010 | 202
21      clu   r1,r0             08      1,0     00 1000 001 000 | 208
22      clu   r1,r2             08      1,2     00 1000 001 010 | 20a
23      clu   r2,r0             08      2,0     00 1000 010 000 | 210
24      clu   r2,r1             08      2,1     00 1000 010 001 | 211

25      jal   r1,(r0)           09      1,0     00 1001 001 000 | 248

26      jmp   (r0)              0a      0,7     00 1010 000 111 | 287
27      jmp   (r1)              0a      1,7     00 1010 001 111 | 28f
28      jmp   (r2)              0a      2,7     00 1010 010 111 | 297

29      la    r0,dddddd         0b      0,7     00 1011 000 111 | 2c7
2a      la    r1,dddddd         0b      1,7     00 1011 001 111 | 2cf
2b      la    r2,dddddd         0b      2,7     00 1011 010 111 | 2d7

2c      lb    r0,dd(r0)         0c      0,0     00 1100 000 000 | 300
2d      lb    r0,dd(r1)         0c      0,1     00 1100 000 001 | 301
2e      lb    r0,dd(r2)         0c      0,2     00 1100 000 010 | 302
2f      lb    r0,dd(fp)         0c      0,3     00 1100 000 011 | 303
30      lb    r1,dd(r0)         0c      1,0     00 1100 001 000 | 308
31      lb    r1,dd(r1)         0c      1,1     00 1100 001 001 | 309
32      lb    r1,dd(r2)         0c      1,2     00 1100 001 010 | 30a
33      lb    r1,dd(fp)         0c      1,3     00 1100 001 011 | 30b
34      lb    r2,dd(r0)         0c      2,0     00 1100 010 000 | 310
35      lb    r2,dd(r1)         0c      2,1     00 1100 010 001 | 311
36      lb    r2,dd(r2)         0c      2,2     00 1100 010 010 | 312
37      lb    r2,dd(fp)         0c      2,3     00 1100 010 011 | 313

38      lbu   r0,dd(r0)         0d      0,0     00 1101 000 000 | 340
39      lbu   r0,dd(r1)         0d      0,1     00 1101 000 001 | 341
3a      lbu   r0,dd(r2)         0d      0,2     00 1101 000 010 | 342
3b      lbu   r0,dd(fp)         0d      0,3     00 1101 000 011 | 343
3c      lbu   r1,dd(r0)         0d      1,0     00 1101 001 000 | 348
3d      lbu   r1,dd(r1)         0d      1,1     00 1101 001 001 | 349
3e      lbu   r1,dd(r2)         0d      1,2     00 1101 001 010 | 34a
3f      lbu   r1,dd(fp)         0d      1,3     00 1101 001 011 | 34b
40      lbu   r2,dd(r0)         0d      2,0     00 1101 010 000 | 350
41      lbu   r2,dd(r1)         0d      2,1     00 1101 010 001 | 351
42      lbu   r2,dd(r2)         0d      2,2     00 1101 010 010 | 352
43      lbu   r2,dd(fp)         0d      2,3     00 1101 010 011 | 353

44      lc    r0,dd             0e      0,7     00 1110 000 111 | 387
45      lc    r1,dd             0e      1,7     00 1110 001 111 | 38f
46      lc    r2,dd             0e      2,7     00 1110 010 111 | 397

47      lcu   r0,dd             0f      0,7     00 1111 000 111 | 3c7
48      lcu   r1,dd             0f      1,7     00 1111 001 111 | 3cf
49      lcu   r2,dd             0f      2,7     00 1111 010 111 | 3d7

4a      lw    r0,dd(r0)         10      0,0     01 0000 000 000 | 400
4b      lw    r0,dd(r1)         10      0,1     01 0000 000 001 | 401
4c      lw    r0,dd(r2)         10      0,2     01 0000 000 010 | 402
4d      lw    r0,dd(fp)         10      0,3     01 0000 000 011 | 403
4e      lw    r1,dd(r0)         10      1,0     01 0000 001 000 | 408
4f      lw    r1,dd(r1)         10      1,1     01 0000 001 001 | 409
50      lw    r1,dd(r2)         10      1,2     01 0000 001 010 | 40a
51      lw    r1,dd(fp)         10      1,3     01 0000 001 011 | 40b
52      lw    r2,dd(r0)         10      2,0     01 0000 010 000 | 410
53      lw    r2,dd(r1)         10      2,1     01 0000 010 001 | 411
54      lw    r2,dd(r2)         10      2,2     01 0000 010 010 | 412
55      lw    r2,dd(fp)         10      2,3     01 0000 010 011 | 413

56      mov   r0,r1             11      0,1     01 0001 000 001 | 441
57      mov   r0,r2             11      0,2     01 0001 000 010 | 442
58      mov   r0,fp             11      0,3     01 0001 000 011 | 443
59      mov   r0,sp             11      0,4     01 0001 000 100 | 444
5a      mov   r1,r0             11      1,0     01 0001 001 000 | 448
5b      mov   r1,r2             11      1,2     01 0001 001 010 | 44a
5c      mov   r1,fp             11      1,3     01 0001 001 011 | 44b
5d      mov   r1,sp             11      1,4     01 0001 001 100 | 44c
5e      mov   r2,r0             11      2,0     01 0001 010 000 | 450
5f      mov   r2,r1             11      2,1     01 0001 010 001 | 451
60      mov   r2,fp             11      2,3     01 0001 010 011 | 453
61      mov   r2,sp             11      2,4     01 0001 010 100 | 454
62      mov   r0,c              11      0,5     01 0001 000 101 | 445
63      mov   r1,c              11      1,5     01 0001 001 101 | 44d
64      mov   r2,c              11      2,5     01 0001 010 101 | 455
65      mov   fp,sp             11      3,4     01 0001 011 100 | 45c
66      mov   sp,r0             11      4,0     01 0001 100 000 | 460
67      mov   sp,r1             11      4,1     01 0001 100 001 | 461
68      mov   sp,r2             11      4,2     01 0001 100 010 | 462
69      mov   sp,fp             11      4,3     01 0001 100 011 | 463

6a      mul   r0,r0             12      0,0     01 0010 000 000 | 480
6b      mul   r0,r1             12      0,1     01 0010 000 001 | 481
6c      mul   r0,r2             12      0,2     01 0010 000 010 | 482
6d      mul   r1,r0             12      1,0     01 0010 001 000 | 488
6e      mul   r1,r1             12      1,1     01 0010 001 001 | 489
6f      mul   r1,r2             12      1,2     01 0010 001 010 | 48a
70      mul   r2,r0             12      2,0     01 0010 010 000 | 490
71      mul   r2,r1             12      2,1     01 0010 010 001 | 491
72      mul   r2,r2             12      2,2     01 0010 010 010 | 492

73      or    r0,r1             13      0,1     01 0011 000 001 | 4c1
74      or    r0,r2             13      0,2     01 0011 000 010 | 4c2
75      or    r1,r0             13      1,0     01 0011 001 000 | 4c8
76      or    r1,r2             13      1,2     01 0011 001 010 | 4ca
77      or    r2,r0             13      2,0     01 0011 010 000 | 4d0
78      or    r2,r1             13      2,1     01 0011 010 001 | 4d1

79      pop     r0              14      0,4     01 0100 000 100 | 504
7a      pop     r1              14      1,4     01 0100 001 100 | 50c
7b      pop     r2              14      2,4     01 0100 010 100 | 514
7c      pop     fp              14      3,4     01 0100 011 100 | 51c

7d      push    r0              15      0,4     01 0101 000 100 | 544
7e      push    r1              15      1,4     01 0101 001 100 | 54c
7f      push    r2              15      2,4     01 0101 010 100 | 554
80      push    fp              15      3,4     01 0101 011 100 | 55c

81      sb    r0,dd(r1)         16      0,1     01 0110 000 001 | 581
82      sb    r0,dd(r2)         16      0,2     01 0110 000 010 | 582
83      sb    r0,dd(fp)         16      0,3     01 0110 000 011 | 583
84      sb    r1,dd(r0)         16      1,0     01 0110 001 000 | 588
85      sb    r1,dd(r2)         16      1,2     01 0110 001 010 | 58a
86      sb    r1,dd(fp)         16      1,3     01 0110 001 011 | 58b
87      sb    r2,dd(r0)         16      2,0     01 0110 010 000 | 590
88      sb    r2,dd(r1)         16      2,1     01 0110 010 001 | 591
89      sb    r2,dd(fp)         16      2,3     01 0110 010 011 | 593

8a      shl   r0,r1             17      0,1     01 0111 000 001 | 5c1
8b      shl   r0,r2             17      0,2     01 0111 000 010 | 5c2
8c      shl   r1,r0             17      1,0     01 0111 001 000 | 5c8
8d      shl   r1,r2             17      1,2     01 0111 001 010 | 5ca
8e      shl   r2,r0             17      2,0     01 0111 010 000 | 5d0
8f      shl   r2,r1             17      2,1     01 0111 010 001 | 5d1

90      sra   r0,r1             18      0,1     01 1000 000 001 | 601
91      sra   r0,r2             18      0,2     01 1000 000 010 | 602
92      sra   r1,r0             18      1,0     01 1000 001 000 | 608
93      sra   r1,r2             18      1,2     01 1000 001 010 | 60a
94      sra   r2,r0             18      2,0     01 1000 010 000 | 610
95      sra   r2,r1             18      2,1     01 1000 010 001 | 611

96      srl   r0,r1             19      0,1     01 1001 000 001 | 641
97      srl   r0,r2             19      0,2     01 1001 000 010 | 642
98      srl   r1,r0             19      1,0     01 1001 001 000 | 648
99      srl   r1,r2             19      1,2     01 1001 001 010 | 64a
9a      srl   r2,r0             19      2,0     01 1001 010 000 | 650
9b      srl   r2,r1             19      2,1     01 1001 010 001 | 651

9c      sub   r0,r1             1a      0,1     01 1010 000 001 | 681
9d      sub   r0,r2             1a      0,2     01 1010 000 010 | 682
9e      sub   r1,r0             1a      1,0     01 1010 001 000 | 688
9f      sub   r1,r2             1a      1,2     01 1010 001 010 | 68a
a0      sub   r2,r0             1a      2,0     01 1010 010 000 | 690
a1      sub   r2,r1             1a      2,1     01 1010 010 001 | 691

a2      sub   sp,dddddd         1b      4,7     01 1011 100 111 | 6e7

a3      sw    r0,dd(r0)         1c      0,0     01 1100 000 000 | 700
a4      sw    r0,dd(r1)         1c      0,1     01 1100 000 001 | 701
a5      sw    r0,dd(r2)         1c      0,2     01 1100 000 010 | 702
a6      sw    r0,dd(fp)         1c      0,3     01 1100 000 011 | 703
a7      sw    r1,dd(r0)         1c      1,0     01 1100 001 000 | 708
a8      sw    r1,dd(r1)         1c      1,1     01 1100 001 001 | 709
a9      sw    r1,dd(r2)         1c      1,2     01 1100 001 010 | 70a
aa      sw    r1,dd(fp)         1c      1,3     01 1100 001 011 | 70b
ab      sw    r2,dd(r0)         1c      2,0     01 1100 010 000 | 710
ac      sw    r2,dd(r1)         1c      2,1     01 1100 010 001 | 711
ad      sw    r2,dd(r2)         1c      2,2     01 1100 010 010 | 712
ae      sw    r2,dd(fp)         1c      2,3     01 1100 010 011 | 713

af      sxt   r0,r0             1d      0,0     01 1101 000 000 | 740
b0      sxt   r0,r1             1d      0,1     01 1101 000 001 | 741
b1      sxt   r0,r2             1d      0,2     01 1101 000 010 | 742
b2      sxt   r1,r0             1d      1,0     01 1101 001 000 | 748
b3      sxt   r1,r1             1d      1,1     01 1101 001 001 | 749
b4      sxt   r1,r2             1d      1,2     01 1101 001 010 | 74a
b5      sxt   r2,r0             1d      2,0     01 1101 010 000 | 750
b6      sxt   r2,r1             1d      2,1     01 1101 010 001 | 751
b7      sxt   r2,r2             1d      2,1     01 1101 010 010 | 752

b8      xor   r0,r1             1e      0,1     01 1110 000 001 | 781
b9      xor   r0,r2             1e      0,2     01 1110 000 010 | 782
ba      xor   r1,r0             1e      1,0     01 1110 001 000 | 788
bb      xor   r1,r2             1e      1,2     01 1110 001 010 | 78a
bc      xor   r2,r0             1e      2,0     01 1110 010 000 | 790
bd      xor   r2,r1             1e      2,1     01 1110 010 001 | 791

be      zxt   r0,r0             1f      0,0     01 1111 000 000 | 7c0
bf      zxt   r0,r1             1f      0,1     01 1111 000 001 | 7c1
c0      zxt   r0,r2             1f      0,2     01 1111 000 010 | 7c2
c1      zxt   r1,r0             1f      1,0     01 1111 001 000 | 7c8
c2      zxt   r1,r1             1f      1,1     01 1111 001 001 | 7c9
c3      zxt   r1,r2             1f      1,2     01 1111 001 010 | 7ca
c4      zxt   r2,r0             1f      2,0     01 1111 010 000 | 7d0
c5      zxt   r2,r1             1f      2,1     01 1111 010 001 | 7d1
c6      zxt   r2,r2             1f      2,2     01 1111 010 010 | 7d2

=== Late additions ===

c7      jmp   dddddd            0b      7,7     00 1011 111 111 | 2ff

[TUHS] Re: Clever code

[Noel Chiappa]

    > From: Bakul Shah

    > one dealt with it by formatting the disk so that the logical blocks N &
    > N+1 (from the OS PoV) were physically more than 1 sector apart. No
    > clever coding needed!

An old hack. ('Nothing new', and all that.) DEC Rx01/02 floppies used the
same thing, circa 1976.

	Noel