[TUHS] Stdin Redirect in Cu History/Alternatives?

[TUHS] Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Good morning all.  I've been doing some historical research on the UUCP cu utility this morning and have come across a little discrepancy between the various UNIX streams I was wondering if someone could illuminate.

So cu as of V7 supported the ~$ escape, a means of calling a local procedure and emitting stdout over the TTY line to the remote machine, all fine and good for packaging a character stream to emit.  However, what I'm not finding in that age of documentation is any means of requesting std*in* from the TTY line as input to a local procedure (in essence running a text filter or handshake-driven protocols over cu).  The context in which I'm researching this is integrating cu into my bare-metal SBC programming using XMODEM so I can rest a little easier my process is based on tools I'll probably find in most places.

So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin.  I did some further digging and it looks like different UUCP implementations cracked this nut with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+.  Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up any command for this.  This is indicative of there never being an agreed-upon mechanism for doing this, although I could see this being a very useful mechanism.

What I'm curious about is if the lack of a bi-directional redirect in early cu is reflective of a lack of need for that sort of functionality at the time or that matters such as that were handled through a different mechanism.  One thought I did have is that there wasn't file locking at the time (right?) and so theoretically nothing would prevent one from using a cu session on one terminal to send interactive commands and then a second using fd redirects in the shell to run filters/protocols separately of the interactive stream.

- Matt G.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3731 bytes --]

On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS <tuhs@tuhs.org> wrote:

> Good morning all.  I've been doing some historical research on the UUCP cu
> utility this morning and have come across a little discrepancy between the
> various UNIX streams I was wondering if someone could illuminate.
>
Maybe. see below....

>
> So cu as of V7 supported the ~$ escape, a means of calling a local
> procedure and emitting stdout over the TTY line to the remote machine, all
> fine and good for packaging a character stream to emit.  However, what I'm
> not finding in that age of documentation is any means of requesting std*in*
> from the TTY line as input to a local procedure (in essence running a text
> filter or handshake-driven protocols over cu).  The context in which I'm
> researching this is integrating cu into my bare-metal SBC programming using
> XMODEM so I can rest a little easier my process is based on tools I'll
> probably find in most places.
>
> So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no
> stdin.  I did some further digging and it looks like different UUCP
> implementations cracked this nut with different escapes, with BSD
> eventually going with ~C and Taylor UUCP opting for ~+.  Checking the
> current illumos manual pages (for a SVR4-ish example) doesn't turn up any
> command for this.  This is indicative of there never being an agreed-upon
> mechanism for doing this, although I could see this being a very useful
> mechanism.
>
maybe -- need to see more of what your session was like.  I never remember
missing anything I needed.

>
> What I'm curious about is if the lack of a bi-directional redirect in
> early cu is reflective of a lack of need for that sort of functionality at
> the time or that matters such as that were handled through a different
> mechanism.

I'm not sure I get the question.  We did all sorts of redirection and
used/abused cu and its friends all the time.    I suspect I'm not
understanding what you are trying to do.

From a history standpoint, cu(1) is just one of many programs in that
family.  In the mid/late 1970s, we used a program called 'connect' for
Sixth Edition at CMU, IIRC the Purdue folks had a similar one which was
called attach(1) and there was tip(1) which was from Case/UCB [Sam
Leffler].  If you look in the USENIX archives, I bet you will find a 1/2
doz or so of programs in the ilk before V7.  With V7 uucp. was delivered,
so cu(1) began to make inroads as it had the advantage that it was set up
to work on concert to uucico(8). Simply, V7 came out, and UUCP started to
used and eventually the 'USENET' born, cu(1) sort of 'won' because most of
the other programs tended to conflict with uucico(8) -  plus since it was
already there, people did not need something else.   But if you had written
one before V7, you often find sites sticking with what they had.

There were a number of UNIX implementations of XMODEM and friends.  The C
version of Kermit  (ckermit) was quite popular plus has
connect(1)/tip(1)/cu(1) style functionality built into it, but ....  IIRC
does not obey the locks that uucico(8) wants so if you used it on TTYs that
had a modem that uucico was trying grab, bad things happened.   That said,
in a microprocessor lab where you often dedicated. serial port to 'target'
micro/pc, kermit worked well.   My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
since I had any use for them, I've forgotten. Look in the both USENIX and
the USENET source archives.

Frankly, the last time I think I was trying to do this sort of thing, I was
using Kermit.

YMMV
Clem
ᐧ

[-- Attachment #2: Type: text/html, Size: 6222 bytes --]

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

[-- Attachment #1: Type: text/plain, Size: 5942 bytes --]

Luckily I can toss you my exact use-case with code.

https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c

Above is a little utility I spit out to very specifically perform XMODEM-CRC for the VisionFive RISC-V single-board computer. Word on the street is their recovery bootloader has a bug that prevents sx and other utilities from working properly. As far as I could tell, they don't send the standard ACK after each packet receipt, but instead some other character. I just opted to check for NAK and proceed if not received.

In any case, the above should achieve XMODEM-CRC transmission to the JH7100 RISC-V chip over UART.

The phenomenon I'm experiencing (and sorry, I'm not trying to turn this into Taylor UUCP troubleshooting, I promise) is that when using the ~$ escape, my code never receives the 'C' from the device to initiate XMODEM-CRC. I know it's reading local stdin still because if I type and submit a 'C', it picks up and starts transferring, likewise requiring me to submit a non-NAK character every packet. Otherwise it does work, so the stdout is going to the remote machine, but at least with ~$, the stdin from the serial line does stop showing on my screen, which made me think the redirect was happening, but it just seems to disappear off into the void (/dev/null?) while the local program reads local stdin and puts data out on the remote stdout. This does hold consistent with the manual verbiage for ~$, it only mentions stdout to remote, doesn't mention any ferrying of remote stdin into the local application being run.

I know my utility works because I've used it in the place of sx in GNU screen and it works like a charm. Plus it does send​ to the device if I provide the expected acknowledgement characters from my local machine in cu.

So this doesn't just turn into troubleshooting, all I hope to highlight here is getting stdin from the remote machine into a program launched via a ~ escape in cu, be it like the ~C option in BSD cu or ~+ option in Taylor. My curiosity is whether such a thing was in historic cu, and, if not, how it might have been accommodated otherwise. As always, thankful for the insight and feedback folks here provide!

- Matt G.
------- Original Message -------
On Saturday, December 10th, 2022 at 4:22 PM, Clem Cole <clemc@ccc.com> wrote:

> On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS <tuhs@tuhs.org> wrote:
>
>> Good morning all. I've been doing some historical research on the UUCP cu utility this morning and have come across a little discrepancy between the various UNIX streams I was wondering if someone could illuminate.
>
> Maybe. see below....
>
>> So cu as of V7 supported the ~$ escape, a means of calling a local procedure and emitting stdout over the TTY line to the remote machine, all fine and good for packaging a character stream to emit. However, what I'm not finding in that age of documentation is any means of requesting std*in* from the TTY line as input to a local procedure (in essence running a text filter or handshake-driven protocols over cu). The context in which I'm researching this is integrating cu into my bare-metal SBC programming using XMODEM so I can rest a little easier my process is based on tools I'll probably find in most places.
>>
>> So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin. I did some further digging and it looks like different UUCP implementations cracked this nut with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+. Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up any command for this. This is indicative of there never being an agreed-upon mechanism for doing this, although I could see this being a very useful mechanism.
>
> maybe -- need to see more of what your session was like. I never remember missing anything I needed.
>
>> What I'm curious about is if the lack of a bi-directional redirect in early cu is reflective of a lack of need for that sort of functionality at the time or that matters such as that were handled through a different mechanism.
>
> I'm not sure I get the question. We did all sorts of redirection and used/abused cu and its friends all the time. I suspect I'm not understanding what you are trying to do.
>
> From a history standpoint, cu(1) is just one of many programs in that family. In the mid/late 1970s, we used a program called 'connect' for Sixth Edition at CMU, IIRC the Purdue folks had a similar one which was called attach(1) and there was tip(1) which was from Case/UCB [Sam Leffler]. If you look in the USENIX archives, I bet you will find a 1/2 doz or so of programs in the ilk before V7. With V7 uucp. was delivered, so cu(1) began to make inroads as it had the advantage that it was set up to work on concert to uucico(8). Simply, V7 came out, and UUCP started to used and eventually the 'USENET' born, cu(1) sort of 'won' because most of the other programs tended to conflict with uucico(8) - plus since it was already there, people did not need something else. But if you had written one before V7, you often find sites sticking with what they had.
>
> There were a number of UNIX implementations of XMODEM and friends. The C version of Kermit (ckermit) was quite popular plus has connect(1)/tip(1)/cu(1) style functionality built into it, but .... IIRC does not obey the locks that uucico(8) wants so if you used it on TTYs that had a modem that uucico was trying grab, bad things happened. That said, in a microprocessor lab where you often dedicated. serial port to 'target' micro/pc, kermit worked well. My memory is there were also a bunch of two letter programs, rx/sx and rz/sz and the like. Frankly its been so long since I had any use for them, I've forgotten. Look in the both USENIX and the USENET source archives.
>
> Frankly, the last time I think I was trying to do this sort of thing, I was using Kermit.
>
> YMMV
> Clem
> ᐧ

[-- Attachment #2: Type: text/html, Size: 10094 bytes --]

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Michael Kjörling]

On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
> My memory is there were also a bunch of two
> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
> since I had any use for them, I've forgotten.

I remember at least sz/rz from my early (for me) forays into UNIX,
back when ZModem was pretty much state of the art at least on micros
and I had files locally that I wanted remotely or vice versa. The
mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
course, being local to the remote host, so the opposite sense of what
one would do with the terminal emulator program that one interacted
with locally. So "sz <some file>" at the prompt, then activate the
"receive ZModem transfer" function locally; or "rz <some file>", then
activate the "send using ZModem" function locally.

I think the host I was on at the time (which appears to have been some
Solaris) also offered XModem and YModem variants as [rs][xy], but I
never used those because someone had at some point told me that ZModem
was better. :-)

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Steve Nickolas]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

On Sun, 11 Dec 2022, Michael Kjörling wrote:

> On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
>> My memory is there were also a bunch of two
>> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
>> since I had any use for them, I've forgotten.
>
> I remember at least sz/rz from my early (for me) forays into UNIX,
> back when ZModem was pretty much state of the art at least on micros
> and I had files locally that I wanted remotely or vice versa. The
> mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
> course, being local to the remote host, so the opposite sense of what
> one would do with the terminal emulator program that one interacted
> with locally. So "sz <some file>" at the prompt, then activate the
> "receive ZModem transfer" function locally; or "rz <some file>", then
> activate the "send using ZModem" function locally.
>
> I think the host I was on at the time (which appears to have been some
> Solaris) also offered XModem and YModem variants as [rs][xy], but I
> never used those because someone had at some point told me that ZModem
> was better. :-)

Kind-of a pain to type as I'm SSHing in from my phone, but they still 
exist and I have "lrzsz" installed on my Debian box.

-uso.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Sun, Dec 11, 2022 at 9:28 AM Steve Nickolas <usotsuki@buric.co> wrote:
> On Sun, 11 Dec 2022, Michael Kjörling wrote:
> > On 10 Dec 2022 19:22 -0500, from clemc@ccc.com (Clem Cole):
> >> My memory is there were also a bunch of two
> >> letter programs, rx/sx and rz/sz and the like.  Frankly its been so long
> >> since I had any use for them, I've forgotten.
> >
> > I remember at least sz/rz from my early (for me) forays into UNIX,
> > back when ZModem was pretty much state of the art at least on micros
> > and I had files locally that I wanted remotely or vice versa. The
> > mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
> > course, being local to the remote host, so the opposite sense of what
> > one would do with the terminal emulator program that one interacted
> > with locally. So "sz <some file>" at the prompt, then activate the
> > "receive ZModem transfer" function locally; or "rz <some file>", then
> > activate the "send using ZModem" function locally.
> >
> > I think the host I was on at the time (which appears to have been some
> > Solaris) also offered XModem and YModem variants as [rs][xy], but I
> > never used those because someone had at some point told me that ZModem
> > was better. :-)
>
> Kind-of a pain to type as I'm SSHing in from my phone, but they still
> exist and I have "lrzsz" installed on my Debian box.

And as I mentioned earlier, they're still very much used.

I recently wrote the initial bootstrap program that runs when the x86
cores on our machine come out of reset
(https://github.com/oxidecomputer/phbl/). However, that's for
production use; for internal engineering use we have _another_ tool,
also written in Rust, that is an interactive standalone program. This
lets us do things like poke at physical memory, read/write MSRs, and
all of that fun stuff, before loading an operating system. So how does
one load the OS? Our machine has a part that includes a 3
MBAUD-capable UART, and we use xmodem to transfer the kernel (and a
ramdisk image!) over that, where the engineering tool can load and
jump into it. The standalone program implements xmodem internally,
whereas we use `sx` on the dev host side. Basically, there is a
built-in commands to perform the transfer, another that effectively
says "there's an ELF image in physical memory $here: load it and
return the entry point" and finally another command that can jump to
an arbitrary virtual address (yes, we load the host OS in 64-bit mode
and treat the kernel like any old "normal" ELF binary).

As the ramdisk image has grown (more and more debugging tools are
included as we move ever forward in the bringup and development
process), latency here is annoying. I added support for compressed
kernel and ramdisk images, which gave a 2.5x speedup (directly
proportional to the usual compression ratio we see on these images)
which ameliorated but did not eliminate the pain. We briefly tossed
around the idea of implementing zmodem, but decided it wasn't worth
it. We _could_, in theory, make use of the enhanced kermit with
sliding windows and so on, but again decided it wasn't worth the
effort for an engineering tool.

Incidentally, my colleague Matt Keeter ran into performance and
reliability issues with xmodem over the USB serial driver in macOS,
and wrote a small shim that side-stepped the host OS device. He did a
nice write-up of it here:
https://www.mattkeeter.com/blog/2022-05-31-xmodem/

I think this is in line with Larry's question about non-legacy
use-cases for these ancient serial transfer protocols in late 2022.

        - Dan C.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 10:04:54AM -0500, Dan Cross wrote:
> I think this is in line with Larry's question about non-legacy
> use-cases for these ancient serial transfer protocols in late 2022.

Indeed, this was just what I was looking for.  I had no idea that 
serial transfer protocols were still useful.  

It makes me wish I had kept the source for quicknet, I wrote my
own file transfer and remote terminal for CP/M in the middle 
1980s.  I did it just because I wanted things to work a certain
way and none of the other options did what I wanted.  quicknet was
really pleasant on a 128K Z80 (not really 128K because the screen
was mapped in there, but more than 64K).  I spent many a pleasant
hour logged in to uwvax in quicknet, got a lot of work done.

Thanks to you and others who had shown that serial transfer still
matters.  Who knew?  You did.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Adam Thornton]

In the late 1990s I did a brief consulting project to move files between sites.  Something to do with customer billing and record transmission, from some large system that was not Internet-connected.  It ended up being a terrible teetering tower of kermit, sz or sx, and expect.  But it got the job done!

The funny thing about that...the project spanned the July 4th weekend.  I didn't get it working before the 4th.  Then when I went in very hungover after the weekend, I couldn't figure out quite what I had been trying to do, because it was something that was attempting to be very clever.  So in my dulled state, I rewrote the bottom layers to be much more straightforward.  Worked the first time.  That taught me the virtue of not overthinking it.

Adam

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Michael Kjörling]

On 11 Dec 2022 10:18 -0700, from athornton@gmail.com (Adam Thornton):
> The funny thing about that...the project spanned the July 4th
> weekend. I didn't get it working before the 4th. Then when I went in
> very hungover after the weekend, I couldn't figure out quite what I
> had been trying to do, because it was something that was attempting
> to be very clever. So in my dulled state, I rewrote the bottom
> layers to be much more straightforward. Worked the first time. That
> taught me the virtue of not overthinking it.

By definition, if you write code as cleverly as you can, then you
aren't clever enough to debug it...

There's something to be said for simplicity unless for some reason you
really need to be clever. Let's save the clever for the, say, 1% where
it's actually needed; premature application of cleverness is the root
of much evil.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

On Sun, 11 Dec 2022, Michael Kjörling wrote:

> By definition, if you write code as cleverly as you can, then you aren't 
> clever enough to debug it...

Indeed...

I've always used the maxim "Write code as though the next person to 
maintain it is an axe-wielding psychopath who knows where you live".

-- Dave

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
> On Sun, 11 Dec 2022, Michael Kj??rling wrote:
> 
> > By definition, if you write code as cleverly as you can, then you aren't 
> > clever enough to debug it...
> 
> Indeed...
> 
> I've always used the maxim "Write code as though the next person to 
> maintain it is an axe-wielding psychopath who knows where you live".

My main job, for the 20 years until I retired, was to keep telling 
people that code that you wrote 6 months ago might as well have been
written by someone else.  Optimize for reading the code, not writing
the code.  It's read many.

99.9% of the time, I detest clever code.  .1% of the time, I need it.
The problem is that smart engineers adore writing clever code.  They
usually, eventually, wise up.

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Readable code also contributes to easier documentation.  It's easier to maintain a manual when you can just translate the code into the local language.  My current team started small and I learned pretty soon that time invested in a habit of readable code meant less time spent providing the person assigned documentation the juicy details.

- Matt G.

------- Original Message -------
On Sunday, December 11th, 2022 at 12:03 PM, Larry McVoy <lm@mcvoy.com> wrote:


> On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
> 
> > On Sun, 11 Dec 2022, Michael Kj??rling wrote:
> > 
> > > By definition, if you write code as cleverly as you can, then you aren't
> > > clever enough to debug it...
> > 
> > Indeed...
> > 
> > I've always used the maxim "Write code as though the next person to
> > maintain it is an axe-wielding psychopath who knows where you live".
> 
> 
> My main job, for the 20 years until I retired, was to keep telling
> people that code that you wrote 6 months ago might as well have been
> written by someone else. Optimize for reading the code, not writing
> the code. It's read many.
> 
> 99.9% of the time, I detest clever code. .1% of the time, I need it.
> The problem is that smart engineers adore writing clever code. They
> usually, eventually, wise up.

[TUHS] Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

On Dec 11, 2022, at 12:04 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> 
> 
> On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
>> On Sun, 11 Dec 2022, Michael Kj??rling wrote:
>> 
>>> By definition, if you write code as cleverly as you can, then you aren't 
>>> clever enough to debug it...
>> 
>> Indeed...
>> 
>> I've always used the maxim "Write code as though the next person to 
>> maintain it is an axe-wielding psychopath who knows where you live".
> 
> My main job, for the 20 years until I retired, was to keep telling 
> people that code that you wrote 6 months ago might as well have been
> written by someone else.  Optimize for reading the code, not writing
> the code.  It's read many.
> 
> 99.9% of the time, I detest clever code.  .1% of the time, I need it.
> The problem is that smart engineers adore writing clever code.  They
> usually, eventually, wise up.

Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).

I do prefer clever/smart design to locally clever/smart code. For example,
using Schönhage-Strassen algorithm for multiplying very large numbers.
Or transforming a problem to use a much more efficient data structure
or making equivalent transforms which may be more efficient to compute.
Such code may not be immediately clear but with proper documentation
it is not difficult + you can solve much larger problems. But agreed
these come up much less often. 

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Steve Nickolas]

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

On Sun, 11 Dec 2022, Bakul Shah wrote:

> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time. As an example, building "Unix as a service" as
> user processes on top of a small microkernel could provide the same
> functionality using much clearer and much less code but it would be
> slower so we don't do it. Plan9 sort of went in that direction and it
> is much simpler (but that could also be because it is not hacked on so
> much).
>
> I do prefer clever/smart design to locally clever/smart code. For example,
> using Schönhage-Strassen algorithm for multiplying very large numbers.
> Or transforming a problem to use a much more efficient data structure
> or making equivalent transforms which may be more efficient to compute.
> Such code may not be immediately clear but with proper documentation
> it is not difficult + you can solve much larger problems. But agreed
> these come up much less often.

My attitude is: if I'm doing an ugly on nonobvious hack, I'll drop a 
comment saying what it does.

Most recently I wrote code that set a "magic flag" - and the comment 
explained why the flag was set (so that it would be immediately altered, 
forcing a redraw).

-uso.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
>
>
> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time. As an example, building "Unix as a service" as
> user processes on top of a small microkernel could provide the same
> functionality using much clearer and much less code but it would be
> slower so we don't do it. Plan9 sort of went in that direction and it
> is much simpler (but that could also be because it is not hacked on so
> much).
>

It's not necessarily true that microkernels are significantly slower.
They mostly got that reputation because of Mach and kernels like it
with their heavyweight IPC. Lightweight microkernels like QNX and the
L4 family generally have significantly better performance (in fact,
QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
a proof-of-concept network driver on a current version of seL4 is
significantly faster than a Linux network driver).

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
> On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
> >
> >
> > Agree that clear code is preferable to complicated code. But in practice
> > people sacrifice clarity for performance improvement all the time. Look
> > at the kernel code of any modern os. Everybody pays lip service to this
> > but most anything other than toy programs ends up getting needlessly
> > complicated over time. As an example, building "Unix as a service" as
> > user processes on top of a small microkernel could provide the same
> > functionality using much clearer and much less code but it would be
> > slower so we don't do it. Plan9 sort of went in that direction and it
> > is much simpler (but that could also be because it is not hacked on so
> > much).
> >
> 
> It's not necessarily true that microkernels are significantly slower.
> They mostly got that reputation because of Mach and kernels like it
> with their heavyweight IPC. Lightweight microkernels like QNX and the
> L4 family generally have significantly better performance (in fact,
> QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
> a proof-of-concept network driver on a current version of seL4 is
> significantly faster than a Linux network driver).

I was friends with one of the 3 people who were allowed to commit to QNX
kernel, Dan Hildebrandt (RIP 1998).  Those 3 people actually understood
the "micro" in "microkernel".   Early versions of QNX kernel fit in
the 4K instruction cache and left space for user programs.  They had
benchmarks that measured cache misses and refused commits that added to
the number of cache misses.

I met Dan because I had 10 terminals on a 80286 with I think 256KB
of memory and we were all editing and compiling and it worked way the
heck better than a 11/780 with 4MB with 30-40 students trying to do the
same thing.  I just wanted to talk to the people who made that possible,
it was after I had made a connection to Dennis.  The early QNX people
were special.  Dan and I had many conversations about the mono kernel
vs micro kernel, it was super fun because we weren't trying to convince
each other, we were comparing approaches.

I've never seen a team like Dan's team that cared as much as they did
about "micro" actually being micro.  That's the only way that that
approach works.  I can not state that enough, if you want to have a
micro kernel with message passing, if you don't fit in L1 cache,
you are a joke compared to a mono kernel.  You have to embrace
micro.

I had hoped that Mach would be like that but it was a giant mess.  I know
my opinion is not liked but I so wanted to like Mach, read the papers,
loved it, then read the code, hated it, a few years ago I was working
with the BSD team at Netflix, looked through the VM system from Mach
and it was a mess, it was the exact opposite of my experience at Sun
where I looked and looked and things came in focus and it made sense.
The Mach stuff has never made sense to me, I know Clem says it was a
research platform, OK, but the platform never got to a nice clean place.

If you think of microkernels and think Mach is it, nope, look at QNX.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Kevin Bowling]

On Sun, Dec 11, 2022 at 8:35 PM Larry McVoy <lm@mcvoy.com> wrote:
>
> On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
> > On 12/11/22, Bakul Shah <bakul@iitbombay.org> wrote:
> > >
> > >
> > > Agree that clear code is preferable to complicated code. But in practice
> > > people sacrifice clarity for performance improvement all the time. Look
> > > at the kernel code of any modern os. Everybody pays lip service to this
> > > but most anything other than toy programs ends up getting needlessly
> > > complicated over time. As an example, building "Unix as a service" as
> > > user processes on top of a small microkernel could provide the same
> > > functionality using much clearer and much less code but it would be
> > > slower so we don't do it. Plan9 sort of went in that direction and it
> > > is much simpler (but that could also be because it is not hacked on so
> > > much).
> > >
> >
> > It's not necessarily true that microkernels are significantly slower.
> > They mostly got that reputation because of Mach and kernels like it
> > with their heavyweight IPC. Lightweight microkernels like QNX and the
> > L4 family generally have significantly better performance (in fact,
> > QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
> > a proof-of-concept network driver on a current version of seL4 is
> > significantly faster than a Linux network driver).
>
> I was friends with one of the 3 people who were allowed to commit to QNX
> kernel, Dan Hildebrandt (RIP 1998).  Those 3 people actually understood
> the "micro" in "microkernel".   Early versions of QNX kernel fit in
> the 4K instruction cache and left space for user programs.  They had
> benchmarks that measured cache misses and refused commits that added to
> the number of cache misses.
>
> I met Dan because I had 10 terminals on a 80286 with I think 256KB
> of memory and we were all editing and compiling and it worked way the
> heck better than a 11/780 with 4MB with 30-40 students trying to do the
> same thing.  I just wanted to talk to the people who made that possible,
> it was after I had made a connection to Dennis.  The early QNX people
> were special.  Dan and I had many conversations about the mono kernel
> vs micro kernel, it was super fun because we weren't trying to convince
> each other, we were comparing approaches.
>
> I've never seen a team like Dan's team that cared as much as they did
> about "micro" actually being micro.  That's the only way that that
> approach works.  I can not state that enough, if you want to have a
> micro kernel with message passing, if you don't fit in L1 cache,
> you are a joke compared to a mono kernel.  You have to embrace
> micro.
>
> I had hoped that Mach would be like that but it was a giant mess.  I know
> my opinion is not liked but I so wanted to like Mach, read the papers,
> loved it, then read the code, hated it, a few years ago I was working
> with the BSD team at Netflix, looked through the VM system from Mach
> and it was a mess, it was the exact opposite of my experience at Sun
> where I looked and looked and things came in focus and it made sense.
> The Mach stuff has never made sense to me, I know Clem says it was a
> research platform, OK, but the platform never got to a nice clean place.

Mach seems not incompatible to me to Windows NT ideas of layering,
especially in modern macOS guise, more so than a microkernel I'd call
it something like a "macrokernel" with a variety of ways to do
things.. dealer's choice for better and worse.  It is interesting to
note the success, often overlooked by more hardline *nix people, that
macOS (and iOS and tvOS and watchOS etc) is indeed mach (more so than
it is BSD as some people claim) and wildly successful.

> If you think of microkernels and think Mach is it, nope, look at QNX.
> --
> ---
> Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/11/22, Larry McVoy <lm@mcvoy.com> wrote:
>
> If you think of microkernels and think Mach is it, nope, look at QNX.
>
And yet, for some reason, QNX has had almost no influence on anything
else besides a few hobby OSes (only one of which ever reached a
reasonably mature state) as far as I can tell. I really don't get why
that is. As far as I know, I'm the only one who's working on a
QNX-like OS that's not QNX itself.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Sun, Dec 11, 2022 at 10:26:37PM -0700, Andrew Warkentin wrote:
> On 12/11/22, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > If you think of microkernels and think Mach is it, nope, look at QNX.
> >
> And yet, for some reason, QNX has had almost no influence on anything
> else besides a few hobby OSes (only one of which ever reached a
> reasonably mature state) as far as I can tell. I really don't get why
> that is. As far as I know, I'm the only one who's working on a
> QNX-like OS that's not QNX itself.

It influenced me.

Lots of things that were the right answer failed to achieve success.
Beta vs VHS.  Plan 9.

Does that mean they were worthless?  I don't think so.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3091 bytes --]

On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
wrote:

> And yet, for some reason, QNX has had almost no influence on anything
>
Be careful with a statement like that.  It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you.   It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.

When a small, very lightweight UNIX-style programming API needed to be
used, QNX was often a favorite.

I sometimes think QNX must have had a really good salesperson in the
'Rust-Belt.'  I know I talked to several fans in companies doing that
work.   I do know of a least one firm that still uses it.  An inexpensive
x86 can be designed into a custom controller, and the only 'development' is
the customer interface to private HW.   The development system is a PC or
Vmware on an engineer's desk.

After Blackberry bought the company, it's interesting that they seem to be
all that is left of BB.  But they are still going strong: QNX Neutrino RTOS
<https://streaklinks.com/BUDwcGeMTFsTSXcFxQzjvNX5/https%3A%2F%2Fwww.qnx.com%2Fproducts%2Fintl%2Fneutrino_rtos%2F>

Funny, I loved Mach (still do), but it tried to be all things to all
people.  The QNX guys did not.   I also think it help in their success --
by the fact that the QNX folks concentrated on RT, while Mach tried to be
the replacement for all of BSD.    They both have their place ...  I'm
typing this on my macOS 13.0.1 (Ventura) M1 system, which is just the
current flavor of Mach.  As Tru64 hacker, as well as one of the folks that
work on Intel Paragon, which was OSF/1, all three are Mach based.

I also did some work with QNX back in the day and, like Larry, was always
very impressed.   At one time, I did some consulting in the Rust-Belt, and
the executive (*i.e.* -- Havard Business. School types) asked me if  *"this QNX
thing their engineers were using -- after all it was not from Microsoft,
IBM or DEC, of course.''* [they had converted/were in the process of
converting from DEC PMAX-based controllers running Ultrix to PCs running
QNX].   My analysis at the time, for a bunch of ex-Fortran Mech E's, had
done extremely well.  I told the execs then that this is good stuff; it's
going to save them buckets of money as it 'just works' (that was the core
SW in the automatic 'sorter' that at the time was being done under contract
for Amazon -- I know the CEO of that firm and they sold the same basic
system to UPS/FedEx/USPS -- they used to have a very cool movie taken
during the testing at FedEx with glasses full of champagne moving at 45
miles through the sorter without spilling -- the PMAX would never have been
able to do that).

Frankly, for anyone learning either about microkernels or RT, I would
certainly tell them to look at QNX.   Neither topic are what we call
'research' projects as much these days, but both have extremely practical
applications.

ᐧ

[-- Attachment #2: Type: text/html, Size: 5329 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Mon, Dec 12, 2022 at 10:30 AM Clem Cole <clemc@ccc.com> wrote:
>
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com> wrote:
>>
>> And yet, for some reason, QNX has had almost no influence on anything
>
> Be careful with a statement like that.  It's likely running in something in your car. and very likely to be running in something in the last Boeing or Airbus-based flight you took, and it was used when Amazon made the last delivery to you.   It has long been popular in process control/materials handling/robotics/fly-by-wire systems.
>
> When a small, very lightweight UNIX-style programming API needed to be used, QNX was often a favorite.
>
> I sometimes think QNX must have had a really good salesperson in the 'Rust-Belt.'  I know I talked to several fans in companies doing that work.   I do know of a least one firm that still uses it.  An inexpensive x86 can be designed into a custom controller, and the only 'development' is the customer interface to private HW.   The development system is a PC or Vmware on an engineer's desk.
>
> After Blackberry bought the company, it's interesting that they seem to be all that is left of BB.  But they are still going strong: QNX Neutrino RTOS

For a while, Neutrino was open source, but that seemed to change
quietly at some point. I can't seem to find the source code online
anywhere, which surprises me a bit: I'd have assumed anyone who got a
copy of the code under the OSS license would still be able to treat it
as open source, but maybe I'm just wrong.

        - Dan C.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole wrote:
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> wrote:
> 
> > And yet, for some reason, QNX has had almost no influence on anything
> >
> Be careful with a statement like that.  It's likely running in something
> in your car. and very likely to be running in something in the last Boeing
> or Airbus-based flight you took, and it was used when Amazon made the last
> delivery to you.   It has long been popular in process control/materials
> handling/robotics/fly-by-wire systems.
> 
> When a small, very lightweight UNIX-style programming API needed to be
> used, QNX was often a favorite.


Thanks for that Clem.  One question though, all those companies want
support.  Is there anyone still providing support?  I see QNX.com is
a thing but is there an actual team of good people working on it?

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

I don't know.  I'll try to poke and ask.  The engineer I used to work with
at the Mill and later Intelligrated has retired (a lot of that with our
generation).   But I still know some of the execs at the former
Intelligrated (now part of Honeywell), so I'll see if I can find out who
owns that product these days.  If I learn anything interesting, I'll pass
it back.

FWIW: BB is offering support: i.e.:   https://blackberry.qnx.com/en/support
claims such, but what does that mean?

Clem
ᐧ

On Mon, Dec 12, 2022 at 11:04 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole wrote:
> > On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> > wrote:
> >
> > > And yet, for some reason, QNX has had almost no influence on anything
> > >
> > Be careful with a statement like that.  It's likely running in something
> > in your car. and very likely to be running in something in the last
> Boeing
> > or Airbus-based flight you took, and it was used when Amazon made the
> last
> > delivery to you.   It has long been popular in process control/materials
> > handling/robotics/fly-by-wire systems.
> >
> > When a small, very lightweight UNIX-style programming API needed to be
> > used, QNX was often a favorite.
>
>
> Thanks for that Clem.  One question though, all those companies want
> support.  Is there anyone still providing support?  I see QNX.com is
> a thing but is there an actual team of good people working on it?
>

[-- Attachment #2: Type: text/html, Size: 2710 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Mon, 12 Dec 2022 at 17:04, Larry McVoy <lm@mcvoy.com> wrote:
>
> Thanks for that Clem.  One question though, all those companies want
> support.  Is there anyone still providing support?  I see QNX.com is
> a thing but is there an actual team of good people working on it?

QNX is alive and well and owned by Blackberry Ltd, the former Research
in Motion. One big Canadian tech co bought another.

They aren't doing _great_ but they're not dead by any means.

https://en.wikipedia.org/wiki/BlackBerry_Limited

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Well this has certainly piqued my curiosity on QNX.  One of my more long-term projects is to write a kernel (micro or otherwise), initially for RISC-V that can, if nothing else, serve as a springboard for my own projects.  I've thus far just been using AT&T and BSD UNIX kernels for research and inspiration, with a peek at Darwin every now and then to see how Apple/Mach does things, but could certainly stand to broaden my horizons.  That said, if I was to straight up port something as an exercise, I'd probably go for V7 given its ubiquity and the fact that work done there would likely serve as a good template for doing the same with other AT&T and BSD variants.  Plus, I think someone's already gone and done a V6 port to RISC-V, so I'd have stuff to refer to.

- Matt G.

------- Original Message -------
On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven@gmail.com> wrote:


> On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm@mcvoy.com wrote:
> 
> > Thanks for that Clem. One question though, all those companies want
> > support. Is there anyone still providing support? I see QNX.com is
> > a thing but is there an actual team of good people working on it?
> 
> 
> QNX is alive and well and owned by Blackberry Ltd, the former Research
> in Motion. One big Canadian tech co bought another.
> 
> They aren't doing great but they're not dead by any means.
> 
> https://en.wikipedia.org/wiki/BlackBerry_Limited
> 
> --
> Liam Proven ~ Profile: https://about.me/liamproven
> Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
> Twitter/LinkedIn: lproven ~ Skype: liamproven
> UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

It's been decades since I've used it, my comments are based on the QNX that
predated their POSIX conformance.  It was extremely light weight, fast,
and used as little memory as possible.  I literally had 10 active logins
(tty) with people editing and compiling on a 256KB 80286.  Nothing else
came close.  But it wasn't POSIX so it might be more bloated today.

On Mon, Dec 12, 2022 at 11:10:08PM +0000, segaloco via TUHS wrote:
> Well this has certainly piqued my curiosity on QNX.  One of my more long-term projects is to write a kernel (micro or otherwise), initially for RISC-V that can, if nothing else, serve as a springboard for my own projects.  I've thus far just been using AT&T and BSD UNIX kernels for research and inspiration, with a peek at Darwin every now and then to see how Apple/Mach does things, but could certainly stand to broaden my horizons.  That said, if I was to straight up port something as an exercise, I'd probably go for V7 given its ubiquity and the fact that work done there would likely serve as a good template for doing the same with other AT&T and BSD variants.  Plus, I think someone's already gone and done a V6 port to RISC-V, so I'd have stuff to refer to.
> 
> - Matt G.
> 
> ------- Original Message -------
> On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven@gmail.com> wrote:
> 
> 
> > On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm@mcvoy.com wrote:
> > 
> > > Thanks for that Clem. One question though, all those companies want
> > > support. Is there anyone still providing support? I see QNX.com is
> > > a thing but is there an actual team of good people working on it?
> > 
> > 
> > QNX is alive and well and owned by Blackberry Ltd, the former Research
> > in Motion. One big Canadian tech co bought another.
> > 
> > They aren't doing great but they're not dead by any means.
> > 
> > https://en.wikipedia.org/wiki/BlackBerry_Limited
> > 
> > --
> > Liam Proven ~ Profile: https://about.me/liamproven
> > Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
> > Twitter/LinkedIn: lproven ~ Skype: liamproven
> > UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/12/22, Clem Cole <clemc@ccc.com> wrote:
> On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> wrote:
>
>> And yet, for some reason, QNX has had almost no influence on anything
>>
> Be careful with a statement like that.  It's likely running in something
> in your car. and very likely to be running in something in the last Boeing
> or Airbus-based flight you took, and it was used when Amazon made the last
> delivery to you.   It has long been popular in process control/materials
> handling/robotics/fly-by-wire systems.
>

I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.

The only OSes with significant QNX influence to reach anything
resembling a mature state of which I am aware are VSTa
<https://vsta.org/> and WiNGs <http://wingsos.org/>, both of which
have been abandoned for quite a while. Besides those two, there was
also OpenBLT, which was able to run a simple shell and a few basic
utilities from its boot image but not much more, and RadiOS, which was
abandoned at a point where it couldn't run much more than a hello
world. I'm also working on my own QNX-like OS
<https://gitlab.com/uxrt/uxrt-toplevel> like I said earlier, although
it doesn't run user programs yet (I'm working on the VFS and IPC
transport layer at the moment).

The extensive commercial success of QNX makes it even more surprising
to me that it has had so little architectural influence. Similarly, I
don't get why people who bash microkernels always seem to think that
all of them are like Mach despite QNX being quite successful.

On 12/12/22, Larry McVoy <lm@mcvoy.com> wrote:
> It's been decades since I've used it, my comments are based on the QNX that
> predated their POSIX conformance.  It was extremely light weight, fast,
> and used as little memory as possible.  I literally had 10 active logins
> (tty) with people editing and compiling on a 256KB 80286.  Nothing else
> came close.  But it wasn't POSIX so it might be more bloated today.
>

QNX 4 and Neutrino are heavier than QNX "Classic", but they're still
fairly lightweight. QNX 4 had the fairly well-known demo disk with a
Photon desktop and browser on a single 1.4M floppy. Neutrino never had
anything quite like that (the smallest official live CD images of the
early versions are something like 60M in size) but it's still possible
to build images with it that are pretty small.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Mon, Dec 12, 2022 at 07:00:34PM -0700, Andrew Warkentin wrote:
> On 12/12/22, Clem Cole <clemc@ccc.com> wrote:
> > On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591@gmail.com>
> > wrote:
> >
> >> And yet, for some reason, QNX has had almost no influence on anything
> >>
> > Be careful with a statement like that.  It's likely running in something
> > in your car. and very likely to be running in something in the last Boeing
> > or Airbus-based flight you took, and it was used when Amazon made the last
> > delivery to you.   It has long been popular in process control/materials
> > handling/robotics/fly-by-wire systems.
> >
> 
> I'm well aware that QNX has been extremely successful commercially and
> can be found in a wide range of embedded systems. I'm specifically
> talking about architectural influence on other OSes.

Minix?  QNX predated that.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
>
>>
>> I'm well aware that QNX has been extremely successful commercially and
>> can be found in a wide range of embedded systems. I'm specifically
>> talking about architectural influence on other OSes.
>
> Minix?  QNX predated that.
>
Yes, QNX predated Minix by several years, but Minix was completely
independent and there was no QNX influence on it at all AFAIK.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew Warkentin wrote:
> On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
> >
> >>
> >> I'm well aware that QNX has been extremely successful commercially and
> >> can be found in a wide range of embedded systems. I'm specifically
> >> talking about architectural influence on other OSes.
> >
> > Minix?  QNX predated that.
> >
> Yes, QNX predated Minix by several years, but Minix was completely
> independent and there was no QNX influence on it at all AFAIK.

Have you talked to Andy and confirmed that?  I'd be quite surprised if
he hadn't played with QNX but who knows.  I wouldn't assume he hadn't.

And forgive me for asking, do you have some axe to grind against QNX
or something?

To me, it's not that surprising that the rest of the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach.  Don't get me started on Mach, it has defenders but I absolutely
hate it.  Mach is more of a distributed research OS that advertised 
itself as a microkernel.  There is _nothing_ micro about Mach.  It's
a big bloated mess.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[segaloco via TUHS]

Has anything material come of later Mach work other than knowledge and influence on other systems?  I don't have a complete picture but it's my understanding that there were plenty of developments after NeXT and OSF had their way with it that aren't reflected in, say, macOS.  Something about said OSs being forked before Mach 3, which is listed in some references as being more true to the micro kernel architecture.

As an aside, I wish Apple would actually engage with their "open source" nature more.  I'd love to plink around in the Darwin kernel but even with Google-fu have never managed to actually get a kernel build all the way through.  Their lack of documentation is a painful matter.  I'd love to bootstrap it on RISC-V if I could get there...

- Matt G.

------- Original Message -------
On Tuesday, December 13th, 2022 at 5:05 PM, Larry McVoy <lm@mcvoy.com> wrote:


> On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew Warkentin wrote:
> 
> > On 12/13/22, Larry McVoy lm@mcvoy.com wrote:
> > 
> > > > I'm well aware that QNX has been extremely successful commercially and
> > > > can be found in a wide range of embedded systems. I'm specifically
> > > > talking about architectural influence on other OSes.
> > > 
> > > Minix? QNX predated that.
> > 
> > Yes, QNX predated Minix by several years, but Minix was completely
> > independent and there was no QNX influence on it at all AFAIK.
> 
> 
> Have you talked to Andy and confirmed that? I'd be quite surprised if
> he hadn't played with QNX but who knows. I wouldn't assume he hadn't.
> 
> And forgive me for asking, do you have some axe to grind against QNX
> or something?
> 
> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach. Don't get me started on Mach, it has defenders but I absolutely
> hate it. Mach is more of a distributed research OS that advertised
> itself as a microkernel. There is nothing micro about Mach. It's
> a big bloated mess.
> --
> ---
> Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Rich Morin]

> On Dec 13, 2022, at 17:40, segaloco via TUHS <tuhs@tuhs.org> wrote:
> 
> ... I wish Apple would actually engage with their "open source" nature more.  I'd love to plink around in the Darwin kernel but even with Google-fu have never managed to actually get a kernel build all the way through.  Their lack of documentation is a painful matter.  I'd love to bootstrap it on RISC-V if I could get there...

You might want to take a look at http://www.puredarwin.org/ ...

-r

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/13/22, Larry McVoy <lm@mcvoy.com> wrote:
>
> Have you talked to Andy and confirmed that?  I'd be quite surprised if
> he hadn't played with QNX but who knows.  I wouldn't assume he hadn't.
>

I haven't actually talked to him about it. He definitely is aware of
QNX since he's mentioned it on a few occasions, but I'm not sure if he
was aware of it when he wrote the first version of Minix.

Personally I don't see a lot of resemblance between the two, besides
both being single-personality Unix-like microkernel OSes with
lightweight IPC. Minix is more akin to a "serverized" conventional
Unix, whereas QNX seems to embrace its microkernel-ness more fully
with its focus on extensibility and its fairly tight integration of
IPC transport layer and filesystem. There may have been a little bit
of influence, but it's not all that obvious to me.

The pre-3.x versions seem especially un-QNX-like with their more or
less closed set of servers. Even in 3.x, the kernel still seems to
have quite a bit of knowledge about what servers are present and what
messages they accept. QNX does colocate the process server in the
kernel, but it makes very few assumptions about user-mode servers.

>
> And forgive me for asking, do you have some axe to grind against QNX
> or something?
>

Quite the opposite, hence why I'm writing my own OS with a similar architecture.

>
> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach.  Don't get me started on Mach, it has defenders but I absolutely
> hate it.  Mach is more of a distributed research OS that advertised
> itself as a microkernel.  There is _nothing_ micro about Mach.  It's
> a big bloated mess.
>

Yes, I agree 100% that Mach is a complete and utter failure as a
microkernel, and seems to have almost single-handedly destroyed the
reputation of microkernels. I don't get why everyone was so focused on
Mach-like kernels when there was a better alternative that had been
around in some form for almost a decade before Mach (QNX wasn't the
first of its kind; it seems to have had pretty significant influence
from Thoth).

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[arnold]

Andrew Warkentin <andreww591@gmail.com> wrote:

> Yes, I agree 100% that Mach is a complete and utter failure as a
> microkernel, and seems to have almost single-handedly destroyed the
> reputation of microkernels. I don't get why everyone was so focused on
> Mach-like kernels when there was a better alternative that had been
> around in some form for almost a decade before Mach (QNX wasn't the
> first of its kind; it seems to have had pretty significant influence
> from Thoth).

I suspect because Mach was available if you had the right Unix licenses
and because it was hot in the research world in the mid 80s. Researchy
types tend to look at what other researchers are doing / using, it seems
to me often without knowledge of or caring about what people are using
in industry.  (My two cents, from having worked at universities.)

Arnold

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Brad Spencer]

arnold@skeeve.com writes:

> Andrew Warkentin <andreww591@gmail.com> wrote:
>
>> Yes, I agree 100% that Mach is a complete and utter failure as a
>> microkernel, and seems to have almost single-handedly destroyed the
>> reputation of microkernels. I don't get why everyone was so focused on
>> Mach-like kernels when there was a better alternative that had been
>> around in some form for almost a decade before Mach (QNX wasn't the
>> first of its kind; it seems to have had pretty significant influence
>> from Thoth).
>
> I suspect because Mach was available if you had the right Unix licenses
> and because it was hot in the research world in the mid 80s. Researchy
> types tend to look at what other researchers are doing / using, it seems
> to me often without knowledge of or caring about what people are using
> in industry.  (My two cents, from having worked at universities.)
>
> Arnold

In that time frame there was a number of microkernel designs.  One that
has not been mentioned was OS-9 for the 6809/68000 processor.  I used it
pretty extensively.  OS-9 was very unix like from the userland POV, when
you consider something like V5 unix, however it didn't share any of the
same command names, just many of the same concepts.  It was close enough
that if you had the C compiler, a very basic K&R compiler, you could get
some of the unix command to compile without too much trouble.  I ported
sed from the DEC user's group source and the termcap library from BSD
and created a varargs library for it.  OS-9 was very microkernel and
nothing like Mach or even Minix.  It was also very much positioned to
real time OS needs of the time and was not really marketed generally and
unless you happened to have a Color Computer from Radio Shack or was a
part of the nitch they served you would probably have never come across
it and it would have seemed to be expensive to acquire.  It was very
clean, but you needed to know 6809 or 68000 assembly to create anything
new for the OS itself, although at least one person had figured out how
to use the C compiler, sort of, to produce assembly that could be
assembled into device managers and device drivers.




-- 
Brad Spencer - brad@anduin.eldar.org - KC8VKS - http://anduin.eldar.org

[TUHS] Re: (TUHS -> COFF?) Re: Clever code

[Michael Kjörling]

On 14 Dec 2022 06:54 -0500, from brad@anduin.eldar.org (Brad Spencer):
> [...] but you needed to know 6809 or 68000 assembly to create anything
> new for the OS itself,

Wasn't that the norm at the time, though? As I recall one of the
things that really set UNIX apart from other operating systems up
until about the early 1990s was precisely how machine-independent it
was by virtue of (with the exception of the early versions) having
been written in something other than assembler.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Microware's OS-9 (was: Clever code)

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 4881 bytes --]

At 2022-12-14T06:54:04-0500, Brad Spencer wrote:
> arnold@skeeve.com writes:
> > I suspect because Mach was available if you had the right Unix
> > licenses and because it was hot in the research world in the mid
> > 80s. Researchy types tend to look at what other researchers are
> > doing / using, it seems to me often without knowledge of or caring
> > about what people are using in industry.  (My two cents, from having
> > worked at universities.)

The UNSW CSE department seemed to be a bit more outward facing than
that, at least in my brief exposure to it, long after the 1980s.

> In that time frame there was a number of microkernel designs.  One
> that has not been mentioned was OS-9 for the 6809/68000 processor.  I
> used it pretty extensively.  OS-9 was very unix like from the userland
> POV, when you consider something like V5 unix, however it didn't share
> any of the same command names, just many of the same concepts.

This is emphatically true.  I used this system as a kid on a 64KiB
machine, and I don't remember even a mention of Unix in the doorstop of
a manual by Dale Puckett and Peter Dibble (who gave you something like 6
chapters of architectural background before introducing the shell
prompt).  Maybe they did mention Unix , but since it had no meaning to
me at the time, it didn't sink in.  I think it is also possible they
avoided any names that they thought might draw legal ire from AT&T.

> It was close enough that if you had the C compiler, a very basic K&R
> compiler, you could get some of the unix command to compile without
> too much trouble.

Years later I went to college, landed on Sun IPC workstations, and
quickly recognized OS-9's "T/S Edit" as a vi clone, and its "T/S Word"
as a version of nroff.  There was also a "T/S Spell" product but I don't
recall it clearly enough to venture whether it was a clone of ispell.

> OS-9 was very microkernel

In that deployment environment, it had to be.

> and nothing like Mach or even Minix.

With the source of all three available, a technical paper analyzing and
contrasting them would be a worthwhile thing to have.  (It's unclear to
me if even a historical version of QNX is available for study.)

> It was also very much positioned to real time OS needs of the time and
> was not really marketed generally and unless you happened to have a
> Color Computer from Radio Shack

Lucky me!  How I yearned for a 128KiB Color Computer 3 so I could
upgrade to OS-9 Level 2 and the windowing system.  (512KiB was
preferred, but there had been a spike in RAM prices right about the time
the machine was released.  Not that greater market success would have
kept Tandy from under-promoting and eventually killing the machine.[1])

> It was very clean, but you needed to know 6809 or 68000 assembly to
> create anything new for the OS itself,

The 6809 was my first exposure to a (relatively) clean ISA design,
having come from the Z80.  It probably helped that I was born with a
big-endian head and thus had an instinctive revulsion to Intel byte
order at an extremely young age.

In the late 1990s, Apple decided they wanted to rebrand their operating
system (still "MacOS [Classic]" at the time), looked at Microware's
name for its system, and said, "right, we'll be having that".
Microware, having apparently so carefully followed the letter of
trademark law with respect to AT&T Unix, sued Apple for peddling "OS/9"
in the operating system market, and promptly got their asses handed to
them by the federal district court, which dutifully honored the foremost
principle of law: big people get to stomp smaller people as often, and
as hard, as they would like.[2]

(Later, apparently, Apple pointed out this precedent to Cisco with a
shark-toothed grin when Apple decided they wanted the name "iOS" for yet
another revitalizing rebrand of familiar technology.  Cisco rolled over
and took some undisclosed amount of money, which they promptly spent on
acquisitions--they then were suddenly startled by the proportion of op
ex going to salaries, and initiated layoffs.)

Apple's never changed its stripes, but OS-9 lives on, as Free Software,
under the name NitrOS-9.[4]

Regards,
Branden

[1] Here's a story you may have to sit down for from Frank Durda IV (now
    deceased) about how the same company knifed their m68k-based
    line--which ran XENIX--in the gut repeatedly.  It's hard to find
    this story via Web search so I've made a Facebook post
    temporarily(?) public.  I'd simply include it, but it's pretty long.

    https://www.facebook.com/g.branden.robinson/posts/pfbid0F8MrvauQ6KPQ1tytme9uDiWGvprXft5dsxUzABYtdTKA9viZhB6Q2nadvtP1aDNQl

[2] https://www.cnn.com/2000/TECH/computing/03/21/os9.suit.idg/index.html
[3] https://appleinsider.com/articles/10/06/08/cisco_licenses_ios_name_to_apple_screenshot_shows_iwork_on_iphone
[4] https://sourceforge.net/projects/nitros9/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Microware's OS-9 (was: Clever code)

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 1673 bytes --]

On Wed, Dec 14, 2022 at 10:15 AM G. Branden Robinson <
g.branden.robinson@gmail.com> wrote:


Microware, having apparently so carefully followed the letter of
> trademark law with respect to AT&T Unix, sued Apple for peddling "OS/9"
> in the operating system market, and promptly got their asses handed to
> them by the federal district court, which dutifully honored the foremost
> principle of law: big people get to stomp smaller people as often, and
> as hard, as they would like.[2]
>


To be fair, the judge decided that there wasn't a whole lot of risk of
consumer confusion between a consumer OS that ran only on Apple Macintoshes
and an OEM OS that did not.  The only actual victims were Mac people who
stumbled into comp.os.os9 and got seriously confused.

In any case, Microsoft v. Lindows pretty much established that sometimes
the little guy wins, even in trademark cases.  Microsoft claimed that
Lindows was infringing their trademark for Windows (it was a Linux distro
that came with Wine and some glue) and lost the case on prior-use grounds
(both Xerox and Apple).  So rather than risking all on a retrial and maybe
losing the Windows trademark altogether, they settled for US$20M and
Lindows changed its corporate and distro names to Linspire.

That said, "Windows" is a descriptive trademark, and those are always shaky
legally.

> Cisco rolled over
> and took some undisclosed amount of money,
>

Money undoubtedly flowed from Apple Computers to Apple Records when the
first company started selling music, too.  This sort of thing is routine.
While I was working at Chase Bank, they paid a small fortune to Chase
Research for the rights to "chase.com".

[-- Attachment #2: Type: text/html, Size: 3165 bytes --]

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Harald Arnesen]

Larry McVoy [14/12/2022 02.05]:

> To me, it's not that surprising that the rest of the world didn't copy
> QNX because the rest of the world was either a mono-kernel or it was
> Mach.  Don't get me started on Mach, it has defenders but I absolutely
> hate it.  Mach is more of a distributed research OS that advertised
> itself as a microkernel.  There is_nothing_  micro about Mach.  It's
> a big bloated mess.

There was a rumour back in the early 90s that a new version of AmigaOS 
would be based on QNX. This was just before Commodore went bust, so 
nothing came out of it. A shame, I rather liked the Amiga and its 
so-called Workbench.
-- 
Hilsen Harald

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Wed, 14 Dec 2022 at 10:47, Harald Arnesen <skogtun@gmail.com> wrote:
>
> There was a rumour back in the early 90s that a new version of AmigaOS
> would be based on QNX.

Oh, it wasn't a rumour. The deal got quite advanced.

My now-employers covered it at the time:
https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/


> This was just before Commodore went bust, so
> nothing came out of it.

It was after. By this point it was Gateway Inc paying for it. There's
an account of what happened here:

https://www.trollaxor.com/2005/06/how-qnx-failed-amiga.html

The next-gen Amiga project was pretty much why QNX gained its GUI, Neutrino:

https://guidebookgallery.org/screenshots/qnx621

This is also the basis of the famous QNX Demo Disk, build by the late
great Dan Hildebrandt:

http://qnx.puslapiai.lt/qnxdemo/qnx_demo_disk.htm

>  A shame, I rather liked the Amiga and its
> so-called Workbench.

It was a good OS in its time and its admirers often call it a
microkernel, but IMVHO if all the code is in the same memory space,
that's not really a true microkernel.

Amiga Inc then went on to nearly do a deal with Tao Group for the much
more advanced Taos, in its later v2 incarnation as Intent/Elate.

https://wiki.c2.com/?TaoIntentOs

https://www.osnews.com/story/157/tao-group-on-elateos-amigade-and-more/

Ars has some info on this nearly-forgotten OS:

https://arstechnica.com/gadgets/2018/03/a-history-of-the-amiga-part-12-red-vs-blue/

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Harald Arnesen]

Liam Proven [15/12/2022 19.33]:

> On Wed, 14 Dec 2022 at 10:47, Harald Arnesen<skogtun@gmail.com>  wrote:
>> There was a rumour back in the early 90s that a new version of AmigaOS
>> would be based on QNX.
> Oh, it wasn't a rumour. The deal got quite advanced.
> 
> My now-employers covered it at the time:
> https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/

Thanks! Very interesting, I didn't know much of this.
-- 
Hilsen Harald

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Fri, 16 Dec 2022 at 11:43, Harald Arnesen <skogtun@gmail.com> wrote:
>
> Thanks! Very interesting, I didn't know much of this.

My pleasure.

I am a daily Unix user but my main interests tend to lie elsewhere.

I don't have any citations for this, but it looked to me, watching in
the magazines at the time, that _before_ the Amiga deal, QNX did run
on PCs for testing purposes, but I am not totally sure if it had a GUI
at all, and little to no multimedia support.

My impression is that QNX implemented that for Amiga Inc and then were
left with it when Amiga turned its gaze on Tao and Elate.

(Some of the best remaining info about Intent and Elate is in HN comments, e.g.

https://news.ycombinator.com/item?id=16053726

https://news.ycombinator.com/item?id=9807269

Which leads to:

http://www.uruk.org/emu/Taos.html )

I am curious to know if the TAOS virtual-processor,
all-binaries-are-CPU-independent, model influenced or inspired
Inferno. I think that Inferno postdates TAOS.

But returning to QNX: my impression is, they implemented a GUI and
multimedia frameworks for Amiga, Amiga changed its mind, and QNX
offered it as a PC dev kit for a while.

E.g.
https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3

But the only mass-market end-user-facing graphical multimedia-capable
QNX devices I know of were the Blackberry X smartphones. (And
cancelled tablet and netbook.)

I owned a Blackberry Passport. A lovely device with a lovely OS... but
too late and it flopped.

So, oddly, and accidentally and unintentionally, Amiga Inc was
directly responsible for the Blackberry smartphone OS.

-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Stuff Received]

On 2022-12-18 09:05, Liam Proven wrote (in part):
[...]
> But returning to QNX: my impression is, they implemented a GUI and
> multimedia frameworks for Amiga, Amiga changed its mind, and QNX
> offered it as a PC dev kit for a while.
> 
> E.g.
> https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3

QNX initially ran only on x86 kit. I do not recall a GUI at that time.

> But the only mass-market end-user-facing graphical multimedia-capable
> QNX devices I know of were the Blackberry X smartphones. (And
> cancelled tablet and netbook.)

The BB10 i/f was designed by TAT 
(https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of-becoming-awesome.html).

> I owned a Blackberry Passport. A lovely device with a lovely OS... but
> too late and it flopped.

Quite so.

N.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Sun, 18 Dec 2022 at 16:08, Stuff Received <stuff@riddermarkfarm.ca> wrote:
>
> The BB10 i/f was designed by TAT
> (https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of-becoming-awesome.html).

Oh, fascinating. I did not know that -- thank you!

> > I owned a Blackberry Passport. A lovely device with a lovely OS... but
> > too late and it flopped.
>
> Quite so.

One of the slightly stranger things in the smartphone OS space
recently, for me, has been watching Huawei's response to the licensing
fight with Google.

Huawei now promotes its devices as running "Harmony OS" with a lot of
hot air about microkernels, convergence, cross-platform, blah blah.

It's nothing of the kind. It's AOSP, the open-source core of Android,
with their own skin on top.

I don't think Huawei has the tech skills to build a new OS as it's
claimed it has...

But it could make some of those claims come true if it licensed BB10
and spent a bit on updating it.



-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/18/22, Liam Proven <lproven@gmail.com> wrote:
>
> I don't have any citations for this, but it looked to me, watching in
> the magazines at the time, that _before_ the Amiga deal, QNX did run
> on PCs for testing purposes, but I am not totally sure if it had a GUI
> at all, and little to no multimedia support.
>
> My impression is that QNX implemented that for Amiga Inc and then were
> left with it when Amiga turned its gaze on Tao and Elate.
>

QNX Classic and 4.x only ran on x86 machines, most of which were
either standard PCs or at least sort of PC-like (although AFAIK the
ability to run without a BIOS was present very early on). It was quite
common to run QNX on desktops as a development host for embedded
systems AFAIK.

GUIs for QNX predate the Amiga deal, and have existed since the late
80s. The original was QNX Windows, which was either a reimplementation
or port (not quite sure which) of Open Look on a custom non-X11 window
server, running on later versions of 2.x and all versions of 4.x.

Later versions of 4.x added Photon 1, which looks like a cross between
Motif and Windows 9x, again based on a custom window server (this time
with a rather unconventional multi-process architecture). The 90s-era
demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
disk back in the 80s but this didn't have a GUI).

6.0 came with Photon 2, which is still Win9x-ish in terms of
organization and is a fairly straightforward evolution of Photon 1,
although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
out slightly after the Amiga deal, so that might be the reason for the
Amiga-like theming.

Neutrino was not the name of a GUI, but rather of the entire OS that
succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
incapable of self-hosting and were developed alongside 4.x; 6.0 was
the first mainline QNX version to be Neutrino-based).

>
> But the only mass-market end-user-facing graphical multimedia-capable
> QNX devices I know of were the Blackberry X smartphones. (And
> cancelled tablet and netbook.)
>

There were also the i-Opener (running 4.25) and Audrey (running 6.0)
internet appliances of the late 90s.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Liam Proven]

On Tue, 20 Dec 2022 at 09:31, Andrew Warkentin <andreww591@gmail.com> wrote:

> QNX Classic and 4.x only ran on x86 machines, most of which were
> either standard PCs or at least sort of PC-like (although AFAIK the
> ability to run without a BIOS was present very early on). It was quite
> common to run QNX on desktops as a development host for embedded
> systems AFAIK.
>
> GUIs for QNX predate the Amiga deal, and have existed since the late
> 80s. The original was QNX Windows, which was either a reimplementation
> or port (not quite sure which) of Open Look on a custom non-X11 window
> server, running on later versions of 2.x and all versions of 4.x.
>
> Later versions of 4.x added Photon 1, which looks like a cross between
> Motif and Windows 9x, again based on a custom window server (this time
> with a rather unconventional multi-process architecture). The 90s-era
> demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
> disk back in the 80s but this didn't have a GUI).

Fascinating. Thanks for setting the record straight, and I apologise
for my inaccurate speculation.
>
> 6.0 came with Photon 2, which is still Win9x-ish in terms of
> organization and is a fairly straightforward evolution of Photon 1,
> although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
> out slightly after the Amiga deal, so that might be the reason for the
> Amiga-like theming.
>
> Neutrino was not the name of a GUI, but rather of the entire OS that
> succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
> incapable of self-hosting and were developed alongside 4.x; 6.0 was
> the first mainline QNX version to be Neutrino-based).

Aha!

> There were also the i-Opener (running 4.25) and Audrey (running 6.0)
> internet appliances of the late 90s.

I did not realise they were QNX devices. I vaguely thought the 3Com
Audrey was based on BeIA, in fact, but I must be confusing it with
something else -- perhaps the Sony eVilla.


-- 
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven@cix.co.uk ~ gMail/gTalk/FB: lproven@gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> 
> It's not necessarily true that microkernels are significantly slower.

uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:

  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc

or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.

At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

Wasn't there some statement that QNX dropped some of these?  Copy plus
context switch?

On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> > 
> > It's not necessarily true that microkernels are significantly slower.
> 
> uKernels are usually quite fast as they do so little. What can be slow
> is emulating a Unix like OS on top due to context switches. For instance,
> a user process doing read() will have the following context switches:
> 
>   userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> 
> or worse (I didn't account for a few things). Because of this even some
> uKernels run a few critical services + drivers in the supervisor mode.
> But overall slowdown of such a unix emulation will very much depend on the
> workload and also what kind of performance improvements you are willing to
> try in a complex kernel vs same services running in user mode.
> 
> At present the linux kernel has about 31+ Million lines (accounting for
> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> This LoC growth is entirely understandable but I wonder how things may
> have turned out in an alternate universe of uKernel based designs....

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Bakul Shah]

Don't see how unless they put multiple related services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.

> On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> Wasn't there some statement that QNX dropped some of these?  Copy plus
> context switch?
> 
> On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
>> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
>>> 
>>> It's not necessarily true that microkernels are significantly slower.
>> 
>> uKernels are usually quite fast as they do so little. What can be slow
>> is emulating a Unix like OS on top due to context switches. For instance,
>> a user process doing read() will have the following context switches:
>> 
>>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
>> 
>> or worse (I didn't account for a few things). Because of this even some
>> uKernels run a few critical services + drivers in the supervisor mode.
>> But overall slowdown of such a unix emulation will very much depend on the
>> workload and also what kind of performance improvements you are willing to
>> try in a complex kernel vs same services running in user mode.
>> 
>> At present the linux kernel has about 31+ Million lines (accounting for
>> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
>> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
>> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
>> This LoC growth is entirely understandable but I wonder how things may
>> have turned out in an alternate universe of uKernel based designs....
> 
> -- 
> ---
> Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> Don't see how unless they put multiple related services in the same
> address space, which reduces context switching but tends toward a
> monokernel (& increased coupling). Unless I am misunderstanding you.

I don't see why two services in a microkernel couldn't arrange to
share a region of memory and implement bidirectional queues
between themselves. With an appropriate signalling mechanism,
you'd still be context switching but avoiding a lot of copying.

        - Dan C.


> > On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > Wasn't there some statement that QNX dropped some of these?  Copy plus
> > context switch?
> >
> > On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> >> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> >>>
> >>> It's not necessarily true that microkernels are significantly slower.
> >>
> >> uKernels are usually quite fast as they do so little. What can be slow
> >> is emulating a Unix like OS on top due to context switches. For instance,
> >> a user process doing read() will have the following context switches:
> >>
> >>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> >>
> >> or worse (I didn't account for a few things). Because of this even some
> >> uKernels run a few critical services + drivers in the supervisor mode.
> >> But overall slowdown of such a unix emulation will very much depend on the
> >> workload and also what kind of performance improvements you are willing to
> >> try in a complex kernel vs same services running in user mode.
> >>
> >> At present the linux kernel has about 31+ Million lines (accounting for
> >> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> >> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> >> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> >> This LoC growth is entirely understandable but I wonder how things may
> >> have turned out in an alternate universe of uKernel based designs....
> >
> > --
> > ---
> > Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat
>

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross wrote:
> On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> > Don't see how unless they put multiple related services in the same
> > address space, which reduces context switching but tends toward a
> > monokernel (& increased coupling). Unless I am misunderstanding you.
> 
> I don't see why two services in a microkernel couldn't arrange to
> share a region of memory and implement bidirectional queues
> between themselves. With an appropriate signalling mechanism,
> you'd still be context switching but avoiding a lot of copying.

My mind went to similar thoughts.  How did QNX manage the page cache?
Did they have mmap?

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Dan Cross]

On Thu, Dec 15, 2022 at 9:07 AM Larry McVoy <lm@mcvoy.com> wrote:
> On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross wrote:
> > On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul@iitbombay.org> wrote:
> > > Don't see how unless they put multiple related services in the same
> > > address space, which reduces context switching but tends toward a
> > > monokernel (& increased coupling). Unless I am misunderstanding you.
> >
> > I don't see why two services in a microkernel couldn't arrange to
> > share a region of memory and implement bidirectional queues
> > between themselves. With an appropriate signalling mechanism,
> > you'd still be context switching but avoiding a lot of copying.
>
> My mind went to similar thoughts.  How did QNX manage the page cache?
> Did they have mmap?

I don't know if they did historically, but I kind of doubt it; QNX predates
4.2BSD, and I imagine there wasn't much influence from TENEX/PMAP).
But they seem to now:
https://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.getting_started/topic/s1_resmgr_io_mmap.html

No idea how they handle page caching. That's an interesting question,
given how they adopted POSIX and try to at least give the outward
appearance of being Unix-y.

Given the emphasis on the message-based architecture, where messages
can be sent between nodes, I doubt there's that much support for shared
memory as a basis for IPC; I think they favored explicit messaging copying
facilitated by the ukernel. Whether you could build something bespoke using
the provided primitives is another matter.

        - Dan C.

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Larry McVoy]

I wasn't clear.  There was someone who said that the copy to another
address space included a context switch to the receiving process
(I think that was it) that resulted in better performance.  Perhaps
the person who said that can elaborate.

On Wed, Dec 14, 2022 at 09:36:12PM -0800, Bakul Shah wrote:
> Don't see how unless they put multiple related services in the same
> address space, which reduces context switching but tends toward a
> monokernel (& increased coupling). Unless I am misunderstanding you.
> 
> > On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > 
> > Wasn't there some statement that QNX dropped some of these?  Copy plus
> > context switch?
> > 
> > On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
> >> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
> >>> 
> >>> It's not necessarily true that microkernels are significantly slower.
> >> 
> >> uKernels are usually quite fast as they do so little. What can be slow
> >> is emulating a Unix like OS on top due to context switches. For instance,
> >> a user process doing read() will have the following context switches:
> >> 
> >>  userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
> >> 
> >> or worse (I didn't account for a few things). Because of this even some
> >> uKernels run a few critical services + drivers in the supervisor mode.
> >> But overall slowdown of such a unix emulation will very much depend on the
> >> workload and also what kind of performance improvements you are willing to
> >> try in a complex kernel vs same services running in user mode.
> >> 
> >> At present the linux kernel has about 31+ Million lines (accounting for
> >> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
> >> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
> >> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
> >> This LoC growth is entirely understandable but I wonder how things may
> >> have turned out in an alternate universe of uKernel based designs....
> > 
> > -- 
> > ---
> > Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?

[Andrew Warkentin]

On 12/14/22, Larry McVoy <lm@mcvoy.com> wrote:
> Wasn't there some statement that QNX dropped some of these?  Copy plus
> context switch?
>

Yeah, QNX/L4-type IPC is usually just copying followed by a context
switch. Some such kernels (like seL4) don't even have full message
queues per endpoint (the scheduler queue sort of functions as an IPC
queue). Mach IPC is slow because of stuff like permission checking,
which I would assume involves iteration over a list of permitted
threads. QNX/L4-type kernels usually either use constant-time
permission checks (like the old "clans and chiefs" model used by L3
and early L4, or the more modern capability-oriented model used by
seL4 and some others), or lack kernel permission checking entirely
leaving it up to servers.

Another issue is that Mach-type kernels don't have what is known as
"direct process switching" AFAIK. When a synchronous message is sent
on a QNX/L4-type kernel, the kernel immediately switches to the
receiving process, bypassing the scheduler queue entirely, with the
remainder of the sender's timeslice being given to the receiver
(depending on the kernel, priorities may factor into this so it isn't
always quite that simple though). Mach-like kernels often require the
sender to wait for the kernel to decide to schedule the receiver based
on the queue, and then once the reply is sent there's another wait for
the kernel to again decide to schedule the sender again, which makes
for rather poor performance.

On 12/14/22, Bakul Shah <bakul@iitbombay.org> wrote:
> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591@gmail.com> wrote:
>>
>> It's not necessarily true that microkernels are significantly slower.
>
> uKernels are usually quite fast as they do so little. What can be slow
> is emulating a Unix like OS on top due to context switches. For instance,
> a user process doing read() will have the following context switches:
>
>   userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
>
> or worse (I didn't account for a few things). Because of this even some
> uKernels run a few critical services + drivers in the supervisor mode.
> But overall slowdown of such a unix emulation will very much depend on the
> workload and also what kind of performance improvements you are willing to
> try in a complex kernel vs same services running in user mode.

Yeah, excessive vertical layering can be bad for performance. QNX
normally follows a process-per-subsystem-instance architecture, so the
chain is just:

client -> (kernel) -> diskServer  -> (kernel) -> client

where the disk server includes the disk driver, partition table driver
(if applicable), and disk filesystem. The VFS layer isn't even
involved at all on reads, writes, and the like (it's pretty much only
there to deal with operations that involve paths and not those that
only involve FDs), whereas some other Unix-like microkernel OSes have
it act as an intermediary on all FS operations. A lot of the time,
protection domains correspond more to subsystem instances rather than
layer instances, so there really isn't much harm in merging all layers
of a subsystem into a single process for the sake of performance. When
there is a benefit to separation of layers into different processes,
it is possible to use tap-type drivers to allow running subsystem
instances that only contain some of the layers (QNX doesn't do this
AFAIK, but my OS will).

[TUHS] Re: Clever code

[Michael Kjörling]

On 11 Dec 2022 18:15 -0800, from bakul@iitbombay.org (Bakul Shah):
> Agree that clear code is preferable to complicated code. But in practice
> people sacrifice clarity for performance improvement all the time. Look
> at the kernel code of any modern os. Everybody pays lip service to this
> but most anything other than toy programs ends up getting needlessly
> complicated over time.

Performant code does not need to stand in opposition to clear code.
And if you are writing performance-critical or size-critical code
(such as the QNX microkernel that someone brought up), then of course
you might end up needing to do things in non-obvious ways; that's not
the point here. Clever is fine IMO _where the cleverness provides an
actual benefit in a real-world scenario_, and an operating system
kernel (and a language standard library) is a rather special type of
program where sometimes you have to do things in somewhat non-obvious
ways because of the environment the code is meant to execute in. But a
significant portion of the time, where I see "clever" code there is
_no_ significant benefit to the cleverness; it's often more about
"showing off", or saving a few source code characters at the expense
of at-a-glance readability, than it is about actual usefulness and
necessity.

Necessary complexity in order to solve the problem, combined with
things like performance requirements, may sometimes require clever
code. At that point, there is a benefit to the cleverness. But one can
still aim to write the clever code _clearly_, with everything from
comments to good variable and function names to formatting that
enhances readability by for example grouping related operations
together in the source code to keeping related parts grouped together
and separate from other code. Just such a simple thing as that I often
add extra parenthesis over and beyond what's actually required based
on operator precedence, because doing so makes it clearer what goes
together and unless the compiler is severely braindead by modern
standards, doing so costs _nothing_ past the compilation stage.

-- 
✍  Michael Kjörling                  🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Dave Horsfall]

On Sun, 11 Dec 2022, Larry McVoy wrote:

> > I've always used the maxim "Write code as though the next person to 
> > maintain it is an axe-wielding psychopath who knows where you live".
> 
> My main job, for the 20 years until I retired, was to keep telling 
> people that code that you wrote 6 months ago might as well have been 
> written by someone else.  Optimize for reading the code, not writing the 
> code.  It's read many.

I should clarify that on the odd occasion that I write obscure code it's 
for efficiency, and I comment it well because *I* might be the one to 
maintain it six months hence :-)

-- Dave

[TUHS] Re: Stdin Redirect in Cu History/Alternatives?

[Chet Ramey]

On 12/12/22 4:34 PM, Dave Horsfall wrote:

>> My main job, for the 20 years until I retired, was to keep telling
>> people that code that you wrote 6 months ago might as well have been
>> written by someone else.  Optimize for reading the code, not writing the
>> code.  It's read many.
> 
> I should clarify that on the odd occasion that I write obscure code it's
> for efficiency, and I comment it well because *I* might be the one to
> maintain it six months hence :-)

Six months? Try ten years. :-)

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/