[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Norman Wilson]

Doug McIlroy:

  This reminds me of how I agonized over Mike Lesk's refusal to remove
  remote execution from uucp.

====

Uux, the remote-execution mechanism I remember from uucp, had
rather better utility than the famous Sendmail back-door: it
was how uucp carried mail, by sending a file to be handed to
mailer on the remote system.  It was clearly dangerous if
the remote site accepted any command, but as shipped in V7
only a short list of remote commands was allowed: mail rmail
lpr opr fsend fget.  (As uucp was used to carry other things
like netnews, the list was later extended by individual sites,
and eventually moved to a file so reconfiguration needn't
recapitulate compilation).

Not the safest of mechanisms, but at least in V7 it had a use
other than Mike fixing your system for you.

Is there some additional history here?  e.g. was the list of
permitted commands added after arguments about safety, or
some magic command that let Mike in removed?  Or was there a
different remote-execution back door I don't remember and don't
see in a quick look at uuxqt.c?

Norman Wilson
Toronto ON

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

There was also a feature Mike Lesk added that allowed a marked line,
something like

%! command

to cause the command to be executed when the recipient read the mail, for
example to demonstrate a feature of a program or teach the recipient
something. He meant well. Dennis had the closest he ever had to a
conniption, and it was taken out post haste. Meaning well is not enough.

-rob


On Mon, Jul 31, 2023 at 4:23 AM Norman Wilson <norman@oclsc.org> wrote:

> Doug McIlroy:
>
>   This reminds me of how I agonized over Mike Lesk's refusal to remove
>   remote execution from uucp.
>
> ====
>
> Uux, the remote-execution mechanism I remember from uucp, had
> rather better utility than the famous Sendmail back-door: it
> was how uucp carried mail, by sending a file to be handed to
> mailer on the remote system.  It was clearly dangerous if
> the remote site accepted any command, but as shipped in V7
> only a short list of remote commands was allowed: mail rmail
> lpr opr fsend fget.  (As uucp was used to carry other things
> like netnews, the list was later extended by individual sites,
> and eventually moved to a file so reconfiguration needn't
> recapitulate compilation).
>
> Not the safest of mechanisms, but at least in V7 it had a use
> other than Mike fixing your system for you.
>
> Is there some additional history here?  e.g. was the list of
> permitted commands added after arguments about safety, or
> some magic command that let Mike in removed?  Or was there a
> different remote-execution back door I don't remember and don't
> see in a quick look at uuxqt.c?
>
> Norman Wilson
> Toronto ON
>

[-- Attachment #2: Type: text/html, Size: 2578 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[George Michaelson]

It must be the fate of all UUCP-like protocols to recapitulate the
life of UUCP. My memory is that ACSNet (quite UUCP like) had both an
execute, and a *@dom.ain and even root@* handling, and that it caused
some DOS consequences.

There's nothing implicitly wrong with remote execution, remote job
entry was a thing back in the coloured book protocols. I guess the
problem inherent in "just do this thing" in UUCP was the permissions
and runtime context. But a chroot() and permissions drop should have
made it less risky. There is the "but anyone can inject it" problem.

Execute on read is just awful. But, now we have HTML to track "they
read it" through URL fetch.

G

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Erik E. Fair]

	Date: Mon, 31 Jul 2023 09:34:56 +1000
	From: George Michaelson <ggm@algebras.org>

	[...]

	Execute on read is just awful. But, now we have HTML to track "they
	read it" through URL fetch.

And then the utterly disastrous: JavaScript. It should be *eliminated*
from the WWW as the gross security violation it is.

	"don't run software from strangers",

	Erik Fair

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

On Sun, Jul 30, 2023, 5:59 PM Erik E. Fair <fair-tuhs@netbsd.org> wrote:

>
>         Date: Mon, 31 Jul 2023 09:34:56 +1000
>         From: George Michaelson <ggm@algebras.org>
>
>         [...]
>
>         Execute on read is just awful. But, now we have HTML to track "they
>         read it" through URL fetch.
>
> And then the utterly disastrous: JavaScript. It should be *eliminated*
> from the WWW as the gross security violation it is.
>
>         "don't run software from strangers",
>


Write once, run everywhere.

Warner

>         Erik Fair
>

[-- Attachment #2: Type: text/html, Size: 1293 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Grant Taylor via TUHS]

On 7/30/23 7:26 PM, Warner Losh wrote:
> Write once, run everywhere.

Wasn't that "Java" proper and not "JavaScript"?



Grant. . . .

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

On Mon, Jul 31, 2023, 4:58 PM Grant Taylor via TUHS <tuhs@tuhs.org> wrote:

> On 7/30/23 7:26 PM, Warner Losh wrote:
> > Write once, run everywhere.
>
> Wasn't that "Java" proper and not "JavaScript"?
>

It was.. but since Javascript has the name Java in it... I thought I'd make
that joke...

Warner

>
> Grant. . . .
>

[-- Attachment #2: Type: text/html, Size: 903 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Grant Taylor via TUHS]

On 7/31/23 6:05 PM, Warner Losh wrote:
> It was.. but since Javascript has the name Java in it... I thought I'd 
> make that joke...

Fair enough.

I thought perhaps I was misremembering things and more caught up in 
trying to correct my mental understanding and failed to see the humor.



-- 
Grant. . . .

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Niklas Karlsson]

[-- Attachment #1: Type: text/plain, Size: 726 bytes --]

Den mån 31 juli 2023 kl 02:27 skrev Warner Losh <imp@bsdimp.com>:

>
>
> On Sun, Jul 30, 2023, 5:59 PM Erik E. Fair <fair-tuhs@netbsd.org> wrote:
>
>>
>>         Date: Mon, 31 Jul 2023 09:34:56 +1000
>>         From: George Michaelson <ggm@algebras.org>
>>
>>         [...]
>>
>>         Execute on read is just awful. But, now we have HTML to track
>> "they
>>         read it" through URL fetch.
>>
>> And then the utterly disastrous: JavaScript. It should be *eliminated*
>> from the WWW as the gross security violation it is.
>>
>>         "don't run software from strangers",
>>
>
>
> Write once, run everywhere.
>

I've seen some cynical people render it as "write once, run away".

Niklas

[-- Attachment #2: Type: text/html, Size: 1559 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Grant Taylor via TUHS]

On 7/31/23 8:51 PM, Niklas Karlsson wrote:
> I've seen some cynical people render it as "write once, run away".

I've heard:

  - Write once and crash everywhere.

  - Just another vulnerability announcement.



-- 
Grant. . . .

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Theodore Ts'o]

On Tue, Aug 01, 2023 at 03:51:28AM +0200, Niklas Karlsson wrote:
> >
> > Write once, run everywhere.
> 
> I've seen some cynical people render it as "write once, run away".

I've always preferred, "Write once, run screaming".  :-)

     	    	       	      	    	- Ted

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[segaloco via TUHS]

> And then the utterly disastrous: JavaScript. It should be eliminated
> from the WWW as the gross security violation it is.
> 
> "don't run software from strangers",
> 
> Erik Fair

The browser is becoming more and more of an OS of its own, it just needs to act like it in the security realm.

- Matt G.

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

Nathaniel (Mr Mime) Borenstein came up with something (atomicmail?) that
was intended to be more functional than raw text but safer than free
execution of unknown code.  I disremember the details.  I don’t think it
ever got traction.

On Sun, Jul 30, 2023 at 7:59 PM Erik E. Fair <fair-tuhs@netbsd.org> wrote:

>
>         Date: Mon, 31 Jul 2023 09:34:56 +1000
>         From: George Michaelson <ggm@algebras.org>
>
>         [...]
>
>         Execute on read is just awful. But, now we have HTML to track "they
>         read it" through URL fetch.
>
> And then the utterly disastrous: JavaScript. It should be *eliminated*
> from the WWW as the gross security violation it is.
>
>         "don't run software from strangers",
>
>         Erik Fair
>
-- 
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Erik E. Fair]

	Date: Tue, 1 Aug 2023 05:22:17 -0400
	From: Marc Donner <marc.donner@gmail.com>

	Nathaniel (Mr Mime) Borenstein came up with something (atomicmail?)
	that was intended to be more functional than raw text but safer
	than free execution of unknown code. I disremember the details.
	I don't think it ever got traction.


You remember correctly. It got stomped by those of us in the IETF MIME working group with approximately the same forceful negative reaction as you've seen here to Mike Lesk's idea of instantly executed Unix commands in e-mail.

I'm hardly innocent of this - while writing & operating the AppleLink/Internet e-mail gateway at Apple in the 1990s, I discovered that I could download the entire user directory from AppleLink (over 50k users: all Apple employees, Apple 3rd-party developers, Apple retail dealers - the whole "Apple Federation" at that time was on AppleLink), which included both usernames and "full name" fields, which could provide the basis for an AppleLink directory lookup service on the Internet. I figured it'd be easy to use FINGER & WHOIS as the protocol ports since the outputs of those are basically unstructured (unspecified) ASCII text, e.g. "finger fair@applelink.apple.com" would return a list of all usernames and full names matching "fair".

I was writing in Perl because e-mail gatewaying is primarily about string handling, and it sucks to write in C for that. The best performing way to implement the text search was to use its eval() function with a regex constructed from the network protocol input. I tested it, and it worked great, but I bet you can guess where this is going - how to perfectly sanitize the search term inputs taken directly from the net so they don't become arbitrary Perl code?

I never deployed it, partly because I couldn't convince myself I'd made the service completely secure, and partly as a privacy matter: finger (especially after the 1989 Morris Worm & the increasing amounts of e-mail spam) was not a service that sites were offering any longer because there were too many bad actors on the Internet, and it just wasn't a good idea to be as open & trusting as the ARPANET had been. I lament the passing of that culture from time to time.

I think anyone with a modicum of experience in computer & systems security can instantly recognize the dangers in executable code transmitted unsolicited to unwary recipients and automatically executed without prior, explicit permission, and works to stop anything along those lines from becoming standard practice because, despite all the protestations that "it's run in a sandbox, it's safe!", the proponents can never prove their case beyond reasonable doubt.

How many bugs were discovered in the "restricted shell" (rsh) over the years?

Sometimes we fail to prevent such bad ideas from being implemented: JavaScript in HTML/HTTP is one such. What concerns me these days is how often JavaScript is showing up in text/html e-mail. At least visiting a website (URL) with a web browser is, to some degree, an act of volition. Particularly with MIME, Internet e-mail has to be parsed and presented (and which HTML parsers these days do not also include a JavaScript interpreter?), not merely spewed to a presumed-ASCII (OK, UTF-8) terminal. Even simple spew could be dangerous: who remembers "intelligent terminal" transmit-back codes and the mischief those caused?

IIRC, the question we posed to Nathaniel was: "do we really want to enable letter bombs?"

Some of us also remembered (and possibly referenced) the UNAbomber.

	Erik

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Dave Horsfall]

On Tue, 1 Aug 2023, Erik E. Fair wrote:

> Even simple spew could be dangerous: who remembers "intelligent 
> terminal" transmit-back codes and the mischief those caused?

ASCII bombs were fairly popular in the old MS-DOS BBS days (format the 
disk, anyone?), and it was possible over packet radio too (sort of a BBS 
via Amateur i.e. "ham" radio).

-- Dave (vk2kfu)

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Ron Natalie]

ASCII Bombs?   Like my Letter Bomb Transport Protocol (LBTP)?

https://groups.google.com/g/net.followup/c/OJBALbzTq4w/m/LoxMnbz0bwMJ

It seems to have lost something in the formatting (the leading spaces 
were all removed).



------ Original Message ------
From "Dave Horsfall" <dave@horsfall.org>
To "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
Date 8/1/23, 8:37:20 PM
Subject [TUHS] Re: Cool talk on Unix and Sendmail history, by Eric 
Allman

>On Tue, 1 Aug 2023, Erik E. Fair wrote:
>
>>  Even simple spew could be dangerous: who remembers "intelligent
>>  terminal" transmit-back codes and the mischief those caused?
>
>ASCII bombs were fairly popular in the old MS-DOS BBS days (format the
>disk, anyone?), and it was possible over packet radio too (sort of a BBS
>via Amateur i.e. "ham" radio).
>
>-- Dave (vk2kfu)

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Grant Taylor via TUHS]

On 8/2/23 9:52 AM, Ron Natalie wrote:
> ASCII Bombs?   Like my Letter Bomb Transport Protocol (LBTP)?
> 
> https://groups.google.com/g/net.followup/c/OJBALbzTq4w/m/LoxMnbz0bwMJ

~chuckle~

> It seems to have lost something in the formatting (the leading spaces 
> were all removed).

Ya....

People think I'm weird for not liking languages that use white space as 
structural definition.  Frequently those people have not experienced 
such a failure as what has happened to your ASCII Bomb.



Grant. . . .

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[segaloco via TUHS]

> People think I'm weird for not liking languages that use white space as
> structural definition.
>
> Grant. . . .

This is my chief gripe with Python, although on the flip side Python isn't the right language anyway for most scenarios where I use whitespace/indentation to imply structure the language itself can't articulate.  It's meant for mainly functional programming as I understand it so the structure does enforce some stylistic practices conducive to good functional programming.  Still a shame to force a particular style approach by default rather than just strongly suggest it.

What strikes me particularly odd about the Python case is that its not like that space-sensitivity evolved out of the same line of reasoning as the compulsory spacing in FORTRAN, COBOL, etc.  It seems mainly to be a way to operate without code blocks, with the "blocks" being implied by indentation rather than braces, parens, or some other delimiter.

In UNIX of course we have our own little variation on this problem with make(1) and the need to tab out the rule definition.  I seem to recall reading somewhere (perhaps Doug's McIlroy's UPM excerpts) that that Stu Feldman considered undoing that but there were already users who that would've caused trouble for, so make's early, entrenched adoption stymied attempts at the time to rectify this.  Anyone with better details feel free to correct me.

- Matt G.

P.S. This answer can be off list or spin off a separate thread for make junkies, but did any AT&T or BSD revision of make(1) support rule names coming from variables rather than explicitly entered?

For instance:

$(BIN): $(OBJS)
    $(CC) $(LDFLAGS) -o $(BIN) $(OBJS) $(LIBS)

I used to use this in makefiles but at some point, I think with one of the BSDs, it balked at the idea of a variable rule name and so it fell out of my practice in trying to avoid GNUisms.

It's been a while but I feel like I ran through and tried this on V7, System III, and PDP-11 System V and all of them were unhappy about that construct.  I can try and get on the LCMs 3B400 later to see what SVR3 does.  I don't remember which of the BSDs (if not multiple) I ran into that issue on initially, but I can't imagine one of the major streams would work that in without the other two wanting to copy their notes.

Maybe an alternative question is if folks are aware of make implementations besides GNU that *do* support that sort of thing.

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 3290 bytes --]

On Wed, Aug 2, 2023 at 4:20 PM segaloco via TUHS <tuhs@tuhs.org> wrote:

> > People think I'm weird for not liking languages that use white space as
> > structural definition.
> >
> > Grant. . . .
>
> This is my chief gripe with Python, although on the flip side Python isn't
> the right language anyway for most scenarios where I use
> whitespace/indentation to imply structure the language itself can't
> articulate.  It's meant for mainly functional programming as I understand
> it so the structure does enforce some stylistic practices conducive to good
> functional programming.  Still a shame to force a particular style approach
> by default rather than just strongly suggest it.
>
> What strikes me particularly odd about the Python case is that its not
> like that space-sensitivity evolved out of the same line of reasoning as
> the compulsory spacing in FORTRAN, COBOL, etc.  It seems mainly to be a way
> to operate without code blocks, with the "blocks" being implied by
> indentation rather than braces, parens, or some other delimiter.
>
> In UNIX of course we have our own little variation on this problem with
> make(1) and the need to tab out the rule definition.  I seem to recall
> reading somewhere (perhaps Doug's McIlroy's UPM excerpts) that that Stu
> Feldman considered undoing that but there were already users who that
> would've caused trouble for, so make's early, entrenched adoption stymied
> attempts at the time to rectify this.  Anyone with better details feel free
> to correct me.
>
> - Matt G.
>
> P.S. This answer can be off list or spin off a separate thread for make
> junkies, but did any AT&T or BSD revision of make(1) support rule names
> coming from variables rather than explicitly entered?
>
> For instance:
>
> $(BIN): $(OBJS)
>     $(CC) $(LDFLAGS) -o $(BIN) $(OBJS) $(LIBS)
>
> I used to use this in makefiles but at some point, I think with one of the
> BSDs, it balked at the idea of a variable rule name and so it fell out of
> my practice in trying to avoid GNUisms.
>

BSD has long supported

PROG=cat

.include <bsd.prog.mk>

to have it deal with all the details. Of course, FreeBSD's is more complex
than that, because nothing is ever simple.

And I think even V7 make supported what you described, as well as implicit
rules for compiling .c into a .o or into a binary.


> It's been a while but I feel like I ran through and tried this on V7,
> System III, and PDP-11 System V and all of them were unhappy about that
> construct.  I can try and get on the LCMs 3B400 later to see what SVR3
> does.  I don't remember which of the BSDs (if not multiple) I ran into that
> issue on initially, but I can't imagine one of the major streams would work
> that in without the other two wanting to copy their notes.
>

They'd likely be happier if you used {} instead of () for variable
expansion.


> Maybe an alternative question is if folks are aware of make
> implementations besides GNU that *do* support that sort of thing.
>

The NetBSD/FreeBSD pmake does, and has since before NetBSD/FreeBSD were a
thing (at least to 4.2BSD, and I think even further back since I'm nearly
positive V7 supported it, though I've not cranked up a V7 VM to chek).

Warner

[-- Attachment #2: Type: text/html, Size: 4146 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Rich Salz]

[-- Attachment #1: Type: text/plain, Size: 177 bytes --]

> [Python is] meant for mainly functional programming as I understand it

Not true. It has some neat functional features (list comprehensions) but
that's not really its intent.

[-- Attachment #2: Type: text/html, Size: 233 bytes --]

[TUHS] Re: python

[Larry McVoy]

On Wed, Aug 02, 2023 at 07:49:18PM -0400, Rich Salz wrote:
> > [Python is] meant for mainly functional programming as I understand it
> 
> Not true. It has some neat functional features (list comprehensions) but
> that's not really its intent.

I've really tried to like python but any language that doesn't has printf
as builtin is not for me.  Yes, I know about their library printf but it
is weird.  
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: python

[George Michaelson]

Python has limited support for map/reduce patterns, and cannot
implement lazy eval.  It's functional support is no different to any
classical language with an ability to apply a function over data and
you can curry to your hearts content if you can define functions over
functions (as arguments)

Guido is on record as saying he didn't aim it as an FP language. I
wouldn't cast him an FP hater, I think thats a silly concept anyway.

I tried to learn a real FP language. It's hard to think in new ways. I
wish I'd started sooner because I do believe the stories about the
upside of investing in strong types (which are not unique to FP but
are a mainstay of most of the FP which have succeeded in breaking
through)

The resurgence of chatter about Common LISP says something to me. I'm
not sure what, possibly "all those LISP-like turn out not to be as
good as they said" but there's also a resurgence in OCAML in the same
places.

I think the fusion of UNIX and FP was sort of a road not taken. Yes,
there's lots of LISP and GHC. No, it's really an ecosystem "on top"
and there are some grumpy edges to coding in FP to deploy on UNIX.

Maybe it's no worse a fit than "containers" and after all, we're in
Kubernetes because of Plan9, right?

G

[TUHS] Re: python

[Bakul Shah]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

On Aug 2, 2023, at 6:20 PM, George Michaelson <ggm@algebras.org> wrote:
> 
> Maybe it's no worse a fit than "containers" and after all, we're in
> Kubernetes because of Plan9, right?

Wait, what? How did Plan9 beget Kubernetes? plan9 namespaces are wonderful, while kubernetes.... 

But this made me do some googling and I found "K9P: Kubernetes as 9P files" YT video.

[-- Attachment #2: Type: text/html, Size: 2117 bytes --]

[TUHS] Re: python

[segaloco via TUHS]

> The resurgence of chatter about Common LISP says something to me.
> 
> G

LISP is one of those things I always say I'm going to go study more, try and understand the implications of, but it continues to be so far afield from the typical realms I work in that I just haven't found that catch point to get me more interested in it.  That said, there is a little voice in that same part of my brain whispering about exploring FP for creating software polygon rendering stuff, which right now I only really work with through the OpenGL and video game console lenses.  Probably the closest I got to something LISPy was futzing with some Guile stuff pertaining to GDB.

Also pardon the whole "Python's a functional language I am very smart about things", that's based on plenty of commentary I've heard and read, not any sort of formal analysis or research.  Feel free to call me out anywhere you think I might be parroting (inaccurate) common sentiment about things I have tangential understanding of, I try not to but sometimes it just still crops up in casual chit chat :P

To attempt a tie-back to TUHS, has anyone here ever tried to bring up bits of Python on a historic UNIX implementation?

- Matt G.

[TUHS] Re: python

[George Michaelson]

Tail recursion not lazy eval.

I wish words meant what I meant "inside" when I think them, not
"outside" what they mean when I write them.

[TUHS] Re: python

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On Wed, Aug 2, 2023, 9:24 PM George Michaelson <ggm@algebras.org> wrote:

> Tail recursion not lazy eval.
>
> I wish words meant what I meant "inside" when I think them, not
> "outside" what they mean when I write them.
>

Python is an ugly ass language to be sure. It's a useful tool. I have this
rather ugly axe. I've split a lot of wood with it. Chopped down nasty
weeds. Pounded in nails, spikes, landscaping staples, and the odd screw or
two. I've cut impromptu joints to make lean too structures that I threw
painter's tarps over to save the tomatoes from frost. And maybe a dozen
other things. But to do framing or naked furniture, I use different tools
more suited for those specific tasks... but the axe still has its uses...
much like python...

Warner

>

[-- Attachment #2: Type: text/html, Size: 1345 bytes --]

[TUHS] Re: python

[Bakul Shah]

python can certainly implement tail call optimization (TCO). Pretty much any language can implement TCO but for some reason people think such programs are harder to debug (and yet they don't similarly complain about loops!). The beauty of Scheme was that it *mandated* tail recursion. 

> On Aug 2, 2023, at 8:24 PM, George Michaelson <ggm@algebras.org> wrote:
> 
> Tail recursion not lazy eval.
> 
> I wish words meant what I meant "inside" when I think them, not
> "outside" what they mean when I write them.

[TUHS] Re: python

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

I once inherited maintenance of a critical piece of infrastructure written
in exquisitely well written, tested, and documented Python. I mean it, it
was really really good.

It crashed about once a week and I had to fix it over and over because in
those exponentially vast combinations of paths through the code would arise
yet another way to turn a string into a list, or something analogous. It
was hell.

Critical code needs static typing.

-rob


On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org> wrote:

> python can certainly implement tail call optimization (TCO). Pretty much
> any language can implement TCO but for some reason people think such
> programs are harder to debug (and yet they don't similarly complain about
> loops!). The beauty of Scheme was that it *mandated* tail recursion.
>
> > On Aug 2, 2023, at 8:24 PM, George Michaelson <ggm@algebras.org> wrote:
> >
> > Tail recursion not lazy eval.
> >
> > I wish words meant what I meant "inside" when I think them, not
> > "outside" what they mean when I write them.
>
>

[-- Attachment #2: Type: text/html, Size: 2032 bytes --]

[TUHS] Re: python

[Bakul Shah]

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]

I have not heard such horror stories about Common Lisp (or may be I have forgotten them!). My impression is that python doesn't quite have the kind of {meta,}programming tools Common Lisp has. CL has been used for large critical programs. Perhaps Von Rossum had more experience with statically typed languages than Lisp (because -- pure speculation here -- if he had used CL enough, he would never have designed python :-)

> On Aug 3, 2023, at 1:32 AM, Rob Pike <robpike@gmail.com> wrote:
> 
> I once inherited maintenance of a critical piece of infrastructure written in exquisitely well written, tested, and documented Python. I mean it, it was really really good.
> 
> It crashed about once a week and I had to fix it over and over because in those exponentially vast combinations of paths through the code would arise yet another way to turn a string into a list, or something analogous. It was hell.
> 
> Critical code needs static typing.
> 
> -rob
> 
> 
> On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org <mailto:bakul@iitbombay.org>> wrote:
>> python can certainly implement tail call optimization (TCO). Pretty much any language can implement TCO but for some reason people think such programs are harder to debug (and yet they don't similarly complain about loops!). The beauty of Scheme was that it *mandated* tail recursion. 
>> 
>> > On Aug 2, 2023, at 8:24 PM, George Michaelson <ggm@algebras.org <mailto:ggm@algebras.org>> wrote:
>> > 
>> > Tail recursion not lazy eval.
>> > 
>> > I wish words meant what I meant "inside" when I think them, not
>> > "outside" what they mean when I write them.
>> 


[-- Attachment #2: Type: text/html, Size: 2835 bytes --]

[TUHS] Re: python

[Dan Halbert]

[-- Attachment #1: Type: text/plain, Size: 2416 bytes --]

Python has optional type annotations. There are batch tools (e.g., MyPy) 
to do type analysis and IDE's also provide help. Example:

def greeting(name: str) -> str:
     return 'Hello ' + name

I found Python to be an enormous improvement over Perl for writing the 
kinds of things I used to write in Perl, with the Perl book at my side. 
I currently make my living working on Python for microcontrollers. 
Neverthless, I am fond of type checking too, and if I were writing a 
large Python system, I would use type annotations.

I have used BCPL too, in the 70's, and we achieved some measure of type 
safety by careful naming.

Dan H.

On 8/3/23 10:19, Bakul Shah wrote:
> I have not heard such horror stories about Common Lisp (or may be I 
> have forgotten them!). My impression is that python doesn't quite have 
> the kind of {meta,}programming tools Common Lisp has. CL has been used 
> for large critical programs. Perhaps Von Rossum had more experience 
> with statically typed languages than Lisp (because -- pure speculation 
> here -- if he had used CL enough, he would never have designed python :-)
>
>> On Aug 3, 2023, at 1:32 AM, Rob Pike <robpike@gmail.com> wrote:
>>
>> I once inherited maintenance of a critical piece of infrastructure 
>> written in exquisitely well written, tested, and documented Python. I 
>> mean it, it was really really good.
>>
>> It crashed about once a week and I had to fix it over and over 
>> because in those exponentially vast combinations of paths through the 
>> code would arise yet another way to turn a string into a list, or 
>> something analogous. It was hell.
>>
>> Critical code needs static typing.
>>
>> -rob
>>
>>
>> On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org> wrote:
>>
>>     python can certainly implement tail call optimization (TCO).
>>     Pretty much any language can implement TCO but for some reason
>>     people think such programs are harder to debug (and yet they
>>     don't similarly complain about loops!). The beauty of Scheme was
>>     that it *mandated* tail recursion.
>>
>>     > On Aug 2, 2023, at 8:24 PM, George Michaelson
>>     <ggm@algebras.org> wrote:
>>     >
>>     > Tail recursion not lazy eval.
>>     >
>>     > I wish words meant what I meant "inside" when I think them, not
>>     > "outside" what they mean when I write them.
>>
>

[-- Attachment #2: Type: text/html, Size: 5369 bytes --]

[TUHS] Re: python

[will.senn]

[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]

Nice. I've never appreciated type checking at 'compile' time, but I understand why others might (ocd). C was my first exposure to blending types, then Perl was fuzzier, then Python was even better at pretending types didn't matter. Now, with Lisp, I am freed from type concerns... until I'm not. Thankfully, it's my choice.

Will

On August 3, 2023 9:56:22 AM CDT, Dan Halbert <halbert@halwitz.org> wrote:
>Python has optional type annotations. There are batch tools (e.g., MyPy) to do type analysis and IDE's also provide help. Example:
>
>def greeting(name: str) -> str:
>    return 'Hello ' + name
>
>I found Python to be an enormous improvement over Perl for writing the kinds of things I used to write in Perl, with the Perl book at my side. I currently make my living working on Python for microcontrollers. Neverthless, I am fond of type checking too, and if I were writing a large Python system, I would use type annotations.
>
>I have used BCPL too, in the 70's, and we achieved some measure of type safety by careful naming.
>
>Dan H.
>
>On 8/3/23 10:19, Bakul Shah wrote:
>> I have not heard such horror stories about Common Lisp (or may be I have forgotten them!). My impression is that python doesn't quite have the kind of {meta,}programming tools Common Lisp has. CL has been used for large critical programs. Perhaps Von Rossum had more experience with statically typed languages than Lisp (because -- pure speculation here -- if he had used CL enough, he would never have designed python :-)
>> 
>>> On Aug 3, 2023, at 1:32 AM, Rob Pike <robpike@gmail.com> wrote:
>>> 
>>> I once inherited maintenance of a critical piece of infrastructure written in exquisitely well written, tested, and documented Python. I mean it, it was really really good.
>>> 
>>> It crashed about once a week and I had to fix it over and over because in those exponentially vast combinations of paths through the code would arise yet another way to turn a string into a list, or something analogous. It was hell.
>>> 
>>> Critical code needs static typing.
>>> 
>>> -rob
>>> 
>>> 
>>> On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org> wrote:
>>> 
>>>     python can certainly implement tail call optimization (TCO).
>>>     Pretty much any language can implement TCO but for some reason
>>>     people think such programs are harder to debug (and yet they
>>>     don't similarly complain about loops!). The beauty of Scheme was
>>>     that it *mandated* tail recursion.
>>> 
>>>     > On Aug 2, 2023, at 8:24 PM, George Michaelson
>>>     <ggm@algebras.org> wrote:
>>>     >
>>>     > Tail recursion not lazy eval.
>>>     >
>>>     > I wish words meant what I meant "inside" when I think them, not
>>>     > "outside" what they mean when I write them.
>>> 
>> 

-- Sent from /e/OS Mail.

[-- Attachment #2: Type: text/html, Size: 5847 bytes --]

[TUHS] Re: python

[Dan Cross]

On Thu, Aug 3, 2023 at 11:21 AM will.senn@gmail.com <will.senn@gmail.com> wrote:
> Nice. I've never appreciated type checking at 'compile' time, but I understand why others might (ocd). C was my first exposure to blending types, then Perl was fuzzier, then Python was even better at pretending types didn't matter. Now, with Lisp, I am freed from type concerns... until I'm not. Thankfully, it's my choice.

Types in programming languages vary across two axes: strong versus
weak, and static versus dynamic. Common Lisp is strongly typed: for
example, you cannot add an integer to a list, or `cons` something onto
a vector (Common Lisp vectors are not lists).

Indeed, exactly the former caused a production outage in a large Lisp
system I worked on for about a year: someone had needed to store a
pair of integers, so they used a CONS cell; after a while, the pair
needed to be expanded to a triple, so someone converted the single
CONS cell into a (proper) list. Consequently, the function for
accessing the second value went from being CDR to CADR (or `SECOND`),
but the programmer missed one place: the value of `(cdr foo)`, now a
list, was passed to some function that expected a fixnum and tried to
add something to it: this, of course, ran afoul of the type system and
raised a condition, which resulted as an ISE in prod. The fix was
trivial (change CDR to SECOND in the right place) but it really struck
me that if the system were statically typed, this would have been
trivially discovered at compile-time.

On the other axis, Lisps are usually dynamically typed, which in this
context, means that the type of a value associated with a symbol may
change over time and one matters when the value is actually used.
Common Lisp does allow you to declare types in some limited regards;
these are usually hints to the compiler for code generation.
Conversely, in statically typed languages, the type of every value is
known at all times (particularly at compile time).

Incidentally, this episode --- along with something similar in a
Python program --- really soured me on dynamically-typed languages.
The python failure was particularly maddening: it was heavily unit
tested (100% coverage) but as soon as we put it out, we immediately
got an error report: a type error with processing the arguments to
`main` (Google had some non-standard internal libraries for that that
were challenging to test). This was highly frustrating: like Rob, I
greatly prefer strong, static typing.

Incidentally, C is weakly (you can add a pointer to an integer: the
result is another pointer), but statically typed.

        - Dan C.

> On August 3, 2023 9:56:22 AM CDT, Dan Halbert <halbert@halwitz.org> wrote:
>>
>> Python has optional type annotations. There are batch tools (e.g., MyPy) to do type analysis and IDE's also provide help. Example:
>>
>> def greeting(name: str) -> str:
>>     return 'Hello ' + name
>>
>> I found Python to be an enormous improvement over Perl for writing the kinds of things I used to write in Perl, with the Perl book at my side. I currently make my living working on Python for microcontrollers. Neverthless, I am fond of type checking too, and if I were writing a large Python system, I would use type annotations.
>>
>> I have used BCPL too, in the 70's, and we achieved some measure of type safety by careful naming.
>>
>> Dan H.
>>
>> On 8/3/23 10:19, Bakul Shah wrote:
>>
>> I have not heard such horror stories about Common Lisp (or may be I have forgotten them!). My impression is that python doesn't quite have the kind of {meta,}programming tools Common Lisp has. CL has been used for large critical programs. Perhaps Von Rossum had more experience with statically typed languages than Lisp (because -- pure speculation here -- if he had used CL enough, he would never have designed python :-)
>>
>> On Aug 3, 2023, at 1:32 AM, Rob Pike <robpike@gmail.com> wrote:
>>
>> I once inherited maintenance of a critical piece of infrastructure written in exquisitely well written, tested, and documented Python. I mean it, it was really really good.
>>
>> It crashed about once a week and I had to fix it over and over because in those exponentially vast combinations of paths through the code would arise yet another way to turn a string into a list, or something analogous. It was hell.
>>
>> Critical code needs static typing.
>>
>> -rob
>>
>>
>> On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org> wrote:
>>>
>>> python can certainly implement tail call optimization (TCO). Pretty much any language can implement TCO but for some reason people think such programs are harder to debug (and yet they don't similarly complain about loops!). The beauty of Scheme was that it *mandated* tail recursion.
>>>
>>> > On Aug 2, 2023, at 8:24 PM, George Michaelson <ggm@algebras.org> wrote:
>>> >
>>> > Tail recursion not lazy eval.
>>> >
>>> > I wish words meant what I meant "inside" when I think them, not
>>> > "outside" what they mean when I write them.
>>>
>>
>>
> -- Sent from /e/OS Mail.

[TUHS] Re: python

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]

On Thu, Aug 3, 2023 at 6:06 PM Dan Cross <crossd@gmail.com> wrote:

> On Thu, Aug 3, 2023 at 11:21 AM will.senn@gmail.com <will.senn@gmail.com>
> wrote:
>


> someone had needed to store a
> pair of integers, so they used a CONS cell;


Of course, that was a bad idea.  The pair of integers should have been a
struct or a class named after whatever its application-level purpose was: a
point, for example.

after a while, the pair
> needed to be expanded to a triple, so someone converted the single
> CONS cell into a (proper) list.


In that case, a derived struct or class should have been created.  The two
classes would use the same accessor methods, just as in C++.

this, of course, ran afoul of the type system and
> raised a condition, which resulted as an ISE in prod. The fix was
> trivial (change CDR to SECOND in the right place) but it really struck
> me that if the system were statically typed, this would have been
> trivially discovered at compile-time.
>

Absolutely, and if the failure was intolerable, CL's static type
declarations would have caught the use of the wrong type.  But you don't
have to declare *everything*.  For that matter, there is nothing in a fully
statically typed system that requires every variable, function, argument,
etc. to be *declared*: type inference is powerful.

Common Lisp does allow you to declare types in some limited regards;
> these are usually hints to the compiler for code generation.
>

They may or may not be, depending on how you set the OPTIMIZE declaration.


> like Rob, I
> greatly prefer strong, static typing.
>

Then why weren't you using mypy?


> Incidentally, C is weakly (you can add a pointer to an integer: the
> result is another pointer), but statically typed.
>

That's not weak typing, it's operator overloading, just as when you add an
int to a double.  C will not let you, e.g., add an int to a function.  Weak
typing is quite rare in high-level languages: PL/I pointer variables are
weakly typed (that is, when you allocate an object you specify the type of
the object and then assign it to the pointer variable), but the rest of the
language is strongly typed.

[-- Attachment #2: Type: text/html, Size: 4681 bytes --]

[TUHS] Re: python

[Dan Cross]

On Thu, Aug 3, 2023 at 8:24 PM John Cowan <cowan@ccil.org> wrote:
> On Thu, Aug 3, 2023 at 6:06 PM Dan Cross <crossd@gmail.com> wrote:
>> someone had needed to store a
>> pair of integers, so they used a CONS cell;
>
> Of course, that was a bad idea.  The pair of integers should have been a struct or a class named after whatever its application-level purpose was: a point, for example.

Probably, but I didn't write the original code, or even make the
change; I did submit the fix, however. :-)

An issue with CLs aggregation primitives is that they can be
expensive, however. QPX was weird; despite things that have been said
about it publicly (http://www.paulgraham.com/carl.html) QPX was not a
"typical" Lisp program: it is more like FORTRAN written in
S-expressions (many thousand-line Lisp functions with jumps between
various points inside of them are common; those functions like, say,
setq [not setf] a boolean on line 1387, and then reference it on
3298...kind of a mess, but that's what you need to make Lisp fast).

>> after a while, the pair
>> needed to be expanded to a triple, so someone converted the single
>> CONS cell into a (proper) list.
>
> In that case, a derived struct or class should have been created.  The two classes would use the same accessor methods, just as in C++.

Of course, we all knew this. But when you've got an O(10^6) line
codebase that's extremely fragile, technology and business limitations
demand tradeoffs that are not always what the engineers would choose.

>> this, of course, ran afoul of the type system and
>> raised a condition, which resulted as an ISE in prod. The fix was
>> trivial (change CDR to SECOND in the right place) but it really struck
>> me that if the system were statically typed, this would have been
>> trivially discovered at compile-time.
>
> Absolutely, and if the failure was intolerable, CL's static type declarations would have caught the use of the wrong type.  But you don't have to declare *everything*.  For that matter, there is nothing in a fully statically typed system that requires every variable, function, argument, etc. to be *declared*: type inference is powerful.

Not really, unless they were used consistently across the entire
codebase (and sadly, they were not).

>> Common Lisp does allow you to declare types in some limited regards;
>> these are usually hints to the compiler for code generation.
>
> They may or may not be, depending on how you set the OPTIMIZE declaration.
>
>> like Rob, I
>> greatly prefer strong, static typing.
>
> Then why weren't you using mypy?

Because it didn't exist at the time (the early 2010s), or if it did,
it was in a very nascent state.

>> Incidentally, C is weakly (you can add a pointer to an integer: the
>> result is another pointer), but statically typed.
>
> That's not weak typing, it's operator overloading, just as when you add an int to a double. C will not let you, e.g., add an int to a function.

I think there's a bit of a debate as to whether C is weakly or
strongly typed (and certainly, these exist on a spectrum), and some
good judges say it's "moderately typed". I've never heard anyone refer
to implicit type conversions as "operator overloading", however.

> Weak typing is quite rare in high-level languages: PL/I pointer variables are weakly typed (that is, when you allocate an object you specify the type of the object and then assign it to the pointer variable), but the rest of the language is strongly typed.

Once `memcpy` and `void *` are in the mix, all bets are off in C.

        - Dan C.

[TUHS] Re: python

[Bakul Shah]

Sorry for beating a dead horse but...

What I was getting at when I invoked lisp is because I was thinking
of the ability to debug live processes as opposed analyzing dead
programs (core dumps). With lisps you can not only access a lot of
the dynamic state of the program at the language level but (in theory)
you may also be able to fix the problem and let the process continue.
Especially when a program crashes rather infrequently, having access
to only a core dump & stack trace can be quite frustrating. Why do we
continue to live in a virtual "batch processing" world when it comes
to dealing with such problems? [I once spent a month analyzing a problem
a customer reported. I had guessed what the problem might be but couldn't
reproduce it. In their live setup, it took 15 minutes to catch the bug
with a logic analyzer!]

The other thing both you and Rob pointed out is that even "well-tested"
programs can crash. While you can use more type annotations even in
dynamic languages to catch type errors, types are not sufficiently
expressive so you always have that potential problem (and we can't 
*prove* large programs to be bug free). It is of course better to do
all the testing you can but that typically occurs in "lab conditions",
not in the real world. That is why many large services can run a small
subset of servers with newer version of s/w as a "canary" to catch
errors early (but that is usually a binary decision -- if the canary
dies, you don't do the software update). With a dynamic language you
can add situation specific tests on the fly if the program misbehaves.

Live debugging should be even easier now -- with the right setup one
should be attach a bot to do some initial checking etc. Instead we
have apps that use 100s of MB to GBs of space but die silently. Guess
we all suffer from Stockholm Syndrome!


> On Aug 3, 2023, at 3:05 PM, Dan Cross <crossd@gmail.com> wrote:
> 
> On Thu, Aug 3, 2023 at 11:21 AM will.senn@gmail.com <will.senn@gmail.com> wrote:
>> Nice. I've never appreciated type checking at 'compile' time, but I understand why others might (ocd). C was my first exposure to blending types, then Perl was fuzzier, then Python was even better at pretending types didn't matter. Now, with Lisp, I am freed from type concerns... until I'm not. Thankfully, it's my choice.
> 
> Types in programming languages vary across two axes: strong versus
> weak, and static versus dynamic. Common Lisp is strongly typed: for
> example, you cannot add an integer to a list, or `cons` something onto
> a vector (Common Lisp vectors are not lists).
> 
> Indeed, exactly the former caused a production outage in a large Lisp
> system I worked on for about a year: someone had needed to store a
> pair of integers, so they used a CONS cell; after a while, the pair
> needed to be expanded to a triple, so someone converted the single
> CONS cell into a (proper) list. Consequently, the function for
> accessing the second value went from being CDR to CADR (or `SECOND`),
> but the programmer missed one place: the value of `(cdr foo)`, now a
> list, was passed to some function that expected a fixnum and tried to
> add something to it: this, of course, ran afoul of the type system and
> raised a condition, which resulted as an ISE in prod. The fix was
> trivial (change CDR to SECOND in the right place) but it really struck
> me that if the system were statically typed, this would have been
> trivially discovered at compile-time.
> 
> On the other axis, Lisps are usually dynamically typed, which in this
> context, means that the type of a value associated with a symbol may
> change over time and one matters when the value is actually used.
> Common Lisp does allow you to declare types in some limited regards;
> these are usually hints to the compiler for code generation.
> Conversely, in statically typed languages, the type of every value is
> known at all times (particularly at compile time).
> 
> Incidentally, this episode --- along with something similar in a
> Python program --- really soured me on dynamically-typed languages.
> The python failure was particularly maddening: it was heavily unit
> tested (100% coverage) but as soon as we put it out, we immediately
> got an error report: a type error with processing the arguments to
> `main` (Google had some non-standard internal libraries for that that
> were challenging to test). This was highly frustrating: like Rob, I
> greatly prefer strong, static typing.
> 
> Incidentally, C is weakly (you can add a pointer to an integer: the
> result is another pointer), but statically typed.
> 
>       - Dan C.
> 
>> On August 3, 2023 9:56:22 AM CDT, Dan Halbert <halbert@halwitz.org> wrote:
>>> 
>>> Python has optional type annotations. There are batch tools (e.g., MyPy) to do type analysis and IDE's also provide help. Example:
>>> 
>>> def greeting(name: str) -> str:
>>>   return 'Hello ' + name
>>> 
>>> I found Python to be an enormous improvement over Perl for writing the kinds of things I used to write in Perl, with the Perl book at my side. I currently make my living working on Python for microcontrollers. Neverthless, I am fond of type checking too, and if I were writing a large Python system, I would use type annotations.
>>> 
>>> I have used BCPL too, in the 70's, and we achieved some measure of type safety by careful naming.
>>> 
>>> Dan H.
>>> 
>>> On 8/3/23 10:19, Bakul Shah wrote:
>>> 
>>> I have not heard such horror stories about Common Lisp (or may be I have forgotten them!). My impression is that python doesn't quite have the kind of {meta,}programming tools Common Lisp has. CL has been used for large critical programs. Perhaps Von Rossum had more experience with statically typed languages than Lisp (because -- pure speculation here -- if he had used CL enough, he would never have designed python :-)
>>> 
>>> On Aug 3, 2023, at 1:32 AM, Rob Pike <robpike@gmail.com> wrote:
>>> 
>>> I once inherited maintenance of a critical piece of infrastructure written in exquisitely well written, tested, and documented Python. I mean it, it was really really good.
>>> 
>>> It crashed about once a week and I had to fix it over and over because in those exponentially vast combinations of paths through the code would arise yet another way to turn a string into a list, or something analogous. It was hell.
>>> 
>>> Critical code needs static typing.
>>> 
>>> -rob
>>> 
>>> 
>>> On Thu, Aug 3, 2023 at 1:56 PM Bakul Shah <bakul@iitbombay.org> wrote:
>>>> 
>>>> python can certainly implement tail call optimization (TCO). Pretty much any language can implement TCO but for some reason people think such programs are harder to debug (and yet they don't similarly complain about loops!). The beauty of Scheme was that it *mandated* tail recursion.
>>>> 
>>>>> On Aug 2, 2023, at 8:24 PM, George Michaelson <ggm@algebras.org> wrote:
>>>>> 
>>>>> Tail recursion not lazy eval.
>>>>> 
>>>>> I wish words meant what I meant "inside" when I think them, not
>>>>> "outside" what they mean when I write them.
>>>> 
>>> 
>>> 
>> -- Sent from /e/OS Mail.

[TUHS] Re: python

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

On Thu, Aug 3, 2023 at 10:20 AM Bakul Shah <bakul@iitbombay.org> wrote:

I have not heard such horror stories about Common Lisp (or may be I have
> forgotten them!).
>

Both Common Lisp and Python have optional static typing that is as rigorous
as you might want.  In the case of Python it is an outboard tool that you
put into your build file; Python itself ignores declarations.  (That means
that the declarations are not used to optimize runtime performance.)  The
mypy tool does extensive type inference; you can get a long way without
ever writing a declaration.

For CL, type checking is part of the compiler; the interpreter generally
ignores type declarations, although Steel Bank CL, the fastest existing
implementation, uses the compiler even in the REPL.

[-- Attachment #2: Type: text/html, Size: 1561 bytes --]

[TUHS] Re: python

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

IMO (Like Larry) no printf stinks.  But the real killer for my sustain for
Python is the use white space and being typeless.   My daughter loves it
for her cloud development and we argue a bit.  But it was the first
language she really mastered in college and she never took a competitive
languages course so I’m not so sure really had experienced much beyond it
for real programs.   Maybe I’m just an old fart but between C, Go and Rust
I’m pretty good.  I do write scripts in Bourne shell and or awk truth be
known.

On Wed, Aug 2, 2023 at 8:51 PM Larry McVoy <lm@mcvoy.com> wrote:

> On Wed, Aug 02, 2023 at 07:49:18PM -0400, Rich Salz wrote:
> > > [Python is] meant for mainly functional programming as I understand it
> >
> > Not true. It has some neat functional features (list comprehensions) but
> > that's not really its intent.
>
> I've really tried to like python but any language that doesn't has printf
> as builtin is not for me.  Yes, I know about their library printf but it
> is weird.
> --
> ---
> Larry McVoy           Retired to fishing
> http://www.mcvoy.com/lm/boat
>
-- 
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 1708 bytes --]

[TUHS] Re: python

[Pete Wright via TUHS]

On 8/2/23 19:07, Clem Cole wrote:
> IMO (Like Larry) no printf stinks.  But the real killer for my sustain 
> for Python is the use white space and being typeless.   My daughter 
> loves it for her cloud development and we argue a bit.  But it was the 
> first language she really mastered in college and she never took a 
> competitive languages course so I’m not so sure really had experienced 
> much beyond it for real programs.   Maybe I’m just an old fart but 
> between C, Go and Rust I’m pretty good.  I do write scripts in Bourne 
> shell and or awk truth be known.
> 

as one of those "new kids" who finds python to be a decent enough 
language for my cloud development tasks - my main observation is that 
having the python REPL was an eye opener.

after fighting the learning curve of C and Java the interactivity of the 
whole thing was a revelation.  being able to open a unix shell, run 
python and have a basic webserver running in a few lines of code was 
fantastic for my young hacker mind.  this also probably helps explain 
the popularity of javascript too.

this isn't to say i think python is the end-all of programming, far from 
it, but it certainly took a lot of the novice programmer anxiety out of 
programming for me.

if only i'd encountered erlang during those formative years...

-pete

-- 
Pete Wright
pete@nomadlogic.org
@nomadlogicLA

[TUHS] Re: python

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

On Wed, Aug 2, 2023 at 8:21 PM Pete Wright via TUHS <tuhs@tuhs.org> wrote:

> this isn't to say i think python is the end-all of programming, far from
> it, but it certainly took a lot of the novice programmer anxiety out of
> programming for be.
>

python doesn't suck because batteries are included and what's not included
is pretty easy to just drag in and get craptops of useful stuff done in a
hurry.
Would I write a kernel in? Hell no. But would I knock together a bit of data
visualization after massaging data in an almost throw-a-way manner? You
betcha. It's stunningly adequate... A latter day awk for junk I need to do
from time to time...

Warner

[-- Attachment #2: Type: text/html, Size: 1086 bytes --]

[TUHS] Re: python

[Mike Markowski]

Clem and all,

I find python string format syntax to be close enough to printf. E.g.,

   print('%.4f ns, (%.4f, %.4fj)' % (tap[0], tap[1].real, tap[1].imag))

However, the example highlights a shortcoming.  While complex numbers 
are supported by the language, there is no formatting support like 
'%.5j' ('j' is my made up format char) to directly format a complex number.

I work in an RF lab focused on work with hardware and lab gear. Some 
points in favor of python are (1) lab gear is controlled by SCPI, (2) 
DSP relies on complex math, and (3) RF propagation modeling is 
computationally intense.

Item (1) is easily performed with python, (2) with python or 
Matlab/octave, and (3) is 'it depends.'  An engineer's friend went from 
slide rule, to calculator, fortran/c (fortran for numbers, c for 
hardware), and now python. A laptop with python or matlab is the new 
'calculator.'  As to (3), if you will use the program for large 
scenarios, use c or fortran. For small runs or to dovetail results with 
control of lab gear python fills the bill.  (I even went to the slightly 
insane length of converting a classic prop model from fortran to python 
for that reason: https://udel.edu/~mm/itm/ )

I agree 110% that python white space formatting is horrible.  I can't 
say many times I took someone else's program, made a quick change, to 
discover one of us used tabs and the other spaces.

Mike Markowski

On 8/2/23 10:07 PM, Clem Cole wrote:
> IMO (Like Larry) no printf stinks.  But the real killer for my sustain 
> for Python is the use white space and being typeless.   My daughter 
> loves it for her cloud development and we argue a bit.  But it was the 
> first language she really mastered in college and she never took a 
> competitive languages course so I’m not so sure really had experienced 
> much beyond it for real programs.   Maybe I’m just an old fart but 
> between C, Go and Rust I’m pretty good.  I do write scripts in Bourne 
> shell and or awk truth be known.
>

[TUHS] Re: python

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]

The idea of indentation defining structure seemed really cool when it
arose. I first saw it in a toy language called Henry, back in the early
1980s.

But over time the notion that invisible characters define program execution
created so many problems that in retrospect it is ill advised despite its
prevalence. In fact its prevalence makes it even less advisable, as it
creates yet more areas for trouble.

-rob


On Thu, Aug 3, 2023 at 10:36 PM Mike Markowski <mike.ab3ap@gmail.com> wrote:

> Clem and all,
>
> I find python string format syntax to be close enough to printf. E.g.,
>
>    print('%.4f ns, (%.4f, %.4fj)' % (tap[0], tap[1].real, tap[1].imag))
>
> However, the example highlights a shortcoming.  While complex numbers
> are supported by the language, there is no formatting support like
> '%.5j' ('j' is my made up format char) to directly format a complex number.
>
> I work in an RF lab focused on work with hardware and lab gear. Some
> points in favor of python are (1) lab gear is controlled by SCPI, (2)
> DSP relies on complex math, and (3) RF propagation modeling is
> computationally intense.
>
> Item (1) is easily performed with python, (2) with python or
> Matlab/octave, and (3) is 'it depends.'  An engineer's friend went from
> slide rule, to calculator, fortran/c (fortran for numbers, c for
> hardware), and now python. A laptop with python or matlab is the new
> 'calculator.'  As to (3), if you will use the program for large
> scenarios, use c or fortran. For small runs or to dovetail results with
> control of lab gear python fills the bill.  (I even went to the slightly
> insane length of converting a classic prop model from fortran to python
> for that reason: https://udel.edu/~mm/itm/ )
>
> I agree 110% that python white space formatting is horrible.  I can't
> say many times I took someone else's program, made a quick change, to
> discover one of us used tabs and the other spaces.
>
> Mike Markowski
>
> On 8/2/23 10:07 PM, Clem Cole wrote:
> > IMO (Like Larry) no printf stinks.  But the real killer for my sustain
> > for Python is the use white space and being typeless.   My daughter
> > loves it for her cloud development and we argue a bit.  But it was the
> > first language she really mastered in college and she never took a
> > competitive languages course so I’m not so sure really had experienced
> > much beyond it for real programs.   Maybe I’m just an old fart but
> > between C, Go and Rust I’m pretty good.  I do write scripts in Bourne
> > shell and or awk truth be known.
> >
>

[-- Attachment #2: Type: text/html, Size: 3576 bytes --]

[TUHS] Re: python

[emanuel stiebler]

On 2023-08-03 09:29, Rob Pike wrote:
> The idea of indentation defining structure seemed really cool when it 
> arose. I first saw it in a toy language called Henry, back in the early 
> 1980s.
> 
> But over time the notion that invisible characters define program 
> execution created so many problems that in retrospect it is ill advised 
> despite its prevalence. In fact its prevalence makes it even less 
> advisable, as it creates yet more areas for trouble.

Most editors have some "beautify modes" which can fix indentation,
so the programmer sees what he is doing in deep nested stuff.

Counting "white spaces" in 21st century?

[TUHS] Re: python

[Steffen Nurpmeso]

emanuel stiebler wrote in
 <99511256-1c0a-732b-4b78-a879a67a2291@e-bbes.com>:
 |On 2023-08-03 09:29, Rob Pike wrote:
 |> The idea of indentation defining structure seemed really cool when it 
 |> arose. I first saw it in a toy language called Henry, back in the early 
 |> 1980s.
 |> 
 |> But over time the notion that invisible characters define program 
 |> execution created so many problems that in retrospect it is ill advised 
 |> despite its prevalence. In fact its prevalence makes it even less 
 |> advisable, as it creates yet more areas for trouble.
 |
 |Most editors have some "beautify modes" which can fix indentation,
 |so the programmer sees what he is doing in deep nested stuff.
 |
 |Counting "white spaces" in 21st century?

Normally a newer C compiler beats unto you if your indentation
does not satisfy him?  (Or was that clang only?  That one maybe
learns to cook your favorite hot beverage if its size exceeds the
second gigabyte.)

 --End of <99511256-1c0a-732b-4b78-a879a67a2291@e-bbes.com>

Ciao.

P.S.: unfair + without any insider knowledge: it must be like that.

  $ ll /usr/ports/built/tcc#20230731-1.pkg.tar.xz
  -rw-rw---- 1 ports ports 250632 Jul 31 21:14 /usr/ports/built/tcc#20230731-1.pkg.tar.xz
  $ xz -l /usr/ports/built/tcc#20230731-1.pkg.tar.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1       1    244.8 KiB   1180.5 KiB  0.207  CRC64   /usr/ports/built/tcc#20230731-1.pkg.tar.xz


No pcc here at the moment :(


  $ ll /usr/ports/built/gcc#12.3.0-2.pkg.tar.xz
  -rw-rw---- 1 ports ports 50662716 May 27 20:45 /usr/ports/built/gcc#12.3.0-2.pkg.tar.xz
  $ xz -l /usr/ports/built/gcc#12.3.0-2.pkg.tar.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1       9     48.3 MiB    206.5 MiB  0.234  CRC64   /usr/ports/built/gcc#12.3.0-2.pkg.tar.xz
  $ prt-get info gcc|grep Depend
  Dependencies: libmpc,zlib,zstd


  $ ll /usr/ports/built/clang#16.0.6-1.pkg.tar.xz
  -rw-rw---- 1 ports ports 103260340 Jun 23 22:19 /usr/ports/built/clang#16.0.6-1.pkg.tar.xz
  $ xz -l /usr/ports/built/clang#16.0.6-1.pkg.tar.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1      42     98.5 MiB    991.7 MiB  0.099  CRC64   /usr/ports/built/clang#16.0.6-1.pkg.tar.xz

  $ prt-get info clang|grep Depend
  Dependencies: compiler-rt

  $ ll /usr/ports/built/compiler-rt#16.0.6-1.pkg.tar.xz
  -rw-rw---- 1 ports ports 3633680 Jun 23 21:04 /usr/ports/built/compiler-rt#16.0.6-1.pkg.tar.xz
  $ xz -l /usr/ports/built/compiler-rt#16.0.6-1.pkg.tar.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1       2   3548.5 KiB     36.3 MiB  0.096  CRC64   /usr/ports/built/compiler-rt#16.0.6-1.pkg.tar.xz

  $ prt-get info compiler-rt|grep Depend
  Dependencies: llvm

  $ ll /usr/ports/built/llvm#16.0.6-1.pkg.tar.xz
  -rw-rw---- 1 ports ports 115485456 Jun 23 21:00 /usr/ports/built/llvm#16.0.6-1.pkg.tar.xz
  $ xz -l /usr/ports/built/llvm#16.0.6-1.pkg.tar.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1      29    110.1 MiB    678.9 MiB  0.162  CRC64   /usr/ports/built/llvm#16.0.6-1.pkg.tar.xz

  $ prt-get info llvm|grep Depend
  Dependencies: cmake,libffi,libxml2,ninja,python3-setuptools

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

[TUHS] Re: python

[Larry McVoy]

I agree with Rob.  I get why Guido did it, he wanted some sane style.  Any 
of us who were kernel engineers and saw a sane C style, then you see GNU
C style and you recoil in horror.  I don't know if that was his path but
I get that you would like some sanity in the formatting.

That said, as a person who thinks of himself as a professional, when I go
in to someone else's code, I adopt their style.  It's really rude to not
do so.  I've written code in GNU C style.

On Thu, Aug 03, 2023 at 11:29:18PM +1000, Rob Pike wrote:
> The idea of indentation defining structure seemed really cool when it
> arose. I first saw it in a toy language called Henry, back in the early
> 1980s.
> 
> But over time the notion that invisible characters define program execution
> created so many problems that in retrospect it is ill advised despite its
> prevalence. In fact its prevalence makes it even less advisable, as it
> creates yet more areas for trouble.
> 
> -rob
> 
> 
> On Thu, Aug 3, 2023 at 10:36???PM Mike Markowski <mike.ab3ap@gmail.com> wrote:
> 
> > Clem and all,
> >
> > I find python string format syntax to be close enough to printf. E.g.,
> >
> >    print('%.4f ns, (%.4f, %.4fj)' % (tap[0], tap[1].real, tap[1].imag))
> >
> > However, the example highlights a shortcoming.  While complex numbers
> > are supported by the language, there is no formatting support like
> > '%.5j' ('j' is my made up format char) to directly format a complex number.
> >
> > I work in an RF lab focused on work with hardware and lab gear. Some
> > points in favor of python are (1) lab gear is controlled by SCPI, (2)
> > DSP relies on complex math, and (3) RF propagation modeling is
> > computationally intense.
> >
> > Item (1) is easily performed with python, (2) with python or
> > Matlab/octave, and (3) is 'it depends.'  An engineer's friend went from
> > slide rule, to calculator, fortran/c (fortran for numbers, c for
> > hardware), and now python. A laptop with python or matlab is the new
> > 'calculator.'  As to (3), if you will use the program for large
> > scenarios, use c or fortran. For small runs or to dovetail results with
> > control of lab gear python fills the bill.  (I even went to the slightly
> > insane length of converting a classic prop model from fortran to python
> > for that reason: https://udel.edu/~mm/itm/ )
> >
> > I agree 110% that python white space formatting is horrible.  I can't
> > say many times I took someone else's program, made a quick change, to
> > discover one of us used tabs and the other spaces.
> >
> > Mike Markowski
> >
> > On 8/2/23 10:07 PM, Clem Cole wrote:
> > > IMO (Like Larry) no printf stinks.  But the real killer for my sustain
> > > for Python is the use white space and being typeless.   My daughter
> > > loves it for her cloud development and we argue a bit.  But it was the
> > > first language she really mastered in college and she never took a
> > > competitive languages course so I???m not so sure really had experienced
> > > much beyond it for real programs.   Maybe I???m just an old fart but
> > > between C, Go and Rust I???m pretty good.  I do write scripts in Bourne
> > > shell and or awk truth be known.
> > >
> >

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: python

[segaloco via TUHS]

> That said, as a person who thinks of himself as a professional, when I go
> in to someone else's code, I adopt their style. It's really rude to not
> do so. I've written code in GNU C style.
> 
> Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat

I've adopted a variation on this in that I try and write additions to existing code stylistically similar to what is there, anything presenting glue logic to some sort of external vendor library in a way resembling their style, and then anything else is mine.

That middle one I've found particularly helpful even for myself over the years as there are plenty of places in the dayjob codebases I can drop in and tell almost immediately "Oh this is a wrapper over so and so based on the variable names" or "Yeah this is an interface to library <xyz> based on the way the operations are named."

Generally the only thing I have a hard time sticking to is casing, I'm a fervent snake caser in my assembly and C code, but then fervent pascal caser in my JavaScript and C#.  Then again, that may also tie into my middle practice in that those are the common cases seen in model examples of those languages.  One of the weirder side effects of that stylistic practice is the rare occasion where I blindly copy something between languages with relatively similar syntax (C to C# or JavaScript for instance) I can tell going back later because there's a hunk of code with snake case smack in the middle of a bunch of pascal case.  I usually go and clean that up though because otherwise the codebase starts to look like a copypaste job from StackOverflow after a while, that stuff drives me up the wall.

- Matt G.

P.S. For TUHS subject appropriateness, I have TUHS to thank for my C style practices.  I learned from KnR 2nd Edition back when I was a kid, but diverged a bit from the typical KnR way of things for a while, when I caught wind of TUHS and started pouring over all the code, unbeknownst to my own consciousness I started absorbing stylistic patterns from UNIX sources.  I'm thankful to Warren and all the others who have facilitated this community, I think TUHS should be in any programmer's bookmark list :)

[TUHS] Re: python

[Adam Thornton]

[-- Attachment #1: Type: text/plain, Size: 2871 bytes --]

What we've done on my current project is pretty much equivalent to the
route Go chose.

Go has go fmt; doesn't matter what you personally believe, just run that
pre-commit, and you get a consistent style.  For Python we use black.  Same
idea.  It's not what everyone would have chosen--in fact, precisely what it
does is not what *anyone* on the project, probably, would have chosen--but
the fact is, it does something sane and pretty readable, and then there's
no fighting over style.

Adam

On Thu, Aug 3, 2023 at 6:28 PM segaloco via TUHS <tuhs@tuhs.org> wrote:

> > That said, as a person who thinks of himself as a professional, when I go
> > in to someone else's code, I adopt their style. It's really rude to not
> > do so. I've written code in GNU C style.
> >
> > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
>
> I've adopted a variation on this in that I try and write additions to
> existing code stylistically similar to what is there, anything presenting
> glue logic to some sort of external vendor library in a way resembling
> their style, and then anything else is mine.
>
> That middle one I've found particularly helpful even for myself over the
> years as there are plenty of places in the dayjob codebases I can drop in
> and tell almost immediately "Oh this is a wrapper over so and so based on
> the variable names" or "Yeah this is an interface to library <xyz> based on
> the way the operations are named."
>
> Generally the only thing I have a hard time sticking to is casing, I'm a
> fervent snake caser in my assembly and C code, but then fervent pascal
> caser in my JavaScript and C#.  Then again, that may also tie into my
> middle practice in that those are the common cases seen in model examples
> of those languages.  One of the weirder side effects of that stylistic
> practice is the rare occasion where I blindly copy something between
> languages with relatively similar syntax (C to C# or JavaScript for
> instance) I can tell going back later because there's a hunk of code with
> snake case smack in the middle of a bunch of pascal case.  I usually go and
> clean that up though because otherwise the codebase starts to look like a
> copypaste job from StackOverflow after a while, that stuff drives me up the
> wall.
>
> - Matt G.
>
> P.S. For TUHS subject appropriateness, I have TUHS to thank for my C style
> practices.  I learned from KnR 2nd Edition back when I was a kid, but
> diverged a bit from the typical KnR way of things for a while, when I
> caught wind of TUHS and started pouring over all the code, unbeknownst to
> my own consciousness I started absorbing stylistic patterns from UNIX
> sources.  I'm thankful to Warren and all the others who have facilitated
> this community, I think TUHS should be in any programmer's bookmark list :)
>

[-- Attachment #2: Type: text/html, Size: 3331 bytes --]

[TUHS] Re: python

[Dan Cross]

On Thu, Aug 3, 2023 at 9:58 PM Adam Thornton <athornton@gmail.com> wrote:
> What we've done on my current project is pretty much equivalent to the route Go chose.
>
> Go has go fmt; doesn't matter what you personally believe, just run that pre-commit, and you get a consistent style.  For Python we use black.  Same idea.  It's not what everyone would have chosen--in fact, precisely what it does is not what *anyone* on the project, probably, would have chosen--but the fact is, it does something sane and pretty readable, and then there's no fighting over style.

This.

I despised Google's C++ style standards, but I respected them because
they allowed Google to scale to hundreds of millions of lines of C++
code that was at least intelligible to more or less anyone who worked
there. After a couple of months, people stop noticing the sharp edges
and differences from their personal styles.

The decades spent arguing over where to put the braces seem wasted, in
retrospect.

        - Dan C.

[TUHS] Re: python

[Larry McVoy]

On Fri, Aug 04, 2023 at 11:04:16AM -0400, Dan Cross wrote:
> On Thu, Aug 3, 2023 at 9:58???PM Adam Thornton <athornton@gmail.com> wrote:
> > What we've done on my current project is pretty much equivalent to the route Go chose.
> >
> > Go has go fmt; doesn't matter what you personally believe, just run that pre-commit, and you get a consistent style.  For Python we use black.  Same idea.  It's not what everyone would have chosen--in fact, precisely what it does is not what *anyone* on the project, probably, would have chosen--but the fact is, it does something sane and pretty readable, and then there's no fighting over style.
> 
> This.
> 
> I despised Google's C++ style standards, but I respected them because
> they allowed Google to scale to hundreds of millions of lines of C++
> code that was at least intelligible to more or less anyone who worked
> there. After a couple of months, people stop noticing the sharp edges
> and differences from their personal styles.
> 
> The decades spent arguing over where to put the braces seem wasted, in
> retrospect.

I can get used to anything, no argument there.  I'm teaching my kid 
some programming and he is eating up my style because I tell him why
it is like that.  For example, 

int
some_func(int some_arg)
{
}

not 

int some_func(int some_arg)
{
}

because I can do

grep '^some_func(' *.c

and find the declaration.

I "won" the braces argument by starting my own company and people were
pretty happy with the resulting code.

--lm

[TUHS] Re: [TULSA] Re: python

[Phil Budne]

I come not to praise Python...

I picked up Python (v1) in the early 'aughts as a more readable
alternative to Perl and Ruby (non-alphanumerical variable names are a
non-starter for me).

I agree that using indentation alone to determine block structure was
an interesting thought experiment, but TERRIBLE in practice (you might
as well throw away any editing buffer where the phone rang after you
cut and paste, but before re-indenting code), BUT it does prevent (in
a rather fascist way) formatting abominations like the C code in
procmail and SimH.

Python continues to evolve, so it's a moving target, and the argument
that there is ONE RIGHT WAY(TM) to do anything is not only long dead,
but now deeply putrid (much as with the multiple doublings in the size
of C++ and later C++ + STL books).

I used to regard the Python2/Python3 world breakage as a fiasco, but
considering how quickly the language is mutating, perhaps is was a
good fiasco if it lowered the mutation rate for number of years.

On the subject of "no printf", there is not one, not two, but THREE
ways to format strings, easily compounded with print:

	print("%s %s" % ("hello", "world"))
	print("{1} {two}".format("hello", two="world"))
	print(f"{greeting} {populace}")

I'm pretty sure the last method (which initially made me vomit, due to
violating my hardwired ideas about complicating the lexer, as if it
can even be thought of as a separate layer), BUT I Seem To Recall that
it allows a class to implement a formatting method, which may (or may
not) work for complex numbers.

Type "hinting" has been mentioned: again, it's almost like a whole new
version of the language.  You need to pick a type checker, and likely
turn up the knobs to achieve any actual safety, and then deal with the
fact that not all packages supply hints out of the box.  It kind of
offers the desert-topping/floor-wax dichotomy: You can easily/quickly
write small programs with dynamic typing for quick one-offs, AND write
better armored code for libraries & production code.

On tabs vs spaces: Python3 forbids mixing them.  Again, fascist, but
"it's for your own good".

So yes, Python sucks, but I continue using it, and unlike sendmail
(which I continue to run, as I have a 200+ line .mc file it would take
me at LEAST a week to replicate the effects of in another MTA), I
don't tell people Python is unsuitable at any speed.

I'd probably happily write Go or Rust, given a situation (existing
project/program) where its use was inevitable, and I DO have a program
I think is a good candidate for rewriting in Go, but on the whole,
it's a useful tool for many kinds of programmers, and in many
situations, which I think is somewhat remarkable.

[TUHS] Re: [TULSA] Re: python

[Rich Salz]

[-- Attachment #1: Type: text/plain, Size: 117 bytes --]

What, we all need something to kick now that we've beaten sendmail?  How
about something unix, ideally a decade old?

[-- Attachment #2: Type: text/html, Size: 148 bytes --]

[TUHS] Split addressing (I/D) space (inspired by the death of the python... thread)

[Will Senn]

Does unix (v7) know about the PDP-11 45's split I/D space through 
configuration or is it convention and programmer's responsibility to 
know and manage what's actually available?

Will

On 8/3/23 12:00, Rich Salz wrote:
> What, we all need something to kick now that we've beaten sendmail?  
> How about something unix, ideally a decade old?

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Kenneth Goodwin]

[-- Attachment #1: Type: text/plain, Size: 2118 bytes --]

At the risk of exposing my ignorance and thus being events long long ago in
history....
And my mind now old and feeble...

😆 🤣

1.  I don't think the 11/45 had split I & d.
But I could be wrong.
That did not appear until the 11/70
And was in the later generation 11/44 several years later.

2. The kernel determined it by MMU type and managed it solely. The
assembler and loader always built the binary object file as the three
sections - instructions,  data and bss spaces so loading an object file
could be done on any platform.
Programmers generally did not worry about the underlying hardware

3. I don't recall if a systype style system call was available in v7 to
give you a machine type to switch off of.

With something like that you could determine memory availability hard
limits on the DATA/bss side if you needed to.

But that was also easily determined by a allocation failure in malloc/sbrk
with an out of memory error.

If you really needed to know availability,  you could have a start up
subroutine that would loop trying to malloc ever decreasing memory sizes
until success and until out of available memory error.
Then release it all back via free(). Or manage it internally.

As I recall however vaguely,  there was an attempt to split the kernel into
two pieces. One running in kernel mode and one running in supervisor mode
in order to double the amount of available  instruction and data spaces for
the operating system. I recall playing around with what was there trying to
get it to work right.
I was trying to support over 200 users on a pdp 11/70 at the time running a
massive insurance database system.

On Thu, Aug 3, 2023, 4:35 PM Will Senn <will.senn@gmail.com> wrote:

> Does unix (v7) know about the PDP-11 45's split I/D space through
> configuration or is it convention and programmer's responsibility to
> know and manage what's actually available?
>
> Will
>
> On 8/3/23 12:00, Rich Salz wrote:
> > What, we all need something to kick now that we've beaten sendmail?
> > How about something unix, ideally a decade old?
>
>

[-- Attachment #2: Type: text/html, Size: 2958 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Ronald Natalie]

[-- Attachment #1: Type: text/plain, Size: 2765 bytes --]

Having cut my UNIX teeth on the JHU 11/45, I can tell you very much that 
it did have split I/D.    V6 supported split I/D for user mode programs. 
   The kernel originally wasn’t split I/D.   Version 7, if I’m recalling 
properly, did run the kernel split I/D on the 45 and 70.



------ Original Message ------
From "Kenneth Goodwin" <kennethgoodwin56@gmail.com>
To "Will Senn" <will.senn@gmail.com>
Cc "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
Date 8/3/23, 5:05:31 PM
Subject [TUHS] Re: Split addressing (I/D) space (inspired by the death 
of the python... thread)

>At the risk of exposing my ignorance and thus being events long long 
>ago in history....
>And my mind now old and feeble...
>
>😆 🤣
>
>1.  I don't think the 11/45 had split I & d.
>But I could be wrong.
>That did not appear until the 11/70
>And was in the later generation 11/44 several years later.
>
>2. The kernel determined it by MMU type and managed it solely. The 
>assembler and loader always built the binary object file as the three 
>sections - instructions,  data and bss spaces so loading an object file 
>could be done on any platform.
>Programmers generally did not worry about the underlying hardware
>
>3. I don't recall if a systype style system call was available in v7 to 
>give you a machine type to switch off of.
>
>With something like that you could determine memory availability hard 
>limits on the DATA/bss side if you needed to.
>
>But that was also easily determined by a allocation failure in 
>malloc/sbrk with an out of memory error.
>
>If you really needed to know availability,  you could have a start up 
>subroutine that would loop trying to malloc ever decreasing memory 
>sizes until success and until out of available memory error.
>Then release it all back via free(). Or manage it internally.
>
>As I recall however vaguely,  there was an attempt to split the kernel 
>into two pieces. One running in kernel mode and one running in 
>supervisor mode in order to double the amount of available  instruction 
>and data spaces for the operating system. I recall playing around with 
>what was there trying to get it to work right.
>I was trying to support over 200 users on a pdp 11/70 at the time 
>running a massive insurance database system.
>
>On Thu, Aug 3, 2023, 4:35 PM Will Senn <will.senn@gmail.com> wrote:
>>Does unix (v7) know about the PDP-11 45's split I/D space through
>>configuration or is it convention and programmer's responsibility to
>>know and manage what's actually available?
>>
>>Will
>>
>>On 8/3/23 12:00, Rich Salz wrote:
>> > What, we all need something to kick now that we've beaten sendmail?
>> > How about something unix, ideally a decade old?
>>

[-- Attachment #2: Type: text/html, Size: 4845 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 3278 bytes --]

2BSD also did split I&D in the kernel (as well as run TCP in supervisor
mode to get another I/D space). A lot of the overlays was done in the
linker, but it wasn't completely automated.
I had to tweak the overlay tables a little as I did the 2.11pl0 work since
the early stuff wasn't exactly careful about distributing the hacks to the
makefile to make it happen...

Warner

On Thu, Aug 3, 2023 at 3:10 PM Ronald Natalie <ron@ronnatalie.com> wrote:

> Having cut my UNIX teeth on the JHU 11/45, I can tell you very much that
> it did have split I/D.    V6 supported split I/D for user mode programs.
> The kernel originally wasn’t split I/D.   Version 7, if I’m recalling
> properly, did run the kernel split I/D on the 45 and 70.
>
>
>
> ------ Original Message ------
> From "Kenneth Goodwin" <kennethgoodwin56@gmail.com>
> To "Will Senn" <will.senn@gmail.com>
> Cc "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
> Date 8/3/23, 5:05:31 PM
> Subject [TUHS] Re: Split addressing (I/D) space (inspired by the death of
> the python... thread)
>
> At the risk of exposing my ignorance and thus being events long long ago
> in history....
> And my mind now old and feeble...
>
> 😆 🤣
>
> 1.  I don't think the 11/45 had split I & d.
> But I could be wrong.
> That did not appear until the 11/70
> And was in the later generation 11/44 several years later.
>
> 2. The kernel determined it by MMU type and managed it solely. The
> assembler and loader always built the binary object file as the three
> sections - instructions,  data and bss spaces so loading an object file
> could be done on any platform.
> Programmers generally did not worry about the underlying hardware
>
> 3. I don't recall if a systype style system call was available in v7 to
> give you a machine type to switch off of.
>
> With something like that you could determine memory availability hard
> limits on the DATA/bss side if you needed to.
>
> But that was also easily determined by a allocation failure in malloc/sbrk
> with an out of memory error.
>
> If you really needed to know availability,  you could have a start up
> subroutine that would loop trying to malloc ever decreasing memory sizes
> until success and until out of available memory error.
> Then release it all back via free(). Or manage it internally.
>
> As I recall however vaguely,  there was an attempt to split the kernel
> into two pieces. One running in kernel mode and one running in supervisor
> mode in order to double the amount of available  instruction and data
> spaces for the operating system. I recall playing around with what was
> there trying to get it to work right.
> I was trying to support over 200 users on a pdp 11/70 at the time running
> a massive insurance database system.
>
> On Thu, Aug 3, 2023, 4:35 PM Will Senn <will.senn@gmail.com> wrote:
>
>> Does unix (v7) know about the PDP-11 45's split I/D space through
>> configuration or is it convention and programmer's responsibility to
>> know and manage what's actually available?
>>
>> Will
>>
>> On 8/3/23 12:00, Rich Salz wrote:
>> > What, we all need something to kick now that we've beaten sendmail?
>> > How about something unix, ideally a decade old?
>>
>>

[-- Attachment #2: Type: text/html, Size: 4995 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Ronald Natalie]

[-- Attachment #1: Type: text/plain, Size: 4236 bytes --]

In fact, it was TCP (mbuf windowing) that killed the non split-I/D 
systems in our installation.    We were already using kernel overlays, 
but with only 8 segment registers combined for code, data, and stack, we 
just ran out of registers.     By then the VAXEN were coming along.  I 
recycled the 11/34’s etc… into LOS/C Internet routers.

The 55 (just a tweaked 45) and later the 44 also had it.   In addition 
the 23/24/J-11 and those derived processors did.


------ Original Message ------
From "Warner Losh" <imp@bsdimp.com>
To "Ronald Natalie" <ron@ronnatalie.com>
Cc "Kenneth Goodwin" <kennethgoodwin56@gmail.com>; "Will Senn" 
<will.senn@gmail.com>; "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
Date 8/3/23, 5:16:25 PM
Subject Re: [TUHS] Re: Split addressing (I/D) space (inspired by the 
death of the python... thread)

>2BSD also did split I&D in the kernel (as well as run TCP in supervisor 
>mode to get another I/D space). A lot of the overlays was done in the 
>linker, but it wasn't completely automated.
>I had to tweak the overlay tables a little as I did the 2.11pl0 work 
>since the early stuff wasn't exactly careful about distributing the 
>hacks to the makefile to make it happen...
>
>Warner
>
>On Thu, Aug 3, 2023 at 3:10 PM Ronald Natalie <ron@ronnatalie.com> 
>wrote:
>>Having cut my UNIX teeth on the JHU 11/45, I can tell you very much 
>>that it did have split I/D.    V6 supported split I/D for user mode 
>>programs.   The kernel originally wasn’t split I/D.   Version 7, if 
>>I’m recalling properly, did run the kernel split I/D on the 45 and 70.
>>
>>
>>
>>------ Original Message ------
>>From "Kenneth Goodwin" <kennethgoodwin56@gmail.com>
>>To "Will Senn" <will.senn@gmail.com>
>>Cc "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
>>Date 8/3/23, 5:05:31 PM
>>Subject [TUHS] Re: Split addressing (I/D) space (inspired by the death 
>>of the python... thread)
>>
>>>At the risk of exposing my ignorance and thus being events long long 
>>>ago in history....
>>>And my mind now old and feeble...
>>>
>>>😆 🤣
>>>
>>>1.  I don't think the 11/45 had split I & d.
>>>But I could be wrong.
>>>That did not appear until the 11/70
>>>And was in the later generation 11/44 several years later.
>>>
>>>2. The kernel determined it by MMU type and managed it solely. The 
>>>assembler and loader always built the binary object file as the three 
>>>sections - instructions,  data and bss spaces so loading an object 
>>>file could be done on any platform.
>>>Programmers generally did not worry about the underlying hardware
>>>
>>>3. I don't recall if a systype style system call was available in v7 
>>>to give you a machine type to switch off of.
>>>
>>>With something like that you could determine memory availability hard 
>>>limits on the DATA/bss side if you needed to.
>>>
>>>But that was also easily determined by a allocation failure in 
>>>malloc/sbrk with an out of memory error.
>>>
>>>If you really needed to know availability,  you could have a start up 
>>>subroutine that would loop trying to malloc ever decreasing memory 
>>>sizes until success and until out of available memory error.
>>>Then release it all back via free(). Or manage it internally.
>>>
>>>As I recall however vaguely,  there was an attempt to split the 
>>>kernel into two pieces. One running in kernel mode and one running in 
>>>supervisor mode in order to double the amount of available  
>>>instruction and data spaces for the operating system. I recall 
>>>playing around with what was there trying to get it to work right.
>>>I was trying to support over 200 users on a pdp 11/70 at the time 
>>>running a massive insurance database system.
>>>
>>>On Thu, Aug 3, 2023, 4:35 PM Will Senn <will.senn@gmail.com> wrote:
>>>>Does unix (v7) know about the PDP-11 45's split I/D space through
>>>>configuration or is it convention and programmer's responsibility to
>>>>know and manage what's actually available?
>>>>
>>>>Will
>>>>
>>>>On 8/3/23 12:00, Rich Salz wrote:
>>>> > What, we all need something to kick now that we've beaten 
>>>>sendmail?
>>>> > How about something unix, ideally a decade old?
>>>>

[-- Attachment #2: Type: text/html, Size: 7172 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Kenneth Goodwin]

[-- Attachment #1: Type: text/plain, Size: 2985 bytes --]

Having worked on the v6 kernel on the
11/70 it was split on that version on that hardware.

On Thu, Aug 3, 2023, 5:10 PM Ronald Natalie <ron@ronnatalie.com> wrote:

> Having cut my UNIX teeth on the JHU 11/45, I can tell you very much that
> it did have split I/D.    V6 supported split I/D for user mode programs.
> The kernel originally wasn’t split I/D.   Version 7, if I’m recalling
> properly, did run the kernel split I/D on the 45 and 70.
>
>
>
> ------ Original Message ------
> From "Kenneth Goodwin" <kennethgoodwin56@gmail.com>
> To "Will Senn" <will.senn@gmail.com>
> Cc "The Eunuchs Hysterical Society" <tuhs@tuhs.org>
> Date 8/3/23, 5:05:31 PM
> Subject [TUHS] Re: Split addressing (I/D) space (inspired by the death of
> the python... thread)
>
> At the risk of exposing my ignorance and thus being events long long ago
> in history....
> And my mind now old and feeble...
>
> 😆 🤣
>
> 1.  I don't think the 11/45 had split I & d.
> But I could be wrong.
> That did not appear until the 11/70
> And was in the later generation 11/44 several years later.
>
> 2. The kernel determined it by MMU type and managed it solely. The
> assembler and loader always built the binary object file as the three
> sections - instructions,  data and bss spaces so loading an object file
> could be done on any platform.
> Programmers generally did not worry about the underlying hardware
>
> 3. I don't recall if a systype style system call was available in v7 to
> give you a machine type to switch off of.
>
> With something like that you could determine memory availability hard
> limits on the DATA/bss side if you needed to.
>
> But that was also easily determined by a allocation failure in malloc/sbrk
> with an out of memory error.
>
> If you really needed to know availability,  you could have a start up
> subroutine that would loop trying to malloc ever decreasing memory sizes
> until success and until out of available memory error.
> Then release it all back via free(). Or manage it internally.
>
> As I recall however vaguely,  there was an attempt to split the kernel
> into two pieces. One running in kernel mode and one running in supervisor
> mode in order to double the amount of available  instruction and data
> spaces for the operating system. I recall playing around with what was
> there trying to get it to work right.
> I was trying to support over 200 users on a pdp 11/70 at the time running
> a massive insurance database system.
>
> On Thu, Aug 3, 2023, 4:35 PM Will Senn <will.senn@gmail.com> wrote:
>
>> Does unix (v7) know about the PDP-11 45's split I/D space through
>> configuration or is it convention and programmer's responsibility to
>> know and manage what's actually available?
>>
>> Will
>>
>> On 8/3/23 12:00, Rich Salz wrote:
>> > What, we all need something to kick now that we've beaten sendmail?
>> > How about something unix, ideally a decade old?
>>
>>

[-- Attachment #2: Type: text/html, Size: 4704 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Ronald Natalie]

Both V6 and V7 used split I/D on the 45 and 70 where it as available.    
  You had to specify that you wanted it in your build, the choices were 
the 407 flagged. a.out, which had a single text/data/bss space 
(unspilt).    410 which still ran in one address space, but put the text 
in read only segments so they could be shared.   411 ran the executable 
in split I and D space.

The original use of the “sticky” bit in the inode mode indicated that a 
410 or 411 program text would be held in swap.

-Ron


------ Original Message ------
From "Will Senn" <will.senn@gmail.com>
To tuhs@tuhs.org
Date 8/3/23, 4:35:09 PM
Subject [TUHS] Split addressing (I/D) space (inspired by the death of 
the python... thread)

>Does unix (v7) know about the PDP-11 45's split I/D space through configuration or is it convention and programmer's responsibility to know and manage what's actually available?
>
>Will
>
>On 8/3/23 12:00, Rich Salz wrote:
>>What, we all need something to kick now that we've beaten sendmail?  How about something unix, ideally a decade old?
>

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1909 bytes --]

Will its a C Compiler switch (-i) and create 411 files instead of 407.
I like to refer to it as the 17th address bit.

I was first brought out with the 11/45 (which was SSI/MSI TTL), and the
biggest differences between it and the 11/40.  They were both early 1970s
and both of these processors were multiple boards. By 1976, the 780 has
started and that sucked off most of the HW folks.   A new hire in 1976,
Jeff Mitchell supposedly had a bet with Bill Strecker that he could
implement an 11 on a single"hex high" CPU board if he got rid of the lights
and switches.  He ran out of room to implement seperate I/D, so it became
an 11/40 class [and it has an 8008-1 that runs the front panel].

The 11/70 came out between the 11/45 and the 34 and had a number of
the STAR folks on it it original but it was also multiple boards.   It was
not until 11/44 that DEC was able to make a hex height implementation of
the 11 that managed to cram a full 11/70 into that system.   The later J-11
chip set took things beyond the 11/70.

If you look at the conf directory in the sys sources for V6, you see m40.s
and m45.s - but if you look at the link line of sys/run the 45 does not
have -i; but if you look in sys/sys1.c you'll see the in the routine
getxfile the support for 0407/0411/0405/0410 files for user mode.

If you look at the conf directory in the sys sources for V7, you see mch.s
and m45.s  its common and the makefile adds -i

ᐧ

On Thu, Aug 3, 2023 at 4:35 PM Will Senn <will.senn@gmail.com> wrote:

> Does unix (v7) know about the PDP-11 45's split I/D space through
> configuration or is it convention and programmer's responsibility to
> know and manage what's actually available?
>
> Will
>
> On 8/3/23 12:00, Rich Salz wrote:
> > What, we all need something to kick now that we've beaten sendmail?
> > How about something unix, ideally a decade old?
>
>

[-- Attachment #2: Type: text/html, Size: 3422 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Will Senn]

[-- Attachment #1: Type: text/plain, Size: 2572 bytes --]

Clem,

Oh, so... Without I/D, you're stuck with 64k max per process, with I/D, 
you can use 64k for I and 64k for D. Was that it, or were there other 
tricks to get even more allocated (didn't the 11 max out at 256k)?

The kernel could be compiled either with, or without separate I/D. The 
only reason not to is if you didn't have more then 64k or were there 
other reasons?

So, besides the kernel what apps tended to be split? If I remember 
correctly, vi was one, pascal another?

Thanks!

Will

On 8/3/23 16:44, Clem Cole wrote:
> Will its a C Compiler switch (-i) and create 411 files instead of 407.
> I like to refer to it as the 17th address bit.
>
> I was first brought out with the 11/45 (which was SSI/MSI TTL), and 
> the biggest differences between it and the 11/40.  They were both 
> early 1970s and both of these processors were multiple boards. By 
> 1976, the 780 has started and that sucked off most of the HW folks.  
>  A new hire in 1976, Jeff Mitchell supposedly had a bet with Bill 
> Strecker that he could implement an 11 on a single"hex high" CPU board 
> if he got rid of the lights and switches.  He ran out of room to 
> implement seperate I/D, so it became an 11/40 class [and it has an 
> 8008-1 that runs the front panel].
>
> The 11/70 came out between the 11/45 and the 34 and had a number of 
> the STAR folks on it it original but it was also multiple boards.   It 
> was not until 11/44 that DEC was able to make a hex height 
> implementation of the 11 that managed to cram a full 11/70 into that 
> system.   The later J-11 chip set took things beyond the 11/70.
>
> If you look at the conf directory in the sys sources for V6, you see 
> m40.s and m45.s - but if you look at the link line of sys/run the 45 
> does not have -i; but if you look in sys/sys1.c you'll see the in the 
> routine getxfile the support for 0407/0411/0405/0410 files for user mode.
>
> If you look at the conf directory in the sys sources for V7, you see 
> mch.s and m45.s  its common and the makefile adds -i
>
> ᐧ
>
> On Thu, Aug 3, 2023 at 4:35 PM Will Senn <will.senn@gmail.com> wrote:
>
>     Does unix (v7) know about the PDP-11 45's split I/D space through
>     configuration or is it convention and programmer's responsibility to
>     know and manage what's actually available?
>
>     Will
>
>     On 8/3/23 12:00, Rich Salz wrote:
>     > What, we all need something to kick now that we've beaten sendmail?
>     > How about something unix, ideally a decade old?
>

[-- Attachment #2: Type: text/html, Size: 5408 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 4384 bytes --]

ᐧ
below... [and I meant to answer the second ½ of your question before]

On Thu, Aug 3, 2023 at 6:09 PM Will Senn <will.senn@gmail.com> wrote:

> Clem,
>
> Oh, so... Without I/D, you're stuck with 64k max per process, with I/D,
> you can use 64k for I and 64k for D.
>
Exactly but ... more in a minute.




> Was that it, or were there other tricks to get even more allocated (didn't
> the 11 max out at 256k)?
>
Different issues... the MMU on the 40 class and the 45/55 allows 256K [18
bits], the MMU for the 70 class is 4M [22 bits], Unibus I/O controllers had
18 bits of address and RH70 controllers could support 22 bits of extended
addresses - see the processor and peripheral handbooks for details [or I
can explain offline].

What the PDP-11 books calls 'pages' are 64-byte segments.  So the MMU is
set up to allow the processor to address 64K or 64KI/64KD at the time,
depending on if you have the I/D hardware, and the MMU is set up as to
which 'pages' are being addressed.

But you could overlay things ... [0405 files] with 'thunks'.

So to allow a process (or the kernel) to have more than 64K, overlays can
be loaded into memory and since the total physical space memory space is
either 18 or 22 bits, if the kernel supports overlays - processes could get
bigger [which is part of your first question].
V7 was when 0405 [text only] overlays were added.   With DEC's release of
v7m - Fred Cantor rewrote the overlay code and they became more general
[and that would go into 2.9BSD].

So the programmer needs to decide what you wanted to put into what
overlay.   For processes, the kernel can swap out segments and replace them
as needed.   The key is that link needs to generate near/far style calls
and it can be a PITA.  If you want to access a routine that is not
currently mapped into memory, the 'thunk' needs to ask the OS to switch it.
Great thought of what was going to be stored where.




>
> The kernel could be compiled either with, or without separate I/D. The
> only reason not to is if you didn't have more then 64k or were there other
> reasons?
>
Well by V6, UNIX needed at least 64K of physical memory and it was really
slow with anything less than 256K.   For the kernel, using I/D allowed the
kernel to grow more easily.  By the time of trying to cram networking into
it, running on anything less than an 11/44 was pretty hard.

That said, Able made an alternate MMU called the ENABLE that allow 4M of
memory on a Unibus system.  It worked at a cache/bus repeater.  So you set
the internal MMU to point to it and then use its MMU.   Very cool and a
soft spot for me. I ran an 11/60 [which is 40 class] with 2M of memory in
Teklabs with the first Enable board.

For whatever its worth, even with 4M the kernel had started to become a
problem for V7 on an 11/70.  Data buffers eat a lot of memory.

>
> So, besides the kernel what apps tended to be split? If I remember
> correctly, vi was one, pascal another?
>
Anything that started to get big ;-)

Ppeople ran out of data space and text space from 64K fairly fast.   With
the 32-bit Vax, the UNIX Small is Beautiful thinking started to fall away.
 Rob has an excellent paper -> "cat -v considered harmful"   BSD UNIX, and
the Vaxen greatly fueled that.   Adding features and thinking less about
what functionality was really needed started to get lost [so now we have
Gnu - but I digress].  Werner and the BSD 2.9 folks are to be commended for
what they did with so few resources.   They moved things back from the Vax
by using the overlays, but if you were to have any semblance of
performance, you need the overlays to stay resident so you need that full
4M of memory.

As for this specific question first two subsystems for the 11 that ran out
of text space were indeed vi and Pascal subsystems (Joy having had his hand
in both, BTW).  But they were hardly the only ones once the genie was out
of the bottle.   Data space quickly became the real issue.  People really
wanted larger heaps in particular.   In fact, by the 1990s, I knew of few
programs that run out of 32-bit worth of text space, but many that started
to run out of 32-bits of data space -> hence Alpha.   But BTW:  DEC took a
performance hit originally, and there was a huge discussion at the time if
64-bits was really needed.
ᐧ

[-- Attachment #2: Type: text/html, Size: 9286 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On Thu, 3 Aug 2023, Clem Cole wrote:

> For whatever its worth, even with 4M the kernel had started to become a 
> problem for V7 on an 11/70.  Data buffers eat a lot of memory.

We at UNSW had a kludge for that; the global symbol "b" (similar to "u") 
was mapped by KISA5 into the current buffer header.  Having dozens of 
buffers on a 40-class machine was really something...

-- Dave

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 5645 bytes --]

I probably should add one more thing...  RT11 and RSX, and in particular
the DEC FTN compiler, supported thunks and overlays - i.e. larger-sized
programs before UNIX did.  IIRC, they needed them because by then the FTN
subsystem needed overlays to run itself. So .. assuming my memory is
correct, this was the reason why Fred rewrote the V7 code, and DEC pushed
it out as part of the v7m release.  Ken's original overlay support code was
not sufficient for the DEC language tools.

I don't remember if Ultrix-11 or v7m for that matter, ultimately got the
FTN compiler.  That said, Paul W might remember as he did the linker work
for Ultrix moving the VMS linker to Ultrix to support the 'Technical
Languages Group' (TLG ).  Utrix-32 certainly did get FTN I think a couple
of other of the RSX and RSTS languages, but my memory of Fred's work was to
support overlays in V7 so it could support it.  This was all during the
Unix wars inside of DEC.  Fred was part of the 'Telephone Industries Group'
(TIG) in MRK.
ᐧ

On Thu, Aug 3, 2023 at 6:54 PM Clem Cole <clemc@ccc.com> wrote:

>
> ᐧ
> below... [and I meant to answer the second ½ of your question before]
>
> On Thu, Aug 3, 2023 at 6:09 PM Will Senn <will.senn@gmail.com> wrote:
>
>> Clem,
>>
>> Oh, so... Without I/D, you're stuck with 64k max per process, with I/D,
>> you can use 64k for I and 64k for D.
>>
> Exactly but ... more in a minute.
>
>
>
>
>> Was that it, or were there other tricks to get even more allocated
>> (didn't the 11 max out at 256k)?
>>
> Different issues... the MMU on the 40 class and the 45/55 allows 256K [18
> bits], the MMU for the 70 class is 4M [22 bits], Unibus I/O controllers
> had 18 bits of address and RH70 controllers could support 22 bits of
> extended addresses - see the processor and peripheral handbooks for details
> [or I can explain offline].
>
> What the PDP-11 books calls 'pages' are 64-byte segments.  So the MMU is
> set up to allow the processor to address 64K or 64KI/64KD at the time,
> depending on if you have the I/D hardware, and the MMU is set up as to
> which 'pages' are being addressed.
>
> But you could overlay things ... [0405 files] with 'thunks'.
>
> So to allow a process (or the kernel) to have more than 64K, overlays can
> be loaded into memory and since the total physical space memory space is
> either 18 or 22 bits, if the kernel supports overlays - processes could get
> bigger [which is part of your first question].
> V7 was when 0405 [text only] overlays were added.   With DEC's release of
> v7m - Fred Cantor rewrote the overlay code and they became more general
> [and that would go into 2.9BSD].
>
> So the programmer needs to decide what you wanted to put into what
> overlay.   For processes, the kernel can swap out segments and replace them
> as needed.   The key is that link needs to generate near/far style calls
> and it can be a PITA.  If you want to access a routine that is not
> currently mapped into memory, the 'thunk' needs to ask the OS to switch it.
> Great thought of what was going to be stored where.
>
>
>
>
>>
>> The kernel could be compiled either with, or without separate I/D. The
>> only reason not to is if you didn't have more then 64k or were there other
>> reasons?
>>
> Well by V6, UNIX needed at least 64K of physical memory and it was really
> slow with anything less than 256K.   For the kernel, using I/D allowed the
> kernel to grow more easily.  By the time of trying to cram networking into
> it, running on anything less than an 11/44 was pretty hard.
>
> That said, Able made an alternate MMU called the ENABLE that allow 4M of
> memory on a Unibus system.  It worked at a cache/bus repeater.  So you set
> the internal MMU to point to it and then use its MMU.   Very cool and a
> soft spot for me. I ran an 11/60 [which is 40 class] with 2M of memory in
> Teklabs with the first Enable board.
>
> For whatever its worth, even with 4M the kernel had started to become a
> problem for V7 on an 11/70.  Data buffers eat a lot of memory.
>
>>
>> So, besides the kernel what apps tended to be split? If I remember
>> correctly, vi was one, pascal another?
>>
> Anything that started to get big ;-)
>
> Ppeople ran out of data space and text space from 64K fairly fast.   With
> the 32-bit Vax, the UNIX Small is Beautiful thinking started to fall away.
>  Rob has an excellent paper -> "cat -v considered harmful"   BSD UNIX, and
> the Vaxen greatly fueled that.   Adding features and thinking less about
> what functionality was really needed started to get lost [so now we have
> Gnu - but I digress].  Werner and the BSD 2.9 folks are to be commended for
> what they did with so few resources.   They moved things back from the Vax
> by using the overlays, but if you were to have any semblance of
> performance, you need the overlays to stay resident so you need that full
> 4M of memory.
>
> As for this specific question first two subsystems for the 11 that ran out
> of text space were indeed vi and Pascal subsystems (Joy having had his hand
> in both, BTW).  But they were hardly the only ones once the genie was out
> of the bottle.   Data space quickly became the real issue.  People really
> wanted larger heaps in particular.   In fact, by the 1990s, I knew of few
> programs that run out of 32-bit worth of text space, but many that
> started to run out of 32-bits of data space -> hence Alpha.   But BTW:
> DEC took a performance hit originally, and there was a huge discussion at
> the time if 64-bits was really needed.
> ᐧ
>

[-- Attachment #2: Type: text/html, Size: 11229 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 510 bytes --]

On Thu, Aug 3, 2023 at 6:54 PM Clem Cole <clemc@ccc.com> wrote:

>
> ᐧ
>    Adding features and thinking less about what functionality was really
> needed started to get lost [so now we have Gnu - but I digress].
>

And a Good Thing Too.  I hate to think how many thousands of times in the
last few decades I have written "awk '{if (m < length) m = length} END
{print m}'" (and wondering for a moment if it's "length" or "len") until
someone mentioned "wc -L" to me just the other day.  Arrgh.

[-- Attachment #2: Type: text/html, Size: 1572 bytes --]

[TUHS] Re: [TULSA] Re: python

[Alejandro Colomar]

On 2023-08-03 18:57, Phil Budne wrote:
> On the subject of "no printf", there is not one, not two, but THREE
> ways to format strings, easily compounded with print:
> 
> 	print("%s %s" % ("hello", "world"))
> 	print("{1} {two}".format("hello", two="world"))
> 	print(f"{greeting} {populace}")
> 
> I'm pretty sure the last method (which initially made me vomit, due to
> violating my hardwired ideas about complicating the lexer, as if it
> can even be thought of as a separate layer), BUT I Seem To Recall that
> it allows a class to implement a formatting method, which may (or may
> not) work for complex numbers.

Actually, there's no need to move away from printf(3) for customization.
glibc provides register_printf_specifier(3), register_printf_modifier(3),
and register_printf_modifier(3), with which you can register %S for
printing your very nice struct.  You can even register modifiers, such as
%vS for printing your structure in a different manner.

Of course, those APIs are not portable, but that's a matter of porting
them; you don't need to invent a completely new formatted print variant
just to do that.

How does one even specify the equivalent of "%+'0#8.5f" in the new {}
formats??

Of course, there may be reasons to move away from printf(3): C++ seems
to have a faster thing with std:format (or so they claim; I didn't try
it).  But if speed is not a problem, I'd keep the good ol' syntax that
everybody knows.  No need to make everybody learn a "cool" new print
function, that probably won't be as tunable as printf(3) is.

Another thing is type safety.  printf(3) is actually type-safe.  At least
as long as you use powerful compilers that can diagnose misuses of
printf-like APIs.  GCC provides [[gnu::format()]] for a reason.  And if
you customize printf with your own formats, Clang has tools for checking
those too.

> 
> Type "hinting" has been mentioned: again, it's almost like a whole new
> version of the language.  You need to pick a type checker, and likely
> turn up the knobs to achieve any actual safety, and then deal with the
> fact that not all packages supply hints out of the box.  It kind of
> offers the desert-topping/floor-wax dichotomy: You can easily/quickly
> write small programs with dynamic typing for quick one-offs, AND write
> better armored code for libraries & production code.
> 
> On tabs vs spaces: Python3 forbids mixing them.  Again, fascist, but
> "it's for your own good".
> 
> So yes, Python sucks, but I continue using it, and unlike sendmail
> (which I continue to run, as I have a 200+ line .mc file it would take
> me at LEAST a week to replicate the effects of in another MTA), I
> don't tell people Python is unsuitable at any speed.
> 
> I'd probably happily write Go or Rust, given a situation (existing
> project/program) where its use was inevitable, and I DO have a program
> I think is a good candidate for rewriting in Go, but on the whole,
> it's a useful tool for many kinds of programmers, and in many
> situations, which I think is somewhat remarkable.

I'll go a little step further, and claim that newlines being significant
is another bad decission.  It's one of the things I dislike from
languages, including sh(1).  My scripts end up having a lot of escaped
newlines, because I find the pipe to be more readable and pleasant as

foo \
| bar

rather than

foo |
bar

While I can forgive that in the shell, because interactive mode is as
important as scripts, I can't do the same with Go.  Go is a programming
language, and having newlines be significant is nonsense.  Saving from
typing ;s is a similar excuse that the one python used for saving typing
{}s.  And the consequence is that due to that fascist rule of go, I
can't put the opening brace of a function declaration in a new line; it
must be sticked to the same line as the function declarator.

I never enjoyed Go, and that was the main reason.  In any case, I never
found a need for it.  Most (all?) of what I do can be done with sh(1)
or C.

Alex

[TUHS] Re: [TULSA] Re: python

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]

On Thu, Aug 3, 2023 at 1:29 PM Alejandro Colomar <alx.manpages@gmail.com>
wrote:

But if speed is not a problem, I'd keep the good ol' syntax that

everybody knows.  No need to make everybody learn a "cool" new print
> function, that probably won't be as tunable as printf(3) is.
>
>
By that argument, there would be no C, only Algol 68 and PL/I, or subsets
of them.

[-- Attachment #2: Type: text/html, Size: 1208 bytes --]

[TUHS] Re: [TULSA] Re: python

[Alejandro Colomar]

On 2023-08-03 19:51, John Cowan wrote:
> On Thu, Aug 3, 2023 at 1:29 PM Alejandro Colomar <alx.manpages@gmail.com>
> wrote:
> 
> But if speed is not a problem, I'd keep the good ol' syntax that
> 
> everybody knows.  No need to make everybody learn a "cool" new print
>> function, that probably won't be as tunable as printf(3) is.
>>
>>
> By that argument, there would be no C, only Algol 68 and PL/I, or subsets
> of them.
> 

I didn't claim that there's never a reason to invent new syntax.  My claim
was rather that in this case, there isn't.

-  printf(3) is more powerful than any other existing formatting function
   that I know of any language --I'm still curious of what's the equivalent
   of "%+'0#8.5f" in other formatting functions--.

-  It is also reasonably fast (at least for such a highly-customizable
   formatting function), and I'd like to see any system beat that while
   keeping the customizability.

-  It is type-safe, with the right tools.

I can understand the need for less-customizable faster formatting functions
for very-high-performance programs, and std::format may fit well there.  But
other than that, I don't see a reason to invent so many different formatting
functions.  Of course, one may do that just for fun, in which case I applaud
that.  But printf(3) is superior to them, IMO.

Alex

[TUHS] Re: [TULSA] Re: python

[Dan Cross]

On Thu, Aug 3, 2023 at 2:05 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
> On 2023-08-03 19:51, John Cowan wrote:
> > On Thu, Aug 3, 2023 at 1:29 PM Alejandro Colomar <alx.manpages@gmail.com>
> > wrote:
> >
> > But if speed is not a problem, I'd keep the good ol' syntax that
> >
> > everybody knows.  No need to make everybody learn a "cool" new print
> >> function, that probably won't be as tunable as printf(3) is.
> >>
> >>
> > By that argument, there would be no C, only Algol 68 and PL/I, or subsets
> > of them.
> >
>
> I didn't claim that there's never a reason to invent new syntax.  My claim
> was rather that in this case, there isn't.
>
> -  printf(3) is more powerful than any other existing formatting function
>    that I know of any language --I'm still curious of what's the equivalent
>    of "%+'0#8.5f" in other formatting functions--.

One issue is that this isn't standard C: the `'` verb for grouping by
thousands is an SUSv2 extension. I just checked the latest C23 draft,
and unless I missed it, it doesn't appear to have made it in.

But things like that are fairly straight-forward in many other
languages. For example, in Go, the format string is nearly identical,
modulo the `'`:

: prithvi; cat print.go
package main

import "fmt"

func main() {
        fmt.Printf("%+0#8.5f\n", 3.1415)
}
: prithvi; go run print.go
+3.14150
: prithvi;

Type safety is achieved through reflection.

In Rust, which has a very pleasant formatted print facility, type
safety is handled by implementing `print` and `println` as macros. I
don't miss `printf` there.

> -  It is also reasonably fast (at least for such a highly-customizable
>    formatting function), and I'd like to see any system beat that while
>    keeping the customizability.

At Google, a group of exceptionally talented engineers wrote a
replacement in C++ for both type safety and because, bluntly, `printf`
(actually `snprintf`) was too slow. I believe the overall mechanism
made it into ABSL.

> -  It is type-safe, with the right tools.

No it's not, and it really can't be. True, there are linters that can
try to match up types _if_ the format string is a constant and all the
arguments are known at e.g. compile time, but C permits one to
construct the format string at run time (or just select between a
bunch of variants); the language gives you no tools to enforce type
safety in a meaningful way once you do that.

> I can understand the need for less-customizable faster formatting functions
> for very-high-performance programs, and std::format may fit well there.  But
> other than that, I don't see a reason to invent so many different formatting
> functions.  Of course, one may do that just for fun, in which case I applaud
> that.  But printf(3) is superior to them, IMO.

That sort of relative value judgement can be difficult to justify
without specifying the criteria that goes into one's judgement. As you
said, it is an opinion and that's fine, but opinions vary. :-)

        - Dan C.

[TUHS] printf (was: python)

[Alejandro Colomar]

Hi Dan,

On 2023-08-03 23:29, Dan Cross wrote:
> On Thu, Aug 3, 2023 at 2:05 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
>> On 2023-08-03 19:51, John Cowan wrote:
>>> On Thu, Aug 3, 2023 at 1:29 PM Alejandro Colomar <alx.manpages@gmail.com>
>>> wrote:
>>>
>>> But if speed is not a problem, I'd keep the good ol' syntax that
>>>
>>> everybody knows.  No need to make everybody learn a "cool" new print
>>>> function, that probably won't be as tunable as printf(3) is.
>>>>
>>>>
>>> By that argument, there would be no C, only Algol 68 and PL/I, or subsets
>>> of them.
>>>
>>
>> I didn't claim that there's never a reason to invent new syntax.  My claim
>> was rather that in this case, there isn't.
>>
>> -  printf(3) is more powerful than any other existing formatting function
>>    that I know of any language --I'm still curious of what's the equivalent
>>    of "%+'0#8.5f" in other formatting functions--.
> 
> One issue is that this isn't standard C: the `'` verb for grouping by
> thousands is an SUSv2 extension. I just checked the latest C23 draft,
> and unless I missed it, it doesn't appear to have made it in.

Being in POSIX.1 it's portable to most (all?) current systems. ISO C
is a baseline for an implementation.  A quality implementation will
go beyond that standard (or will be rather useless).  POSIX.1 is more
of a useful thing.

But yeah, we can remove that "'" to get the idea.

> 
> But things like that are fairly straight-forward in many other
> languages. For example, in Go, the format string is nearly identical,
> modulo the `'`:

Yup; I like go in that sense.

[...]

> 
>> -  It is also reasonably fast (at least for such a highly-customizable
>>    formatting function), and I'd like to see any system beat that while
>>    keeping the customizability.
> 
> At Google, a group of exceptionally talented engineers wrote a
> replacement in C++ for both type safety and because, bluntly, `printf`
> (actually `snprintf`) was too slow. I believe the overall mechanism
> made it into ABSL.

I think you mean absl::StrFormat().  It has printf(3)-like syntax, so
I can't say say much against it.  I don't know the details of how they
achieved the claimed 2x ~ 3x performance compared to snprintf(3).  I'm
curious to know if it's an inherent limitation of snprintf(3), or if
it's just that glibc is very unoptimized --which is true anyway, because
no-one has really maintained the printf(3) code in a long time--.

It's interesting, because then std::format() is not that miraculous
compared to snprintf(3).

> 
>> -  It is type-safe, with the right tools.
> 
> No it's not, and it really can't be. True, there are linters that can
> try to match up types _if_ the format string is a constant and all the
> arguments are known at e.g. compile time, but C permits one to
> construct the format string at run time (or just select between a
> bunch of variants); the language gives you no tools to enforce type
> safety in a meaningful way once you do that.

Isn't a variable format string a security vulnerability?  Where do you
need it?

Thanks,
Alex

[TUHS] Re: printf (was: python)

[Dan Cross]

On Thu, Aug 3, 2023 at 7:55 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
> On 2023-08-03 23:29, Dan Cross wrote:
> > On Thu, Aug 3, 2023 at 2:05 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
> >> On 2023-08-03 19:51, John Cowan wrote:
> >>> On Thu, Aug 3, 2023 at 1:29 PM Alejandro Colomar <alx.manpages@gmail.com>
> >>> wrote:
> >>>
> >>> But if speed is not a problem, I'd keep the good ol' syntax that
> >>>
> >>> everybody knows.  No need to make everybody learn a "cool" new print
> >>>> function, that probably won't be as tunable as printf(3) is.
> >>>>
> >>>>
> >>> By that argument, there would be no C, only Algol 68 and PL/I, or subsets
> >>> of them.
> >>>
> >>
> >> I didn't claim that there's never a reason to invent new syntax.  My claim
> >> was rather that in this case, there isn't.
> >>
> >> -  printf(3) is more powerful than any other existing formatting function
> >>    that I know of any language --I'm still curious of what's the equivalent
> >>    of "%+'0#8.5f" in other formatting functions--.
> >
> > One issue is that this isn't standard C: the `'` verb for grouping by
> > thousands is an SUSv2 extension. I just checked the latest C23 draft,
> > and unless I missed it, it doesn't appear to have made it in.
>
> Being in POSIX.1 it's portable to most (all?) current systems. ISO C
> is a baseline for an implementation.  A quality implementation will
> go beyond that standard (or will be rather useless).  POSIX.1 is more
> of a useful thing.

I don't know that that's true, but I can see how one could get into a
"no true Scotsman" fallacy pretty quickly arguing over it.

> But yeah, we can remove that "'" to get the idea.
>
> > But things like that are fairly straight-forward in many other
> > languages. For example, in Go, the format string is nearly identical,
> > modulo the `'`:
>
> Yup; I like go in that sense.
>
> [...]
>
> >
> >> -  It is also reasonably fast (at least for such a highly-customizable
> >>    formatting function), and I'd like to see any system beat that while
> >>    keeping the customizability.
> >
> > At Google, a group of exceptionally talented engineers wrote a
> > replacement in C++ for both type safety and because, bluntly, `printf`
> > (actually `snprintf`) was too slow. I believe the overall mechanism
> > made it into ABSL.
>
> I think you mean absl::StrFormat().  It has printf(3)-like syntax, so
> I can't say say much against it.  I don't know the details of how they
> achieved the claimed 2x ~ 3x performance compared to snprintf(3).  I'm
> curious to know if it's an inherent limitation of snprintf(3), or if
> it's just that glibc is very unoptimized --which is true anyway, because
> no-one has really maintained the printf(3) code in a long time--.

I don't recall the details now, but I seem to remember that much of it
was moving the burden of parsing the formatting directives to compile
time (though I may be misremembering).

> It's interesting, because then std::format() is not that miraculous
> compared to snprintf(3).
>
> >
> >> -  It is type-safe, with the right tools.
> >
> > No it's not, and it really can't be. True, there are linters that can
> > try to match up types _if_ the format string is a constant and all the
> > arguments are known at e.g. compile time, but C permits one to
> > construct the format string at run time (or just select between a
> > bunch of variants); the language gives you no tools to enforce type
> > safety in a meaningful way once you do that.
>
> Isn't a variable format string a security vulnerability?  Where do you
> need it?

It _can_ be a security vulnerability, but it doesn't necessarily
_need_ to be. If one is careful in how one constructs it, such things
can be very safe indeed.

As to where one needs it, there are examples like `vsyslog()`, but
that's almost besides the point, which is that given that you _can_ do
things like that, the language can't really save you by type-checking
the arguments to printf; and once varargs are in the mix? Forget about
it.

        - Dan C.

[TUHS] Re: printf (was: python)

[Alejandro Colomar]

Hello Dan,

On 2023-08-04 18:06, Dan Cross wrote:
> On Thu, Aug 3, 2023 at 7:55 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
>> On 2023-08-03 23:29, Dan Cross wrote:
>>> On Thu, Aug 3, 2023 at 2:05 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
>>>> -  It is type-safe, with the right tools.
>>>
>>> No it's not, and it really can't be. True, there are linters that can
>>> try to match up types _if_ the format string is a constant and all the
>>> arguments are known at e.g. compile time, but C permits one to
>>> construct the format string at run time (or just select between a
>>> bunch of variants); the language gives you no tools to enforce type
>>> safety in a meaningful way once you do that.
>>
>> Isn't a variable format string a security vulnerability?  Where do you
>> need it?
> 
> It _can_ be a security vulnerability, but it doesn't necessarily
> _need_ to be. If one is careful in how one constructs it, such things
> can be very safe indeed.
> 
> As to where one needs it, there are examples like `vsyslog()`,

I guessed you'd mention v*() formatting functions, as that's the only
case where a variable format string is indeed necessary (or kind of).

I'll simplify your example to vwarnx(3), from the BSDs, which does less
job, but has a similar API regarding our discussion.

I'm not sure if you meant vsyslog() uses or its implementation, but
I'll cover both (but for vwarnx(3)).

Uses:

This function (and all v*() functions) will be used to implement a
wrapper variadic function, like for example warnx(3).  It's there, in
the variadic function, where the string /must be/ a literal, and where
the arguments are checked.  There's never a good reason to use a
non-literal there (AFAIK), and there are compiler warnings and linters
to enforce that.  Since those args have been previously checked, you
should just pass the va_list pristine to other formatting functions.
Then, as long as libc doesn't have bugs, you're fine.

In the implementation of a v*() function:

Do /not/ touch the va_list.  Just pass it to the next function.  Of
course, in the end, libc will have to iterate over it and do the job,
but that's not the typical programmer's problem.  Here's the libbsd
implementation of vwarnx(3), which does exactly that: no messing with
the va_list.

$ grepc vwarnx
./include/bsd/err.h:63:
void vwarnx(const char *format, va_list ap)
	__printflike(1, 0);


./src/err.c:97:
void
vwarnx(const char *format, va_list ap)
{
	fprintf(stderr, "%s: ", getprogname());
	if (format)
		vfprintf(stderr, format, ap);
	fprintf(stderr, "\n");
}


Just put a [[gnu::format(printf)]] in the outermost wrapper, which
should be using a string literal, and you'll be fine.

> but
> that's almost besides the point, which is that given that you _can_ do
> things like that, the language can't really save you by type-checking
> the arguments to printf; and once varargs are in the mix? Forget about
> it.

Not really.  You can do that _only_ if you really want.  If you want to
not be able, you can "drop privileges" by adding a few flags to your
compiler, such as -Werror=format-security -Werror=format-nonliteral,
and add a bunch of linters to your build system for more redundancy,
and voila, your project is now safe.

Alex

> 
>         - Dan C.

[TUHS] Re: printf (was: python)

[Dan Cross]

On Fri, Aug 4, 2023 at 12:57 PM Alejandro Colomar
<alx.manpages@gmail.com> wrote:
> On 2023-08-04 18:06, Dan Cross wrote:
> > On Thu, Aug 3, 2023 at 7:55 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
> >> On 2023-08-03 23:29, Dan Cross wrote:
> >>> On Thu, Aug 3, 2023 at 2:05 PM Alejandro Colomar <alx.manpages@gmail.com> wrote:
> >>>> -  It is type-safe, with the right tools.
> >>>
> >>> No it's not, and it really can't be. True, there are linters that can
> >>> try to match up types _if_ the format string is a constant and all the
> >>> arguments are known at e.g. compile time, but C permits one to
> >>> construct the format string at run time (or just select between a
> >>> bunch of variants); the language gives you no tools to enforce type
> >>> safety in a meaningful way once you do that.
> >>
> >> Isn't a variable format string a security vulnerability?  Where do you
> >> need it?
> >
> > It _can_ be a security vulnerability, but it doesn't necessarily
> > _need_ to be. If one is careful in how one constructs it, such things
> > can be very safe indeed.
> >
> > As to where one needs it, there are examples like `vsyslog()`,
>
> I guessed you'd mention v*() formatting functions, as that's the only
> case where a variable format string is indeed necessary (or kind of).

I think you are conflating "necessary" with "possible."

> I'll simplify your example to vwarnx(3), from the BSDs, which does less
> job, but has a similar API regarding our discussion.
>
> I'm not sure if you meant vsyslog() uses or its implementation, but
> I'll cover both (but for vwarnx(3)).
>
> Uses:
>
> This function (and all v*() functions) will be used to implement a
> wrapper variadic function, like for example warnx(3).  It's there, in
> the variadic function, where the string /must be/ a literal, and where

No, the format string does not need to be a literal at all: it can be
constructed at runtime. Is that a good idea? Perhaps not. Is it
possible? Yes. Can the compiler type-check it in that case? No, it
cannot (since it hasn't been constructed at compile time).  Consider
this program:

: chandra; cat warn.c
#include <err.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int
main(void)
{
        char buf[1024];

        strlcpy(buf, "%s ", sizeof(buf));
        strlcat(buf, "%s ", sizeof(buf));
        strlcat(buf, "%d", sizeof(buf));

        warnx(buf, "Hello", "World", 42);

        return EXIT_SUCCESS;
}
: chandra; cc -Wall -Werror -o warn warn.c
: chandra; ./warn
warn: Hello World 42
: chandra;

That's a perfectly legal C program, even if it is a silly one. "Don't
do that" isn't a statement about the language, it's a statement about
programmer practice, which is the point.

> the arguments are checked.  There's never a good reason to use a
> non-literal there (AFAIK),

I believe that you believe that. You may even be right. However,
that's not how the language works.

> and there are compiler warnings and linters
> to enforce that.  Since those args have been previously checked, you
> should just pass the va_list pristine to other formatting functions.

I'm afraid that this reasonable advice misses the point: there's
nothing in the language that says you _have_ to do it this way. Some
tools may _help_, but they cannot cover all (reasonable) situations.

Here again `syslog()` is an interesting example, as it supports the
`%m` formatting verb. _An_ implementation of this may work by
interpreting the format string and constructing a new one,
substituting `strerror(errno)` whenever it hits "%m" and then using
`snprintf` (or equivalent) to create the file string that is sent to
`syslogd`. You may argue that programmers should only pass constant
strings (left deliberately vague since there are reasonable cases
where named string constants may be passed as a format string argument
in lieu of a literal) that can be checked by clang and gcc, but again,
nothing in the language _requires_ that, but the implementation of
`vsyslog` that actually implements that logic has no way of knowing
that its caller has done this correctly.

Similarly, someone may choose to implement a templating language that
converts a custom format to a new format string, but assumes that the
arguments are in a `va_list` or similar. Bad idea? Probably. Legal in
C? Yes.

> Then, as long as libc doesn't have bugs, you're fine.

That's a tall order.

> In the implementation of a v*() function:
>
> Do /not/ touch the va_list.  Just pass it to the next function.  Of
> course, in the end, libc will have to iterate over it and do the job,
> but that's not the typical programmer's problem.  Here's the libbsd
> implementation of vwarnx(3), which does exactly that: no messing with
> the va_list.
>
> $ grepc vwarnx
> ./include/bsd/err.h:63:
> void vwarnx(const char *format, va_list ap)
>         __printflike(1, 0);
>
>
> ./src/err.c:97:
> void
> vwarnx(const char *format, va_list ap)
> {
>         fprintf(stderr, "%s: ", getprogname());
>         if (format)
>                 vfprintf(stderr, format, ap);
>         fprintf(stderr, "\n");
> }
>
>
> Just put a [[gnu::format(printf)]] in the outermost wrapper, which
> should be using a string literal, and you'll be fine.

Using a number of extensions aside here, again, that's just (sadly)
not how the language works.

> > but
> > that's almost besides the point, which is that given that you _can_ do
> > things like that, the language can't really save you by type-checking
> > the arguments to printf; and once varargs are in the mix? Forget about
> > it.
>
> Not really.  You can do that _only_ if you really want.

Yes, that's the point: if we're talking about language-level
guarantees, the language can't help you here. It can try, and it can
hit a lot of really useful cases, but not all. By contrast, formatting
in Go and Rust is type-safe by construction.

> If you want to
> not be able, you can "drop privileges" by adding a few flags to your
> compiler, such as -Werror=format-security -Werror=format-nonliteral,
> and add a bunch of linters to your build system for more redundancy,
> and voila, your project is now safe.

Provided that you use a compiler that provides those options, or that
those linters are viable in your codebase. ;-)

        - Dan C.

[TUHS] Re: [TULSA] Re: python

[Steffen Nurpmeso]

Phil Budne wrote in
 <202308031657.373GvVvW008640@ultimate.com>:
 |I come not to praise Python...
 |
 |I picked up Python (v1) in the early 'aughts as a more readable
 |alternative to Perl and Ruby (non-alphanumerical variable names are a
 |non-starter for me).

That thrilled me!  empty? instead of is_empty(), d! instead of
log_this_string_if_debug_is_enabled() (which could be
LogThisStringIfDebugIsEnabled() even, or another damage)!
It (i did not know anything about Plan9 and people who use greek
letters and other math symbols as variable or function names)
inspired me sustainably:

  au Filetype c,cpp :iab <buffer>   $!    /*!  \_*/<Esc><Insert><Left>
  au Filetype c     :iab <buffer>   $!W   su_log_write(su_LOG_WARN, "");<Left><Left><Left><C-R>=__gobble()<CR>
  au Filetype cpp   :iab <buffer>   $!W   log::write(log::warn, "");<Left><Left><Left><C-R>=__gobble()<CR>

(Not a vim master.)
Their yaml help files were a pain (for me; especially since the
structure was there, yet the content was missing).

  ..

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

[TUHS] Re: [TULSA] Re: python

[Larry McVoy]

On Thu, Aug 03, 2023 at 12:57:31PM -0400, Phil Budne wrote:
> On the subject of "no printf", there is not one, not two, but THREE
> ways to format strings, easily compounded with print:
> 
> 	print("%s %s" % ("hello", "world"))
> 	print("{1} {two}".format("hello", two="world"))
> 	print(f"{greeting} {populace}")
> 
> I'm pretty sure the last method (which initially made me vomit, due to

All of those make me vomit.  Pretty much every sane language has printf.

[TUHS] Re: [TULSA] Re: python

[Will Senn]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

On 8/3/23 18:47, Larry McVoy wrote:
> On Thu, Aug 03, 2023 at 12:57:31PM -0400, Phil Budne wrote:
>> On the subject of "no printf", there is not one, not two, but THREE
>> ways to format strings, easily compounded with print:
>>
>> 	print("%s %s" % ("hello", "world"))
>> 	print("{1} {two}".format("hello", two="world"))
>> 	print(f"{greeting} {populace}")
>>
>> I'm pretty sure the last method (which initially made me vomit, due to
> All of those make me vomit.  Pretty much every sane language has printf.
Tell us how you really feel :). But, I couldn't agree more. I worked 
really hard to get printf and sprintf down and then folks started doing 
it "right" and "better" or renaming it but watering it down - ugh! I 
say, we picket.

[-- Attachment #2: Type: text/html, Size: 1252 bytes --]

[TUHS] Re: python

[Ed Bradford]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

To all:

Python 3's fstrings
  https://realpython.com/python-f-strings/
seem to me to be a huge improvement over printf
and its close relatives.

What are people's views about the pro's and con's and
how do print and strings compare in usability?

Ed


On Wed, Aug 2, 2023 at 9:07 PM Clem Cole <clemc@ccc.com> wrote:

> IMO (Like Larry) no printf stinks.  But the real killer for my sustain for
> Python is the use white space and being typeless.   My daughter loves it
> for her cloud development and we argue a bit.  But it was the first
> language she really mastered in college and she never took a competitive
> languages course so I’m not so sure really had experienced much beyond it
> for real programs.   Maybe I’m just an old fart but between C, Go and Rust
> I’m pretty good.  I do write scripts in Bourne shell and or awk truth be
> known.
>
> On Wed, Aug 2, 2023 at 8:51 PM Larry McVoy <lm@mcvoy.com> wrote:
>
>> On Wed, Aug 02, 2023 at 07:49:18PM -0400, Rich Salz wrote:
>> > > [Python is] meant for mainly functional programming as I understand it
>> >
>> > Not true. It has some neat functional features (list comprehensions) but
>> > that's not really its intent.
>>
>> I've really tried to like python but any language that doesn't has printf
>> as builtin is not for me.  Yes, I know about their library printf but it
>> is weird.
>> --
>> ---
>> Larry McVoy           Retired to fishing
>> http://www.mcvoy.com/lm/boat
>>
> --
> Sent from a handheld expect more typos than usual
>


-- 
Advice is judged by results, not by intentions.
  Cicero

[-- Attachment #2: Type: text/html, Size: 3568 bytes --]

[TUHS] Re: python

[Larry McVoy]

We did something sort of like that in Little, all double quoted strings
look for ${anything} in the string and evaluates it and prints.  Like
fstrings, you can do puts("Print 5 + 7 = ${5 + 7}");  

We used single quoted strings as pure strings.

It's handy and sometimes more readable than printf.

http://www.little-lang.org/little.html#String_interpolation

On Fri, Aug 04, 2023 at 02:20:06PM -0500, Ed Bradford wrote:
> To all:
> 
> Python 3's fstrings
>   https://realpython.com/python-f-strings/
> seem to me to be a huge improvement over printf
> and its close relatives.
> 
> What are people's views about the pro's and con's and
> how do print and strings compare in usability?
> 
> Ed
> 
> 
> On Wed, Aug 2, 2023 at 9:07???PM Clem Cole <clemc@ccc.com> wrote:
> 
> > IMO (Like Larry) no printf stinks.  But the real killer for my sustain for
> > Python is the use white space and being typeless.   My daughter loves it
> > for her cloud development and we argue a bit.  But it was the first
> > language she really mastered in college and she never took a competitive
> > languages course so I???m not so sure really had experienced much beyond it
> > for real programs.   Maybe I???m just an old fart but between C, Go and Rust
> > I???m pretty good.  I do write scripts in Bourne shell and or awk truth be
> > known.
> >
> > On Wed, Aug 2, 2023 at 8:51 PM Larry McVoy <lm@mcvoy.com> wrote:
> >
> >> On Wed, Aug 02, 2023 at 07:49:18PM -0400, Rich Salz wrote:
> >> > > [Python is] meant for mainly functional programming as I understand it
> >> >
> >> > Not true. It has some neat functional features (list comprehensions) but
> >> > that's not really its intent.
> >>
> >> I've really tried to like python but any language that doesn't has printf
> >> as builtin is not for me.  Yes, I know about their library printf but it
> >> is weird.
> >> --
> >> ---
> >> Larry McVoy           Retired to fishing
> >> http://www.mcvoy.com/lm/boat
> >>
> > --
> > Sent from a handheld expect more typos than usual
> >
> 
> 
> -- 
> Advice is judged by results, not by intentions.
>   Cicero

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: python

[Ed Bradford]

[-- Attachment #1: Type: text/plain, Size: 3001 bytes --]

Thanks Larry. I've found printf a jump in usability among the
languages I've used (FORTRAN, Visual Basic, Java, PHP and a few more).
Python3's
f-strings are a major readability/suportability advance over printf. There
is nothing I know of as usable, readable, and supportable.
I have yet to find anything I can't do
with f-strings than I could with printf.

There must be some one who has pros and cons of f-strings on this list?
I would like to hear thoughts.

Ed





On Fri, Aug 4, 2023 at 2:47 PM Larry McVoy <lm@mcvoy.com> wrote:

> We did something sort of like that in Little, all double quoted strings
> look for ${anything} in the string and evaluates it and prints.  Like
> fstrings, you can do puts("Print 5 + 7 = ${5 + 7}");
>
> We used single quoted strings as pure strings.
>
> It's handy and sometimes more readable than printf.
>
> http://www.little-lang.org/little.html#String_interpolation
>
> On Fri, Aug 04, 2023 at 02:20:06PM -0500, Ed Bradford wrote:
> > To all:
> >
> > Python 3's fstrings
> >   https://realpython.com/python-f-strings/
> > seem to me to be a huge improvement over printf
> > and its close relatives.
> >
> > What are people's views about the pro's and con's and
> > how do print and strings compare in usability?
> >
> > Ed
> >
> >
> > On Wed, Aug 2, 2023 at 9:07???PM Clem Cole <clemc@ccc.com> wrote:
> >
> > > IMO (Like Larry) no printf stinks.  But the real killer for my sustain
> for
> > > Python is the use white space and being typeless.   My daughter loves
> it
> > > for her cloud development and we argue a bit.  But it was the first
> > > language she really mastered in college and she never took a
> competitive
> > > languages course so I???m not so sure really had experienced much
> beyond it
> > > for real programs.   Maybe I???m just an old fart but between C, Go
> and Rust
> > > I???m pretty good.  I do write scripts in Bourne shell and or awk
> truth be
> > > known.
> > >
> > > On Wed, Aug 2, 2023 at 8:51 PM Larry McVoy <lm@mcvoy.com> wrote:
> > >
> > >> On Wed, Aug 02, 2023 at 07:49:18PM -0400, Rich Salz wrote:
> > >> > > [Python is] meant for mainly functional programming as I
> understand it
> > >> >
> > >> > Not true. It has some neat functional features (list
> comprehensions) but
> > >> > that's not really its intent.
> > >>
> > >> I've really tried to like python but any language that doesn't has
> printf
> > >> as builtin is not for me.  Yes, I know about their library printf but
> it
> > >> is weird.
> > >> --
> > >> ---
> > >> Larry McVoy           Retired to fishing
> > >> http://www.mcvoy.com/lm/boat
> > >>
> > > --
> > > Sent from a handheld expect more typos than usual
> > >
> >
> >
> > --
> > Advice is judged by results, not by intentions.
> >   Cicero
>
> --
> ---
> Larry McVoy           Retired to fishing
> http://www.mcvoy.com/lm/boat
>


-- 
Advice is judged by results, not by intentions.
  Cicero

[-- Attachment #2: Type: text/html, Size: 5553 bytes --]

[TUHS] Re: Cool talk on Unix and Sendmail history, by Eric Allman

[Dave Horsfall]

On Wed, 2 Aug 2023, Ron Natalie wrote:

> ASCII Bombs?   Like my Letter Bomb Transport Protocol (LBTP)?

Some "smart" terminals could have their function keys programmed by escape 
characters, then subsequently invoked...

> https://groups.google.com/g/net.followup/c/OJBALbzTq4w/m/LoxMnbz0bwMJ
> 
> It seems to have lost something in the formatting (the leading spaces 
> were all removed).

Aha...  Much better with a bit of imagination :-)

Which reminds me: the reason why I refuse to use Python is that white 
space is part of the syntax; the last language I used with that property 
was FORTRAN (and I also had to learn COBOL as part of my CS degree, but 
fortunately never had to use it).

-- Dave

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Noel Chiappa]

    > From: Will Senn

    > Does unix (v7) know about the PDP-11 45's split I/D space through
    > configuration or is it convention and programmer's responsibility to
    > know and manage what's actually available?

There are two different cases: i) support of split I+D in the kernel, and
ii) support of split I+D in  user processes. Both arrived with V6; the
V5 source:

 https://minnie.tuhs.org/cgi-bin/utree.pl?file=V5/usr/sys/conf/mch.s
 https://minnie.tuhs.org/cgi-bin/utree.pl?file=V5/usr/sys/ken/main.c

(former for kernel; later for users) shows no sign of it.


    > From: Kenneth Goodwin <kennethgoodwin56@gmail.com>

    > 1.  I don't think the 11/45 had split I & d.
    > But I could be wrong.
    > That did not appear until the 11/70

You are wrong.

The chief differences between the KB11-A&-D of the -11/45 and the -B&-C of
the -11/70 were i) the latter had a cache, and ii) the latter used the 32-bit
wide Main Memory Bus, which also allowed up to 4 Mbytes of main memory.
Detail here:

  https://gunkies.org/wiki/PDP-11/70

along with a couple of lesser differences.


    > From: "Ronald Natalie"

    > with only 8 segment registers combined for code, data, and stack

I think you meant for code, data, and user block.

    > The 55 (just a tweaked 45)

The /50 and /55 had the identical KB11-A&-D of the /45; the difference was
that they came pre-configured with Fastbus memory.

    > In addition the 23/24/J-11 and those derived processors did.

No; the F-11 processors did not support I&D, the J-11 did.


	Noel

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Noel Chiappa]

    > From: Clem Cole

    > A new hire in 1976, Jeff Mitchell supposedly had a bet with Bill
    > Strecker that he could implement an 11 on a single"hex high" CPU board
    > if he got rid of the lights and switches. He ran out of room to
    > implement seperate I/D, so it became an 11/40 class [and it has an
    > 8008-1 that runs the front panel].

I don't know about the Strecker story, but the first PDP-11 CPU on a single
card (a hex card) was the KD11-D:

  https://gunkies.org/wiki/KD11-D_CPU

of the -11/04. It didn't have _any_ memory management, though (or a front
panel; to get that, you had to use the KY"11-LB:

  https://gunkies.org/wiki/KY11-LB_Programmer%27s_Console

which added another quad card). The first -11 CPU i) on a single card and
ii) with memory management was the FDF11-A:

  https://gunkies.org/wiki/KDF11-A_CPU

The first -11 CPU i) on a single card and ii) with split I+D memory
management was the KDJ11-A.


    > It was not until 11/44 that DEC was able to make a hex height
    > implementation of the 11 that managed to cram a full 11/70 into that
    > system.

I'm not sure what your point is here? The KD11-Z CPU of the -11/44:

  https://gunkies.org/wiki/KD11-Z_CPU

was a _minimum_ of five hex boards; a little smaller than a KB11-B (15 hex
cards). Floating point was an extra card; CIS was 2 more. 


    > if you look at the link line of sys/run the 45 does not have -i

Split I+D for the kernel was not supported by the linker in V6; a combination
of 'sysfix' (a special post-processor, which took as input a relocatable
linked image) and code in m45.s was needed.

  https://gunkies.org/wiki/Upgrading_UNIX_Sixth_Edition
  https://gunkies.org/wiki/UNIX_V6_memory_layout

The code in m45.s to handle split I+D in the kernel:

  https://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/sys/conf/m45.s

starts at 'start:' and is adequately commented to tell you what it's doing
when it plays around with kernel memory.



    > From: Will Senn

    > with I/D, you can use 64k for I and 64k for D. Was that it, or were
    > there other tricks to get even more allocated

I have this vague memory that someone (Berkeley, I think?) added support for
automatic code overlays in user processes. The programmer had to decide which
modules went in which overlays, but after that it was all automagic. There
was a 4xx code allocated to them.

I think the support for that (in e.g. the linker) was somehow involved with
the use of overlays in the kernel, but I don't know the details (nothing
after V6 is at all interesting to me).

    > didn't the 11 max out at 256k)?

You need to distinguish between i) the amount of memory the machine could
handle, and ii) the amount of memory that running code could 'see' at any
instant. The latter was always either 64KB, or 64KB+64KB (with split I+D
turned on, on CPUs which supported it).

The former, it's complicated. Mostly, on UNIBUS only machines, it was 256KB.
(Although there was the Able ENABLE:

  https://gunkies.org/wiki/Able_ENABLE

which added an Extended UNIBUS, and could take them up to 4MB.) The -11/70,
as mentioned, had a Main Memory Bus, and could handle up to 4MB. The -11/44
had an Extended UNIBUS, and could also handle up to 4MB (but only after the
MS11-P became available; there were only 4 main memory slots in the
backplane, and MS11-M cards were only 256KB.) On QBUS achines, after the
KB11-A (Revision A), which only suppported 256 KB, all later revisions and
CPUs could also handle up to 4MB.

	Noel

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 3939 bytes --]

On Thu, Aug 3, 2023, 5:10 PM Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

>     > From: Clem Cole
>
>     > A new hire in 1976, Jeff Mitchell supposedly had a bet with Bill
>     > Strecker that he could implement an 11 on a single"hex high" CPU
> board
>     > if he got rid of the lights and switches. He ran out of room to
>     > implement seperate I/D, so it became an 11/40 class [and it has an
>     > 8008-1 that runs the front panel].
>
> I don't know about the Strecker story, but the first PDP-11 CPU on a single
> card (a hex card) was the KD11-D:
>
>   https://gunkies.org/wiki/KD11-D_CPU
>
> of the -11/04. It didn't have _any_ memory management, though (or a front
> panel; to get that, you had to use the KY"11-LB:
>
>   https://gunkies.org/wiki/KY11-LB_Programmer%27s_Console
>
> which added another quad card). The first -11 CPU i) on a single card and
> ii) with memory management was the FDF11-A:
>
>   https://gunkies.org/wiki/KDF11-A_CPU
>
> The first -11 CPU i) on a single card and ii) with split I+D memory
> management was the KDJ11-A.
>
>
>     > It was not until 11/44 that DEC was able to make a hex height
>     > implementation of the 11 that managed to cram a full 11/70 into that
>     > system.
>
> I'm not sure what your point is here? The KD11-Z CPU of the -11/44:
>
>   https://gunkies.org/wiki/KD11-Z_CPU
>
> was a _minimum_ of five hex boards; a little smaller than a KB11-B (15 hex
> cards). Floating point was an extra card; CIS was 2 more.
>
>
>     > if you look at the link line of sys/run the 45 does not have -i
>
> Split I+D for the kernel was not supported by the linker in V6; a
> combination
> of 'sysfix' (a special post-processor, which took as input a relocatable
> linked image) and code in m45.s was needed.
>
>   https://gunkies.org/wiki/Upgrading_UNIX_Sixth_Edition
>   https://gunkies.org/wiki/UNIX_V6_memory_layout
>
> The code in m45.s to handle split I+D in the kernel:
>
>   https://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/sys/conf/m45.s
>
> starts at 'start:' and is adequately commented to tell you what it's doing
> when it plays around with kernel memory.
>
>
>
>     > From: Will Senn
>
>     > with I/D, you can use 64k for I and 64k for D. Was that it, or were
>     > there other tricks to get even more allocated
>
> I have this vague memory that someone (Berkeley, I think?) added support
> for
> automatic code overlays in user processes. The programmer had to decide
> which
> modules went in which overlays, but after that it was all automagic. There
> was a 4xx code allocated to them.
>
> I think the support for that (in e.g. the linker) was somehow involved with
> the use of overlays in the kernel, but I don't know the details (nothing
> after V6 is at all interesting to me).
>

V7 had a number of patches for MENLO_LINKER or MENLO_OVERLAY that Berkeley
integrated into the tree, but I'm not sure they wrote it. Their hacks were
usually UCB_something...

Warner

    > didn't the 11 max out at 256k)?
>
> You need to distinguish between i) the amount of memory the machine could
> handle, and ii) the amount of memory that running code could 'see' at any
> instant. The latter was always either 64KB, or 64KB+64KB (with split I+D
> turned on, on CPUs which supported it).
>
> The former, it's complicated. Mostly, on UNIBUS only machines, it was
> 256KB.
> (Although there was the Able ENABLE:
>
>   https://gunkies.org/wiki/Able_ENABLE
>
> which added an Extended UNIBUS, and could take them up to 4MB.) The -11/70,
> as mentioned, had a Main Memory Bus, and could handle up to 4MB. The -11/44
> had an Extended UNIBUS, and could also handle up to 4MB (but only after the
> MS11-P became available; there were only 4 main memory slots in the
> backplane, and MS11-M cards were only 256KB.) On QBUS achines, after the
> KB11-A (Revision A), which only suppported 256 KB, all later revisions and
> CPUs could also handle up to 4MB.
>
>         Noel
>

[-- Attachment #2: Type: text/html, Size: 5787 bytes --]

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Steve Simon]

not quite split i and d but i do have a memory of some code which could run in three parts as overlays.

this could have been through exec’ing the next overlay with an appropriate argv and a pipe or file of data, or, perhaps there was some kernel support for overlays in early unix.

anyone seen evidence of this?

sadly i cannot remember where i saw it, i want to say it was a versatex printer driver but i am pretty sure that is rubbish.

-Steve

[TUHS] Re: Split addressing (I/D) space (inspired by the death of the python... thread)

[Noel Chiappa]

    > From: Clem Cole

    > first two subsystems for the 11 that ran out of text space were indeed
    > vi and Pascal subsystems 

Those were at Berkeley. We've established that S-I&D were in V6 when it was
released in May, 1975 - so my question is 'what was Bell doing in 1975 that
needed more than 64KB?'

The kernel, yeah, it could definitely use S-I&D on a larger system
(especially when you remember that stock V6 didn't play any tricks with
overlays, and also dedicated one segment - the correct term, used in the 1972
-11/45 processor manual - to the user structure, and one to the I/O page,
limiting the non-S-I&D kernel to 48KB). But what user commands?


It happens that I have a complete dump of one of the MIT systems, so I had a
look to see what _we_ were running S-I&D on. Here's the list from /bin (for
some reason that machine doesn't have a /usr/bin):

  a68
  a86
  c86
  emacs
  lisp
  ndd
  send
  teco

The lisp wasn't a serious use; I think the only thing we ever used it for was
'doctor'. So, two editors, a couple of language tools, an email tool (not
sure why that one got it - maybe for creating large outgoing messages). (The
ndd is probably to allow the biggest possible buffers.)

Nothing in /etc, and in /lib, just lint1 and lint2 (lint, AFAICT, post-dates
V6). Not a lot.


So now I'm really curious what Bell was using S-I&D for. (If I weren't lazy,
I'd pull the V6 distro - which is only available as RK images, and individual
files, alas - and look in /bin and everywhere and see if I can find anything.
I suspect not, though.)

Anyone have any guesses/suggestions? Maybe some custom applications?

	Noel