[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[segaloco via COFF]

On Wednesday, July 10th, 2024 at 4:00 PM, John Levine <johnl@taugh.com> wrote:

> It appears that Al Kossow aek@bitsavers.org said:
> 
> > On 7/10/24 1:53 PM, Dan Cross wrote:
> > 
> > > The idea of writing simulators for machines clearly dates to before
> > > (or near) the beginning of TAOCP.
> 
> 
> Sure, but the topic of interest here is compiling machine code from one
> machine to another. You know like Rosetta does for x86 code running on
> my Macbook (obUnix: whose OS is descended from FreeBSD and Mach and does
> all the Posix stuff) which has an M2 ARM chip.
> 
> We know that someone did it in 1967 from 709x to GE 635, which I agree
> was quite a trick since really the only thing the two machines had in
> common was a 36 bit word size. I was wondering if anyone did machine
> code translation as opposed to instruction at a time simulation before that.
> 

Attempting once again to COFF this thread as I am quite interested in the
discussion of this sort of emulation/simulation matter outside of the
confines of UNIX history as well.

To add to the discussion, while not satisfying the question of "where did
this sort of thing begin", the 3B20 was another machine that provided some
means of emulating another architecture via microcode, although what I know
about this is limited to discussions about emulating earlier ESS machines
to support existing telecom switching programs.  I've yet to find any
literature suggesting this was ever used to emulate other general-purpose
computers such as IBM, DEC, etc. but likewise no suggestion that it *couldn't*
be used this way.

- Matt G.

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[Dan Cross]

[TUHS to Bcc:, +COFF]

On Wed, Jul 10, 2024 at 5:26 PM John Levine <johnl@taugh.com> wrote:
> It appears that Noel Chiappa <jnc@mercury.lcs.mit.edu> said:
> >    > From: Dan Cross
> >
> >    > These techniques are rather old, and I think go back much further than
> >    > we're suggesting. Knuth mentions nested translations in TAOCP ..
> >    > suggesting the technique was well-known as early as the mid-1960s.
>
> Knuth was talking about simulating one machine on another, interpreting
> one instruction at a time.  As he notes, the performance is generally awful,
> although IBM did microcode emulation of many of their second generation
> machines on S/360 which all (for business reasons) ran faster than the
> real machines.  Unsurprisingly, you couldn't emulate a 7094 on anything
> smaller than a 360/65.

It's not clear to me why you suggest with such evident authority that
Knuth was referring only to serialized instruction emulation and not
something like JIT'ed code; true, he doesn't specify one way or the
other, but I find it specious to conclude that that implies the
technique wasn't already in use, or at least known.  But certainly by
then JIT'ing techniques for "interpreted" programming languages were
known; it doesn't seem like a great leap to extend that to binary
translation.  Of course, that's speculation on my part, and I could
certainly be wrong.

> We've been discussing batch or JIT translation of code which gives
> much better performance without a lot of hardware help.

JIT'd performance of binary transliteration is certainly going to be
_better_ than strict emulation, but it is unlikely to be _as good_ as
native code. Indeed, this is still an active area of research; e.g.,
luajit; https://www.mattkeeter.com/blog/2022-10-04-ssra/ (disclaimer:
Matt's a colleague of mine), etc.

        - Dan C.

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[Dan Cross]

On Wed, Jul 10, 2024 at 9:54 PM John R Levine <johnl@taugh.com> wrote:
> On Wed, 10 Jul 2024, Dan Cross wrote:
> > It's not clear to me why you suggest with such evident authority that
> > Knuth was referring only to serialized instruction emulation and not
> > something like JIT'ed code; true, he doesn't specify one way or the
> > other, but I find it specious to conclude that that implies the
> > technique wasn't already in use, or at least known.
>
> The code on pages 205 to 211 shows an instruction by instruction
> interpreter.  I assume Knuth knew about JIT compiling since Lisp systems
> had been doing it since the 1960s, but that's not what this section of the
> book is about.

Sure.  But we're trying to date the topic here; my point is that
JITing was well known, and simulation was similarly well known; we
know when work on those books started; it doesn't seem that odd to me
that combining the two would be known around that time as well.

> One of the later volumes of TAOCP was supposed to be about
> compiling, but it seems unlikely he'll have time to write it.

Yes; volumes 5, 6 and 7 are to cover parsing, languages, and compilers
(more or less respectively).  Sadly, I suspect you are right that it's
unlikely he will have time to write them.

> >> We've been discussing batch or JIT translation of code which gives
> >> much better performance without a lot of hardware help.
> >
> > JIT'd performance of binary transliteration is certainly going to be
> > _better_ than strict emulation, but it is unlikely to be _as good_ as
> > native code.
>
> Well, sure, except in odd cases like the Vax compiler and reoptimizer
> someone mentioned a few messages back.

I think the point about the VAX compiler is that it's an actual
compiler and that the VAX MACRO-32 _language_ is treated as a "high
level" programming language, rather than as a macro assembly language.
That's not doing binary->binary translation, that's doing
source->binary compilation.  It's just that, in this case, the source
language happens to look like assembler for an obsolete computer.

        - Dan C.

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 4459 bytes --]

On Wed, Jul 10, 2024 at 10:12 PM Dan Cross <crossd@gmail.com> wrote:

> I think the point about the VAX compiler is that it's an actual
> compiler and that the VAX MACRO-32 _language_ is treated as a "high
> level" programming language, rather than as a macro assembly language.
> That's not doing binary->binary translation, that's doing
> source->binary compilation.  It's just that, in this case, the source
> language happens to look like assembler for an obsolete computer.
>

Yes, that was precisely my point, and thank you for stating it more clearly
and concisely than I did.  The VAX MACRO compiler takes in VAX assembly
source code, not binary VAX instructions.

The discrete transistor -> integrated circuit transition (2nd generation->
3rd generation) eventually resulted in enough of a speedup that microcode
implementations of instruction sets such as System/360 offered acceptable
performance at a low manufacturing cost.  IIRC System/360 models 75 and up
were done completely in hardware--no microcode.  S/360 models 67 and down
were microcoded.  The lowest end S/360, the model 25, was actually a 16-bit
machine.   All the S/360s from the 65 and down had microcoded emulators for
2nd generation IBM architectures such as the 1400.  The emulators usually
ran faster than the real hardware.

The emulators were there to accommodate data centers that had lost their
source code for mission critical applications.  There was also translation
software available that would translate 2nd generation machine code into
COBOL.  I remember their ads appearing weekly in ComputerWorld.  It showed
a data center manager smacking his forehead.  The headline of the ad read,
"1400 emulation?  In your 19xx budget?"  I never had personal experience
with the translator, but I'm told that the resulting COBOL was hard to
maintain.

Another example of using instruction set emulation was the IBM 5100
Portable Computer.  Released in 1975, this was a desktop-sized machine with
an attached keyboard and small CRT screen.  One interacted with it in
either APL or BASIC, the language selectable by a toggle switch.  64K of
memory was available to the user.  Internally the machine had a CPU
codenamed Palm that was used in many of IBM's peripheral controllers.  The
hardware had two 64K banks of ROM and one 64K bank of RAM.  One of the ROM
banks held the BASIC interpreter and the other bank the APL interpreter.
The front panel toggle switch selected which ROM bank was in use.

The BASIC interpreter was implemented in native Palm code.  But the APL
interpreter was APL\360, in S/360 machine code.  There was a S/360
instruction set emulator written in Palm code that interpreted the APL\360
interpreter.  This sort of situation delivers horrible performance.

During grad school I interned for a few years at IBM's Cambridge Scientific
Center (CSC) in Cambridge, MA, the birthplace of CP-67, the virtual machine
software system, and CMS (Cambridge Monitor System), an interactive OS that
ran in CP-67 virtual machines.  The CSC folks got hold of a 5100 and were
fascinated in its S/360 instruction set emulation facility.  The first
versions of CMS ran in less than 64K on a S/360 model 40.  CSC had a
project to take modern CMS, put it on a bit of a diet, and then run it on
the 5100, thus creating an interactive, desktop S/370.  Project programming
was done on a large S/370 mainframe.  The problem was getting the code onto
the 5100.  They eventually wrote an OS/VS link editor in APL that ran on
the 5100 and linked OS/VS object files into executable images stored on the
5100's floppy disk.  They wrote a APL shared variable module called
delta-S360 that caused the Palm processor's S/360 emulator to stop
emulating the APL interpreter and instead emulate instructions loaded into
an APL variable passed as a parameter to delta-S360.

The APL link editor ran as slow as death.  It processed one object file
card image every 10 seconds or so and took hours to link a program.  I was
given the task of writing an OS/360 link editor that could run standalone
on the 5100's S/360 emulator.  It was invoked from the APL interpreter by a
delta-S360 call.  With the APL interpreter out of the picture, my link
editor ran the floppy disk as fast as it could go and reduced link time
from hours to minutes.

Knuth was right about multiple levels of emulation/interpretaton.

-Paul W.

[-- Attachment #2: Type: text/html, Size: 5002 bytes --]

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[Dave Horsfall]

On Thu, 11 Jul 2024, Paul Winalski wrote:

> The lowest end S/360, the model 25, was actually a 16-bit machine.

Wouldn't that be the /20?  16-bit, 8 registers, no FP (but did have a HALT
instruction), etc...

-- Dave

[COFF] Re: [TUHS] Re: machine code translation,as mental architecture models

[John R Levine]

On Thu, 11 Jul 2024, Paul Winalski wrote:
> Yes, that was precisely my point, and thank you for stating it more clearly
> and concisely than I did.  The VAX MACRO compiler takes in VAX assembly
> source code, not binary VAX instructions.

Does anyone know how extensively they used the macro facilities?  You can 
write much higher level stuff as macros than as single instructions, which 
makes it a lot easier to do efficient translation.  For example, on OS/360 
you'd write a GET macro to retrieve the next record (or a pointer to it) 
from a file, which was a lot easier to figure out than the control blocks 
and subroutine calls the macros expanded into.

> were done completely in hardware--no microcode.  S/360 models 67 and down
> were microcoded.  The lowest end S/360, the model 25, was actually a 16-bit
> machine.

The /25 and /30 were 8 bits internally and as slow as you would expect, 
but were still full implementations of S/360.  IBM sold a lot of them. 
The /40 was 16 bits, /50 32 bits, and /65 64 bits.  The later /85 was 
roughly a /65 reimplemented in faster logic with a cache and 128 bit 
memory, making it as fast as the more expensive /91 for programs that 
didn't use a lot of floating point.  The /85 was microcoded and could 
emulate a 7094.

   All the S/360s from the 65 and down had microcoded emulators for
> 2nd generation IBM architectures such as the 1400.  The emulators usually
> ran faster than the real hardware.

They always ran faster, company policy.  That's why you needed a 360/65 to 
emulate a 7094.

> The emulators were there to accommodate data centers that had lost their
> source code for mission critical applications.

Partly that, more that the emulators allowed the customers to spread the 
conversion work out partly before they got the new machine, partly after. 
Remember that the new machine was faster and had better peripherals. 
Since it was built with more modern components it may well have been 
cheaper to rent.

> Knuth was right about multiple levels of emulation/interpretaton.

The 5100 was indeed a marvel, but it was a very slow one.

Regards,
John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

[COFF] Re: machine code translation,as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 2368 bytes --]

On Fri, Jul 12, 2024 at 12:23 PM John R Levine <johnl@taugh.com> wrote:

> On Thu, 11 Jul 2024, Paul Winalski wrote:
> > Yes, that was precisely my point, and thank you for stating it more
> clearly
> > and concisely than I did.  The VAX MACRO compiler takes in VAX assembly
> > source code, not binary VAX instructions.
>
> Does anyone know how extensively they used the macro facilities?  You can
> write much higher level stuff as macros than as single instructions, which
> makes it a lot easier to do efficient translation.  For example, on OS/360
> you'd write a GET macro to retrieve the next record (or a pointer to it)
> from a file, which was a lot easier to figure out than the control blocks
> and subroutine calls the macros expanded into.
>
> Macros were used very extensively in VAX MACRO, both for user programming
and in the operating system.  All of the low-level system calls for user
programs were implemented and documented as macros.  The OS assembly code
made heavy use of macros as well.

Outside the VAX/VMS development group, BLISS was DEC's standard
implementation language.  In the development organizations I worked in
(software development tools and compilers), we did almost zero programming
in assembly code.  The only assembly-level programming I ever did was the
innermost loop of my VAX/VMS Mandelbrot set  program and a device driver to
implement Unix-style pipes on VMS.  I did the latter in assembly code only
because there was no documented HLL interface for writing VAX/VMS device
drivers.

As Clem Cole observed, Dave Cutler hated BLISS with a passion.  All of his
work on the VAX/VMS operating system was in assembly code.  Because of this
there is still a large amount of OpenVMS still written in VAX MACRO.  But
the RMS (Record Management Services, the VMS file system user interface)
group wrote all of their code in BLISS.

After leaving the VMS group Dave went on to design and implement a common
compiler back end for VAX, the VAX Code Generator (VCG), which was the
optimizer and back end for VAX PL/I, VAX C, and VAX Ada.  The earlier
generations of VAX/VMS compilers (VAX Fortran, VAX Pascal, VAX COBOL) each
had their own back ends.  Cutler later did get HLL religion.  His real-time
operating system for the VAX, VAXeln, was written almost entirely in Pascal.

-Paul W.

[-- Attachment #2: Type: text/html, Size: 2758 bytes --]

[COFF] Re: [TUHS] Re: history of machine code translation,as mental architecture models

[John R Levine]

On Wed, 10 Jul 2024, Dan Cross wrote:
>< [[ Knuth described instruction by instruction simulation ]]
> Sure.  But we're trying to date the topic here; my point is that
> JITing was well known, and simulation was similarly well known; we
> know when work on those books started; it doesn't seem that odd to me
> that combining the two would be known around that time as well.

Simulators that interpreted a pseudo-machine code date from the early 
1950s.  They were popular before compilers existed.  Fortran and 
Flow-matic arrived in 1954, translating languages that were somewhat 
familiar to people into machine code, mathematical formulas for Fortran, 
business English for Flow-matic.  But it would be rather a stretch to 
assert that anyone was doing machine code translation in 1954.  It was a 
while until enough machine code existed for it to be worth thinking about, 
and the jump from tranlating from human-ish languages to translating from 
machine languages was likely far less obvious at the time than it is in 
retrosoect.

On the other hand, there was UNCOL.  In the 1950s people quckly realized 
that building a compiler for every language for every machine would be a 
lot of work, particularly in an era when there was such extreme variation 
in machine architecture.  So someone came up with UNCOL which was supposed 
to be a UNiversal Computer Oriented Language, or what we would call a 
universal intermediate language.  All of the compilers would translate the 
original language to UNCOL, and there wold be an UNCOL to machine language 
translator for each machine.  This reduced the NxM compiler problem to N+M 
if it worked, which of course it didn't because the languages and the 
machines were both too varied.  But UNCOL to machine code is in the same 
ballpark as machine to machine.

CACM articles on UNCOL here:

https://dl.acm.org/doi/10.1145/368892.368915

https://dl.acm.org/doi/10.1145/368919.3165711

I would still love to hear if there are any actual reports or papers or 
anecdotes of people doing machine code translation before the 1967 one 
that started this discussion.

R's,
John

[COFF] Re: machine code translation,as mental architecture models

[John R Levine]

On Fri, 12 Jul 2024, Paul Winalski wrote:
>> Macros were used very extensively in VAX MACRO, both for user programming
> and in the operating system.  All of the low-level system calls for user
> programs were implemented and documented as macros.  The OS assembly code
> made heavy use of macros as well.

Oh, no wonder the translator worked so well.

> Outside the VAX/VMS development group, BLISS was DEC's standard
> implementation language.  In the development organizations I worked in
> (software development tools and compilers), we did almost zero programming
> in assembly code. ...

When I was at Yale we did a fair amount of programming in BLISS-36 which 
was a pretty nice language once you wrapped your brain around some of its 
quirks like needing a dot for every memory reference.

Our Vaxes ran Unix so it was all C other than a few things like tracking 
down a bug in the 11/750's microcode that broke an instruction in the 
inner loop of printf().  I had managed to get a cross-compiling 
environment working an a PDP-11 but Bill Joy found the bug at the same 
time so we used his patched version.

Regards,
John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

[COFF] Re: machine code translation,as mental architecture models

[Stuff Received]

On 2024-07-12 12:51, Paul Winalski wrote (in part):
> 
> Macros were used very extensively in VAX MACRO, both for user 
> programming and in the operating system.  All of the low-level system 
> calls for user programs were implemented and documented as macros.  The 
> OS assembly code made heavy use of macros as well.

I recall one place that used macros that were hundreds of lines long. 
They were a nightmare to maintain as they were so rigid -- not the right 
way write macros.

S.

[COFF] Re: machine code translation,as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]

On Fri, Jul 12, 2024 at 1:03 PM Stuff Received <stuff@riddermarkfarm.ca>
wrote:

>
> I recall one place that used macros that were hundreds of lines long.
> They were a nightmare to maintain as they were so rigid -- not the right
> way write macros.
>
> DEC BLISS has the most powerful macro facility that I've ever seen in any
programming language.  Ward Clark of DEC's Development Methods and Software
Tools Group wrote a set of macros called "Portable BLISS"  that provided a
uniform means to access basic operating system services such as sequential
file I/O and heap memory management.  There were Portable BLISS
implementations for much of the vast zoo of DEC operating systems (RSX-11,
RSTS, RT-11, TOPS-10, TOPS-20, VMS).  Some of the Portable BLISS macros
were over a thousand lines long.  They were all obscure and hard to
maintain.

Unix users will be familiar with the Obfuscated C Contest.  Within DEC
Engineering there was an Obfuscated BLISS Contest one year.  The winner was
Stan Rabinowitz, who used macros to produce a text that looked nothing like
BLISS syntax.  Stan said it would compile cleanly except for one semantic
error and challenged us to figure out what that was.  It turned out to be
an illegal uplevel reference.

-Paul W.

[-- Attachment #2: Type: text/html, Size: 1619 bytes --]

[COFF] Re: machine code translation,as mental architecture models

[Aron Insinga]

Outside of product development, the internal DECSIM multi-level logic 
simulator and its CLI and compilers were also written in BLISS.  Most of 
these started out in BLISS-36 but they were all brought together with a 
fault simulation kernel on the VAX to take advantage of the larger 
address space.

Some other internal VLSI CAD tools were also written in BLISS, although 
a few were written in more common languages.  The first (AFAIK) 
retargetable microcode assembler at DEC, MICRO, was written in MACRO-10; 
sources are on the net.  The next generation MICRO2 was written in BLISS 
and AFAIK the binary is available but the sources have sadly been lost.  
<Insert obvious plea here.>

Re: Paul's comment about the portable I/O library in BLISS: When I first 
encountered C, I used the Unix system calls to do I/O.  The "standard 
I/O" library was a very important addition to C's usability.  Similarly, 
when I first encountered BLISS-10, I had to type in a package from the 
DECUS documentation to call TOPS-10 to do I/O.  Getting the "XPORT" I/O 
(etc.) library for Common BLISS was invaluable for the CAD tools.

People have repeatedly omitted I/O from the initial design of a language 
and relegated it to a library, but even if the language is intended to 
be used for a kernel, people learning the language ("Hello, world!") or 
using the language above the kernel for utilities will need to do I/O, 
so this library should be included in the language design from the very 
beginning.  I think this lesson has been learned by now.  Its fallout is 
recognized in Stroustrup's quote of a Bell Labs axiom, "Library design 
is language design, and vice versa."

- Aron


On 7/12/24 12:51, Paul Winalski wrote:
> Outside the VAX/VMS development group, BLISS was DEC's standard 
> implementation language.  In the development organizations I worked in 
> (software development tools and compilers), we did almost zero 
> programming in assembly code.

[COFF] Re: machine code translation,as mental architecture models

[John Levine]

It appears that Paul Winalski <paul.winalski@gmail.com> said:
>> DEC BLISS has the most powerful macro facility that I've ever seen in any
>programming language. ...

Take a look at PL/I. Its preprocessor lets you use a large subset of
PL/I including if, goto, and do loops and most of the string and
arithmetic operators to write functions that run at compile time and
put their results into the source file.

And don't forget the IBM macro assembler. In macros along with all of
the usual loops and local and global variables and subscripted
parameters and conditional expressions and string processing, it had
the AREAD op which read the next line from the source file and put it
in a macro-time variable, thereby letting the macro define its own
syntax from scratch if you wanted. There was also PUNCH to write a
line directly to the object file.

[COFF] Re: machine code translation,as mental architecture models

[Dave Horsfall]

On Sat, 12 Jul 2024, John R Levine wrote:

> Our Vaxes ran Unix so it was all C other than a few things like tracking 
> down a bug in the 11/750's microcode that broke an instruction in the 
> inner loop of printf().  [...]

Do tell...

-- Dave

[COFF] Re: machine code translation,as mental architecture models

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

On Fri, 12 Jul 2024, Aron Insinga wrote:

> Re: Paul's comment about the portable I/O library in BLISS: When I first 
> encountered C, I used the Unix system calls to do I/O.  The "standard 
> I/O" library was a very important addition to C's usability.  [...]

As was the "Portable I/O Library" preceding it.

-- Dave

[COFF] Re: machine code translation,as mental architecture models

[John Levine]

It appears that Dave Horsfall <dave@horsfall.org> said:
>On Sat, 12 Jul 2024, John R Levine wrote:
>
>> Our Vaxes ran Unix so it was all C other than a few things like tracking 
>> down a bug in the 11/750's microcode that broke an instruction in the 
>> inner loop of printf().  [...]
>
>Do tell...

The details are a litle dim after 45 years, but there was a MOVTUC
instruction in the inner loop of printf that scanned for the null at
the end of the string.  The /750 had a microcode bug that didn't
matter for the way DEC's software used it but broke the libc and
I think also the kernel version.  MOVTUC sets six registers and
we probably used one they didn't.

Bill replaced it with a few simpler instructions and the comment

	; Comet sucks

R's,
John

PS: For you young folks, Comet was DEC's internal project name for the /750.

[COFF] Re: machine code translation,as mental architecture models

[Aron Insinga]

I think my work was in the summer of 1979 or possibly 1980, before the 
FRS in Oct. 1980.  People were happy because mass production of the 
ucode ROMs hadn't started yet.  I was using one of the 2~4 11/750 
prototypes which were on concrete blocks in a lab, and the 11/730 
breadboard was standing up on a table (no box) with its boards fanned 
out.  And it was during the gasoline shortage.  Maybe I'll find 
something about it later.

- Aron


On 7/12/24 18:03, John Levine wrote:
> It appears that Dave Horsfall <dave@horsfall.org> said:
>> On Sat, 12 Jul 2024, John R Levine wrote:
>>
>>> Our Vaxes ran Unix so it was all C other than a few things like tracking
>>> down a bug in the 11/750's microcode that broke an instruction in the
>>> inner loop of printf().  [...]
>> Do tell...
> The details are a litle dim after 45 years, but there was a MOVTUC
> instruction in the inner loop of printf that scanned for the null at
> the end of the string.  The /750 had a microcode bug that didn't
> matter for the way DEC's software used it but broke the libc and
> I think also the kernel version.  MOVTUC sets six registers and
> we probably used one they didn't.
>
> Bill replaced it with a few simpler instructions and the comment
>
> 	; Comet sucks
>
> R's,
> John
>
> PS: For you young folks, Comet was DEC's internal project name for the /750.

[COFF] Re: machine code translation,as mental architecture models

[Dave Horsfall]

On Sat, 12 Jul 2024, John Levine wrote:

> The details are a litle dim after 45 years, but there was a MOVTUC 
> instruction in the inner loop of printf that scanned for the null at the 
> end of the string.  The /750 had a microcode bug that didn't matter for 
> the way DEC's software used it but broke the libc and I think also the 
> kernel version.  MOVTUC sets six registers and we probably used one they 
> didn't.

I am reminded of the time when overlapped seeks on the RK-11 were 
implemented in Unix, per the Peripherals Handbook.

They didn't work.

DEC responded with "Well, you were running Unix!"...

We then ran "DECEX" (their own diagnostic tool), which indeed failed
on overlapped seekson the RK-11.

It turned out that DEC never used overlapped seeks in their own stuff...

-- Dave

[COFF] Re: machine code translation,as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On Fri, Jul 12, 2024 at 6:10 PM John Levine <johnl@taugh.com> wrote:

>
> PS: For you young folks, Comet was DEC's internal project name for the
> /750.
>

The first generation of VAXen were the 11-780 (codenamed Star), the 11/750
(Comet) and the 11/730 (Nebula).

The 11/780's successor was to be implemented in ECL and was codenamed
Venus.  Along with Venus were a two-board VAX implementation (Gemini) and a
one-board VAX (Scorpio).  Gemini was cancelled.  Venus had lots of schedule
slippage but was eventually released, but under the new 4-digit VAX
branding system (it was originally slated to be called the 11-790).
Scorpio became the VAX 8000.

-Paul W

[-- Attachment #2: Type: text/html, Size: 1003 bytes --]

[COFF] Re: machine code translation,as mental architecture models

[John Levine]

According to Dave Horsfall <dave@horsfall.org>:
>I am reminded of the time when overlapped seeks on the RK-11 were 
>implemented in Unix, per the Peripherals Handbook.
>
>They didn't work.

Yup, I recall looking at the disk drivers which had a comment saying
that overlapped seeks on RK disks didn't work.

At Yale our PDP-11 had a pair of 2314-style RP drives. Its controller
did work and I stayed up all night one weekend debugging a driver that
queued and sorted requests for the two drives separately and did
separate seeks. It made a lot of difference, something like 30% more
transfers/minute.

-- 
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

[COFF] Re: machine code translation, as mental architecture models

[Ralph Corderoy]

Hi,

John Levine wrote:
> Take a look at PL/I.  Its preprocessor lets you use a large subset of
> PL/I including if, goto, and do loops and most of the string and
> arithmetic operators to write functions that run at compile time and
> put their results into the source file.

Another approach is for the language to offer evaluation at compile
time.

- FORTH had COMPILE and IMMEDIATE to switch modes; there's POSTPONE as
  well now.

- Zig has comptime.
  https://ziglang.org/documentation/master/#comptime
  https://ziglang.org/documentation/master/#Case-Study-print-in-Zig

-- 
Cheers, Ralph.

[COFF] Re: machine code translation, as mental architecture models

[Ralph Corderoy]

Hi,

John and Paul wrote:
> > > > The VAX MACRO compiler takes in VAX assembly source code, not
> > > > binary VAX instructions.
> > >
> > > Does anyone know how extensively they used the macro facilities?
> > > You can write much higher level stuff as macros than as single
> > > instructions, which makes it a lot easier to do efficient
> > > translation.
> >
> > Macros were used very extensively in VAX MACRO, both for user
> > programming and in the operating system.  All of the low-level
> > system calls for user programs were implemented and documented as
> > macros.  The OS assembly code made heavy use of macros as well.
>
> Oh, no wonder the translator worked so well.

Well, doesn't it depend on whether VAX MACRO kept the macros as
high-level entities when translating them, or if it processed macros in
the familiar way into instructions that sat at the same level as
hand-written ‘assembler’.  I don't think this thread has made that clear
so far.

-- 
Cheers, Ralph.

[COFF] Re: machine code translation, as mental architecture models

[Dan Cross]

On Sat, Jul 13, 2024 at 5:57 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> John and Paul wrote:
> > > > > The VAX MACRO compiler takes in VAX assembly source code, not
> > > > > binary VAX instructions.
> > > >
> > > > Does anyone know how extensively they used the macro facilities?
> > > > You can write much higher level stuff as macros than as single
> > > > instructions, which makes it a lot easier to do efficient
> > > > translation.
> > >
> > > Macros were used very extensively in VAX MACRO, both for user
> > > programming and in the operating system.  All of the low-level
> > > system calls for user programs were implemented and documented as
> > > macros.  The OS assembly code made heavy use of macros as well.
> >
> > Oh, no wonder the translator worked so well.
>
> Well, doesn't it depend on whether VAX MACRO kept the macros as
> high-level entities when translating them, or if it processed macros in
> the familiar way into instructions that sat at the same level as
> hand-written ‘assembler’.  I don't think this thread has made that clear
> so far.

The non-VAX compilers for Macro-32 do expansion.  That is, they don't
just recognize macros and treat them as intrinsics or primitives that
are specially optimized. Macro is not C, where you can treat "library"
functions defined in the standard specially.  John Reagan, of VSI, has
posted about XMACRO (the Macro compiler for x86) in various places.
For instance, here:
https://comp.os.vms.narkive.com/F05qmMaD/x86-cross-tools-kit#post4

This post, and others, suggests to me that XMACRO is definitely doing
macro expansion before compilation proper.

        - Dan C.

[COFF] Re: machine code translation, as mental architecture models

[Aron Insinga]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

On 7/13/24 10:25, Dan Cross wrote:
> On Sat, Jul 13, 2024 at 5:57 AM Ralph Corderoy<ralph@inputplus.co.uk>  wrote:
>> John and Paul wrote:
>> Well, doesn't it depend on whether VAX MACRO kept the macros as
>> high-level entities when translating them, or if it processed macros in
>> the familiar way into instructions that sat at the same level as
>> hand-written ‘assembler’.  I don't think this thread has made that clear
>> so far.

The DEC assemblers [sic] for the PDP-11 and VAX-11 did macro expansion 
as text substitution.

AA-V027A-TC PDP-11 MACRO-11 Language Reference Manual
Section 7.1 (PDF file p 115, original document p 7-1) says
"Macro expansion is the insertion of the macro source lines into the 
main program."
Note: This manual also discusses concatenation with macro arguments and 
(on p 120 or p 7-6) macros that define other macros.

AA-D032B-TE VAX-11 MACRO Language Reference Manual
The beginning of chapter 6 (p 119 or p 6-1) says the same thing.

At least in those assemblers, macros were not inline procedures.

Google can find manuals for other assemblers.  For even earlier history, see
https://github.com/PDP-10/its/issues/2032
http://www.bitsavers.org/pdf/mit/rle_pdp1/memos/PDP-1_MIDAS.pdf

[-- Attachment #2: Type: text/html, Size: 2206 bytes --]

[COFF] Re: machine code translation, as mental architecture models

[John Levine]

According to Ralph Corderoy <ralph@inputplus.co.uk>:
>> Oh, no wonder the translator worked so well.
>
>Well, doesn't it depend on whether VAX MACRO kept the macros as
>high-level entities when translating them, or if it processed macros in
>the familiar way into instructions that sat at the same level as
>hand-written ‘assembler’.  I don't think this thread has made that clear
>so far.

It was a macro assembler. The macros generated assembler statements
that got assembled the normal way. I agree with the person that every
macro assembler I've ever seen did that. A semi-exception is the IBM
assembler that also had a PUNCH statement that put records into the
object file but I think that was only used to pass commands to the
linker.

The more relevant question is how they used the macros. If the macros
were used consistently for semantically higher level things, the
translator can use the semantics of the macros when translating.

-- 
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

[COFF] Re: machine code translation, as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 943 bytes --]

On Sat, Jul 13, 2024 at 2:50 PM John Levine <johnl@taugh.com> wrote:

> The more relevant question is how they used the macros. If the macros
> were used consistently for semantically higher level things, the
> translator can use the semantics of the macros when translating.
>
> There is an assembly macro for each of the user-mode VMS system services
These are wrappers around the RTL subroutines.  So, for example, the $QIO
(queue I/O) macro is a wrapper around a call to the SYS$QIO runtime library
routine.  There is a similar set of macros for the RMS (record management
services--the VMS file I/O system) data structures and routines.

The VAX MACRO compiler could perhaps use the semantics of the macros when
translating, but it doesn't.  It has to support semantics-free macro
expansion since users can write their own macros, so there is little point
in building semantic knowledge into the translator.

-Paul W.

[-- Attachment #2: Type: text/html, Size: 1257 bytes --]

[COFF] Re: machine code translation, as mental architecture models

[Dan Cross]

On Sat, Jul 13, 2024 at 2:50 PM John Levine <johnl@taugh.com> wrote:
> According to Ralph Corderoy <ralph@inputplus.co.uk>:
> >> Oh, no wonder the translator worked so well.
> >
> >Well, doesn't it depend on whether VAX MACRO kept the macros as
> >high-level entities when translating them, or if it processed macros in
> >the familiar way into instructions that sat at the same level as
> >hand-written ‘assembler’.  I don't think this thread has made that clear
> >so far.
>
> It was a macro assembler. The macros generated assembler statements
> that got assembled the normal way. I agree with the person that every
> macro assembler I've ever seen did that. A semi-exception is the IBM
> assembler that also had a PUNCH statement that put records into the
> object file but I think that was only used to pass commands to the
> linker.

This is conflating two different things over the life of the MACRO-32
language.  It certainly started out as a macro assembler, but with the
introduction of the Alpha, DEC turned it into a true compiler (where
the source language happens to be VAX assembly language) that
generated native code for whatever machine it was compiled for: VAX or
Alpha; using the GEM backends.

When Compaq started the port to Itanium, they employed the same
technique; this was maintained by HP.

VSI has shifted to using LLVM as their compiler backend for the x86_64
port, with ELF as the executable file format (and presumably for
object files as well).  As I understand the current state of affairs,
there is a GEM to LLVM layer that interfaces between the still-GEM
output of compiler frontends and the LLVM backend, that performs
native code generation.  Macro, in particular, bypasses most of the
LLVM optimization layer.

> The more relevant question is how they used the macros. If the macros
> were used consistently for semantically higher level things, the
> translator can use the semantics of the macros when translating.

As has been mentioned, they don't do this.  The output of fully
expanded MACRO-32 is basically VAX assembly language, which is then
compiled in a manner similar to how one would compile preprocessed C.

        - Dan C.

[COFF] Re: [TUHS] Re: history of machine code translation,as mental architecture models

[Dan Cross]

On Sat, Jul 13, 2024 at 1:35 PM John R Levine <johnl@taugh.com> wrote:
> On Sat, 13 Jul 2024, Dan Cross wrote:
> > Honeywell was doing it with their "Liberator" software on the
> > Honeywell 200 computer in, at least, 1966:
> > https://bitsavers.org/pdf/honeywell/series200/charlie_gibbs/012_Series_200_Summary_Description.pdf
> > (See the section on, "Conversion Compatibility."). Given that that
> > document was published in February of 1966, it stands to reason work
> > started on that earlier, in at least 1965 if not before ...
>
> Good thought.  Now that you mention it, I recall that there were a lot of
> Autocoder to X translators, where X was anything from another machine
> to Cobol.  Of course I can't find any of them now but they must have been
> around the same time.
>
> R's,
> John
>
> PS: For you young folks, Autocoder was the IBM 1401 assembler.  There were
> other Autocoders but that was by far the most popular because the 1401 was
> the most popular computer of the late 1950s.

Oops, it appears that I inadvertently forgot to Cc: COFF in my earlier
reply to John.  Mea culpa.
For context, here's my complete earlier message; the TL;DR is that
Honeywell was doing binary translation from the 1401 to the H-200
sometime in 1965 or earlier; possibly as early as 1963, according to
some sources.

-->BEGIN<--
Honeywell was doing it with their "Liberator" software on the
Honeywell 200 computer in, at least, 1966:
https://bitsavers.org/pdf/honeywell/series200/charlie_gibbs/012_Series_200_Summary_Description.pdf
(See the section on, "Conversion Compatibility."). Given that that
document was published in February of 1966, it stands to reason work
started on that earlier, in at least 1965 if not before (how much
earlier is unclear). According to Wikipedia, that machine was
introduced in late 1963; it's unclear whether the Liberator software
was released at the same time, however.  Ease of translation of IBM
1401 instructions appears to have been a design goal.  At least some
sources suggest that Liberator shipped with the H-200 in 1963
(https://ibm-1401.info/1401-Competition.html#UsingLib).

It seemed like what Doug was describing earlier was still
source->binary translation, using some clever macro packages.
-->END<--

        - Dan C.

[COFF] Re: machine code translation, as mental architecture models

[Aron Insinga]

Back to topic of machine code translation, I stumbled over this paper 
describing DEC's migration from VAX (VMS) or MIPS (Ultrix) to Alpha (VMS 
or OSF/1, respectively) for user-mode applications.  The paper states 
that this included hand-coded assembly language applications.

https://web.stanford.edu/class/cs343/resources/binary-translation.pdf

[COFF] Re: machine code translation, as mental architecture models

[Aron Insinga]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On 7/13/24 15:42, Dan Cross wrote:

> The output of fully expanded MACRO-32 is basically VAX assembly language, which is then
> compiled in a manner similar to how one would compile preprocessed C.
>
>          - Dan C.

Although, a minor point, the macro definition & expansion, conditional 
assembly, etc. are all done on the fly as the assembly language source 
file is read.  There's no separate macro processing pass like on Unix 
when, if the first character of the .c file was '#', cpp read the .c 
file and generated a .i file to pass to the compiler proper (which let 
cpp be used on Pascal, assembler, and who knows what else).

- Aron

[-- Attachment #2: Type: text/html, Size: 1155 bytes --]

[COFF] Re: machine code translation, as mental architecture models

[Paul Winalski]

[-- Attachment #1: Type: text/plain, Size: 2868 bytes --]

On Sat, Jul 13, 2024 at 4:00 PM Dan Cross <crossd@gmail.com> wrote:

>
> This is conflating two different things over the life of the MACRO-32
> language.  It certainly started out as a macro assembler, but with the
> introduction of the Alpha, DEC turned it into a true compiler (where
> the source language happens to be VAX assembly language) that
> generated native code for whatever machine it was compiled for: VAX or
> Alpha; using the GEM backends.
>
> The confusion arises because the name VAX MACRO was used for two
completely independent and very different software tools.

The first VAX MACRO was the conventional assembler for VAX/VMS, developed
in the mid-1970s.  It had conventional macroinstruction capabilities, hence
the name.

An inconveniently large amount of the VAX/VMS operating system code was
written in VAX assembly language.  Dave Cutler didn't believe in using
higher level languages for system code, and he despised BLISS (DEC's
standard HLL language for software product development) in particular.  To
migrate VMS to Alpha it was decided that the quickest and least error-prone
path was to write a compiler front end that takes VAX MACRO syntax and
generates expanded intermediate language (EIL) for the GEM compiler back
end.  GEM then generates the Alpha code and writes it into an object file.
This work was done circa 1990.  The name "VAX MACRO" was used for this
compiler as well as for the previous assembler.


> When Compaq started the port to Itanium, they employed the same
> technique; this was maintained by HP.
>
> No further work needed to be done in the VAX MACRO compiler front end to
support Itanium.  All that needed to be done was to add Itanium code
generation capability to the GEM back end.  After that all GEM-based
compilers acquired Itanium code generation capability.  That is the big
advantage of having a common back end for all compilers.  Over its lifetime
GEM could generate object files for the MIPS, Alpha, and IA-32 machine
architectures and for the VMS, Ultrix, OSF-1, and Windows operating
systems.

VSI has shifted to using LLVM as their compiler backend for the x86_64
> port, with ELF as the executable file format (and presumably for
> object files as well).  As I understand the current state of affairs,
> there is a GEM to LLVM layer that interfaces between the still-GEM
> output of compiler frontends and the LLVM backend, that performs
> native code generation.  Macro, in particular, bypasses most of the
> LLVM optimization layer.
>
> VAX MACRO also bypassed most of GEM's optimization layer as well.  And
yes, most (if not all) of the legacy OpenVMS compilers use a GEM-to-LLVM IL
translator.  VSI has more than enough on their plate without taking on a
wholesale rewrite of the IL generators in the compiler front ends.

-Paul W.

[-- Attachment #2: Type: text/html, Size: 3568 bytes --]