[TUHS] First supercomputer UNIX: The Denelcor HEP (was Early multiprocessor Unix)

["Ron Natalie" <ron@ronnatalie.com>]

In 1982, I returned to BRL (having spent a year working for Martin 
Marietta on  project involving a dual process RSX-11M system.    I did 
spend some time setting up the PWB UNIX tools with the QA guys while 
there).

BRL had commissioned the Denver Electronics Corporation (i.e., Denelcor) 
to build a MIMD system called the HEP, ostensibly to do fluid dynamics 
problems.   It was looking like the manufacturer wasn’t going to have a 
useful OS for it so Mike Muss did his usual thing and suggested he could 
put UNIX on it (he had done this twice previously with orphaned 
machines:  some 11/34 graphics stations that were supposed to be 
connected to the CDC7600 and converting the University of Illinois 
Arpanet Terminal Server (ANTS) to a host that could support long 
leaders.

I was part of Mike’s team.   We split up various aspects of the project. 
   Mike ported PCC.   I ported ld, and F77 (as we figured the 
“scientists” would probably want that).    Mike did the core of the 
kernel.  I handled all the I/O system because I was the “device driver” 
guy at JHU and continued on with this at BRL.     Our initial work was 
done by traveling out to Denelcor’s facility where our machine was 
assembled on their floor.   They had started their operating system work 
(not UNIX), so we took the machine after hours.   Amusingly, at BRL in 
the room the HEP was going in, the cabinet layout was taped out on the 
floor around a central pillar.   The tape marked with the component that 
went there.   Out in the Denver machine room, the equipment was there 
but there was a square taped out that said “BRL post” so they knew not 
to put/route anything there.

The software being ported was 4BSD.    I believe we started with the 
George Gobels mods of the dual VAX (BRL already had built one of these). 
   In working on that I found there was a rather serious portability 
problem.    The kernel did what I refer to as “conversion by union” 
where they stuck a pointer of one type into one element of the union and 
retrieved it by a different pointer type union element.   Well, such is 
all well and good on the VAX, but the HEP encoded the partial word size 
into the low order bits of the pointer.   This meant that you ended up 
with pointers that would access the wrong size operand.    I spent a few 
days chasing down all these things and fixing them.

The HEP had an 11/34 as a control processor that allowed you to 
configure the switch system that interconnected the memory in the 
processors.    It did this via a special feature called (aptly) “The Low 
Speed Bus” (more on that later).   But once running, everything was self 
contained on the HEP hardware itself.    Other amusing things about the 
HEP is it had a lot of registers.   It also had this stuff called “table 
memory” which essentially were read-only registers.  You could but a 
variety of constants there (we had all the integers from 0-255 and then 
powers of two after that).    The rest of the memory had an extra bit 
per 64-bit word called the “full empty” bit.   While the thing was 
capable of running the equivalent of 32 UNIX processes simultaneously, 
it also had the ability within each of those to hardware schedule 
threads.  Many instructions had features that used the extra bit like 
“WAIT FULL, READ, SET EMPTY” or “WRITE, SET FULL”.

The whole thing was built out of Motorola 10800 ECL.   All hand 
assembled.   Quite impressive.

After we got the system up, and Mike tried to port his ray tracing 
software to it, we found the I/O system was beastly slow.  The I/O 
system was a cute device.  It had a bunch traditional memory (no 
full/empty bits) connected to the main memory via the switch but also to 
32 individual UNIBUSes.   The only problem was that in order to whack 
the registers on the UNIBUS you had to do it through the “Low Speed 
Bus.”    This was the performance bottleneck.    The original designer 
of this thing, Burton Smith and I sat down at the Golden Corral 
Steakhouse in Aberdeen and worked on a new plan.   The key was to allow 
the kernel tasks to talk directly to the UNIBUS registers via the 
regular memory switch.    Fortunately, we had an extra switch node in 
spare parts.    We did however need software and another PDP-11 to 
control the thing.   Fortunately, I had spares from my building of the 
early internet router: BRL-GATEWAY.   I also used the little OS on it 
(LOS) to write the control software.   In a couple of days, we had 
rebuilt the I/O system out of essentially spare parts and it flew.

Amusingly, this was the first of two UNIX ports I worked on that didn’t 
use the standard ISR thing that most of the kernels those days did.    
After the device driver on the HEP had started the IO up, that thread 
just ended.    When the UNIBUS interrupt came in, a new thread was 
created to service it.    I later did a similar strategy with the 
message passing coprocessor on the Multibus II.

This was a fun waste of about a year.    By the time we got this ready 
for prime time, the Army had decided they didn’t have any supercomputers 
(other than HEP) and immediately used defense priority to snag the next 
Cray X/MP off the assembly line (which had been slated for Apple).    
Mike got one run of his raytracer to make a movie:  The artillery 
shell’s view of collision with a tank.   The HEP could generate the 
frames about 60:1 (took a minute to make one second of movie time) 
faster than anything else we had in the day.    Of course, it pales 
compared to your modern video games.   After that, the HEP was 
dismantled (Denelcor had gone out of business) to make room for the 
Cray.   I recently just found pictures of us all sitting on the “seats” 
of the Cray and one of me peering out from the core of the machine.

The Army went on for plans for multiple Cray 2 purchases.   After a lot 
of amusing “systems engineering” work that originally slated for all of 
them to go in Oak Ridge Tennessee, and then all of them to go at BRL, 
they decided to spread them around the country (what I told them they 
should do in the first place).    My last months at BRL, I signed my 
name to the $25 million dollar Cray 2 procurement, but it got delivered 
after I left (I visited it a few years later).

Somewhere I still have the name plates for the Denelcor HEP H1000.   
They, in fact, were never actually installed on the machine,