Re: [TUHS] Looking for final C compiler by Dennis Ritchie

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Paul Ruizendaal]

> Thank you for the info - I will certainly look at the USENIX tapes.
> 
> I will try to port the C compiler to amd64 - while preserving as much of
> the original code as I can. But not sure if this is even feasible.
> 
> Thanks and Regards
> Dibyendu

If that is your goal, you might want to start with the version included with 2.11BSD. It is essentially the same as the version from V7, but with 15 more years of bug fixes. I used that source to port V6 Unix to the TI990 architecture back in 2014/2015 and the good thing about it is that it still compiles with a modern gcc.

For your project, I think you would be able to use the first pass ‘c0’ almost unchanged. The second pass ‘c1’ would need major restructuring. It mainly builds a tree for each expression and then performs various transformations, many of which are PDP11 specific (but also portable ones, like handling of constant expressions). It then covers the tree with code fragments selected from a library. This library (‘optable') would need a full rewrite as well. The last pass ‘c2’ is the optimiser and is also highly PDP11 specific. It reads the assembler output of ‘c1’ function by function, building an instruction list. It then performs some portable optimisations (eliminating unnecessary jumps, etc.) and also more PDP11 specific optimisations (the most complex being removing redundant register loads - the concept of which would be reusable).

There are about 12,000 lines of code and as a rough guess I would say that some 40% needs rewriting. A new code fragment library would probably be some 2 to 3 thousand lines.

I recall reading about a project to revive the Ritchie C compiler one or two years ago, but a quick web search came up dry. Anybody else remember reading that?

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

On 20 July 2018 at 11:04, Paul Ruizendaal <pnr@planet.nl> wrote:
>> I will try to port the C compiler to amd64 - while preserving as much of
>> the original code as I can. But not sure if this is even feasible.
>>
> If that is your goal, you might want to start with the version included with 2.11BSD. It is essentially the same as the version from V7, but with 15 more years of bug fixes. I used that source to port V6 Unix to the TI990 architecture back in 2014/2015 and the good thing about it is that it still compiles with a modern gcc.
>

Thank you for the tip - very helpful.


> For your project, I think you would be able to use the first pass ‘c0’ almost unchanged. The second pass ‘c1’ would need major restructuring. It mainly builds a tree for each expression and then performs various transformations, many of which are PDP11 specific (but also portable ones, like handling of constant expressions). It then covers the tree with code fragments selected from a library. This library (‘optable') would need a full rewrite as well. The last pass ‘c2’ is the optimiser and is also highly PDP11 specific. It reads the assembler output of ‘c1’ function by function, building an instruction list. It then performs some portable optimisations (eliminating unnecessary jumps, etc.) and also more PDP11 specific optimisations (the most complex being removing redundant register loads - the concept of which would be reusable).
>
> There are about 12,000 lines of code and as a rough guess I would say that some 40% needs rewriting. A new code fragment library would probably be some 2 to 3 thousand lines.
>

Thank you again for all this info.

> I recall reading about a project to revive the Ritchie C compiler one or two years ago, but a quick web search came up dry. Anybody else remember reading that?
>

I thought that the revival only meant being able to run code in PDP11 emulator?

Thanks and Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

Hi Paul,

On 20 July 2018 at 11:04, Paul Ruizendaal <pnr@planet.nl> wrote:
>> I will try to port the C compiler to amd64 - while preserving as much of
>> the original code as I can. But not sure if this is even feasible.
>
> If that is your goal, you might want to start with the version included with 2.11BSD. It is essentially the same as the version from V7, but with 15 more years of bug fixes. I used that source to port V6 Unix to the TI990 architecture back in 2014/2015 and the good thing about it is that it still compiles with a modern gcc.

I found similarities between the 2.11BSD and 10th edition sources -
presumably 2.11BSD was based on (or incorporated) changes in the 10th
edition?

BTW did you publish your work?

> For your project, I think you would be able to use the first pass ‘c0’ almost unchanged.

I have been able to get c0 working. I had to work around the use of
sbrk() as this is not supported on Windows. I also changed the output
to text format - mainly so that I can see what is being generated.

> The second pass ‘c1’ would need major restructuring. It mainly builds a tree for each expression and then performs various transformations, many of which are PDP11 specific (but also portable ones, like handling of constant expressions). It then covers the tree with code fragments selected from a library. This library (‘optable') would need a full rewrite as well.

This part will be more difficult I suspect - but I will only start on
this after I have converted c0 to output 64-bit compliant intermediate
code.

It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
Compiler' which is quite useful in understanding the compiler. I have
converted the doc to Markdown format as well which makes it easier to
read.

Thanks and Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Paul Ruizendaal]

> On 22 Jul 2018, at 22:49, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> 
>>> I will try to port the C compiler to amd64 - while preserving as much of
>>> the original code as I can. But not sure if this is even feasible.
>> 
>> If that is your goal, you might want to start with the version included with 2.11BSD. It is essentially the same as the version from V7, but with 15 more years of bug fixes. I used that source to port V6 Unix to the TI990 architecture back in 2014/2015 and the good thing about it is that it still compiles with a modern gcc.
> 
> I found similarities between the 2.11BSD and 10th edition sources -
> presumably 2.11BSD was based on (or incorporated) changes in the 10th
> edition?

No, it derives from V7, with much stuff from 4BSD back-ported to the PDP11. See the “Early BSDs” section of the Unix Tree (https://minnie.tuhs.org//cgi-bin/utree.pl). Although V8 also has some 4BSD roots, the V8/V9/V10 lineage is different from the 2.8-2.11 BSD lineage. There was a lot of code sharing back then, it is not a simple picture.

> BTW did you publish your work?

https://1587660.websites.xs4all.nl/cgi-bin/9995/dir?ci=tip

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[arnold]

FYI, Steve Johnson's Portable C Compiler has been revived and updated
to C89 and maybe even C99.  It has code generators for x86 both 32
and 64 bit mode.

Home page is at http://pcc.ludd.ltu.se/.  I maintain a git mirror
(the defalt is CVS) at https://github.com/arnoldrobbins/pcc-revived.

Arnold

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Sun, Jul 22, 2018 at 4:49 PM, Dibyendu Majumdar <mobile@majumdar.org.uk>
wrote:

>
>
> It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
> Compiler' which is quite useful in understanding the compiler. I have
> converted the doc to Markdown format as well which makes it easier to
> read.


https://s3-us-west-2.amazonaws.com/belllabs-microsite-plan9/7thEdMan/bswv7.html

​You might try downloading the PDF or PS versions of the V7 documents from
amazon.   It seems like a lot less work work and will be a lot easier to
read.   Also Gnu groff runs on pretty much everything these days, so going
from the V7 troff sources yourself is not difficult; in fact its pretty
smart.   Its still the best 'pure document compiler' out there.  And it
means you can add your additions/ corrections / notes to what Dennis did
right inline.
​
ᐧ

[-- Attachment #2: Type: text/html, Size: 1917 bytes --]

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Larry McVoy]

On Mon, Jul 23, 2018 at 11:12:49AM -0400, Clem Cole wrote:
> On Sun, Jul 22, 2018 at 4:49 PM, Dibyendu Majumdar <mobile@majumdar.org.uk>
> wrote:
> 
> >
> >
> > It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
> > Compiler' which is quite useful in understanding the compiler. I have
> > converted the doc to Markdown format as well which makes it easier to
> > read.
> 
> 
> https://s3-us-west-2.amazonaws.com/belllabs-microsite-plan9/7thEdMan/bswv7.html
> 
> ???You might try downloading the PDF or PS versions of the V7 documents from
> amazon.   It seems like a lot less work work and will be a lot easier to
> read.   Also Gnu groff runs on pretty much everything these days, so going
> from the V7 troff sources yourself is not difficult; in fact its pretty
> smart.   Its still the best 'pure document compiler' out there.  And it
> means you can add your additions/ corrections / notes to what Dennis did
> right inline.

I actually wacked a bunch of the Unix docs to make them look a little
better, I should see if I can find that.

I agree that roff is awesome, it's a bummer that Latex seems to be
the winner (which I think is purely because the roff/eqn/pic/etc
docs weren't widely available back in the day).

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

[-- Attachment #1: Type: text/plain, Size: 1221 bytes --]

On 23 July 2018 at 16:12, Clem Cole <clemc@ccc.com> wrote:

>
>
> On Sun, Jul 22, 2018 at 4:49 PM, Dibyendu Majumdar <mobile@majumdar.org.uk
> > wrote:
>
>>
>>
>> It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
>> Compiler' which is quite useful in understanding the compiler. I have
>> converted the doc to Markdown format as well which makes it easier to
>> read.
>
>
> https://s3-us-west-2.amazonaws.com/belllabs-microsite-plan9/7thEdMan/
> bswv7.html
>
> ​You might try downloading the PDF or PS versions of the V7 documents from
> amazon.   It seems like a lot less work work and will be a lot easier to
> read.   Also Gnu groff runs on pretty much everything these days, so going
> from the V7 troff sources yourself is not difficult; in fact its pretty
> smart.   Its still the best 'pure document compiler' out there.  And it
> means you can add your additions/ corrections / notes to what Dennis did
> right inline.
> ​
> ᐧ
>

Hi - thank you for the links. I am not familiar with troff; I amanged to
convert to Markdown from a text version.
Here it is:

https://github.com/dibyendumajumdar/C/blob/master/docs/ctour.md

Thanks and Regards
Dibyendu

[-- Attachment #2: Type: text/html, Size: 2798 bytes --]

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

Hi Arnold,

On 23 July 2018 at 11:28,  <arnold@skeeve.com> wrote:
> FYI, Steve Johnson's Portable C Compiler has been revived and updated
> to C89 and maybe even C99.  It has code generators for x86 both 32
> and 64 bit mode.
>
> Home page is at http://pcc.ludd.ltu.se/.  I maintain a git mirror
> (the defalt is CVS) at https://github.com/arnoldrobbins/pcc-revived.
>

Very nice - thank you.
I hope to do the same for the C compiler by Dennis Ritchie.

Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Bakul Shah]

On Jul 23, 2018, at 8:12 AM, Clem Cole <clemc@ccc.com> wrote:
> 
> On Sun, Jul 22, 2018 at 4:49 PM, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> 
> > It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
> > Compiler' which is quite useful in understanding the compiler. I have
> > converted the doc to Markdown format as well which makes it easier to
> > read.
> 
> https://s3-us-west-2.amazonaws.com/belllabs-microsite-plan9/7thEdMan/bswv7.html
> 
> ​You might try downloading the PDF or PS versions of the V7 documents from amazon.
> It seems like a lot less work work and will be a lot easier to read.


Links in the above webpage don't work. Try https://9p.io/7thEdMan/bswv7.html

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

On 23 July 2018 at 21:12, Bakul Shah <bakul@bitblocks.com> wrote:
> On Jul 23, 2018, at 8:12 AM, Clem Cole <clemc@ccc.com> wrote:
>>
>> On Sun, Jul 22, 2018 at 4:49 PM, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
>>
>> > It seems that Dennis Ritchie wrote a paper 'A Tour through the Unix C
>> > Compiler' which is quite useful in understanding the compiler. I have
>> > converted the doc to Markdown format as well which makes it easier to
>> > read.
>>
>> https://s3-us-west-2.amazonaws.com/belllabs-microsite-plan9/7thEdMan/bswv7.html
>>
>> You might try downloading the PDF or PS versions of the V7 documents from amazon.
>> It seems like a lot less work work and will be a lot easier to read.
>
>
> Links in the above webpage don't work. Try https://9p.io/7thEdMan/bswv7.html

Nice, thank you.

Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[arnold]

Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:

> Hi Arnold,
>
> On 23 July 2018 at 11:28,  <arnold@skeeve.com> wrote:
> > FYI, Steve Johnson's Portable C Compiler has been revived and updated
> > to C89 and maybe even C99.  It has code generators for x86 both 32
> > and 64 bit mode.
> >
> > Home page is at http://pcc.ludd.ltu.se/.  I maintain a git mirror
> > (the defalt is CVS) at https://github.com/arnoldrobbins/pcc-revived.
> >
>
> Very nice - thank you.
> I hope to do the same for the C compiler by Dennis Ritchie.
>
> Regards
> Dibyendu

Best of luck to you!  Should be interesting to see what you get.

Arnold

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

On 19 July 2018 at 20:23, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> I was also wondering if the C pre-processor included in lcc project
> was written by Dennis:
> https://github.com/drh/lcc/tree/master/cpp
>
> I seem to have read somewhere that it was - but I cannot locate any
> reference to that.
>

Found it - its directly from Dennis Ritchie!
https://groups.google.com/d/msg/comp.std.c/Xw5H8TlQ4W4/deq8iZ6exvUJ

Thanks and Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

[-- Attachment #1: Type: text/plain, Size: 2406 bytes --]

On 19 July 2018 at 15:50, Clem Cole <clemc@ccc.com> wrote:

>
> On Wed, Jul 18, 2018 at 5:37 PM, Dibyendu Majumdar <mobile@majumdar.org.uk
> > wrote:
>
>> I am interested in finding out if the last C compiler code (not the
>> earliest versions which I know
>> are available) written by Dennis Ritchie is available somewhere. I
>> assume that the C compiler in V7 code was written by him?
>>
>> ​I'm not sure if this is the last.   This is a pointer to the V7 Ritchie
> Compiler:  https://minnie.tuhs.org//cgi-bin/utree.pl?file=V7/usr/src/cmd/c
> - The sources and the makefile build the three passes /lib/c[012]
> This should be a good starting point/base line.
>
>
Many thanks - I will compare this with
https://github.com/eunuchs/unix-archive/tree/master/PDP-11/Trees/V7/usr/src/cmd/c
which is what I have been looking at.

I guess that by this time the work had transitioned to pcc so probably
there isn't a later version available?


> Be careful because the Johnson Compiler (pcc) was also included with V7
> and is a different technology.
>

Yes understood.


>
> This is important because their are modifications to both the Ritchie and
> Johnson compilers 'in-the-wild' for other back-ends and new optimizations.
>   I for instance, re-targeted the Ritchie compiler to what would become the
> 68000 (it was not yet numbered, it was an experimental chip when we had
> access to it in the late 1970s in Tek Labs - mine was a 16 bit 'int' as I
> was coming primarily from the PDP-11 at the time and the chip was a 16 bit
> chip internally - so the code was tight and clean and I basically
> substituted PDP-11 instruction sequences for 68000 sequences).   IIRC, Jack
> Test's 68000 compiler from MIT which was about 18 mons later was based on
> the Johnson compiler but he used a 32 bit 'int' which proved easier for
> porting programs from the Vax, as the chip supported 32 bit words even
> though it took 2 ticks to do anything [so Jack's compiler generated slower
> code for many simple ops].
>
> I recommend, that google for the old USENIX tapes and see what you turn up
> and compare.
>
> ᐧ
>

Thank you for the info - I will certainly look at the USENIX tapes.

I will try to port the C compiler to amd64 - while preserving as much of
the original code as I can. But not sure if this is even feasible.

Thanks and Regards
Dibyendu

[-- Attachment #2: Type: text/html, Size: 5055 bytes --]

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

On 19 July 2018 at 06:41,  <arnold@skeeve.com> wrote:
> Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
>> I am interested in finding out if the last C compiler code (not the
>> earliest versions which I know
>> are available) written by Dennis Ritchie is available somewhere. I
>> assume that the C compiler in V7 code was written by him?
>>
> Yes - but take note, that would be 'cc'.  The 'pcc' command, also in
> V7, is Steve Johnson's Portable C Compiler.
>

Many thanks.

I was also wondering if the C pre-processor included in lcc project
was written by Dennis:
https://github.com/drh/lcc/tree/master/cpp

I seem to have read somewhere that it was - but I cannot locate any
reference to that.

Thanks and Regards
Dibyendu

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1768 bytes --]

On Wed, Jul 18, 2018 at 5:37 PM, Dibyendu Majumdar <mobile@majumdar.org.uk>
wrote:

> Hi
>
> I am interested in finding out if the last C compiler code (not the
> earliest versions which I know
> are available) written by Dennis Ritchie is available somewhere. I
> assume that the C compiler in V7 code was written by him?
>
> Thanks and Regards
> Dibyendu
>
​I'm not sure if this is the last.   This is a pointer to the V7 Ritchie
Compiler:  https://minnie.tuhs.org//cgi-bin/utree.pl?file=V7/usr/src/cmd/c
- The sources and the makefile build the three passes /lib/c[012]
This should be a good starting point/base line.

Be careful because the Johnson Compiler (pcc) was also included with V7 and
is a different technology.

This is important because their are modifications to both the Ritchie and
Johnson compilers 'in-the-wild' for other back-ends and new optimizations.
  I for instance, re-targeted the Ritchie compiler to what would become the
68000 (it was not yet numbered, it was an experimental chip when we had
access to it in the late 1970s in Tek Labs - mine was a 16 bit 'int' as I
was coming primarily from the PDP-11 at the time and the chip was a 16 bit
chip internally - so the code was tight and clean and I basically
substituted PDP-11 instruction sequences for 68000 sequences).   IIRC, Jack
Test's 68000 compiler from MIT which was about 18 mons later was based on
the Johnson compiler but he used a 32 bit 'int' which proved easier for
porting programs from the Vax, as the chip supported 32 bit words even
though it took 2 ticks to do anything [so Jack's compiler generated slower
code for many simple ops].

I recommend, that google for the old USENIX tapes and see what you turn up
and compare.

ᐧ

[-- Attachment #2: Type: text/html, Size: 3497 bytes --]

Re: [TUHS] Looking for final C compiler by Dennis Ritchie

[arnold]

Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:

> Hi
>
> I am interested in finding out if the last C compiler code (not the
> earliest versions which I know
> are available) written by Dennis Ritchie is available somewhere. I
> assume that the C compiler in V7 code was written by him?
>
> Thanks and Regards
> Dibyendu

Yes - but take note, that would be 'cc'.  The 'pcc' command, also in
V7, is Steve Johnson's Portable C Compiler.

Arnold

[TUHS] Looking for final C compiler by Dennis Ritchie

[Dibyendu Majumdar]

Hi

I am interested in finding out if the last C compiler code (not the
earliest versions which I know
are available) written by Dennis Ritchie is available somewhere. I
assume that the C compiler in V7 code was written by him?

Thanks and Regards
Dibyendu