[TUHS] C++ / Kernel

[TUHS] C++ / Kernel

[ron]

> Has anyone experimented with building Unix using C++, to take
> advantage of strong typing?  My guess is no--it would be a Herculean task
likely to introduce more bugs than it would fix.

I'm not sure what "strong typing" gain you expect to find.   With the
exception of the void* difference, C++ isn't much more "type strong" than C.
A lot of the type issues you can find on the Kernel just by turning up the
compiler warnings (or use lint).

The biggest issue we had was BSD didn't use void* at all.   Had they
converted pointers to void*, which is in implementation necessarily the same
as char*,
C would have done the right thing.   The problem is they did what I call
"Conversion by union."    They would store a char* into one element of a
union and read it out of another typed pointer.
This works fine for a VAX where all pointers are the same format, but fails
on word address machines (notably in our case the HEP where the target size
is encoded in the low order bits of the pointer).
Still, running around and fixing those was only a few hours work.

The DevSwitch is the beginnings of an object oriented philosophy.   Alas,
the original UNIX used it only for mapping major dev numbers to functions.
It got some later use for other things like multi filesystemsupport.

The scary supportability thing in the kernel, isn't so much typing, but the
failuer to "hide" implementation details of one part of the kernel from the
other.

Re: [TUHS] C++ / Kernel

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3179 bytes --]

On Thu, Aug 23, 2018 at 10:43 AM <ron@ronnatalie.com> wrote:

> I'm not sure what "strong typing" gain you expect to find.   With the
> exception of the void* difference, C++ isn't much more "type strong" than
> C.
> A lot of the type issues you can find on the Kernel just by turning up the
> compiler warnings (or use lint).
>
100% agree...



>
> The biggest issue we had was BSD didn't use void* at all.   Had they
> converted pointers to void*, which is in implementation necessarily the
> same
> as char*,
> C would have done the right thing.   The problem is they did what I call
> "Conversion by union."
>
I just put it as 'silent architectural assumptions' in system's
programming.   In Henry's 10 commandments, he referred to this as assuming
'all the world is a Vax.'  (today the assumption is all the world is
INTEL*64).


The DevSwitch is the beginnings of an object oriented philosophy.   Alas,
> the original UNIX used it only for mapping major dev numbers to functions.
> It got some later use for other things like multi filesystem
> support.
>
Fair enough -- again.  As much as I love BSD, I'm quick to knock it (and
now Linux) a few pegs for two issues in this light.  At the time, adding
something like the file system switch was engious and novel.   It took a
couple of different schemes in different UNIX kernels (Sun, Masscomp, Eight
edition and later System V) and then a couple of arguments on how to do
interposition to stack them to settle on the proper functions needed for
the FS, but eventually we mostly have them.   But that took years, we still
don't have something like the Locus vproc work for the process layer,
although it was added to Linux as well as BSD/Mach [Linux rejected it - see
the OpenSSI.org work].

To me, if we had done the same thing with the process layer that we did to
FS's we would be better off.  But the reason is off course the lack of an
indirection layer which object give you.




>
> The scary supportability thing in the kernel, isn't so much typing, but the
> failuer to "hide" implementation details of one part of the kernel from the
> other.
>
Again, I agree.  But this argument screams in favor of Dykstra THE style
layering (e.g. micro-kernel) approach.   I always thought it was a better
way to go, but in the end, people always picked pure performance over the
safety that "information hiding" gives you.

It's tough, I've been on both sides of this debate and have empathy for
both positions.   As a supplier, it is all about being as fast as possible,
because the customers don't care how hard you have to work, they just want
as much speed as possible (in fact the super computer folks would really
rather no OS at all).  So the 'less is more' monolithic / hack /
architectural specific ideas make a lot of sense.    But some of the things
we are talking about here are ideas that aid us in the development side on
making 'better' or 'cleaner' systems - ones that are more maintainable and
easier to ensure are 'correct' - which often the custom will not pay for,
or worse shun because it end up being a little slower.

Clem


ᐧ

[-- Attachment #2: Type: text/html, Size: 5597 bytes --]

Re: [TUHS] C++ / Kernel

[Bakul Shah]

On Aug 23, 2018, at 7:42 AM, ron@ronnatalie.com wrote:
> 
> The biggest issue we had was BSD didn't use void* at all.   Had they
> converted pointers to void*, which is in implementation necessarily the same
> as char*,
> C would have done the right thing.   The problem is they did what I call
> "Conversion by union."    They would store a char* into one element of a
> union and read it out of another typed pointer.

I haven't done much BSD kernel programming in last 15 years
but this is not my recollection. BSD used caddr_t, typedefed
to char*, sort of as void *. IIRC void* came in common use
after BSD unix first came about. Why use a union when a cast
will do? :-) The union trick is more likely to be used for
esoteric things (like getting at fl.pt. bytes) or for more
complex types or probably by people with lots of programming
experience in pascal and not so much in C (in Pascal you *had*
to use untagged variant records if you wanted to cheat!). In C,
all that void* does is allow you to avoid casts in some cases.

> This works fine for a VAX where all pointers are the same format, but fails
> on word address machines (notably in our case the HEP where the target size
> is encoded in the low order bits of the pointer).
> Still, running around and fixing those was only a few hours work.
> 
> The DevSwitch is the beginnings of an object oriented philosophy.   Alas,
> the original UNIX used it only for mapping major dev numbers to functions.
> It got some later use for other things like multi filesystemsupport.

{b,c}devsw are closer to an interface (or in C++ terminology 
an abstract class that contains nothing but pure virtual
functions). Go got it right.  I had to build something like
devsw for some IOT devices in Go and it worked very well. And
you don't need to manually stuff relevant functions in a
struct so that {bdev,cdev,line}sw can be properly initialized
as in C.
 
Not having this feature in C meant dev switches got a bit 
impure (in 4.xBSD there was only a buf ptr for bdevsw and a
tty ptr in cdesw).  In modern BSD the cdev struct contains all
sorts of cruft.
 
So in this sense C++ got it wrong, at least initially. Code 
for each disk or network device would be device specific but
in so far as possible we want to access devices of a specific
class the same way. Classical OO would not help here.

[Aside: What I would really like is if Go interfaces can be
somehow turned into abstract data types by adding an algebraic
specification. For example, at the source level you should
be able to say don't allow read/write once  device is closed.
Adding this sort of assertions/axioms at the interface level
can make your code less cluttered (no need to check at every
point if the device is still open) and safer (in case you forget
to check).]


> The scary supportability thing in the kernel, isn't so much typing, but the
> failuer to "hide" implementation details of one part of the kernel from the
> other.

Lack of modularity or more fine grained trust boundaries.
Apropos this point:

https://threatpost.com/researchers-blame-monolithic-linux-code-base-for-critical-vulnerabilities/136785/

  In an exhaustive study of critical Linux vulnerabilities, a 
  team of academic and government-backed researchers claim to
  have proven that almost all flaws could be mitigated to less
  than critical severity - and that 40 percent could be
  completely eliminated - with an OS design based on a
  verified microkernel. Though I tend to think the "verified"
  part doesn't play as strong a role as the modularization
  forced by microkernels.

[Ties in with the formal methods thread in here and in COFF!]

Re: [TUHS] C++ / Kernel

[ron]

> I haven't done much BSD kernel programming in last 15 years but this is
not my recollection. BSD used caddr_t, typedefed to char*, sort of as void
*. IIRC void* 
>came in common use after BSD unix first came about. Why use a union when a
cast will do? :-) The union trick is more likely to be used for esoteric
things 
> (like getting at fl.pt. bytes) or for more complex types or probably by
people with lots of programming experience in pascal and not so much in C
(in Pascal 
>you *had* to use untagged variant records if you wanted to cheat!). In C,
all that void* does is allow you to avoid casts in some cases.

Your recollections are certainly wrong.    I spent a lot of time tracing
down why the Kernel crashed doing I/O and traced it to this and spent a
while undoing it as I stated.
This union was right in the middle of the buf struct:

	union {
	    caddr_t b_addr;		/* low order core address */
	    int	*b_words;		/* words for clearing */
	    struct filsys *b_filsys;	/* superblocks */
	    struct dinode *b_dino;	/* ilist */
	    daddr_t *b_daddr;		/* indirect block */
	} b_un;
There were a number of other places that did the same thing.   It's
OFFICIALLY now in undefined behavior by the standard (though of course that
didn't exist in the BSD days) ,
to store in one element of the union and retrieve it via another.   This is
one of the reasons why.

This isn't the only place it occurs.

Void* came out with the V7 compiler, if I recall properly.   The BSD kernel
looks as if it requires such a later compiler (it uses bit fields which the
earlier compilers didn't support).
But it doesn't matter.   You are right char* (or caddr_t) would work just
fine for this albeit with some explicit casting.

Re: [TUHS] C++ / Kernel

[Nevin Liber]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On Thu, Aug 23, 2018 at 5:17 PM <ron@ronnatalie.com> wrote:

> This union was right in the middle of the buf struct:
>
>         union {
>             caddr_t b_addr;             /* low order core address */
>             int *b_words;               /* words for clearing */
>             struct filsys *b_filsys;    /* superblocks */
>             struct dinode *b_dino;      /* ilist */
>             daddr_t *b_daddr;           /* indirect block */
>         } b_un;
> There were a number of other places that did the same thing.   It's
> OFFICIALLY now in undefined behavior by the standard (though of course that
> didn't exist in the BSD days) ,
> to store in one element of the union and retrieve it via another.


It was still kinda frowned upon in K&R1:  "It is the responsibility of the
programmer to keep track of what type is currently stored in a union; the
results are machine dependent if something is stored as one type and
extracted as another."
-- 
 Nevin ":-)" Liber  <mailto:nevin@eviloverlord.com>  +1-847-691-1404

[-- Attachment #2: Type: text/html, Size: 1683 bytes --]

Re: [TUHS] C++ / Kernel

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1884 bytes --]

On Thu, Aug 23, 2018 at 6:29 PM Nevin Liber <nevin@eviloverlord.com> wrote:

> It was still kinda frowned upon in K&R1:  "It is the responsibility of the
> programmer to keep track of what type is currently stored in a union; the
> results are machine dependent if something is stored as one type and
> extracted as another."
>

No.  It was not frowned upon, it was widely used.  The message was just
just Dennis and Brian giving you fair warning. Remember K&R1 came out
approx the same time as 7th edition [I saw the proofs for the book with
UNIX/TS which was a little earlier, but pretty much the same kernel].

From V7th editions buf.h (which I show a date of May 5, 1979) and Ron is
correct void*, caddr_t, daddr_t, dev_t and the like; were all 7th
edition-isms that BSD picked up)
...
struct buf
{
        int     b_flags;                /* see defines below */
        struct  buf *b_forw;            /* headed by d_tab of conf.c */
        struct  buf *b_back;            /*  "  */
        struct  buf *av_forw;           /* position on free list, */
        struct  buf *av_back;           /*     if not BUSY*/
        dev_t   b_dev;                  /* major+minor device name */
        unsigned b_bcount;              /* transfer count */
        union {
            caddr_t b_addr;             /* low order core address */
            int *b_words;               /* words for clearing */
            struct filsys *b_filsys;    /* superblocks */
            struct dinode *b_dino;      /* ilist */ls -l buf.h
            daddr_t *b_daddr;           /* indirect block */
        } b_un;
        daddr_t b_blkno;                /* block # on device */
        char    b_xmem;                 /* high order core address */
        char    b_error;                /* returned after I/O */
        unsigned int b_resid;           /* words not transferred after
error */
};
...

[-- Attachment #2: Type: text/html, Size: 5705 bytes --]

Re: [TUHS] C++ / Kernel

[Steffen Nurpmeso]

ron@ronnatalie.com wrote in <151301d43b2f$07881ed0$16985c70$@ronnatalie.com>:
  ...
 |This union was right in the middle of the buf struct:
 |
 | union {
 |     caddr_t b_addr;          /* low order core address */
 |     int      *b_words;               /* words for clearing */
 |     struct filsys *b_filsys; /* superblocks */
 |     struct dinode *b_dino;   /* ilist */
 |     daddr_t *b_daddr;                /* indirect block */
 |} b_un;
 |There were a number of other places that did the same thing.   It's
 |OFFICIALLY now in undefined behavior by the standard (though of course that
 |didn't exist in the BSD days) ,
 |to store in one element of the union and retrieve it via another.   This is
 |one of the reasons why.

Luckily "type punning" is possible.  Ach, what do i know;
nonetheless, i think it is a terrible direction that C goes to,
now you often need memcpy() to local storage instead of a simple
assignment (with a cast) in order to avoid undefined behaviour
that allows compilers to do bad things.  On a NetBSD list there
was a link to such a story not too long ago, the resulting
C standard compliant code was ridiculous, the last string of fun
has been screwed out of C.  Unfortunately i have not saved the
message, but Christos Zoulas has been the author.

Regarding "type punning" the standard C 2011 says (6.5.2.3)

  If the member used to read the contents of a union object is not
  the same as the member last used to store a value in the object,
  the appropriate part of the object representation of the value
  is reinterpreted as an object representation in the new type as
  described in 6.2.6 (a process sometimes called ‘‘type
  punning’’). This might be a trap representation.

This has not changed in C 2018 (though i have only version 2017
ballot).  So as long as the type sizes are identical anything
should be fine.  Just as you say.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] C++ / Kernel

[Bakul Shah]

> On Aug 23, 2018, at 3:17 PM, <ron@ronnatalie.com> <ron@ronnatalie.com> wrote:
> 
> 
> 
>> I haven't done much BSD kernel programming in last 15 years but this is
> not my recollection. BSD used caddr_t, typedefed to char*, sort of as void
> *. IIRC void* 
>> came in common use after BSD unix first came about. Why use a union when a
> cast will do? :-) The union trick is more likely to be used for esoteric
> things 
>> (like getting at fl.pt. bytes) or for more complex types or probably by
> people with lots of programming experience in pascal and not so much in C
> (in Pascal 
>> you *had* to use untagged variant records if you wanted to cheat!). In C,
> all that void* does is allow you to avoid casts in some cases.
> 
> Your recollections are certainly wrong.    I spent a lot of time tracing
> down why the Kernel crashed doing I/O and traced it to this and spent a
> while undoing it as I stated.
> This union was right in the middle of the buf struct:
> 
> 	union {
> 	    caddr_t b_addr;		/* low order core address */
> 	    int	*b_words;		/* words for clearing */
> 	    struct filsys *b_filsys;	/* superblocks */
> 	    struct dinode *b_dino;	/* ilist */
> 	    daddr_t *b_daddr;		/* indirect block */
> 	} b_un;
> There were a number of other places that did the same thing.   It's
> OFFICIALLY now in undefined behavior by the standard (though of course that
> didn't exist in the BSD days) ,
> to store in one element of the union and retrieve it via another.   This is
> one of the reasons why.

Note that this is a legitimate use of union. That is,
unless I misunderstood what you meant by it, there is
no "conversion by union" as you call it or "cheating"
as I call it or type punning. There is no put one thing
in and take another thing out. Now it may be that
someone misused such a union. This is easy to do as,
unlike Pascal, C has no tagged variant record & it is
user's responsibility to use it right.

> 
> This isn't the only place it occurs.
> 
> Void* came out with the V7 compiler, if I recall properly.   The BSD kernel
> looks as if it requires such a later compiler (it uses bit fields which the
> earlier compilers didn't support).

From what I recall {c,m,re}alloc() returned a char*, not a void *.
I don't have K&R1 handy at the moment so can't recall if void* was
mentioned in the book (if not, that could be one reason for a lack of
its use).

Re: [TUHS] C++ / Kernel

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Thu, Aug 23, 2018 at 6:17 PM <ron@ronnatalie.com> wrote:

> [snip]
>
> Void* came out with the V7 compiler, if I recall properly.   The BSD kernel
> looks as if it requires such a later compiler (it uses bit fields which the
> earlier compilers didn't support).
> But it doesn't matter.   You are right char* (or caddr_t) would work just
> fine for this albeit with some explicit casting.
>

This appears to be incorrect, unfortunately. I just tested on the PDP-11/70
running 7th Edition at the Living Computer Museum (I've got an account
there) and it appears that neither `cc` nor `pcc` understand `void`.

Perhaps Steve Johnson can chime in on this? I suspect he'd know the history
here well.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 1084 bytes --]

Re: [TUHS] C++ / Kernel

[Clem cole]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

Interesting. Void is missing, not just void*? At one point void worked but the void * idiom was buggy/missing    

The problem I have is the compiler was changing in small ways with each version and the differences run together 


Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

> On Aug 23, 2018, at 9:58 PM, Dan Cross <crossd@gmail.com> wrote:
> 
>> On Thu, Aug 23, 2018 at 6:17 PM <ron@ronnatalie.com> wrote:
>> [snip]
>> 
>> Void* came out with the V7 compiler, if I recall properly.   The BSD kernel
>> looks as if it requires such a later compiler (it uses bit fields which the
>> earlier compilers didn't support).
>> But it doesn't matter.   You are right char* (or caddr_t) would work just
>> fine for this albeit with some explicit casting.
> 
> This appears to be incorrect, unfortunately. I just tested on the PDP-11/70 running 7th Edition at the Living Computer Museum (I've got an account there) and it appears that neither `cc` nor `pcc` understand `void`.
> 
> Perhaps Steve Johnson can chime in on this? I suspect he'd know the history here well.
> 
>         - Dan C.
> 

[-- Attachment #2: Type: text/html, Size: 1792 bytes --]

Re: [TUHS] C++ / Kernel

[Pete Turnbull]

On 24/08/2018 02:41, Bakul Shah wrote:

>From what I recall {c,m,re}alloc() returned a char*, not a void *. > I don't have K&R1 handy at the moment so can't recall if void* was
> mentioned in the book (if not, that could be one reason for a lack of
> its use).

Yes, K&R1 specifically states that alloc(n) returns a char* (page 97), 
calloc(n, sizeof(obj)) returns a pointer to whatever type you specify 
(page 157), and void isn't mentioned anywhere in the book that I can see.

-- 
Pete
Pete Turnbull

Re: [TUHS] C++ / Kernel

[ron]

>> 	union {
>> 	    caddr_t b_addr;		/* low order core address */
>> 	    int	*b_words;		/* words for clearing */
>> 	    struct filsys *b_filsys;	/* superblocks */
>> 	    struct dinode *b_dino;	/* ilist */
>> 	    daddr_t *b_daddr;		/* indirect block */
>> 	} b_un;

>  Note that this is a legitimate use of union. That is, unless I
misunderstood what you meant by it, there is no "conversion by union" as you
call it or "cheating"
> as I call it or type punning. There is no put one thing in and take
another thing out. Now it may be that someone misused such a union. This is
easy to do as, unlike Pascal, C has no tagged variant record & it is user's
responsibility to use it right.

It's not legitimate.   It's undefined behavior in the standards and as
someone has already posted, Ritchie warned about the issue at the time.
It is undefined (currently) or in the past (machine dependent) behavior  to
store in one element of a union and retrieve it by another.

For instance:
       int x;
       daddr_t* y;

      u.b_un.b_words = &x;
      y = u.b_un.b_daddr;

This is syntactically correct, but the code will blow up bizarrely on
machines where the int* and long* pointers are not in the same format. 
Now if the datatype was a void* and we did this:

      u.b_vp = & x;
      y = u.b_vp;

Provided there weren't any alignment issues, this would work as the
conversion to and from void* would result in the right pointer value.

I can think of three different machines I've worked on that this is a
problem.   The Delelcor HEP was the one I got caught on because I was
porting 4.2BSD to it.
The Univac (er, um, Sperry) 1100 series and I think the DEC10 by derivation
have the partial word (i.e., anything less than a full 36 bits) encoded in
the pointers.
The Crays (being word addressed machines, character types are "simulated")
have distinct word and char pointers.

void* has to be format the same as char* (don't get me started on this
idiocy, while it worked for the PDP-11, it's an inane assumption that the
basic character type is the same as
the smallest addressable storage unit.    THIS is one of the things that
should have been fixed about the time void* was conceived).

Re: [TUHS] C++ / Kernel

[Derek Fawcus]

On Thu, Aug 23, 2018 at 10:42:40AM -0400, ron@ronnatalie.com wrote:
> 
> > Has anyone experimented with building Unix using C++, to take
> > advantage of strong typing?  My guess is no--it would be a Herculean task
> likely to introduce more bugs than it would fix.

I recall that there was an attempt to do this with the Linux kernel,
but that it was abandoned before it was completed, this would have
been pre '98. Probably pre 2.4 Kernel.

I have vague memories of discussions about it on the (kernel) mailing
list at the time, and that it was the original cause of the 'asmlinkage'
annotation on the C level syscalls - basically to turn off C++ name
mangling for when the assembler syscall table referenced the C routines.

DF

Re: [TUHS] C++ / Kernel

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

On Thu, Aug 23, 2018 at 11:05 PM Clem cole <clemc@ccc.com> wrote:

> Interesting. Void is missing, not just void*? At one point void worked but
> the void * idiom was buggy/missing
>

Correct. Neither cc nor pcc on 7th edition will accept:

void
sideeffect()
{
        printf("Hi\n");
}

As far as I can tell, both are treating `void` in this short program as an
identifier. The string "void" doesn't appear in the sources for either
compiler.

The problem I have is the compiler was changing in small ways with each
> version and the differences run together
>

It's my subjective impression, based largely on what I read here on TUHS,
that there was quite a lot of activity and cross-pollination in and out of
Bell Labs at the time, so I'm not surprised that the details here are fuzzy.

        - Dan C.

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not
> quite.
>
> On Aug 23, 2018, at 9:58 PM, Dan Cross <crossd@gmail.com> wrote:
>
> On Thu, Aug 23, 2018 at 6:17 PM <ron@ronnatalie.com> wrote:
>
>> [snip]
>>
>> Void* came out with the V7 compiler, if I recall properly.   The BSD
>> kernel
>> looks as if it requires such a later compiler (it uses bit fields which
>> the
>> earlier compilers didn't support).
>> But it doesn't matter.   You are right char* (or caddr_t) would work just
>> fine for this albeit with some explicit casting.
>>
>
> This appears to be incorrect, unfortunately. I just tested on the
> PDP-11/70 running 7th Edition at the Living Computer Museum (I've got an
> account there) and it appears that neither `cc` nor `pcc` understand `void`.
>
> Perhaps Steve Johnson can chime in on this? I suspect he'd know the
> history here well.
>
>         - Dan C.
>
>

[-- Attachment #2: Type: text/html, Size: 3098 bytes --]

Re: [TUHS] C++ / Kernel

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in <20180823231413.QeahD%steffen@sdaoden.eu>:
 |ron@ronnatalie.com wrote in <151301d43b2f$07881ed0$16985c70$@ronnatalie.\
 |com>:
  ...
 ||This union was right in the middle of the buf struct:
  ...
 ||
 || union {
 ||     caddr_t b_addr;          /* low order core address */
 ||     int      *b_words;               /* words for clearing */
 ||     struct filsys *b_filsys; /* superblocks */
 ||     struct dinode *b_dino;   /* ilist */
 ||     daddr_t *b_daddr;                /* indirect block */
 ||} b_un;
 ||There were a number of other places that did the same thing.   It's
 ||OFFICIALLY now in undefined behavior by the standard (though of course \
 ||that
 ||didn't exist in the BSD days) ,
 ||to store in one element of the union and retrieve it via another. \
 ||  This is
 ||one of the reasons why.
 |
 |Luckily "type punning" is possible.  Ach, what do i know;
 |nonetheless, i think it is a terrible direction that C goes to,
 |now you often need memcpy() to local storage instead of a simple
 |assignment (with a cast) in order to avoid undefined behaviour
 |that allows compilers to do bad things.  On a NetBSD list there
 |was a link to such a story not too long ago, the resulting
 |C standard compliant code was ridiculous, the last string of fun
 |has been screwed out of C.  Unfortunately i have not saved the
 |message, but Christos Zoulas has been the author.

I have found the link in the history of my browser, the story was

  http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] C++ / Kernel

[ron]

That's a different issue (as I alluded to in my post).   Alignment on
machines is a different problem.   Type "punning" isn't the problem, the
compiler is propely converting one
pointer type to another but the operand is not correctly aligned.   

In the case I'm referring to, the Univac and the HEP encode partial word
sizes in the pointer.    If you take the literal bits from one pointer to
another (as if you stored it in a union),
then you end up referring to the wrong sized operand in the subsequent
operation.    Had you converted it to void* or explicitly forced a
conversion wih the cast, the compiler 
would have taken care of converting the pointer for you.

It's very odd when you find the machine storing the WRONG SIZE operand from
the pointer type in use.
As I said, it wasn't hard to fix (just grep for all the afflicted unions),
but it took a bit of long kernel debug sessions to figure out what was
happening.

> -----Original Message-----
> From: Steffen Nurpmeso <steffen@sdaoden.eu>
> I have found the link in the history of my browser, the story was
> 
>   http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html
> 

Re: [TUHS] C++ / Kernel

[Steffen Nurpmeso]

Derek Fawcus wrote in <20180824132244.GA2979@accordion.employees.org>:
 |On Thu, Aug 23, 2018 at 10:42:40AM -0400, ron@ronnatalie.com wrote:
 |> 
 |>> Has anyone experimented with building Unix using C++, to take
 |>> advantage of strong typing?  My guess is no--it would be a Herculean \
 |>> task
 |> likely to introduce more bugs than it would fix.
 |
 |I recall that there was an attempt to do this with the Linux kernel,
 |but that it was abandoned before it was completed, this would have
 |been pre '98. Probably pre 2.4 Kernel.
 |
 |I have vague memories of discussions about it on the (kernel) mailing
 |list at the time, and that it was the original cause of the 'asmlinkage'
 |annotation on the C level syscalls - basically to turn off C++ name
 |mangling for when the assembler syscall table referenced the C routines.

Also the often seen C paradigm of using "enum bla" in prototypes
results in C++ compilation warnings/errors if the enum is actually
used to define flags rather than a true enumeration, and there are
not "bit enumerations" which would accomplish that.  So you need
to use an integer for the prototype and have lost all checking
capabilities with a glance.  I have never understood this
deficiency.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] C++ / Kernel

[Steffen Nurpmeso]

ron@ronnatalie.com wrote in <1f6201d43bb7$3b9ebde0$b2dc39a0$@ronnatalie.com>:
 |That's a different issue (as I alluded to in my post).   Alignment on
 |machines is a different problem.   Type "punning" isn't the problem, the
 |compiler is propely converting one
 |pointer type to another but the operand is not correctly aligned.   

The story told here is more about how the peculiarities of
a standard text and compiler developers come together to something
very unfortunate.

 |In the case I'm referring to, the Univac and the HEP encode partial word
 |sizes in the pointer.    If you take the literal bits from one pointer to
 |another (as if you stored it in a union),
 |then you end up referring to the wrong sized operand in the subsequent
 |operation.    Had you converted it to void* or explicitly forced a

Ah, i am sorry.  Indeed i misunderstood that.

 |conversion wih the cast, the compiler 
 |would have taken care of converting the pointer for you.

Though even more unfortunate i am, since this union trick is often
the only way to be able to do proper a.k.a. allowed type
conversion, where the standard text forbids something quick and
easy, casting of or to function pointers in C++ comes to mind
spontaneously.  (The linked story also tries to go via (char*) to
a desired type, but the compiler seems to be too neat.  And even
if this is a bug, of course...)

 |It's very odd when you find the machine storing the WRONG SIZE operand from
 |the pointer type in use.
 |As I said, it wasn't hard to fix (just grep for all the afflicted unions),
 |but it took a bit of long kernel debug sessions to figure out what was
 |happening.
 |
 |> -----Original Message-----
 |> From: Steffen Nurpmeso <steffen@sdaoden.eu>
 |> I have found the link in the history of my browser, the story was
 |> 
 |>   http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html
 --End of <1f6201d43bb7$3b9ebde0$b2dc39a0$@ronnatalie.com>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] C++ / Kernel

[Bakul Shah]

On Aug 24, 2018, at 5:17 AM, ron@ronnatalie.com wrote:
> 
> For instance:
>       int x;
>       daddr_t* y;
> 
>      u.b_un.b_words = &x;
>      y = u.b_un.b_daddr;

One would typically *not* do this.

> 
> This is syntactically correct, but the code will blow up bizarrely on
> machines where the int* and long* pointers are not in the same format. 
> Now if the datatype was a void* and we did this:
> 
>      u.b_vp = & x;
>      y = u.b_vp;
> 
> Provided there weren't any alignment issues, this would work as the
> conversion to and from void* would result in the right pointer value.

For example, given

	union {
	    caddr_t b_addr;		/* low order core address */
	    int	*b_words;		/* words for clearing */
	    struct filsys *b_filsys;	/* superblocks */
	    struct dinode *b_dino;	/* ilist */
	    daddr_t *b_daddr;		/* indirect block */
	} b_un;

You'd use it something like this:

	struct b_un x, y, z, *w;
	struct filesys fs;
	daddr_t d;
	...
	x.b_words = &b;
	y.b_filsys = &fs;
	z.b_addr = &d;
	w = &x;

These come about because in different contexts different ptrs
may be used but one would not do any type conversion. Said another
way, if you replaced the union with a struct, type punning code
would fall apart but legit code would continue working (except
in the case where the *size* of the object is significant).

Re: [TUHS] C++ / Kernel

[ron]

> >
> > For instance:
> >       int x;
> >       daddr_t* y;
> >
> >      u.b_un.b_words = &x;
> >      y = u.b_un.b_daddr;
> 
> One would typically *not* do this.

One would think not, but the BSD kernel did.   That's the whole point.

Re: [TUHS] C++ / Kernel

[Paul Winalski]

On 8/24/18, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
>
> Though even more unfortunate i am, since this union trick is often
> the only way to be able to do proper a.k.a. allowed type
> conversion, where the standard text forbids something quick and
> easy, casting of or to function pointers in C++ comes to mind
> spontaneously.  (The linked story also tries to go via (char*) to
> a desired type, but the compiler seems to be too neat.  And even
> if this is a bug, of course...)

One of the complaints about DEC's C99 compiler for Ultrix was that it
was too didactic in its enforcement of the C99 standard.  One customer
called it the Rush Limbough of compilers, because it was extremely
conservative and you couldn't argue with it.

Function pointers can be tricky, as they might not be simple pointers
to a sequence of instructions.  On Itanium, for example, a C
pointer-to-function points to a descriptor for the function, not the
function code itself.

-Paul W.

Re: [TUHS] C++ / Kernel

[Steffen Nurpmeso]

Paul Winalski wrote in <CABH=_VT5Yi-6uRz_-045cmf-4kbgTSHZhsCLLracoFXNDwS\
AYQ@mail.gmail.com>:
 |On 8/24/18, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
 |> Though even more unfortunate i am, since this union trick is often
 |> the only way to be able to do proper a.k.a. allowed type
 |> conversion, where the standard text forbids something quick and
 |> easy, casting of or to function pointers in C++ comes to mind
 |> spontaneously.  (The linked story also tries to go via (char*) to
 |> a desired type, but the compiler seems to be too neat.  And even
 |> if this is a bug, of course...)
 |
 |One of the complaints about DEC's C99 compiler for Ultrix was that it
 |was too didactic in its enforcement of the C99 standard.  One customer
 |called it the Rush Limbough of compilers, because it was extremely
 |conservative and you couldn't argue with it.
 |
 |Function pointers can be tricky, as they might not be simple pointers
 |to a sequence of instructions.  On Itanium, for example, a C
 |pointer-to-function points to a descriptor for the function, not the
 |function code itself.

They can be as tricky as they want as long as i can go and use
them, wouldn't you agree?  Not only for simple callbacks but for
"manual VTables" you simply need those dynamic callbacks.  Like
you have said, you need a way to change the instruction location.
As far as i know this is what actually happens, and i want to be
able to access this mechanism without being prevented from some
language rule which is backed by nothing.  That is all.  Whatever
high language construct there may be.  To me this was one of the
properties of the C language.

I can give you another example.  I have a C++ library which uses
an approach to events where you (can) define slots, as in

  pub Misc::Sender<IOEvent> onInput;

You then say XY.onInput.connect(MYHOOK), where MYHOOK can be one
of four different types, which return "event consumed:

  pub typedef boolean (   *Slot1)(Event *);
  pub typedef boolean (   *Slot2)(void *, Event *);
  pub typedef boolean (   EventListener::*Slot3)(Event *);

The first is a PTF callback without user provided data, the second
adds that; the third is a PointerToMember.  The third defines
a PTM to a subclass of EventListener, which is an empty class but
with VTable (protected destructor), provided for the sole purpose
of offering a cheap event slot without marshalling object, managed
by an equal number of connection types, superclass:

  // actual connection structures
  // note we use the lower three bits of Conn::flags for Flags (including
  // ConnType) as below.  this works because the memory cache has an
  // alignment of 8 (or 16, and only).
  pri struct Conn{
    union{
      uir flags;
      Conn *right;
    };
  } SF_CC_PACKED;

E.g., the third

  pri struct ConnML{ // EventListener PTM
    Conn base;
    EventListener *elobj;
    Slot3 slot;
  } SF_CC_PACKED;

So far so good.  But what to do with free-form PTM hooks?
You need templates for that:

  pub template<class OBJ>
  struct ConnMT{ // free-form PTM (template indirection)
    typedef boolean (OBJ::*Slot4)(Event *);
    ConnFO cfo;
    OBJ *tobj;
    Slot4 slot;
  } SF_CC_PACKED;

But that induces several problems, one of them is

    - 2.95.2 and 2.95.3 compile the library without errors.
      The problem with them is that they are not able to handle the
      free-from template pointer-to-member (PTM),
      as well as the EventListener PTM specialization of
      SF::Sys::Misc::Sender correctly.
      This is a no-go for other libraries, which use events as
      a key concept (and base upon EventListener to get rid of the
      purely template PTM based connection overhead), but is
      otherwise no problem.
      (Some of the tests in test/ will not compile, though.)
      (Note: 2.95.2 has not been tested for a long time.
      2.95.3 is part of the test-suite at the time of this writing
      aka TAG 0.6.0.)

The other is that you cannot store ConnMT in generic code -- the
C++ standard does not offer this possibility, even though Slot4 is
a PTM just the very same way that Slot3 is, right?  PTMs are
a wild mix of VTable and offset, right.  But luckily it can be
done, with a very wild hack, but which still works after almost
twenty years, whether gcc or clang is used, at least the tests
ran just fine last time i have tried:

  // disconnecting of those is hard; try it like that (doc/cppcreq.txt).
  pri struct ConnMX{
    ConnFO cfo;
    void *tobj;
    ui1 ptmbuf[szof(Slot3)];
  } SF_CC_PACKED;

Nonetheless you need marshalling objects to dispatch events
through such slots, which is very painful and expensive, as well
as completely unnecessary yet imposed solely by language speech,
but at least all the list handling and such works just fine
(though using memcmp on ptmbuf is really wild).

  _pro template<class OBJ>
  void _Sender::connect4(typename ConnMT<OBJ>::Slot4 _Slot,
      OBJ *_tobj) const{
    ConnMT<OBJ> *cmt;
    cmt = SF_talloc(ConnMT<OBJ>, 1);
    cmt->cfo.base.flags = ((R(uir,m_slots) & ~f_mask) | ct_ptm_t);
    m_slots = &cmt->cfo.base;
    cmt->cfo.slot = &_Sender::_Dispatch<OBJ>;
    cmt->cfo.obj = cmt;
    cmt->tobj = _tobj;
    cmt->slot = _Slot;
  }

  _pri template<class OBJ>
  _sta boolean _Sender::_Dispatch(void *_tobj, Event *_ev){
    ConnMT<OBJ> *cmt = S(ConnMT<OBJ>*,_tobj);
    return (cmt->tobj->*cmt->slot)(_ev);
  }

And of course this is a primitive event mechanism which requires
an Event class to be used.  

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] C++ / Kernel

[ron]

> Yep.  Im pretty sure I remember void being in typesetter C also.   IIRC the differences between that version of Dennis’s compiler and what was included in 7th Edition was mostly in the libraries ie stdio was first released as part of the typesetter compiler but it was still a work in progress.   

My biggest recollection was the beginning of flipping =+ to += (It supported both but warned you about the former), and fixing the structure elements to not be global.

Lots of odd quirks in the kernel due to early C limitations.    PS->integ always has a fond part of my memory.
	

[TUHS] C++ / Kernel

[Paul Ruizendaal]

> Perhaps Steve Johnson can chime in on this? I suspect he'd know the history
> here well.

The origins of void were discussed 4-7 nov 2017 on this list.

Re: [TUHS] C++ / Kernel

[Clem cole]

As I said I’m not sure but I thought I remember them being there.  I no longer have that distribution tape so I cannot diff it.  

If i recall the order the three Ritchie compilers we had that supported stdio were typesetter, Unix/TS and then finally 7th Edition.  I remember they we all slightly different but don’t remember what they the differences were.  

As Steve has previously said Dennis was constantly improving the compiler, so I suspect it was just the state of the world when those snapshots were taken.  

The big changes to the language were between 6th Edition and Typesetter which were done in concert if not to support Brian’s work on the troff rewrite.  Plus the first draft of book was being written around then also. 

The book definitely predates 7th edition’s release which I date because we did not get Seventh Edition until spring 79 and Ted had xeroxgraphic copies of the book at least a year earlier, which was when we were running TS which he had brought from the Labs.  

Clem

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

On Aug 23, 2018, at 9:27 PM, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

>> From: Clem Cole
> 
>> Im pretty sure I remember void being in typesetter C also.
> 
> Hmm. In the two original 'help' files I have about the changes to C (the term
> 'typesetter C' doesn't appear, but it's pretty clear that's what the subject
> is), available through here:
> 
>  http://gunkies.org/wiki/Typesetter_C
> 
> the term 'void' does not appear (although most other stuff - e.g. unions, bit
> fields, typedef, yadda yadda - does).
> 
>    Noel

Re: [TUHS] C++ / Kernel

[Bakul Shah]

In Henry Spencer's v7 sources there is a "struct void" in
cmd/sh/mode.h -- may have to do Bourne's fascination with
Algol68!

Bitfields were available but they were not used in v7 sources.
May be because v7 pcc generated slow code for bitfields access.
[Gosling's emacs used bitfields extensively. I managed to
compile it but it ran extremely slowly on a 5.6Mhz Fortune
box. That was the main reason I didn't switch to emacs. Then
Yost brought along the Rand editor and I used it until it
would no longer compile on *BSDs.]

> On Aug 23, 2018, at 5:30 PM, Clem Cole <clemc@ccc.com> wrote:
> 
> Yep.  Im pretty sure I remember void being in typesetter C also.   IIRC the differences between that version of Dennis’s compiler and what was included in 7th Edition was mostly in the libraries ie stdio was first released as part of the typesetter compiler but it was still a work in progress.   
> 
> On Thu, Aug 23, 2018 at 7:42 PM <ron@ronnatalie.com> wrote:
> Void probably showed up their first as well    It was more like modern C
> than the V6 compiler.
> 
> -----Original Message-----
> From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Noel Chiappa
> Sent: Thursday, August 23, 2018 7:30 PM
> To: tuhs@minnie.tuhs.org
> Cc: jnc@mercury.lcs.mit.edu
> Subject: Re: [TUHS] C++ / Kernel
> 
>     > From: Ron Natalie
> 
>     > The BSD kernel looks as if it requires such a later compiler (it uses
>     > bit fields which the earlier compilers didn't support).
> 
> Umm, minor nit: bit fields were added to/as of 'Typesetter C', which I
> gather was intermediate between V6 and V7.
> 
>         Noel
> 
> -- 
> Sent from a handheld expect more typos than usual

Re: [TUHS] C++ / Kernel

[Noel Chiappa]

    > From: Clem Cole

    > Im pretty sure I remember void being in typesetter C also.

Hmm. In the two original 'help' files I have about the changes to C (the term
'typesetter C' doesn't appear, but it's pretty clear that's what the subject
is), available through here:

  http://gunkies.org/wiki/Typesetter_C

the term 'void' does not appear (although most other stuff - e.g. unions, bit
fields, typedef, yadda yadda - does).

    Noel

Re: [TUHS] C++ / Kernel

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

Yep.  Im pretty sure I remember void being in typesetter C also.   IIRC the
differences between that version of Dennis’s compiler and what was included
in 7th Edition was mostly in the libraries ie stdio was first released as
part of the typesetter compiler but it was still a work in progress.

On Thu, Aug 23, 2018 at 7:42 PM <ron@ronnatalie.com> wrote:

> Void probably showed up their first as well    It was more like modern C
> than the V6 compiler.
>
> -----Original Message-----
> From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Noel Chiappa
> Sent: Thursday, August 23, 2018 7:30 PM
> To: tuhs@minnie.tuhs.org
> Cc: jnc@mercury.lcs.mit.edu
> Subject: Re: [TUHS] C++ / Kernel
>
>     > From: Ron Natalie
>
>     > The BSD kernel looks as if it requires such a later compiler (it uses
>     > bit fields which the earlier compilers didn't support).
>
> Umm, minor nit: bit fields were added to/as of 'Typesetter C', which I
> gather was intermediate between V6 and V7.
>
>         Noel
>
> --
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 1698 bytes --]

Re: [TUHS] C++ / Kernel

[ron]

Void probably showed up their first as well    It was more like modern C
than the V6 compiler.

-----Original Message-----
From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Noel Chiappa
Sent: Thursday, August 23, 2018 7:30 PM
To: tuhs@minnie.tuhs.org
Cc: jnc@mercury.lcs.mit.edu
Subject: Re: [TUHS] C++ / Kernel

    > From: Ron Natalie

    > The BSD kernel looks as if it requires such a later compiler (it uses
    > bit fields which the earlier compilers didn't support).

Umm, minor nit: bit fields were added to/as of 'Typesetter C', which I
gather was intermediate between V6 and V7.

	Noel

Re: [TUHS] C++ / Kernel

[Noel Chiappa]

    > From: Ron Natalie

    > The BSD kernel looks as if it requires such a later compiler (it uses
    > bit fields which the earlier compilers didn't support).

Umm, minor nit: bit fields were added to/as of 'Typesetter C', which I gather
was intermediate between V6 and V7.

	Noel