[9fans] structure allocation.

[9fans] structure allocation.

[erik quanstrom]

the open-source drivers for the DAC960 have a number
of structures declared like

typedef struct {
	char f;
	ulong l;
	char g;
	ulong m;
} __attribute__((packed)) Dac960fu;

with comments such as:

/*
 * Command result buffers, as placed in system memory by the controller.
 */

how is this handled in plan 9?

- erik

Re: [9fans] structure allocation.

[geoff]

You declare it like this:

typedef struct {
	char	f;
	uchar	l[4];
	char	g;
	uchar	m[4];
} Dac960fu;

and convert l and m using the usual integer-to-bytes and
bytes-to-integers macros or functions, presumably the little-endian
versions.  Mylex were crazy to lay the data out this way.

Re: [9fans] structure allocation.

[erik quanstrom]

okay.

many things about this interface don't make sense to me.
but it's hard at this point to tell if driver is obfuscated or what.

the space/time tradeoff here seems interesting. is pushing 3 more bytes
over a pci bus really slower than doing the alignment on an ad-hoc basis?

the trouble is these structures go on forever and have many
bitfields. (in one i count 46 fields, all integers of random width,
randomly intermixed.)

- erik

geoff@collyer.net writes

|
| You declare it like this:
|
| typedef struct {
| 	char	f;
| 	uchar	l[4];
| 	char	g;
| 	uchar	m[4];
| } Dac960fu;
|
| and convert l and m using the usual integer-to-bytes and
| bytes-to-integers macros or functions, presumably the little-endian
| versions.  Mylex were crazy to lay the data out this way.

Re: [9fans] structure allocation.

[Russ Cox]

you can put #pragma pack on / #pragma pack off
around the offending structures too, but we won't
put code using that into the distribution.

russ

Re: [9fans] structure allocation.

[geoff]

I suspect that the difference in time to access unaligned packed data
vs. aligned data is irrelevant when compared with the general
complexity of running a modern host adaptor.

A more rational layout would be

typedef struct {
	uchar	l[4];
	uchar	m[4];
	char	f;
	char	g;
} Dac960fu;

which takes 12 bytes (including padding at the end) vs. 10 for the
unaligned packed version.  Since PCI buses transfer whole longs, it
would take 3 transfers in either case.  If you've got 46 integers of
variable size, it makes sense to sort them by size, at least as a
first cut, to minimise space wasted by padding.  You can also
sometimes pack more tightly than a simple sort would suggest (e.g., if
you have shorts and chars).

It sounds like the folks who designed the dac960 either didn't think
much about how drivers would access it, or they were hog wild over
gcc's packed data attribute (does microsoft's compiler have something
similar?).

Re: [9fans] structure allocation.

[Ronald G Minnich]

geoff@collyer.net wrote:
> I suspect that the difference in time to access unaligned packed data
> vs. aligned data is irrelevant when compared with the general
> complexity of running a modern host adaptor.
>

It can get worse. Clever use of align and packed resulted, in Xen, in
longword accesses which were odd-byte-aligned half the time, odd-short
aligned 1/4 the time; i.e., misaligned 3/4 of the time. All this from a
goal to be efficient! I kind of wonder how much this packed/aligned
mania results in unaligned accesses in the end ...

This did get fixed. But it's a real warning against the kind of
cleverness that people resort to with gcc nowadays -- just wander
through the linux kernel sometime -- grep unlikely if you wish.

> It sounds like the folks who designed the dac960 either didn't think
> much about how drivers would access it, or they were hog wild over
> gcc's packed data attribute (does microsoft's compiler have something
> similar?).
>

Packed has been around in compilers for realtime and embedded for 30
years -- dare I admit to using it at some time? Nah. You were none of
you born then anyway, so you can't prove a thing. That code is on a
magtape in a landfill somewhere in Pennsylvania.

ron

Re: [9fans] structure allocation.

[C H Forsyth]

> This did get fixed. But it's a real warning against the kind of
> cleverness that people resort to with gcc nowadays -- just wander
> through the linux kernel sometime -- grep unlikely if you wish.

yes, people do all that micromanagement, and then wade through pages of code
to get anywhere

Re: [9fans] structure allocation.

[erik quanstrom]

what compiler were you using with packed structures in 1976?

- erik

Ronald G Minnich <rminnich@lanl.gov> writes

| Packed has been around in compilers for realtime and embedded for 30
| years -- dare I admit to using it at some time? Nah. You were none of
| you born then anyway, so you can't prove a thing. That code is on a
| magtape in a landfill somewhere in Pennsylvania.
|
| ron

Re: [9fans] structure allocation.

[Ronald G Minnich]

erik quanstrom wrote:
> what compiler were you using with packed structures in 1976?
>
> - erik
>

It was 1978. It was called Microl, and was developed by someone I know
at HP.

We used it to write code for Z80s. It also had bit fields that were NOT
a botch and a type-safe linker. In fact, it had some features that I was
not to see in the C world until several decades later.

ron

Re: [9fans] structure allocation.

[geoff]

I was implicitly referring to C compilers.  Heck, Pascal had packed
data in the early 1970s, possibly even the late 1960s.

Re: [9fans] structure allocation.

[Bruce Ellis]

i still like hjdicks  it is obscure enough that no-one would think it's
a feature (or guess it).  it was required because we had a large
slab of 3rd-party code that assumed it could read packets off the
wire (assuming correct endian) and do no marshaling.

#pragam pack

looks like a feature.

i was there when it happened (after a nice italian meal).

ken asked "Do i really have to do this?"

P: Yes, there's buckets of code that rely on it.

K: *some expression of disbelief*

P: well hj are just dicks

done deal

it also turned out to be important for inferno on machines with
greater than 32 bit alignment requirements.  the 64 bit mips
is an example.  took but a recompile with hjdicks in the right
place (it takes an optional alignment parameter).  same with
the ps2 which has 128 bit issues.

thanks for telling me that it has been changed.

brucee

On 3/8/06, geoff@collyer.net <geoff@collyer.net> wrote:
> I was implicitly referring to C compilers.  Heck, Pascal had packed
> data in the early 1970s, possibly even the late 1960s.

Re: [9fans] structure allocation.

[Russ Cox]

> thanks for telling me that it has been changed.

You knew it was changed; you just blocked out the trauma.
http://marc.theaimsgroup.com/?t=111559007700007&r=1&w=2

Russ

Re: [9fans] structure allocation.

[Bruce Ellis]

oh that, great - doesn't help at all if you can't specify alignment.

brucee

On 3/8/06, Russ Cox <rsc@swtch.com> wrote:
> > thanks for telling me that it has been changed.
>
> You knew it was changed; you just blocked out the trauma.
> http://marc.theaimsgroup.com/?t=111559007700007&r=1&w=2
>
> Russ

Re: [9fans] structure allocation.

[Russ Cox]

> oh that, great - doesn't help at all if you can't specify alignment.

who said you can't?  it's not mentioned in the thread
because it wasn't relevant, but the old code is there,
just now accessed as #pragma pack and in every
compiler instead of just vc.

russ

Re: [9fans] structure allocation.

[Bruce Ellis]

who said you could?

brucee

On 3/8/06, Russ Cox <rsc@swtch.com> wrote:
> > oh that, great - doesn't help at all if you can't specify alignment.
>
> who said you can't?  it's not mentioned in the thread
> because it wasn't relevant, but the old code is there,
> just now accessed as #pragma pack and in every
> compiler instead of just vc.
>
> russ

Re: [9fans] structure allocation.

[Russ Cox]

>>> oh that, great - doesn't help at all if you can't specify alignment.
>> who said you can't?  it's not mentioned in the thread
> who said you could?

http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cc/dpchk.c

Re: [9fans] structure allocation.

[Bruce Ellis]

ok, sorry for not reading the source.  i have my own.

brucee

On 3/8/06, Russ Cox <rsc@swtch.com> wrote:
> >>> oh that, great - doesn't help at all if you can't specify alignment.
> >> who said you can't?  it's not mentioned in the thread
> > who said you could?
>
> http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cc/dpchk.c

Re: [9fans] structure allocation.

[Brantley Coile]

> I was implicitly referring to C compilers.  Heck, Pascal had packed
> data in the early 1970s, possibly even the late 1960s.

The first Pascal compiler became operational in 1970, on a CDC 6400.
For those born after Nixon, the CDC 6000 series was a 60 bit word
addressable machine.  That explains the packed keyword.

Implementing a recursive programing language on a machine that stores
the return address of a procedure call in the first word of the
procedure was a trick.  The speed of the original Pascal compiler on
the CDC machine wasn't that good.  I used the compiler at the
University of Georgia in the late 1970's.  Software to talk to our
packet network was written in it.

The packed keyword is still in the Pascal spec, but was dropped in Modula-2,
and Oberon.

Re: [9fans] structure allocation.

[Bakul Shah]

> It sounds like the folks who designed the dac960 either didn't think
> much about how drivers would access it, or they were hog wild over
> gcc's packed data attribute (does microsoft's compiler have something
> similar?).

I suspect that as usual either the h/w designers weren't
thinking about the driver guys or didn't listen to them, or
the driver guys didn't come on board until after the "design
was done".  The h/w logic is probably doing the equivalent of

	for (i = 0; i < 10; i++) {
		data_bus <= byte[i];
		assert ready;
		wait ack;
	}

In the scheme of things this particular register layout may
have come about because that is how the design evolved or it
was an arbitrary choice or it minimized the number of logic
gates to fit in the fewest number of PALs (likely the DAC960
still carries some logic originally designed for a board full
of PALs and what not).

Re: [9fans] structure allocation.

[Charles Forsyth]

> the trouble is these structures go on forever and have many
> bitfields. (in one i count 46 fields, all integers of random width,
> randomly intermixed.)

they shouldn't rely on bitfields.  in particular, the language does
not guarantee which order they are assigned to bits in a larger
unit (ie, left to right or right to left).

the nice thing is that in the plan 9 environment, you get to
fix such mistakes and do much better (although it's always
tedious to discover that late in the day)

Re: [9fans] structure allocation.

[erik quanstrom]

thanks to all for the information.

i was suprised that the plan9 c compiler supports bit fields.

- erik

Charles Forsyth <forsyth@terzarima.net> writes

|
| > the trouble is these structures go on forever and have many
| > bitfields. (in one i count 46 fields, all integers of random width,
| > randomly intermixed.)
|
| they shouldn't rely on bitfields.  in particular, the language does
| not guarantee which order they are assigned to bits in a larger
| unit (ie, left to right or right to left).
|
| the nice thing is that in the plan 9 environment, you get to
| fix such mistakes and do much better (although it's always
| tedious to discover that late in the day)

Re: [9fans] structure allocation.

[jmk]

On Tue Mar  7 06:55:24 EST 2006, quanstro@quanstro.net wrote:
> thanks to all for the information.
>
> i was suprised that the plan9 c compiler supports bit fields.
>
> - erik

they were only added under duress.

Re: [9fans] structure allocation.

["Nils O. Selåsdal"]

erik quanstrom wrote:
> thanks to all for the information.
>
> i was suprised that the plan9 c compiler supports bit fields.
Just pretend they arn't there ? :-)
You can't set them as part of a struct initializer btw.

Re: [9fans] structure allocation.

[uriel]

> erik quanstrom wrote:
>> thanks to all for the information.
>>
>> i was suprised that the plan9 c compiler supports bit fields.
> Just pretend they arn't there ? :-)
> You can't set them as part of a struct initializer btw.

If I recall correctly #pragma pack used to be #pragma hjdicks,
I liked better the old name.  Damned political correctness.

uriel

Re: [9fans] structure allocation.

[Russ Cox]

> If I recall correctly #pragma pack used to be #pragma hjdicks,
> I liked better the old name.  Damned political correctness.

I made the change, and I did it because pack is a better name.
It is less obscure and accurately describes the effect of the
pragma.  If it had been named #pragma xyzzy, I would have
renamed it for exactly the same reasons.

Russ

Re: [9fans] structure allocation.

[C H Forsyth]

> If I recall correctly #pragma pack used to be #pragma hjdicks,
> I liked better the old name.  Damned political correctness.

better still is just to forget about it and not use it.
there is little point to it since it will trap on enough platforms
to make it useless in portable code.  the one minor thing it does that won't trap
in some cases (changing the rounding size for structs, provided
they are never put in an array) can be done other ways.
use it only in similar ways to those originally intended: you've got a big lump of imported protocol
code that's made unfortunate assumptions and you've got an architecture
that won't trap unaligned accesses, and the ends justify the means.

Re: [9fans] structure allocation.

[erik quanstrom]

i've done well in ignoring them so far. ;-)

"Nils O. Selåsdal" <NOS@Utel.no> writes

|
| erik quanstrom wrote:
| > thanks to all for the information.
| >
| > i was suprised that the plan9 c compiler supports bit fields.
| Just pretend they arn't there ? :-)
| You can't set them as part of a struct initializer btw.