Re: [9fans] how to avoid a memset() optimization

Re: [9fans] how to avoid a memset() optimization

[presotto]

My probably incorrect 2 cents.  I think the basic problem is
bigger than volatile's power to solve.  There is no silver bullet.

My only experience with volatile was trying to make inferno emu code
run when compiled with non-plan9 compilers.  I never did figure out
everywhere it was needed.  Our waserror()/nexterror() stuff completely
blew away optimizing compilers since they are essentially longjumps.
Therefore, the compilers didn't know the active scope of a variable
so that registerization (and other opts) broke the code.  The appropriate
application of volatile fixed the problem though appropriate seemed
to be different for each compiler.  The general rule of thumb was that
any variable used by the waserror block should be volatile.  This seemed
to work in most instances but not all.  We had afternoons of sitting around
the Unix room trying to define exactly what it was that volatile
did in different compilers.  I never felt safe about the emu code
because of it.

My three main conclusions from this were:
1) waserror/nexterror is definitely evil unless understood by the
 compiler and perhaps even then.  It's a step outside the language
 definition and therefore a dangerous step to take.  We were
 comfortable with it in Plan 9 because we controlled the compilers
 but it became tortuous when someone else controlled them.

2) Volatile's meaning needs to grow to encompass the optimization
 du jour (don't registerize, don't optimize away memory accesses,
 don't optimize away condition code changes, don't inline code
 that contains it, ...) or we need more constructs.  It's hard to
 not screw up when using a construct whose meaning requires an in
 depth knowledge of what the compiler does.  To a certain extent, knowing
 the compiler's properties is a prerequisite to working in a kernel
 but there's a limit to what you have to understand.

 I'ld be happy if volatile just meant what I think it was originally
 intended for: don't optimize away or reorder any memory references.
 However, that in itself has myriad side effects.
 People will start to use it to avoid loop unrollings etc not envisioned
 by the compiler writer.

3) I'm too stupid to understand what C has become.  Perhaps I should go back
 to assembler.  Oops, I'm way to stupid to understand what ken's
 assembler does, I should go back to 1's and 0's.

The idea that compiler optimization is a knob that you turn till some
assumption you made becomes incorrect is scarey to me.  Very few
people understand the languages they use well enough to know when
they're treading on dangerous ground.  Even fewer testing environments
are complete enough to notice when something really has broken such
as the inadvertant creation of covert channels that got this started.
Luckily incorrect behavior in most programs doesn't really matter
because what most programs do is pretty ill defined.

Re: [9fans] how to avoid a memset() optimization

[Douglas A. Gwyn]

presotto@plan9.bell-labs.com wrote:
> The idea that compiler optimization is a knob that you turn
> till some assumption you made becomes incorrect is scarey to me.

A major purpose of the language standard is to define the
"treaty point": what a careful programmer has a right to expect
from the implementation and what latitude the implementation is
allowed.  The problem with assumptions is that there is no
standard for them.

Re: [9fans] how to avoid a memset() optimization

[William Josephson]

On Fri, Nov 15, 2002 at 10:50:38AM +0000, Douglas A. Gwyn wrote:
> presotto@plan9.bell-labs.com wrote:
> > The idea that compiler optimization is a knob that you turn
> > till some assumption you made becomes incorrect is scarey to me.
>
> A major purpose of the language standard is to define the
> "treaty point": what a careful programmer has a right to expect
> from the implementation and what latitude the implementation is
> allowed.  The problem with assumptions is that there is no
> standard for them.

While this is a fine sentiment, it is, in practice, not so easily
applied by the programmer.  In general, it is easier for run of
the mill applications and much harder for operating systems.  Can
you name any major operating system written in C that actually
compiles reliably with more than one compiler?  How many even work
with different major releases of the compiler?  Some of this is
probably due to sloppiness, but I don't think it is reasonable to
claim that it is only due to sloppiness.  The fact of the matter
seems to be that in the real world the standard, although important,
isn't as useful as you're claiming.

Re: [9fans] how to avoid a memset() optimization

[Douglas A. Gwyn]

William Josephson wrote:
> Can you name any major operating system written in C that actually
> compiles reliably with more than one compiler?

Unix used to; the real-time embedded OS I'm using (RTXC) does.
All my embedded systems code does.  It's not hard, really, if
you code to the standard.  There are certain operations that
can be difficult on some architectures, such as the VAX and
PDP-11 access pattern requirements that we've discussed, but
(some) newer architectures are more robustly designed.

Re: [9fans] how to avoid a memset() optimization

[Ronald G. Minnich]

On Mon, 18 Nov 2002, Douglas A. Gwyn wrote:

> William Josephson wrote:
> > Can you name any major operating system written in C that actually
> > compiles reliably with more than one compiler?
>
> Unix used to; the real-time embedded OS I'm using (RTXC) does.

wow, missed that question and the answer.

Things have sure gone downhill, eh? Our OS source code, written in C, is
less portable now than 14 years ago. Oh, well, what's a little of this
among friends:

#define ____cacheline_aligned __attribute__((__aligned__(SMP_CACHE_BYTES)))

doesn't every C compiler support that?

I, sir, am a curmudgeon. I have it on the best authority.

ron

Re: [9fans] how to avoid a memset() optimization

[Roman V. Shaposhnick]

On Thu, Nov 14, 2002 at 11:47:11AM -0500, presotto@plan9.bell-labs.com wrote:
> Our waserror()/nexterror() stuff completely
> blew away optimizing compilers since they are essentially longjumps.
........
> My three main conclusions from this were:
> 1) waserror/nexterror is definitely evil unless understood by the
>  compiler and perhaps even then.  It's a step outside the language
>  definition and therefore a dangerous step to take.  We were
>  comfortable with it in Plan 9 because we controlled the compilers
>  but it became tortuous when someone else controlled them.

  That's a very interesting remark, since I've always wondered why don't
  you use waserror()/nexterror() in the rest of the Plan9 source tree
  as a regular error handling mechanism.

  Personally I'm still struggling with developing an error handling
  policy that I'd feel comfortable with. Of course, good old:

       if ((a = do_it()) == BadThingHappened)
            return TellEmAboutIt;

  works, but costs a little bit too much when clarity is needed. Especially
  when a "transaction" like pattern is needed ( e.g. I need to
  to have o1 = f1(); o2 = f2(); o3 = f3(); but if any of f* fails,
  I have to destroy what I've got so far ).

  C++ style exceptions are nice, but easily abused, and I haven't seen
  any way of emulating then in pure C.

  So, what your experience has been with error handling ( granted, you
  must've had plenty ;-) ) and what would you consider the most
  comfortable one ?

Thanks,
Roman.

Re: [9fans] how to avoid a memset() optimization

[Douglas A. Gwyn]

Roman V. Shaposhnick wrote:
>   C++ style exceptions are nice, but easily abused,
 > and I haven't seen any way of emulating then in pure C.

Nearly any programming facility can be abused.
Of course C++ exceptions cannot be exactly emulated
in C, because there is no possible way to tie the
handling to the type as in C++.  However, similar
facilities can be built on top of setjmp/longjmp,
and several people have done so.  Send me e-mail at
my work address and I'll be happy to send you my
implementation which I call the "Ex" package.

Re: [9fans] how to avoid a memset() optimization

[Roman V. Shaposhnick]

[ DISCLAIMER: sorry for wide distribution, but I had no luck contacting
  Douglas at his e-mail ].

On Wed, Nov 20, 2002 at 09:47:53AM +0000, Douglas A. Gwyn wrote:
> Roman V. Shaposhnick wrote:
> >   C++ style exceptions are nice, but easily abused,
>  > and I haven't seen any way of emulating then in pure C.
>
> Nearly any programming facility can be abused.

  Oh, that's for sure. I guess I just have a "sausage
  problem" with C++ ( you, know, people who make 'em,
  can't eat 'em ) since I've been involved in C++ compiler
  development for quite some time. Since then, I'm
  a pure C diehard but I miss very basic functionality
  of what was known as "C with classes" from time to time.

> Of course C++ exceptions cannot be exactly emulated
> in C, because there is no possible way to tie the
> handling to the type as in C++.  However, similar
> facilities can be built on top of setjmp/longjmp,
> and several people have done so.  Send me e-mail at
> my work address and I'll be happy to send you my
> implementation which I call the "Ex" package.

  That'll be wonderful. Thank you.

Thanks,
Roman.

Re: [9fans] how to avoid a memset() optimization

[Douglas A. Gwyn]

"Roman V. Shaposhnick" wrote:
> [ DISCLAIMER: sorry for wide distribution, but I had no luck
> contacting Douglas at his e-mail ].

I got the e-mail, but the package I sent in response got
queued up due to inability to connect directly to your
host.  If it doesn't arrive within a couple of days
send me another note.

Re: [9fans] how to avoid a memset() optimization

[Ralph Corderoy]

Hi Roman,

> Of course, good old:
>      
>      if ((a = do_it()) == BadThingHappened)
>           return TellEmAboutIt;
> 
> works, but costs a little bit too much when clarity is needed.
> Especially when a "transaction" like pattern is needed ( e.g. I need
> to to have o1 = f1(); o2 = f2(); o3 = f3(); but if any of f* fails, I
> have to destroy what I've got so far ). 

A common way of doing this is using goto.

        failed = 1;

        if ((res1 = lock(A) == 0) {
            goto releaseA;
        }
        if ((res2 = lock(B) == 0) {
            goto releaseB;
        }
        if ((res3 = lock(C) == 0) {
            goto releaseC;
        }

        foo(res1, res2, res3);
        failed = 0;

    releaseC:
        release(C);
    releaseB:
        release(B);
    releaseA:
        release(A);

        return failed;

Cheers,

-- 
Ralph Corderoy.      http://inputplus.co.uk/ralph/     http://troff.org/

Re: [9fans] how to avoid a memset() optimization

[Sam]

or:

{
	int m=0;

	if(++m && (res1 = lock(A)))
	if(++m && (res2 = lock(B)))
	if(++m && (res3 = lock(C))) {
		foo();
		return success;
	}
	switch(m) {
	case 3:	 release(B);
	case 2:  release(A);
	default: return failure;
	}
}

you just need a marker to see how far you got. (I'm not even sure
what the context of this message is, but every time i see the goto
solution to this I have to say something). ;)

Cheers,

Sam

>
>         failed = 1;
>
>         if ((res1 = lock(A) == 0) {
>             goto releaseA;
>         }
>         if ((res2 = lock(B) == 0) {
>             goto releaseB;
>         }
>         if ((res3 = lock(C) == 0) {
>             goto releaseC;
>         }
>
>         foo(res1, res2, res3);
>         failed = 0;
>
>     releaseC:
>         release(C);
>     releaseB:
>         release(B);
>     releaseA:
>         release(A);
>
>         return failed;
>
> Cheers,
>
>

Re: [9fans] how to avoid a memset() optimization

[Russ Cox]

> you just need a marker to see how far you got. (I'm not even sure
> what the context of this message is, but every time i see the goto
> solution to this I have to say something). ;)

the goto seems much clearer to me.
also easier to edit.  what happens when
you have to lock A½?  you have to rewrite
all your case statements!

g% grep goto /sys/games/lib/fortunes |sed 1q
If you want to go somewhere, goto is the best way to get there. K Thompson
g% 

also you can't always write the code in your if-style,
but you can always use a goto.  for example, you can't
rewrite this without goto (or at least, if you do, it ends
up a lot less clear or a lot more duplicated):

static int
p9skclient(Conv *c)
{
	char *user;
	char cchal[CHALLEN];
	uchar secret[8];
	char buf[MAXAUTH];
	int speakfor, ret;
	Attr *a;
	Authenticator au;
	Key *k;
	Ticket t;
	Ticketreq tr;

	ret = -1;
	a = nil;
	k = nil;

	/* p9sk1: send client challenge */
	if(c->proto == &p9sk1){
		c->state = "write challenge";
		memrandom(cchal, CHALLEN);
		if(convwrite(c, cchal, CHALLEN) < 0)
			goto out;
	}

	/* read ticket request */
	c->state = "read tickreq";
	if(convread(c, buf, TICKREQLEN) < 0)
		goto out;
	convM2TR(buf, &tr);

	/* p9sk2: use server challenge as client challenge */
	if(c->proto == &p9sk2)
		memmove(cchal, tr.chal, CHALLEN);

	/*
	 * find a key.
	 *
	 * if the user is the factotum owner, any key will do.
	 * if not, then if we have a speakfor key,
	 * we will only vouch for the user's local identity.
	 *
	 * this logic is duplicated in p9any.c
	 */
	user = strfindattr(c->attr, "user");
	a = delattr(copyattr(c->attr), "role");
	a = addattr(a, "proto=p9sk1");

	if(strcmp(c->sysuser, owner) == 0){
		speakfor = 0;
		a = addattr(a, "proto=p9sk1 user? dom=%q", tr.authdom);
	}else if(user==nil || strcmp(c->sysuser, user)==0){
		speakfor = 1;
		a = delattr(a, "user");
		a = addattr(a, "proto=p9sk1 user? dom=%q role=speakfor", tr.authdom);
	}else{
		werrstr("will not authenticate for %q as %q", c->sysuser, user);
		goto out;
	}

	for(;;){
		c->state = "find key";
		k = keyfetch(c, "%A", a);
		if(k == nil)
			goto out;
		
		/* relay ticket request to auth server, get tickets */
		strcpy(tr.hostid, strfindattr(k->attr, "user"));
		if(speakfor)
			strcpy(tr.uid, c->sysuser);
		else
			strcpy(tr.uid, tr.hostid);

		c->state = "get tickets";
		if(gettickets(&tr, buf, k) < 0)
			goto out;

		convM2T(buf, &t, k->priv);
		if(t.num == AuthTc)
			break;

		/* we don't agree with the auth server about the key; try again */
		c->state = "replace key";
		if((k = keyreplace(c, k, "key mismatch with auth server")) == nil){
			werrstr("key mismatch with auth server");
			goto out;
		}
	}

	/* send second ticket and authenticator to server */
	c->state = "write ticket+auth";
	memmove(buf, buf+TICKETLEN, TICKETLEN);
	au.num = AuthAc;
	memmove(au.chal, tr.chal, CHALLEN);
	au.id = 0;
	convA2M(&au, buf+TICKETLEN, t.key);
	if(convwrite(c, buf, TICKETLEN+AUTHENTLEN) < 0)
		goto out;

	/* read authenticator from server */
	c->state = "read auth";
	if(convread(c, buf, AUTHENTLEN) < 0)
		goto out;
	convM2A(buf, &au, t.key);
	if(au.num != AuthAs || memcmp(au.chal, cchal, CHALLEN) != 0 || au.id != 0){
		werrstr("server lies through his teeth");
		goto out;
	}

	/* success */
	c->attr = addcap(c->attr, c->sysuser, &t);
	des56to64((uchar*)t.key, secret);
	c->attr = addattr(c->attr, "secret=%.8H", secret);
	ret = 0;

out:
	freeattr(a);
	keyclose(k);
	return ret;
}

[9fans] Fs Kernel

[Conor Williams]

which kernel mkfile do I use for the file server?
thanks
will551

Re: [9fans] how to avoid a memset() optimization

[David Presotto]

There's always waserror/nexterror

Re: [9fans] how to avoid a memset() optimization

[Sam]

ooh, a challenge.  Nothing pleases me more than proving myself
wrong.  be back in a few. ;)

btw - i don't fear the goto.  I use it a lot in small loop
situations to avoid empty for loops and such.  I'll play
with this function and see what I find. ;)

Cheers,

Sam

On Mon, 6
Jan 2003, Russ Cox wrote:

> > you just need a marker to see how far you got. (I'm not even sure
> > what the context of this message is, but every time i see the goto
> > solution to this I have to say something). ;)
>
> the goto seems much clearer to me.
> also easier to edit.  what happens when
> you have to lock A½?  you have to rewrite
> all your case statements!
>
> g% grep goto /sys/games/lib/fortunes |sed 1q
> If you want to go somewhere, goto is the best way to get there. K Thompson
> g%
>
> also you can't always write the code in your if-style,
> but you can always use a goto.  for example, you can't
> rewrite this without goto (or at least, if you do, it ends
> up a lot less clear or a lot more duplicated):
>
> static int
> p9skclient(Conv *c)
> {
> 	char *user;
> 	char cchal[CHALLEN];
> 	uchar secret[8];
> 	char buf[MAXAUTH];
> 	int speakfor, ret;
> 	Attr *a;
> 	Authenticator au;
> 	Key *k;
> 	Ticket t;
> 	Ticketreq tr;
>
> 	ret = -1;
> 	a = nil;
>  	k = nil;
>
> 	/* p9sk1: send client challenge */
> 	if(c->proto == &p9sk1){
> 		c->state = "write challenge";
> 		memrandom(cchal, CHALLEN);
> 		if(convwrite(c, cchal, CHALLEN) < 0)
> 			goto out;
> 	}
>
> 	/* read ticket request */
> 	c->state = "read tickreq";
> 	if(convread(c, buf, TICKREQLEN) < 0)
> 		goto out;
> 	convM2TR(buf, &tr);
>
> 	/* p9sk2: use server challenge as client challenge */
> 	if(c->proto == &p9sk2)
> 		memmove(cchal, tr.chal, CHALLEN);
>
> 	/*
> 	 * find a key.
> 	 *
> 	 * if the user is the factotum owner, any key will do.
> 	 * if not, then if we have a speakfor key,
> 	 * we will only vouch for the user's local identity.
> 	 *
> 	 * this logic is duplicated in p9any.c
> 	 */
> 	user = strfindattr(c->attr, "user");
> 	a = delattr(copyattr(c->attr), "role");
> 	a = addattr(a, "proto=p9sk1");
>
> 	if(strcmp(c->sysuser, owner) == 0){
> 		speakfor = 0;
> 		a = addattr(a, "proto=p9sk1 user? dom=%q", tr.authdom);
> 	}else if(user==nil || strcmp(c->sysuser, user)==0){
> 		speakfor = 1;
> 		a = delattr(a, "user");
> 		a = addattr(a, "proto=p9sk1 user? dom=%q role=speakfor", tr.authdom);
> 	}else{
> 		werrstr("will not authenticate for %q as %q", c->sysuser, user);
> 		goto out;
> 	}
>
> 	for(;;){
> 		c->state = "find key";
> 		k = keyfetch(c, "%A", a);
> 		if(k == nil)
> 			goto out;
>
> 		/* relay ticket request to auth server, get tickets */
> 		strcpy(tr.hostid, strfindattr(k->attr, "user"));
> 		if(speakfor)
> 			strcpy(tr.uid, c->sysuser);
> 		else
> 			strcpy(tr.uid, tr.hostid);
>
> 		c->state = "get tickets";
> 		if(gettickets(&tr, buf, k) < 0)
> 			goto out;
>
> 		convM2T(buf, &t, k->priv);
> 		if(t.num == AuthTc)
> 			break;
>
> 		/* we don't agree with the auth server about the key; try again */
> 		c->state = "replace key";
> 		if((k = keyreplace(c, k, "key mismatch with auth server")) == nil){
> 			werrstr("key mismatch with auth server");
> 			goto out;
> 		}
> 	}
>
> 	/* send second ticket and authenticator to server */
> 	c->state = "write ticket+auth";
> 	memmove(buf, buf+TICKETLEN, TICKETLEN);
> 	au.num = AuthAc;
> 	memmove(au.chal, tr.chal, CHALLEN);
> 	au.id = 0;
> 	convA2M(&au, buf+TICKETLEN, t.key);
> 	if(convwrite(c, buf, TICKETLEN+AUTHENTLEN) < 0)
> 		goto out;
>
> 	/* read authenticator from server */
> 	c->state = "read auth";
> 	if(convread(c, buf, AUTHENTLEN) < 0)
> 		goto out;
> 	convM2A(buf, &au, t.key);
> 	if(au.num != AuthAs || memcmp(au.chal, cchal, CHALLEN) != 0 || au.id != 0){
> 		werrstr("server lies through his teeth");
> 		goto out;
> 	}
>
> 	/* success */
> 	c->attr = addcap(c->attr, c->sysuser, &t);
> 	des56to64((uchar*)t.key, secret);
> 	c->attr = addattr(c->attr, "secret=%.8H", secret);
> 	ret = 0;
>
> out:
> 	freeattr(a);
> 	keyclose(k);
> 	return ret;
> }
>

Re: [9fans] Fs Kernel

[C H Forsyth]

[-- Attachment #1: Type: text/plain, Size: 40 bytes --]

see /sys/src/fs/words
for a few hints

[-- Attachment #2: Type: message/rfc822, Size: 1389 bytes --]

From: nigel@9fs.org
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Fs Kernel
Date: Mon, 6 Jan 2003 17:22:33 0000
Message-ID: <fa0a29ffa78d7a4f560ed841a58a2020@9fs.org>

> which kernel mkfile do I use for the file server?
> thanks
> will551

/sys/src/fs/*/mkfile

Re: [9fans] Fs Kernel

[nigel]

> which kernel mkfile do I use for the file server?
> thanks
> will551

/sys/src/fs/*/mkfile