9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14 16:47 presotto
  2002-11-15 10:50 ` Douglas A. Gwyn
  2002-11-19  7:38 ` Roman V. Shaposhnick
  0 siblings, 2 replies; 92+ messages in thread
From: presotto @ 2002-11-14 16:47 UTC (permalink / raw)
  To: 9fans

My probably incorrect 2 cents.  I think the basic problem is
bigger than volatile's power to solve.  There is no silver bullet.

My only experience with volatile was trying to make inferno emu code
run when compiled with non-plan9 compilers.  I never did figure out
everywhere it was needed.  Our waserror()/nexterror() stuff completely
blew away optimizing compilers since they are essentially longjumps.
Therefore, the compilers didn't know the active scope of a variable
so that registerization (and other opts) broke the code.  The appropriate
application of volatile fixed the problem though appropriate seemed
to be different for each compiler.  The general rule of thumb was that
any variable used by the waserror block should be volatile.  This seemed
to work in most instances but not all.  We had afternoons of sitting around
the Unix room trying to define exactly what it was that volatile
did in different compilers.  I never felt safe about the emu code
because of it.

My three main conclusions from this were:
1) waserror/nexterror is definitely evil unless understood by the
 compiler and perhaps even then.  It's a step outside the language
 definition and therefore a dangerous step to take.  We were
 comfortable with it in Plan 9 because we controlled the compilers
 but it became tortuous when someone else controlled them.

2) Volatile's meaning needs to grow to encompass the optimization
 du jour (don't registerize, don't optimize away memory accesses,
 don't optimize away condition code changes, don't inline code
 that contains it, ...) or we need more constructs.  It's hard to
 not screw up when using a construct whose meaning requires an in
 depth knowledge of what the compiler does.  To a certain extent, knowing
 the compiler's properties is a prerequisite to working in a kernel
 but there's a limit to what you have to understand.

 I'ld be happy if volatile just meant what I think it was originally
 intended for: don't optimize away or reorder any memory references.
 However, that in itself has myriad side effects.
 People will start to use it to avoid loop unrollings etc not envisioned
 by the compiler writer.

3) I'm too stupid to understand what C has become.  Perhaps I should go back
 to assembler.  Oops, I'm way to stupid to understand what ken's
 assembler does, I should go back to 1's and 0's.

The idea that compiler optimization is a knob that you turn till some
assumption you made becomes incorrect is scarey to me.  Very few
people understand the languages they use well enough to know when
they're treading on dangerous ground.  Even fewer testing environments
are complete enough to notice when something really has broken such
as the inadvertant creation of covert channels that got this started.
Luckily incorrect behavior in most programs doesn't really matter
because what most programs do is pretty ill defined.


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2003-01-06 10:47 nigel
  2003-01-06 11:15 ` Geoff Collyer
  0 siblings, 1 reply; 92+ messages in thread
From: nigel @ 2003-01-06 10:47 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

Or

        failed = 1;
        if ((res1 = lock(A) != 0)
	        if ((res2 = lock(B) != 0) {
		        if ((res3 = lock(C) != 0) {
				foo(res1, res2, res3);
				failed = 0;
			}
	if (res3 != 0)
        		release(C);
	if (res2 != 0)
        		release(B);
	if (res1 != 0)
        		release(A);

        return failed;

avoiding the goto.

Further, if release(nil) is implemented as a noop you can have

        failed = 1;
        if ((res1 = lock(A) != 0)
	        if ((res2 = lock(B) != 0) {
		        if ((res3 = lock(C) != 0) {
				foo(res1, res2, res3);
				failed = 0;
			}
	release(C);
	release(B);
	release(A);

        return failed;

[-- Attachment #2: Type: message/rfc822, Size: 2464 bytes --]

From: Ralph Corderoy <ralph@inputplus.co.uk>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] how to avoid a memset() optimization
Date: Mon, 6 Jan 2003 10:18:19 GMT
Message-ID: <3afa.3e16dd11.67d1a@blake.inputplus.co.uk>

Hi Roman,

> Of course, good old:
>      
>      if ((a = do_it()) == BadThingHappened)
>           return TellEmAboutIt;
> 
> works, but costs a little bit too much when clarity is needed.
> Especially when a "transaction" like pattern is needed ( e.g. I need
> to to have o1 = f1(); o2 = f2(); o3 = f3(); but if any of f* fails, I
> have to destroy what I've got so far ). 

A common way of doing this is using goto.

        failed = 1;

        if ((res1 = lock(A) == 0) {
            goto releaseA;
        }
        if ((res2 = lock(B) == 0) {
            goto releaseB;
        }
        if ((res3 = lock(C) == 0) {
            goto releaseC;
        }

        foo(res1, res2, res3);
        failed = 0;

    releaseC:
        release(C);
    releaseB:
        release(B);
    releaseA:
        release(A);

        return failed;

Cheers,

-- 
Ralph Corderoy.      http://inputplus.co.uk/ralph/     http://troff.org/

^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-19 14:32 presotto
  2002-11-20  7:24 ` Tomas
  0 siblings, 1 reply; 92+ messages in thread
From: presotto @ 2002-11-19 14:32 UTC (permalink / raw)
  To: 9fans

The exceptional style has its merits.  If C had a resonable multivalued
return mechanism like alef or limbo, I think it would have been a better
solution.

There's no reason other than personal preference to not use the same
model at user level, it's just a stack of long jmp variables.  However,
I haven't found the need.  Return values seem to work just as well
though they are much wordier.


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-19  8:21 Fco.J.Ballesteros
  0 siblings, 0 replies; 92+ messages in thread
From: Fco.J.Ballesteros @ 2002-11-19  8:21 UTC (permalink / raw)
  To: 9fans

>   That's a very interesting remark, since I've always wondered why don't
>   you use waserror()/nexterror() in the rest of the Plan9 source tree
>   as a regular error handling mechanism.

But they still control the compilers for Plan 9 non-kernel code.
In fact, I'm using the same kind of stuff for error handling in
several programs.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* [9fans] how to avoid a memset() optimization
@ 2002-11-18 21:36 Joel Salomon
  0 siblings, 0 replies; 92+ messages in thread
From: Joel Salomon @ 2002-11-18 21:36 UTC (permalink / raw)
  To: 9fans

Andrew Simmons wrote:
>http://groups.google.com/groups?threadm=oA47GsAf7M29EwY0%40robinton.demon.co.uk
this did not work for me, try:
http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&threadm=3DD35AAB.AA37D5CA%40monitorbm.co.nz&rnum=1
(should be all on one line)

--Joel
______________________________________________________
Due to economic circumstances, the light at the end of
the tunnel has been turned off.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-18 20:42 Andrew Simmons
  0 siblings, 0 replies; 92+ messages in thread
From: Andrew Simmons @ 2002-11-18 20:42 UTC (permalink / raw)
  To: 9fans

I was perturbed by Ron's remarks on structure layout in C++, and
posted a query on one of the C++ groups. I wasn't intending to draw
attention to it here, but the discussion has reached a stage that
might appeal to connoisseurs of arcana:

http://groups.google.com/groups?threadm=oA47GsAf7M29EwY0%40robinton.de
mon.co.uk


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-18 14:19 C H Forsyth
  0 siblings, 0 replies; 92+ messages in thread
From: C H Forsyth @ 2002-11-18 14:19 UTC (permalink / raw)
  To: 9fans


>>#define ____cacheline_aligned __attribute__((__aligned__(SMP_CACHE_BYTES)))

oh dear.  how do they come up with this stuff?

i think it's a Martian invasion, actually.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-15  1:56 Dennis Ritchie
  2002-11-15 10:51 ` Douglas A. Gwyn
  2002-11-15 12:03 ` Boyd Roberts
  0 siblings, 2 replies; 92+ messages in thread
From: Dennis Ritchie @ 2002-11-15  1:56 UTC (permalink / raw)
  To: 9fans

Gwyn remarked

 > C as you imagine it was never really in existence; even on
 > the PDP-11, certain access patterns (such as dereference
 >  of constants in the range 0160000-017777t) were specially
 > (kludge) excluded from optimization, so that device drivers
 > wouldn't be broken by non-1-to-1 translations from source
 > code to accesses.

This wasn't true for the PDP-11 compiler, which
(approximately) treated everything as volatile in the
current sense, though it could do some elementary
dead-code elimination.  The (early) BSD/VAX assembly
optimizer did have something like this, however, in that it
avoided certain instructions that wouldn't work in the I/O
space.

 > "volatile" addresses such issues as
 > well as we could with a simple portable mechanism.

The C90 and C99 semantics of volatile remain a bit flabby,
though it would take a lot of work to get a standard
that captured the behavior.  It's been tried.

The intent of volatile was to capture appropriate
behavior of memory-mapped registers and similar
things (like a clock in user space updated by the
OS.)  So, things like

	*p = 0;
	*p = 0;

should generate two stores if p is volatile *int.

One thing added in C90 was volatile's use in proper preservation
of values of automatics in the presence of
setjmp/longjmp. This was a non-issue for the PDP-11
because of details of its calling sequence, harder but
not impossible for the VAX because of its more complicated
(and slow) calling sequence, a real mess with the
advent of RISCs and optimization.  This is presumably
what hit Dave etc. with the setjmp-like waserror/nexterror
(the compiler didn't trigger on those words the
way they're supposed to on setjmp/longjmp).

	Dennis


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14 18:55 jmk
  2002-11-14 22:23 ` Steve Kilbane
  0 siblings, 1 reply; 92+ messages in thread
From: jmk @ 2002-11-14 18:55 UTC (permalink / raw)
  To: 9fans

On Thu Nov 14 13:32:18 EST 2002, tad@entrisphere.com wrote:
>
> someone mentioned something along the lines of
>
> void*
> secmemset(void *buf, int v, int len)
> {
> 	return memset((volatile void*)buf, v, len);
> }
>
> However, isn't GCC going to complain that you're passing
> a volatile to a function that isn't expecting a volatile?
>
> -Tad

Jarring chord. The door flies open and Cardinal Ximinez of Spain enters
flanked by two junior cardinals. Cardinal Biggles has goggles pushed over
his forehead. Cardinal Fang is just Cardinal Fang.

Ximinez:
	Nobody expects the Volatile Inquisition. Our chief weapon is surprise
	that your code doesn't work ... surprise and fear that you forgot an
	optimisation parameter... fear and surprise ... our two weapons are fear
	and surprise ... and ruthless efficiency instead of correctness. Our
	THREE weapons are fear, surprise and ruthless efficiency and an almost
	fanatical devotion to RMS ... Our FOUR ... no ... AMONGST our weaponry
	are such elements as fear, surprise ... I'll come in again.
	(exit and exeunt)


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14 18:17 presotto
  0 siblings, 0 replies; 92+ messages in thread
From: presotto @ 2002-11-14 18:17 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 107 bytes --]

nah, the inability to do a good job of something has
never stopped us before.  it's just lack of interest.

[-- Attachment #2: Type: message/rfc822, Size: 2113 bytes --]

From: Joel Salomon <salomo3@cooper.edu>
To: 9fans@cse.psu.edu
Subject: [9fans] how to avoid a memset() optimization
Date: Thu, 14 Nov 2002 13:11:17 -0500 (EST)
Message-ID: <Pine.SOL.3.96.1021114130542.22864A-100000@robin.cooper.edu>

> Anyone have any numbers showing that, "without optimizations,
> this simply wouldn't be tolerable," in the face of well
> written code?
I believe this is the case with the IA-64 architecture. I don't have any
experience writing compilers, but I would imagine that compile-time
concurrency analysis (necessary for *decent* performance on IA-64)
requires very clever optimizers.
Come to think of it, I don't believe that plan9's comilers have been
ported to IA-64 - is this the reason?

 --Joel
______________________________________________________
Due to economic circumstances, the light at the end of
the tunnel
has been
turned off.

^ permalink raw reply	[flat|nested] 92+ messages in thread
* [9fans] how to avoid a memset() optimization
@ 2002-11-14 18:11 Joel Salomon
  2002-11-14 18:26 ` William Josephson
  0 siblings, 1 reply; 92+ messages in thread
From: Joel Salomon @ 2002-11-14 18:11 UTC (permalink / raw)
  To: 9fans

> Anyone have any numbers showing that, "without optimizations,
> this simply wouldn't be tolerable," in the face of well
> written code?
I believe this is the case with the IA-64 architecture. I don't have any
experience writing compilers, but I would imagine that compile-time
concurrency analysis (necessary for *decent* performance on IA-64)
requires very clever optimizers.
Come to think of it, I don't believe that plan9's comilers have been
ported to IA-64 - is this the reason?

 --Joel
______________________________________________________
Due to economic circumstances, the light at the end of
the tunnel
has been
turned off.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14 17:44 rog
  2002-11-15 10:50 ` Douglas A. Gwyn
  0 siblings, 1 reply; 92+ messages in thread
From: rog @ 2002-11-14 17:44 UTC (permalink / raw)
  To: 9fans

> memset((volatile void*)p,n,0) would be similar if it were allowed.

that's odd.

i thought it had originally been suggested that:

void
f(void)
{
	volatile void *buf = malloc(n);

	memset(buf, n, 0);
}

was the "right" way to do it.
but now it appears that's not allowed...
does that mean that in fact there's *no* portable way
of doing it?!



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14 17:28 Russ Cox
  0 siblings, 0 replies; 92+ messages in thread
From: Russ Cox @ 2002-11-14 17:28 UTC (permalink / raw)
  To: 9fans

> Do you really want the compiler to always generate code
> 	MOV	R1,#0
> 	MOV	R2,#0
> 	AND	R2,R1
> 	MOV	R2,X(SP)
> ?  Almost any modern compiler will peform the AND at
> compiler time (constant folding), and most customers want
> that.  Anyway, I don't see why you need to AND in this
> context.  Are you talking about the C and V bits?  If some
> other process can inspect those for your process I'd think
> it could inspect a whole lot more..

i _said_ it was a stupid argument.  my point was more
that volatile addresses memory optimizations but not
necessarily other things which might matter just as much.



^ permalink raw reply	[flat|nested] 92+ messages in thread
[parent not found: <nemo@plan9.escet.urjc.es>]
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14  6:53 Russ Cox
  2002-11-14 10:22 ` Douglas A. Gwyn
  2002-11-19  7:20 ` Roman V. Shaposhnick
  0 siblings, 2 replies; 92+ messages in thread
From: Russ Cox @ 2002-11-14  6:53 UTC (permalink / raw)
  To: 9fans

>   He doesn't need to know about volatile -- that's the whole point.
>   Once "experienced" programmer has set the buffer that way, the
>   less experienced just gets less chance to screw himself up

I'd still rather see it closer to the use that needs it.
If you're used to seeing secmemset then seeing
memset is a big red flag.  If you're used to seeing
memset with a volatile declaration, then you have
to seek out the volatile declaration, which might not
be anywhere near the call to memset.

This is a stupid argument anyway.  Volatile is a crock.
Here's another problem.  Suppose I'm worried that
leaking the processor flags at the end of my computation
might reveal something about the secret key.  I want to
clear them.  I could do something like:

	x = 0 & 0;

which in this particular case will clear the appropriate
flags.  But now the compiler comes along and optimizes
away the whole expression.  Where do I put my volatile
to make that stay?  Hmm?  The answer is not more crappy
qualifiers.  The answer is clearly dumber compilers.

Russ



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-14  2:48 Dennis Ritchie
  2002-11-14  4:23 ` Ronald G. Minnich
  0 siblings, 1 reply; 92+ messages in thread
From: Dennis Ritchie @ 2002-11-14  2:48 UTC (permalink / raw)
  To: 9fans

Minnich remarked

 > I feel the need. Linuxbios no longer works on gcc 3.2 because of some
 > bizarre optimisations that have gone in that break mptable parsing. One
 > person claims that you can no longer count on this:

 > struct x {
 > 	int a;
 > 	int b;
 > };

 > resulting in code in which a and b are laid out in memory in the same
 > order as in the structure (this due to C++). Structure members can get
 > reordered.

 > I can't believe this is true, but someone claims it is and I have not had
 > time to verify it.

Dunno about C++, but C99 still specifies that
structure members are laid out in the order
they are declared.

	Dennis


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 18:58 Rob `Commander' Pike
  0 siblings, 0 replies; 92+ messages in thread
From: Rob `Commander' Pike @ 2002-11-13 18:58 UTC (permalink / raw)
  To: 9fans

> Yep, we need a C that works for its original purpose.

You, sir, are a curmudgeon.

-rob



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 14:40 C H Forsyth
  2002-11-13 15:54 ` rob pike
  0 siblings, 1 reply; 92+ messages in thread
From: C H Forsyth @ 2002-11-13 14:40 UTC (permalink / raw)
  To: 9fans

	It's not a bug, it's an allowed optimization.

no doubt that's true in this case, but i did think the phrase had
broader application, so i put it in my fortunes file.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 14:14 Skip Tavakkolian
  0 siblings, 0 replies; 92+ messages in thread
From: Skip Tavakkolian @ 2002-11-13 14:14 UTC (permalink / raw)
  To: 9fans

> How would Limbo do as a systems language?

Limbo does things for you that one wouldn't want in a systems
language, and neither is Alef, although the name would fit.

Nowadays do we need anything more than a Dis translator?  Everyone
could learn Dis or MMIX (Knuth's) and be done with it.  After all,
portability was the main goal.

Maybe Forth is the answer ☺



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 13:55 rog
  0 siblings, 0 replies; 92+ messages in thread
From: rog @ 2002-11-13 13:55 UTC (permalink / raw)
  To: 9fans

> For one thing, it doesn't work (unless you force
> volatile qualification within secmemset)

which seems reasonable to me.

and for barely more runtime cost, one could define:

void*
secmemset(void *ap, int c, ulong n)
{
	memset((volatile void *)ap, c, n);
}

(or is that "void * volatile"?  i never sussed this C type qualifier
stuff).

is that not sufficient to stymie any legitimate optimisations that GCC
might try to do, while maintaining comparable performance?

(not to mention making sure that you don't kill the performance of any
crypto tight loops that might be running on the original buffer by
declaring it volatile).

  cheers,
    rog.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 13:38 Skip Tavakkolian
  2002-11-13 16:25 ` Boyd Roberts
  0 siblings, 1 reply; 92+ messages in thread
From: Skip Tavakkolian @ 2002-11-13 13:38 UTC (permalink / raw)
  To: 9fans

> gcc(1) is a treasure hunt; there's all sorts of stuff buried in there.

There should be some sort of certification one could get for
knowing all gcc options.

On the issue of optimization, I think if the programmer has to figure
out what the compiler has done through exegesis or inference, then
there is something wrong with the compiler or the specification;

I feel I should only be required to know a compiler the way I know my
car.  I expect it to behave according to the manual.  I shouldn't be
required to know a compiler the way I know a close family member.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13 10:43 C H Forsyth
  2002-11-14 10:21 ` Douglas A. Gwyn
  0 siblings, 1 reply; 92+ messages in thread
From: C H Forsyth @ 2002-11-13 10:43 UTC (permalink / raw)
  To: 9fans

>>No, it mens "compiler, the access must really be
>>performed".  It is the programmer who decides whether
>>that is a requirement, and it could be for numerous
>>reasons, such as to assist in debugging, because the
"location" is a portal to an external device register,
>>because the object is shared among threads, etc.  In
>>this particular case it would be because the programmer
>>really wants the memory overwrite to occur regardless
>>of whether it is used any further.

so, in the case of memset applied to such a thing,
are those accesses that `must really be performed'
byte accesses, word accesses, quadword?



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13  6:52 Geoff Collyer
  2002-11-13 10:13 ` Boyd Roberts
  0 siblings, 1 reply; 92+ messages in thread
From: Geoff Collyer @ 2002-11-13  6:52 UTC (permalink / raw)
  To: 9fans

When I'm stuck with using gcc, I invoke it with this script:

#!/bin/sh
# agcc - invoke gcc in ansi mode
# the --writable-strings is for pre-ansi programs like some versions of mk
exec gcc -ansi -pipe -fvolatile --writable-strings "$@"

-fvolatile makes gcc treat all data as if declared volatile.  I
sometimes add (usually to the ?akefile) `-Dconst=' to deal with const
poisoning.

gcc(1) is a treasure hunt; there's all sorts of stuff buried in there.



^ permalink raw reply	[flat|nested] 92+ messages in thread
* [9fans] how to avoid a memset() optimization
@ 2002-11-13  6:34 Andrew Simmons
  2002-11-13  6:43 ` Doc Shipley
  0 siblings, 1 reply; 92+ messages in thread
From: Andrew Simmons @ 2002-11-13  6:34 UTC (permalink / raw)
  To: 9fans

>We don't optimize out subroutine calls that are in live
>codd, mostly because they might have side effects.

A wise policy. I tend to avoid all subroutine calls involving live fish,
especially the tench, roach, bream, and, most dangerous of all, the gudgeon.


^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13  1:47 Russ Cox
  2002-11-13 10:16 ` Douglas A. Gwyn
  2002-11-14  1:46 ` Roman V. Shaposhnick
  0 siblings, 2 replies; 92+ messages in thread
From: Russ Cox @ 2002-11-13  1:47 UTC (permalink / raw)
  To: 9fans

>   Putting aside issues with optimizing compilers, I don't see a "volatile"
>   qualifier as something that "hurts readability and clarity", I even
>   would say, that it is exactly the case where its use is justified.
>   Wouldn't you agree ?

My problem with volatile is that it means ``something
weird is going on here'' but doesn't tell you what.
It's also not near the weirdness.  Using smemset says
``this is a security-critical memset; it cannot get
optimized away.''  Otherwise I just see a memset
of a buffer that happens to be volatile.  What does
that tell me?  Very little.  And the volatile tag might
be miles away from the memset call, making it even
easier to forget that they're related.

>   You're right here, but for me defining secmemset is much bigger evil
>   than just adding volatile to the buffer.

Why?



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13  0:31 Russ Cox
  2002-11-13  1:26 ` Roman V. Shaposhnick
  2002-11-13 10:15 ` Douglas A. Gwyn
  0 siblings, 2 replies; 92+ messages in thread
From: Russ Cox @ 2002-11-13  0:31 UTC (permalink / raw)
  To: 9fans

> Plan9 has its own C compiler ) but seeing how gcc is making its
> way, it seems that declaring every buffer volatile ( as John the
> moderator has suggested ) wouldn't hurt either.

I disagree.  It hurts readability and clarity, all
to avoid a bug in gcc.  If I asked it to zero the
memory, then, damn it!, zero the memory.

The right fix is to avoid gcc.

After accepting the fact that gcc cannot be avoided,
cannot be fixed everywhere at once, and might not
ever be fixed anywhere at all, the right fix is to declare
a ``secure memset'' function:

	void*
	smemset(void *ap, int c, ulong n)
	{
		char *p;

		p = ap;
		while(n > 0) {
			*p++ = c;
			n--;
		}
		return ap;
	}

The only difference between this memset and the
standard portable libc memset is the name.
The point is that this only happens because gcc
is inlining the memset and then treating it as a
sequence of unused assignments.  Gcc ``knows'' too
much about memset.  If you call it something else,
that special knowledge goes away, as does the bug,
especially if smemset is in its own source file.

Russ



^ permalink raw reply	[flat|nested] 92+ messages in thread
* Re: [9fans] how to avoid a memset() optimization
@ 2002-11-13  0:20 presotto
  0 siblings, 0 replies; 92+ messages in thread
From: presotto @ 2002-11-13  0:20 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 163 bytes --]

We don't optimize out subroutine calls that are in live
codd, mostly because they might have side effects.  We
do optimize out expressions that don't get called.

[-- Attachment #2: Type: message/rfc822, Size: 3132 bytes --]

From: "Roman V. Shaposhnick" <vugluskr@unicorn.math.spbu.ru>
To: 9fans@cse.psu.edu
Subject: [9fans] how to avoid a memset() optimization
Date: Wed, 13 Nov 2002 01:42:17 +0300
Message-ID: <20021113014217.A5718@unicorn.math.spbu.ru>

There's quite a lively discussion on comp.compilers about memset()
use in security related applications and compiler optimization.
This idiom of memset'ing secure stuff with 0 is not new, and I've
spotted it in Plan9 sources several times -- don't know whether
it should be a concern for Plan9 users and developers ( after all,
Plan9 has its own C compiler ) but seeing how gcc is making its
way, it seems that declaring every buffer volatile ( as John the
moderator has suggested ) wouldn't hurt either.

Thanks,
Roman.

From: "Francis Wai" <fwai@rsasecurity.com>
Newsgroups: comp.compilers
Subject: how to avoid a memset() optimization
Date: 7 Nov 2002 00:51:51 -0500

In a recent article (http://online.securityfocus.com/archive/82/297827),
Peter Gutmann raised a concern which has serious implications in
secure programming. His example, along the lines of,

int main()
{
    char key[16];
    strcpy(key, "whatever");
    encrpts(key);
    memset(key, 0, 16);
}

where memset() was optimized away because memset() is the last
expression before the next sequence point and that its side-effect is
not needed and that the subject of memset() is an auto variable. The
compiler sees that it is legitimate to optimize it away. This is _bad_
news for anyone concerns with sensitive data being left lying around
in memory.

Various suggestions have been made, such as declaring the variable
volatile and having a scrub memory function in a file of its own. I'm
wondering if there are better ways such as telling the compiler not to
optimize away a function call.
[Declaring the array volatile is the right way to do it.  The reason
volatile exists is to tell the compiler not to do otherwise valid
optimizations. -John]

^ permalink raw reply	[flat|nested] 92+ messages in thread
* [9fans] how to avoid a memset() optimization
@ 2002-11-12 22:42 Roman V. Shaposhnick
  0 siblings, 0 replies; 92+ messages in thread
From: Roman V. Shaposhnick @ 2002-11-12 22:42 UTC (permalink / raw)
  To: 9fans

There's quite a lively discussion on comp.compilers about memset()
use in security related applications and compiler optimization.
This idiom of memset'ing secure stuff with 0 is not new, and I've
spotted it in Plan9 sources several times -- don't know whether
it should be a concern for Plan9 users and developers ( after all,
Plan9 has its own C compiler ) but seeing how gcc is making its
way, it seems that declaring every buffer volatile ( as John the
moderator has suggested ) wouldn't hurt either.

Thanks,
Roman.

From: "Francis Wai" <fwai@rsasecurity.com>
Newsgroups: comp.compilers
Subject: how to avoid a memset() optimization
Date: 7 Nov 2002 00:51:51 -0500

In a recent article (http://online.securityfocus.com/archive/82/297827),
Peter Gutmann raised a concern which has serious implications in
secure programming. His example, along the lines of,

int main()
{
    char key[16];
    strcpy(key, "whatever");
    encrpts(key);
    memset(key, 0, 16);
}

where memset() was optimized away because memset() is the last
expression before the next sequence point and that its side-effect is
not needed and that the subject of memset() is an auto variable. The
compiler sees that it is legitimate to optimize it away. This is _bad_
news for anyone concerns with sensitive data being left lying around
in memory.

Various suggestions have been made, such as declaring the variable
volatile and having a scrub memory function in a file of its own. I'm
wondering if there are better ways such as telling the compiler not to
optimize away a function call.
[Declaring the array volatile is the right way to do it.  The reason
volatile exists is to tell the compiler not to do otherwise valid
optimizations. -John]


^ permalink raw reply	[flat|nested] 92+ messages in thread

end of thread, other threads:[~2003-01-06 16:02 UTC | newest]

Thread overview: 92+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-11-14 16:47 [9fans] how to avoid a memset() optimization presotto
2002-11-15 10:50 ` Douglas A. Gwyn
2002-11-15 16:51   ` William Josephson
2002-11-18 10:38     ` Douglas A. Gwyn
2002-11-18 12:34       ` Ronald G. Minnich
2002-11-19  7:38 ` Roman V. Shaposhnick
2002-11-20  9:47   ` Douglas A. Gwyn
2002-11-21 20:55     ` Roman V. Shaposhnick
2002-11-22  9:59       ` Douglas A. Gwyn
2003-01-06 10:18   ` Ralph Corderoy
2003-01-06 15:42     ` Sam
2003-01-06 15:49       ` Russ Cox
2003-01-06 15:58         ` David Presotto
2003-01-06 16:02         ` Sam
2003-01-06 15:58     ` [9fans] Fs Kernel Conor Williams
  -- strict thread matches above, loose matches on Subject: below --
2003-01-06 10:47 [9fans] how to avoid a memset() optimization nigel
2003-01-06 11:15 ` Geoff Collyer
2002-11-19 14:32 presotto
2002-11-20  7:24 ` Tomas
2002-11-20 16:38   ` Douglas A. Gwyn
2002-11-19  8:21 Fco.J.Ballesteros
2002-11-18 21:36 Joel Salomon
2002-11-18 20:42 Andrew Simmons
2002-11-18 14:19 C H Forsyth
2002-11-15  1:56 Dennis Ritchie
2002-11-15 10:51 ` Douglas A. Gwyn
2002-11-15 12:03 ` Boyd Roberts
2002-11-14 18:55 jmk
2002-11-14 22:23 ` Steve Kilbane
2002-11-14 18:17 presotto
2002-11-14 18:11 Joel Salomon
2002-11-14 18:26 ` William Josephson
2002-11-14 17:44 rog
2002-11-15 10:50 ` Douglas A. Gwyn
2002-11-14 17:28 Russ Cox
     [not found] <nemo@plan9.escet.urjc.es>
2002-11-14 15:38 ` Fco.J.Ballesteros
2002-11-14 16:24   ` Scott Schwartz
2002-11-14  6:53 Russ Cox
2002-11-14 10:22 ` Douglas A. Gwyn
2002-11-14 13:20   ` Sam
2002-11-14 15:20     ` Scott Schwartz
2002-11-14 15:26       ` Boyd Roberts
2002-11-14 15:34         ` plan9
2002-11-14 15:59           ` Sam
2002-11-14 18:57         ` Steve Kilbane
2002-11-15 10:51           ` Douglas A. Gwyn
2002-11-14 15:50       ` Dan Cross
2002-11-14 17:21         ` Douglas A. Gwyn
2002-11-14 18:51           ` Dan Cross
2002-11-14 15:50     ` Douglas A. Gwyn
2002-11-19  7:20 ` Roman V. Shaposhnick
2002-11-14  2:48 Dennis Ritchie
2002-11-14  4:23 ` Ronald G. Minnich
2002-11-13 18:58 Rob `Commander' Pike
2002-11-13 14:40 C H Forsyth
2002-11-13 15:54 ` rob pike
2002-11-13 16:05   ` andrey mirtchovski
2002-11-13 16:32   ` Ronald G. Minnich
2002-11-14 10:21     ` Douglas A. Gwyn
2002-11-14 17:07       ` Ronald G. Minnich
2002-11-22  9:59     ` Clint Olsen
2002-11-13 16:56   ` William K. Josephson
2002-11-14 10:21     ` Douglas A. Gwyn
2002-11-14 16:48       ` William Josephson
2002-11-14 10:21   ` Douglas A. Gwyn
2002-11-14 14:46     ` Dan Cross
2002-11-14 16:59       ` Douglas A. Gwyn
2002-11-14 18:31         ` Tad Hunt
2002-11-15 10:50           ` Douglas A. Gwyn
2002-11-18 14:27         ` Aharon Robbins
2002-11-13 14:14 Skip Tavakkolian
2002-11-13 13:55 rog
2002-11-13 13:38 Skip Tavakkolian
2002-11-13 16:25 ` Boyd Roberts
2002-11-13 10:43 C H Forsyth
2002-11-14 10:21 ` Douglas A. Gwyn
2002-11-13  6:52 Geoff Collyer
2002-11-13 10:13 ` Boyd Roberts
2002-11-13  6:34 Andrew Simmons
2002-11-13  6:43 ` Doc Shipley
2002-11-13  1:47 Russ Cox
2002-11-13 10:16 ` Douglas A. Gwyn
2002-11-14  1:46 ` Roman V. Shaposhnick
2002-11-14  1:52   ` William Josephson
2002-11-14  6:42     ` Roman V. Shaposhnick
2002-11-13  0:31 Russ Cox
2002-11-13  1:26 ` Roman V. Shaposhnick
2002-11-13 10:15   ` Douglas A. Gwyn
2002-11-14  1:42     ` Roman V. Shaposhnick
2002-11-13 10:15 ` Douglas A. Gwyn
2002-11-13  0:20 presotto
2002-11-12 22:42 Roman V. Shaposhnick

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).