Re: Difference between -O2 and -g

[Brian Wang <brian.wang.0721@gmail.com>]

On Fri, Oct 26, 2012 at 11:19 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Fri, Oct 26, 2012 at 11:01:47AM +0800, Brian Wang wrote:
>> On Fri, Oct 26, 2012 at 10:55 AM, Rich Felker <dalias@aerifal.cx> wrote:
>> > On Fri, Oct 26, 2012 at 10:47:57AM +0800, Brian Wang wrote:
>> >> On Fri, Oct 26, 2012 at 10:32 AM, Rich Felker <dalias@aerifal.cx> wrote:
>> >> > On Fri, Oct 26, 2012 at 10:04:46AM +0800, Brian Wang wrote:
>> >> >> > One very simple way to get a picture of what's going on in a program
>> >> >> > is to run it under strace. Try saving strace logs for both the working
>> >> >> > version and the broken version and comparing them either manually or
>> >> >> > with the diff utility (although the latter may be difficult unless you
>> >> >> > filter out the addresses and other contnets that will naturally
>> >> >> > differ, so it might be easier to visually inspect). If you don't
>> >> >> > already have an strace built for your target, I think Aboriginal Linux
>> >> >> > has static binaries you can use.
>> >> >>
>> >> >> I have previously built my static strace.
>> >> >> I could not decipher what went wrong.  Please find the strace logs for
>> >> >> the three binaries in question.
>> >> >> The source code is basically the same, except for the musl ones,
>> >> >> printf calls are sprinkled here and there
>> >> >> as my desperate attempt.
>> >> >
>> >> > The good and bad traces diverge at this line, which only happens in
>> >> > the good one:
>> >> >
>> >> > writev(2, [{"CreateColormap : good end\n", 26}, {NULL, 0}], 2) = 26
>> >> >
>> >> > So search the source for that string and see what condition is causing
>> >> > that code to be reached or not reached.
>> >>
>> >> Thank you for reading through them. :-)
>> >>
>> >> The failed call (XaceHook) is:
>> >> -----------
>> >>     /*
>> >>      * Security creation/labeling check
>> >>      */
>> >>     i = XaceHook(XACE_RESOURCE_ACCESS, clients[client], mid, RT_COLORMAP,
>> >>                pmap, RT_NONE, NULL, DixCreateAccess);
>> >>     if (i != Success) {
>> >>       fprintf(stderr, "%s : 9\n", __func__);
>> >>       FreeResource(mid, RT_NONE);
>> >>       return i;
>> >>     }
>> >> -----------
>> >
>> > It would help a lot if I knew what source you were using, or at least
>> > the part of the tree it corresponds to in the upstrea/latest X source
>> > tree, so I could take a look at it.
>>
>> I am using a mildly patched one based on:
>> http://xorg.freedesktop.org/archive/individual/xserver/xorg-server-1.5.1.tar.bz2
>> The patch is about displaying a background ppm image and should be
>> after the error path.
>>
>> The failed call is within dix/colormap.c:CreateColormap().
>>
>> Thanks for helping.  I really appreciate it.
>
> I would check out Xext/security.c SecurityResource() and see if you
> can figure out what it's doing. That appears to be the callback that's
> getting called. You might want to check and see if there are any
> others that could be registered; this could be done by grepping for
> XaceRegister.*RESOURCE, which I didn't do because I'm browsing the
> source online and the web interface seems to lack grep.
>

With a bit of digging, I found the source code of this oldish xserver
may trigger an undefined behaviour.
When calling XaceHook(int hook, ...), the switch case looks like this:
---------
switch (hook)
    {
	case XACE_RESOURCE_ACCESS: {
	    XaceResourceAccessRec rec = {
		va_arg(ap, ClientPtr),
		va_arg(ap, XID),
		va_arg(ap, RESTYPE),
		va_arg(ap, pointer),
		va_arg(ap, RESTYPE),
		va_arg(ap, pointer),
		va_arg(ap, Mask),
		Success /* default allow */
	    };
	    calldata = &rec;
	    prv = &rec.status;
	    break;
	}
---------
I think gcc-4.7.2 looks at 'rec' and thinks it is local to the switch
case and optimizes it away.  However, 'prv' will be accessed
down below outside the case.
I do not know whose fault this is (most likely this piece of code),
but certainly not musl's. :-)

I looked at the later version of xserver and it puts the rec variable
at the top, outside the switch, to avoid the ambiguity.  I copied this
particular piece over and recompiled Xfbdev with -O2.  Now it at least
starts up!  Finally I can move my hobby project a bit forward.

P.S.  Trying out a new C library is definitely scary and will somehow
cloud people's judgement, at least mine. :-)

Thank you for the help.


Brian

-- 
brian
------------------

Cool-Karaoke - The smallest recording studio, in your palm, open-sourced
http://cool-idea.com.tw/

iMaGiNaTiOn iS mOrE iMpOrTaNt tHaN kNoWlEdGe