RE: [musl] RE: [EXTERNAL] Re: [musl] [PATCH] fix avoidable segfault in catclose

[Andy Caldwell <andycaldwell@microsoft.com>]

> > > > > And it has been musl policy to crash on invalid args since the beginning.
> > > >
> > > > The current implementation doesn't (necessarily) crash/trap on an
> > > > invalid argument, instead it invokes (C-language spec-defined) UB
> > > > itself (it dereferences `(uint32_t*)((char*)cat) + 8)`, which, in
> > > > the case of the `-1` handle is the address 0x7, which in turn, not
> > > > being a valid address, is UB to dereference). If you're lucky (or
> > > > are compiling without optimizations/inlining) the compiler will
> > > > emit a MOV that will trigger an access violation and hence a SEGV,
> > > > if
> > >
> > > In general, it's impossible to test for "is this pointer valid?"
> > >
> > > There are certain special cases we could test for, but unless there
> > > is a particularly convincing reason that they could lead to runaway
> > > wrong execution/vulnerabilities prior to naturally trapping, we have
> > > not considered littering the code with these kinds of checks to be a
> worthwhile trade-off.
> > >
> > > > you're unlucky the compiler will make wild assumptions about the
> > > > value of the variable passed as the arg (and for example in your
> > > > first code snippet, simply delete the `if` statement, meaning
> > > > `use_cat` gets called even when `catopen` fails potentially
> > > > corrupting user data/state).
> > >
> > > I have no idea what you're talking about there. The compiler cannot
> > > make that kind of transformation (lifting code that could produce
> > > undefined behavior, side effects, etc. out of a conditional).
> >
> > It's a hypothetical, but something like the following is valid for the compiler to
> do:
> >
> > * inline the catclose (e.g. in LTO for a static link)
> > * consider the `if` statement and ask "what if `cat` is `-1`
> > * look forward to the pointer dereference (confirming that `cat` can't
> > change in the interim)
> > * realise that `0x7` is not a valid pointer on the target platform so
> > UB is inevitable if `cat` is `-1`
> > * optimize out the comparison since UB frees the compiler of any
> > responsibilities
> 
> You have the logic backwards. In the case where cat==(cat_t)-1, catclose is not
> called on the abstract machine, so no conclusions can be drawn from anything
> catclose would do.

The original code I was working from was:

```
nl_catd cat = catopen(...);
if (cat != (nl_catd)-1) {
    use_cat(cat);
}
catclose(cat);
```

(i.e. an incorrect use of the APIs, but not UB in a "C99 spec" sense).  In that code the `catclose` call is provably inevitable, allowing the compiler to infer properties of `cat` from it.