Re: TLS (thread-local storage) support

[Rich Felker <dalias@aerifal.cx>]

On Wed, Oct 17, 2012 at 01:39:49AM +0200, boris brezillon wrote:
> >> 4) Compile musl with '-fsplit-stack' and add no_split_stack attribute
> >> to appropriate functions (at least all functions called before
> >> pthread_self_init because %gs or %fs register is unusable before this
> >> call).
> >
> > This is definitely not desirable, at least not by default. It hurts
> > performance, possibly a lot, and destroys async-signal-safety. Also I
> > doubt it's needed. As long as split stack mode leaves at least ~8k
> > when calling a new function, most if not all functions in musl should
> > run fine without needing support for enlarging the stack.
> I agree. This should be made optional. But if we don't compile libc
> with fsplit-stack (-fnosplit-stack).
> Each call to a libc func from an external func compiled with split
> stack may lead to a 64K stack chunk alloc.

Where does this allocation take place from? There should simply be a
way to inhibit it.

> >> 6) add no-split-stack note to every asm file.
> >
> > I'm against this, or any boilerplate clutter. If it's really needed,
> > it should be possible with CFLAGS (or "ASFLAGS"), rather than
> > modifying every file, and if there's no way to do it with command line
> > options, that's a bug in gas.
> Not supported in gas, already tried.

That's frustrating..

> > Basically, the whole idea of split-stack is antithetical to the QoI
> > guarantees of musl. A program using split-stack can crash at any time
> > due to out-of-memory, and there is no reliable/portable way to recover
> > from this condition. It's much like the following low-quality aspects
> > of glibc and default Linux config:
> The same program may crash because of stack overflow (segfault) or
> worst : corrupt memory.

Only if written improperly. A correctly written program has bounded
stack usage that's easily proven correct with static analysis.
Unbounded stack usage is a bug, plain and simple, because there's no
way to safely and portably handle the runtime error of running out of
memory.

> At best the split stack provides a way to increase the thread without
> crashing the whole process.

If you're comparing the behavior of a program with initial
thread-stack size N and no-split-stack to a program with initial
thread-stack size N that can also obtain additional stack space with
split-stack, and you don't have static bounds on your stack usage that
keep it below N, then I agree that the latter will succeed in cases
where the former crashes. On the other hand, both programs WILL CRASH
under appropriate conditions, and as such, they are both buggy
programs.

> At worst it crash the program but never corrupt the memory.

Memory corruption will not happen without split stack either unless
you turn off guard pages or use functions with huge stack frames
without the -fstack-check option.

> > As such, I'm willing to add whatever inexpensive support framework is
> > needed so that people who want to use split-stack can use it, but I'm
> > very wary of invasive or costly changes to support a feature which I
> > believe is fundamentally misguided (and, for 64-bit targets, utterly
> > useless).
> 
> I understand.

Getting into it more, I think split-stack is a lot harder to support
than anybody has considered, especially if you want to still have a
POSIX conforming environment. There are all sorts of nasty cases
connected to signal handlers, async-signal-safety,
async-cancel-safety, longjmp, and thread cancellation where I know at
the very least you would need some ugly bloated hacks with unwinding
to get them right, and where I'm doubtful you even _can_ make them
100% conforming. Getting this stuff right is highly non-trivial to
begin with, even without split-stack (and glibc doesn't really even
try) so I'm doubtful that the architects of split-stack even thought
about it before throwing their experiment out there for everybody to
use...

Rich