Re: Solving the recursive memcpy/memset/etc. issue

[Luca Barbato <lu_zero@gentoo.org>]

On 01/08/13 08:20, Rich Felker wrote:
> On Thu, Aug 01, 2013 at 08:05:01AM +0200, Luca Barbato wrote:
>>> The only fully viable option I see is replacing the code for
>>> these functions with code that uses volatile objects so as to
>>> make optimization utterly impossible. This will of course make
>>> them incredibly slow, but at least we would have safe, working C
>>> code, and we could add asm for each supported arch.
>> 
>> Not exactly great.
> 
> Well, we really need to add the arch asm anyway, as ugly as it is. 
> Right now most archs have memcpy running 2-5x slower than it should.
> I could _try_ writing C to handle the unaligned (hard) cases well, 
> basically mimicing what the proposed asm for arm does, but I don't 
> think it will be competitive, just "not as slow". And we'd still
> have to worry about it getting miscompiled...

I trust you on that.

>>> An alternative might be to test the compiler in configure to
>>> determine if, with the selected CFLAGS, it generates recursive
>>> code for these functions, and if so, defining a macro that causes
>>> musl to revert to the volatile code.
>> 
>> Sounds much better.
> 
> Well, it would be an ugly heuristic like running cc -S -o - on 
> src/string/memcpy.c, with -Dmemcpy=noname or something, and grepping 
> the output for memcpy...

Indeed.

>>> Other ideas? For now, if -fno-tree-loop-distribute-patterns fixes
>>> it (still waiting on confirmation for this) I'm going to commit
>>> that to configure, but it doesn't seem like a viable long-term
>>> solution.
>> 
>> I'd rather check and error out reporting the compiler is broken.
>> Then have an explicit configure option to try to workaround it.
> 
> If it were just a temporary regression, I would agree, but I think
> the GCC position is that this is not a bug...

Given that even glibc breaks on that...

> I figured someone would say that, and almost put a preemptive note
> in my post. clang/LLVM was the first to have this sort of bug of
> ignoring -ffreestanding, only much worse, making invalid assumptions
> about the result value of malloc inside the malloc implementation...

Understandable, clang is a "young" project. It is like expecting cparser
to be able to compile Libav.

> Competition is unfortunately the source of our woes, not the
> solution. GCC and clang/LLVM are facing competition to be the best at
> compiling application code, and since compiling the implementation
> itself is an unusual, unexciting usage case, nobody's really watching
> out for how they break that one in the race to have the fastest
> application code...

Implementors and distributors might have to say a thing or two about
that. If they manage to break singlehandedly musl, uclibc, glibc and
bionic and refuse to acknowledge the bug then I guess some people might
question their direction.

lu