Re: [musl] Wrong rounding in printf when precision is not set to max

[Rich Felker <dalias@libc.org>]

On Mon, Jan 01, 2024 at 02:43:09PM +0100, Szabolcs Nagy wrote:
> * John M <johnm@braincalibration.de> [2024-01-01 10:11:09 +0100]:
> > I noticed printf rounding wrongly when FPU Control Word[PC] != 11.
> > 
> > I do not think my workaround is a fix, it just serves to show where the
> > reduced precision breaks the rounding.
> > 
> > Do you consider this a bug?
> 
> the precision control setting of x87 on i386 and x86-64 is ABI
> and on linux must be double extended (64bit pc=0b11, not 53bit
> pc=0b10 nor 24bit pc=0b00)
> 
> if you change the setting then the behaviour is undefined, it's
> not just the libc that may misbeahave, but the compiler and
> other libraries too.

Exactly. It's UB to call any standard library code (regardless of
whether you expect it uses floating point or not) if the fpu control
word is set to a non-ABI-conforming value. If anything wants to use
alternate modes that don't match the ABI, it needs to
save/reset/restore the value around every point at which it uses
library functions.

Note that, even then, there are lots of gotchas. The x87's "single"
and "double" modes are not actually IEEE single and double, but
nonstandard types with the IEEE significand precision but excess
exponent range that's truncated via double-rounding whenever the
intermediate is spilled to memory. This will induce UB-like runaway
wrong behavior in code generated by a compiler that's not aware of
these issues, since it will effectively be able to prove 1.0==0.0. So
it is probably strongly advisable never to set these modes at all.

Rich