Re: [musl] [PATCH] Fix printf hex float formatting with precision

[Rich Felker <dalias@libc.org>]

[-- Attachment #1: Type: text/plain, Size: 900 bytes --]

On Fri, Apr 12, 2024 at 03:57:28PM -0400, Rich Felker wrote:
> On Thu, Apr 11, 2024 at 06:17:25PM -0700, Peter Ammon wrote:
> > if (p>=0 && p<(LDBL_MANT_DIG-1+3)/4) {
> >     int re = LDBL_MANT_DIG-1-(p*4);
> >     long double round = 1ULL<<re;
> 
> This expression overflows. re can be as large as 108 but 1ULL has
> 64-bit type.
> 
> There's probably some reasonable way to write it that works for
> reasonable sizes of long double (I think it's safe to assume IEEE quad
> is the max that will ever be supported), but I presume I wrote it with
> the original inefficient loop to be fully general to arbitrary
> precision.
> 
> It could just be written to use scalbn, I think. At the time I
> probably was trying to avoid dependency on a libm that could have been
> separate from libc (bad historical thing). IIRC there was a time when
> frexp was not used either.

Does the attached look ok?

Rich

[-- Attachment #2: pct_a.diff --]
[-- Type: text/plain, Size: 652 bytes --]

diff --git a/src/stdio/vfprintf.c b/src/stdio/vfprintf.c
index 497c5e19..dc648e7e 100644
--- a/src/stdio/vfprintf.c
+++ b/src/stdio/vfprintf.c
@@ -211,18 +211,11 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 	if (y) e2--;
 
 	if ((t|32)=='a') {
-		long double round = 8.0;
-		int re;
-
 		if (t&32) prefix += 9;
 		pl += 2;
 
-		if (p<0 || p>=LDBL_MANT_DIG/4-1) re=0;
-		else re=LDBL_MANT_DIG/4-1-p;
-
-		if (re) {
-			round *= 1<<(LDBL_MANT_DIG%4);
-			while (re--) round*=16;
+		if (p>=0 && p<(LDBL_MANT_DIG-1+3)/4) {
+			double round = scalbn(1, LDBL_MANT_DIG-1-(p*4));
 			if (*prefix=='-') {
 				y=-y;
 				y-=round;