mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Why thousands of digits in floatscan.c?
@ 2022-10-09  8:45 Markus Wichmann
  2022-10-09 10:35 ` Szabolcs Nagy
  0 siblings, 1 reply; 3+ messages in thread
From: Markus Wichmann @ 2022-10-09  8:45 UTC (permalink / raw)
  To: musl

Hi all,

so I recently read this wonderful paper: https://dl.acm.org/doi/10.1145/103162.103163

Great read, I encourage it for everyone, even though some things are not
directly applicable to the C application programmer.

Anyway, towards the end of the paper, Goldberg makes the argument that a
9 digit decimal number is always enough to uniquely identify a
single-precision number. Generalizing out from his argument, I
calculated that 36 decimal digits are needed for a quad-precision number
(the digit count for a w bit mantissa is roundup(w * lg 2) + 1, which is
9 for single-prec, 17 for double-prec, 21 for double-extended, and 33
for double-double).

Now, I looked again at floatscan.c. The algorithm in decfloat() mostly
consists of reading all the decimal digits into a base 1 billion number,
which could have any length at all, then aligning the radix point with a
word boundary, and then shifting that number such that exactly one
mantissa worth of bits are left of the radix point. For the most part,
the algorithm does not care about the remaining bits except for two bits
of information: If the tail is equal to 0, or else how it compares to
0.5. So now I am left wondering why then, if only the first
LDBL_MANT_DIG bits of the number, plus two more for the tail, ever count
for anything, the algorithm makes room for thousands of decimal digits.
Would it not be enough to save a few dozen? What purpose do the far-back
digits have?

To be clear, I am totally aware of why there needs to be room for
thousands of digits in fmt_fp(): All of those digits are in the number
and could be requested. The smallest representable positive number on
the PC is 2^-16447 (that is, it has an exponent of -16384 and a
mantissa of 2^-63), and that is a number with 16447 decimal places
(though the first 4950 of those are 0 and don't need to be saved
explicitly, but then you'd need wrap-around semantics there as well).

Ciao,
Markus

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-10-11 17:47 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-09  8:45 [musl] Why thousands of digits in floatscan.c? Markus Wichmann
2022-10-09 10:35 ` Szabolcs Nagy
2022-10-11 17:46   ` Rich Felker

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).