RE: [Caml-list] Approximations when converting from string to float

["Soegtrop, Michael" <michael.soegtrop@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]

Dear Jacques-Henri,

if the number is cut off after n digits, the upper and lower bounds I suggested are the (truncated integer)*10^(#cut-off-digits) and (truncated integer+1)*10^(#cut-off-digits). The true number is obviously between these two. Since the integer has higher precision than the mantissa in the case of cut off, this is only a fraction of a mantissa bit. The errors you get by multiplying with the powers of 10 is likely larger in most cases.

In the case you mentioned, 2^70=1180591620717411303424 and 32 bit one would truncate after

1180591620 * 10^12

Maxima gives

is(1180591620 * 10^12 <= 2^70);
true

is(1180591621 * 10^12 >= 2^70);
true

As I said, this method doesn’t give the tightest bounds possible, but it gives you true upper and lower bounds without multi precision arithmetic. It gives 0 intervals e.g. for integers which fit in the mantissa, but not for large exact powers of 2.

> Moreover, it is not possible to implement interval arithmetic with OCaml, since you cannot change the rounding mode without a bit of C...

It should be possible to make the powers of 10 tables such that this works even with round to nearest, but it would likely be easier to make a C library for this.

Of cause it is a good question if I/O is the right place to save performance and waste precision. On the other hand you lose only 2 or 3 bits and you know what the precision is. I would think most applications are such that the required precision is not exactly an IEEE precision, so that these 2 or 3 bits are ok. I used this method once in an embedded platform, where multi precision arithmetic was not an option, and this was a good compromise.

Best regards,

Michael
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

[-- Attachment #2: Type: text/html, Size: 9577 bytes --]