* [9fans] printing floating point numbers
@ 2010-03-12 19:54 Tristan Plumb
2010-03-12 21:37 ` Russ Cox
0 siblings, 1 reply; 2+ messages in thread
From: Tristan Plumb @ 2010-03-12 19:54 UTC (permalink / raw)
To: 9fans
I'm working on (re)writing some utilies for geospatial work on Plan9,
there are lots of floating point numbers (a file outlining the USGS
quadrangles for the state of Maine has about 60k). With the current
implementation this takes a long time. Maybe I should switch to a binary
format (but that's part of what I'm trying to get away from!).
Basically, do folks consider it possible to improve speed while keeping
or increasing the precision? A very simple implementation is accurate to
~15 digits, the current one is to 18 digits (though there are some
rounding errors).
In ways it would be nice to print stored numbers exactly. Yet numbers
like 2⁻¹⁰²²+2⁻¹⁰⁷⁴ (767 digits) are stored exactly...
2.2250738585072018771558785585789482407880088486837041956131300312119688603996006965297904292212628858639037013670281908017171296072711910355127227413175152199055740043138804567803233377539881639177387328959246074229270113078053813397081653361296447449529789521218979090783852583365901851789618799885150427514782636076021680436220311292700454832073964845713103912225963935608322440623896907276890186717054549275173986589324810401738228328251245795065655738191038008646911615828719989708647293221449796971546706720399791990809160347625980385995424739847678861180095072511543762389603716215171729816011544604359531284325406441938645324905389137795680915804792405099227413854274942620542640408839836919187418172987793340279242767544565229087538682506419718265533447265625e-308
The current implementation just prints the first 18 digits.
Thoughts?
Tristan
--
All original matter is hereby placed immediately under the public domain.
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [9fans] printing floating point numbers
2010-03-12 19:54 [9fans] printing floating point numbers Tristan Plumb
@ 2010-03-12 21:37 ` Russ Cox
0 siblings, 0 replies; 2+ messages in thread
From: Russ Cox @ 2010-03-12 21:37 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
> The current implementation just prints the first 18 digits.
18 digits are sufficient to reconstruct exactly the same bit
representation when you run the text back through strtod.
All those extra digits may be precise but they're not necessary
and probably not accurate.
Russ
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2010-03-12 21:37 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-03-12 19:54 [9fans] printing floating point numbers Tristan Plumb
2010-03-12 21:37 ` Russ Cox
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).