9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* [9fans] printing floating point numbers
@ 2010-03-12 19:54 Tristan Plumb
  2010-03-12 21:37 ` Russ Cox
  0 siblings, 1 reply; 2+ messages in thread
From: Tristan Plumb @ 2010-03-12 19:54 UTC (permalink / raw)
  To: 9fans

I'm working on (re)writing some utilies for geospatial work on Plan9,
there are lots of floating point numbers (a file outlining the USGS
quadrangles for the state of Maine has about 60k). With the current
implementation this takes a long time. Maybe I should switch to a binary
format (but that's part of what I'm trying to get away from!).

Basically, do folks consider it possible to improve speed while keeping
or increasing the precision? A very simple implementation is accurate to
~15 digits, the current one is to 18 digits (though there are some
rounding errors).

In ways it would be nice to print stored numbers exactly. Yet numbers
like 2⁻¹⁰²²+2⁻¹⁰⁷⁴ (767 digits) are stored exactly...

2.2250738585072018771558785585789482407880088486837041956131300312119688603996006965297904292212628858639037013670281908017171296072711910355127227413175152199055740043138804567803233377539881639177387328959246074229270113078053813397081653361296447449529789521218979090783852583365901851789618799885150427514782636076021680436220311292700454832073964845713103912225963935608322440623896907276890186717054549275173986589324810401738228328251245795065655738191038008646911615828719989708647293221449796971546706720399791990809160347625980385995424739847678861180095072511543762389603716215171729816011544604359531284325406441938645324905389137795680915804792405099227413854274942620542640408839836919187418172987793340279242767544565229087538682506419718265533447265625e-308

The current implementation just prints the first 18 digits.

Thoughts?
Tristan

--
All original matter is hereby placed immediately under the public domain.



^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2010-03-12 21:37 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-03-12 19:54 [9fans] printing floating point numbers Tristan Plumb
2010-03-12 21:37 ` Russ Cox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).