From: Tristan Plumb <9p-st@imu.li>
To: 9fans@9fans.net
Subject: [9fans] printing floating point numbers
Date: Fri, 12 Mar 2010 14:54:33 -0500 [thread overview]
Message-ID: <2576e4.40c323e4.IOOu.mx@tumtum.plumbweb.net> (raw)
I'm working on (re)writing some utilies for geospatial work on Plan9,
there are lots of floating point numbers (a file outlining the USGS
quadrangles for the state of Maine has about 60k). With the current
implementation this takes a long time. Maybe I should switch to a binary
format (but that's part of what I'm trying to get away from!).
Basically, do folks consider it possible to improve speed while keeping
or increasing the precision? A very simple implementation is accurate to
~15 digits, the current one is to 18 digits (though there are some
rounding errors).
In ways it would be nice to print stored numbers exactly. Yet numbers
like 2⁻¹⁰²²+2⁻¹⁰⁷⁴ (767 digits) are stored exactly...
2.2250738585072018771558785585789482407880088486837041956131300312119688603996006965297904292212628858639037013670281908017171296072711910355127227413175152199055740043138804567803233377539881639177387328959246074229270113078053813397081653361296447449529789521218979090783852583365901851789618799885150427514782636076021680436220311292700454832073964845713103912225963935608322440623896907276890186717054549275173986589324810401738228328251245795065655738191038008646911615828719989708647293221449796971546706720399791990809160347625980385995424739847678861180095072511543762389603716215171729816011544604359531284325406441938645324905389137795680915804792405099227413854274942620542640408839836919187418172987793340279242767544565229087538682506419718265533447265625e-308
The current implementation just prints the first 18 digits.
Thoughts?
Tristan
--
All original matter is hereby placed immediately under the public domain.
next reply other threads:[~2010-03-12 19:54 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-03-12 19:54 Tristan Plumb [this message]
2010-03-12 21:37 ` Russ Cox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2576e4.40c323e4.IOOu.mx@tumtum.plumbweb.net \
--to=9p-st@imu.li \
--cc=9fans@9fans.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).