9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
From: Tristan Plumb <9p-st@imu.li>
To: 9fans@9fans.net
Subject: [9fans] printing floating point numbers
Date: Fri, 12 Mar 2010 14:54:33 -0500	[thread overview]
Message-ID: <2576e4.40c323e4.IOOu.mx@tumtum.plumbweb.net> (raw)

I'm working on (re)writing some utilies for geospatial work on Plan9,
there are lots of floating point numbers (a file outlining the USGS
quadrangles for the state of Maine has about 60k). With the current
implementation this takes a long time. Maybe I should switch to a binary
format (but that's part of what I'm trying to get away from!).

Basically, do folks consider it possible to improve speed while keeping
or increasing the precision? A very simple implementation is accurate to
~15 digits, the current one is to 18 digits (though there are some
rounding errors).

In ways it would be nice to print stored numbers exactly. Yet numbers
like 2⁻¹⁰²²+2⁻¹⁰⁷⁴ (767 digits) are stored exactly...

2.2250738585072018771558785585789482407880088486837041956131300312119688603996006965297904292212628858639037013670281908017171296072711910355127227413175152199055740043138804567803233377539881639177387328959246074229270113078053813397081653361296447449529789521218979090783852583365901851789618799885150427514782636076021680436220311292700454832073964845713103912225963935608322440623896907276890186717054549275173986589324810401738228328251245795065655738191038008646911615828719989708647293221449796971546706720399791990809160347625980385995424739847678861180095072511543762389603716215171729816011544604359531284325406441938645324905389137795680915804792405099227413854274942620542640408839836919187418172987793340279242767544565229087538682506419718265533447265625e-308

The current implementation just prints the first 18 digits.

Thoughts?
Tristan

--
All original matter is hereby placed immediately under the public domain.



             reply	other threads:[~2010-03-12 19:54 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-03-12 19:54 Tristan Plumb [this message]
2010-03-12 21:37 ` Russ Cox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2576e4.40c323e4.IOOu.mx@tumtum.plumbweb.net \
    --to=9p-st@imu.li \
    --cc=9fans@9fans.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).