* 5c ulong/vlong -> float64 bug
@ 2010-10-27 2:12 Russ Cox
2010-10-27 2:26 ` [9fans] " erik quanstrom
0 siblings, 1 reply; 2+ messages in thread
From: Russ Cox @ 2010-10-27 2:12 UTC (permalink / raw)
To: 9fans, inferno-list
For those people using 5c, I found a bug recently in the
conversion of ulong to float64. The bug is that it's not
implemented at all; instead 5c uses the long to float64 code,
so converting 0x80000000 produces -2^31 not 2^31.
This matters in _v2d, which uses it to convert vlong to float64.
If the low 32 bits of the vlong have the high bit set,
the conversion ends up off by 2^32.
http://codereview.appspot.com/2726041 has a
straightforward but perhaps inefficient fix.
Russ
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2010-10-27 2:26 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-10-27 2:12 5c ulong/vlong -> float64 bug Russ Cox
2010-10-27 2:26 ` [9fans] " erik quanstrom
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).