* 5c ulong/vlong -> float64 bug
@ 2010-10-27 2:12 Russ Cox
2010-10-27 2:26 ` [9fans] " erik quanstrom
0 siblings, 1 reply; 2+ messages in thread
From: Russ Cox @ 2010-10-27 2:12 UTC (permalink / raw)
To: 9fans, inferno-list
For those people using 5c, I found a bug recently in the
conversion of ulong to float64. The bug is that it's not
implemented at all; instead 5c uses the long to float64 code,
so converting 0x80000000 produces -2^31 not 2^31.
This matters in _v2d, which uses it to convert vlong to float64.
If the low 32 bits of the vlong have the high bit set,
the conversion ends up off by 2^32.
http://codereview.appspot.com/2726041 has a
straightforward but perhaps inefficient fix.
Russ
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [9fans] 5c ulong/vlong -> float64 bug
2010-10-27 2:12 5c ulong/vlong -> float64 bug Russ Cox
@ 2010-10-27 2:26 ` erik quanstrom
0 siblings, 0 replies; 2+ messages in thread
From: erik quanstrom @ 2010-10-27 2:26 UTC (permalink / raw)
To: 9fans
On Tue Oct 26 22:13:45 EDT 2010, rsc@swtch.com wrote:
> For those people using 5c, I found a bug recently in the
> conversion of ulong to float64. The bug is that it's not
> implemented at all; instead 5c uses the long to float64 code,
> so converting 0x80000000 produces -2^31 not 2^31.
> This matters in _v2d, which uses it to convert vlong to float64.
> If the low 32 bits of the vlong have the high bit set,
> the conversion ends up off by 2^32.
>
> http://codereview.appspot.com/2726041 has a
> straightforward but perhaps inefficient fix.
already reported here with fix:
http://9fans.net/archive/2010/09/334
- erik
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2010-10-27 2:26 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-10-27 2:12 5c ulong/vlong -> float64 bug Russ Cox
2010-10-27 2:26 ` [9fans] " erik quanstrom
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).