[Caml-list] Int overflow in literals

[Caml-list] Int overflow in literals

[Marcin 'Qrczak' Kowalczyk]

I understand that int overflow is not checked on arithmetic for
efficiency reasons, but IMHO it would be better if it was checked
at least in literals. When someone writes 10000000000, he certainly
does not mean -737418240.

It caused confusion in a class when someone was interactively testing
a function with larger and larger inputs.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Alex Baretta]

Marcin 'Qrczak' Kowalczyk wrote:
> I understand that int overflow is not checked on arithmetic for
> efficiency reasons, but IMHO it would be better if it was checked
> at least in literals. When someone writes 10000000000, he certainly
> does not mean -737418240.
> 
> It caused confusion in a class when someone was interactively testing
> a function with larger and larger inputs.
> 

I bet the official answer is "It's can't be done because native integers 
are 31 bits on 32 bit platforms, 63 bits on 64 bit platforms, so what is 
parser supposed to do? Especially in the case of the bytecode compiler."
Although, I suppose something could be done in the toplevel.

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Frederic van der Plancke]

Alex Baretta wrote:
> 
> Marcin 'Qrczak' Kowalczyk wrote:
> > I understand that int overflow is not checked on arithmetic for
> > efficiency reasons, but IMHO it would be better if it was checked
> > at least in literals. When someone writes 10000000000, he certainly
> > does not mean -737418240.
> >
> > It caused confusion in a class when someone was interactively testing
> > a function with larger and larger inputs.
> >
> 
> I bet the official answer is "It's can't be done because native integers
> are 31 bits on 32 bit platforms, 63 bits on 64 bit platforms, so what is
> parser supposed to do? Especially in the case of the bytecode compiler."
> Although, I suppose something could be done in the toplevel.

I think int_of_string is the culprit and must change (last time I checked,
int_of_string "10000000000" returned -737418240).
(sorry if that's already been done in 3.07, I haven't tried it yet.)

There's not much execution time to gain by not letting it check the overflow,
and there's no easy way to write a "correct" safety wrapper for that function
(as it would have to accept arguments like "-0000000123456789", it
would be easier to rewrite it from scratch.)

Ditto for int_of_float (int_of_float 10000000000.0 = -737418240), even
though there the safety wrapper is easier to write (and the relative loss of
execution time would probably be greater.))

I think that understanding "language safety" as "never triggers access
violation" is a tad restrictive...

Frédéric

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Oleg Trott]

I'm also curious if raising an exception in case of arithmetic overflows
instead of silently accepting them would have a significant performance
impact on modern CPUs - few, if anyone, mean to do mod 2^31 
arithmetic.
-- 
Oleg Trott <oleg_trott@columbia.edu>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Marcin 'Qrczak' Kowalczyk]

W liście z czw, 30-10-2003, godz. 20:20, Oleg Trott pisze:

> I'm also curious if raising an exception in case of arithmetic overflows
> instead of silently accepting them would have a significant performance
> impact on modern CPUs - few, if anyone, mean to do mod 2^31 
> arithmetic.

Perhaps it should be like array bounds checking: separate functions for
both variants, with + resolving to either depending on -unsafe option?

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Issac Trotts]

On Thu, Oct 30, 2003 at 02:53:32PM +0100, Marcin 'Qrczak' Kowalczyk wrote:
> I understand that int overflow is not checked on arithmetic for
> efficiency reasons, but IMHO it would be better if it was checked
> at least in literals. When someone writes 10000000000, he certainly
> does not mean -737418240.

If you want to be sure that the number is correctly stored, you can use
Int64:

    Int64.of_string "10000000000"  

Issac

-- 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Oleg Trott]

On Thursday 30 October 2003 03:05 pm, Issac Trotts wrote:
> On Thu, Oct 30, 2003 at 02:53:32PM +0100, Marcin 'Qrczak' Kowalczyk wrote:
> > I understand that int overflow is not checked on arithmetic for
> > efficiency reasons, but IMHO it would be better if it was checked
> > at least in literals. When someone writes 10000000000, he certainly
> > does not mean -737418240.
>
> If you want to be sure that the number is correctly stored, you can use
> Int64:
>
>     Int64.of_string "10000000000"
>
> Issac

Or, better yet, use Big_int:

        Objective Caml version 3.06

# Int64.of_string "10000000000000000000";;
- : int64 = <int64 -8446744073709551616>
# #load "nums.cma";;
# Big_int.big_int_of_string "10000000000000000000";;

-- 
Oleg Trott <oleg_trott@columbia.edu>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Kenneth Knowles]

On Thu, Oct 30, 2003 at 04:14:02PM -0500, Oleg Trott wrote:
> Or, better yet, use Big_int:

But one should note that the Num library, and thus Big_int, is under a different
license than the ocaml distribution, and may be incompatible with your project.

-Kenn

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Jacques Garrigue]

From: Kenneth Knowles <kknowles@uclink.berkeley.edu>
> On Thu, Oct 30, 2003 at 04:14:02PM -0500, Oleg Trott wrote:
> > Or, better yet, use Big_int:
> 
> But one should note that the Num library, and thus Big_int, is under
> a different license than the ocaml distribution, and may be
> incompatible with your project.

If this is the only thing that bothers you, get the CVS version of
ocaml: bignum has just be rewritten from scratch, and is now available
under the same license as other libraries.

Funny how Xavier had to rewrite both regexp and bignum just for
questions of license...  the BSD ideal of freely reusable software is
really far away (and ocaml itself is not 100% free either).
(IIRC improving efficiency was another reason in both cases.)

Concerning the discussion 63-bit vs 31-bit, this is not directly
relevant as the ocaml compiler on 32-bit platforms does not put
63-bit integers in the code. At least there could be a warning for
constant overflow.

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Kenneth Knowles]

> If this is the only thing that bothers you, get the CVS version of
> ocaml: bignum has just be rewritten from scratch, and is now available
> under the same license as other libraries.

Fantastic!  It is not only a "bother" but rather it is impossible in many
situations (such as on the job) to use it.  This is a huge help, thanks for the
info.

-Kenn

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[skaller]

On Fri, 2003-10-31 at 08:14, Oleg Trott wrote:
> On Thursday 30 October 2003 03:05 pm, Issac Trotts wrote:

> 
> Or, better yet, use Big_int:

Felix does that, and i'm very happy with the idea.
Constant folding is done in "infinite" precision.
The check for size, if done, would occur in the backend
code generator .. not the parser.


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Brian Hurt]

On 3 Nov 2003, skaller wrote:

> On Fri, 2003-10-31 at 08:14, Oleg Trott wrote:
> > On Thursday 30 October 2003 03:05 pm, Issac Trotts wrote:
> 
> > 
> > Or, better yet, use Big_int:
> 
> Felix does that, and i'm very happy with the idea.
> Constant folding is done in "infinite" precision.
> The check for size, if done, would occur in the backend
> code generator .. not the parser.
> 

This is a bad idea.  I'd like the constant folding to be done in the word 
length that the code will be executed in- this way, there is no difference 
between code that is executed at run time and code that is executed at 
compile time.

Now, being able to have bignum constants...

Brian


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Marcin 'Qrczak' Kowalczyk]

W liście z nie, 02-11-2003, godz. 17:23, Brian Hurt pisze:

> This is a bad idea.  I'd like the constant folding to be done in the word 
> length that the code will be executed in- this way, there is no difference 
> between code that is executed at run time and code that is executed at 
> compile time.

Constant folding is different from interpreting literals.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[skaller]

On Mon, 2003-11-03 at 03:23, Brian Hurt wrote:
> On 3 Nov 2003, skaller wrote:
> 
> > On Fri, 2003-10-31 at 08:14, Oleg Trott wrote:
> > > On Thursday 30 October 2003 03:05 pm, Issac Trotts wrote:
> > 
> > > 
> > > Or, better yet, use Big_int:
> > 
> > Felix does that, and i'm very happy with the idea.
> > Constant folding is done in "infinite" precision.
> > The check for size, if done, would occur in the backend
> > code generator .. not the parser.
> > 
> 
> This is a bad idea.  I'd like the constant folding to be done in the word 
> length that the code will be executed in- this way, there is no difference 
> between code that is executed at run time and code that is executed at 
> compile time.
> 

The results will be the same, so its a good idea :-)

The Felix types 'int', 'long' etc have the same sizes
as the corresponding C types. But the semantics are
NOT the same. Bitwise operations are not permitted,
and the result is undefined on overflow.

The Felix type 'int32' on the other hand is required
to be two's complement 32 bit, and allows overflow
as in C. Bitwise operations are not allowed.

The type 'uint32' allows bitwise operations.

The constant folder respects these semantics.
Actually only int type is folded .. and if the
result would overflow at run time then the behaviour
of the program is undefined, and so the well defined
result of constant folding is correct, since it is
an element of the set 'undefined' of all possible results :-)



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Frederic van der Plancke]

Issac Trotts wrote:
> 
> On Thu, Oct 30, 2003 at 02:53:32PM +0100, Marcin 'Qrczak' Kowalczyk wrote:
> > I understand that int overflow is not checked on arithmetic for
> > efficiency reasons, but IMHO it would be better if it was checked
> > at least in literals. When someone writes 10000000000, he certainly
> > does not mean -737418240.
> 
> If you want to be sure that the number is correctly stored, you can use
> Int64:
> 
>     Int64.of_string "10000000000"
> 
> Issac

That was not my problem. My problem was to be able to read a list of integers from a file and be warned in case of overflow. And to be able to rely on int_of_string for that purpose. I got hit... of course now I know, but other innocent programmers may get hit in the future as well. (Not to speak of the not-so-innocent people who wrote this nice OCaml compiler ;-)

Frédéric.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Issac Trotts]

On Thu, Oct 30, 2003 at 10:36:24PM +0100, Frederic van der Plancke wrote:
> 
> 
> Issac Trotts wrote:
> > 
> > On Thu, Oct 30, 2003 at 02:53:32PM +0100, Marcin 'Qrczak' Kowalczyk wrote:
> > > I understand that int overflow is not checked on arithmetic for
> > > efficiency reasons, but IMHO it would be better if it was checked
> > > at least in literals. When someone writes 10000000000, he certainly
> > > does not mean -737418240.
> > 
> > If you want to be sure that the number is correctly stored, you can use
> > Int64:
> > 
> >     Int64.of_string "10000000000"
> > 
> > Issac
> 
> That was not my problem. My problem was to be able to read a list of integers from a file and be warned in case of overflow. And to be able to rely on int_of_string for that purpose. I got hit... of course now I know, but other innocent programmers may get hit in the future as well. (Not to speak of the not-so-innocent people who wrote this nice OCaml compiler ;-)

Here's a way to do what you want, at least for numbers 
betweeen -9223372036854775808 and 9223372036854775807 :

let safe_int_of_string str =
    let i = int_of_string str in
    let i64 = Int64.of_string str in
    if i64 = Int64.of_int i then i else failwith "overflow"

# safe_int_of_string "10000000000";;
Exception: Failure "overflow".
# safe_int_of_string (string_of_int 1073741823)
    ;;
    - : int = 1073741823

-ijt 

> 
> Fr?d?ric.
> 
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> 

-- 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Oleg Trott]

On Thursday 30 October 2003 04:36 pm, Frederic van der Plancke wrote:
> Issac Trotts wrote:
> > On Thu, Oct 30, 2003 at 02:53:32PM +0100, Marcin 'Qrczak' Kowalczyk wrote:
> > > I understand that int overflow is not checked on arithmetic for
> > > efficiency reasons, but IMHO it would be better if it was checked
> > > at least in literals. When someone writes 10000000000, he certainly
> > > does not mean -737418240.
> >
> > If you want to be sure that the number is correctly stored, you can use
> > Int64:
> >
> >     Int64.of_string "10000000000"
> >
> > Issac
>
> That was not my problem. My problem was to be able to read a list of
> integers from a file and be warned in case of overflow. And to be able to
> rely on int_of_string for that purpose. I got hit... of course now I know,
> but other innocent programmers may get hit in the future as well. (Not to
> speak of the not-so-innocent people who wrote this nice OCaml compiler ;-)

... and this is what Issac (and myself) suggested (by implication):

let int_of_string s =
  Big_int.int_of_big_int (Big_int.big_int_of_string s)

We addressed your specific problem. But I agree with you that it would be
nice to have the safe behavior in OCaml by default (at least for parsing,
where performance wouldn't be affected)

-- 
Oleg Trott <oleg_trott@columbia.edu>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Xavier Leroy]

> I understand that int overflow is not checked on arithmetic for
> efficiency reasons, but IMHO it would be better if it was checked
> at least in literals. When someone writes 10000000000, he certainly
> does not mean -737418240.
> It caused confusion in a class when someone was interactively testing
> a function with larger and larger inputs.

This is a very good suggestion.  There are several ways to go about this:

- The lexer emits a warning in case of overflow, and proceeds with
  the value modulo the size of the type.
- The lexer emits an error on overflow.
- The int_of_string functions raise an exception on overflow.

Based on the comments posted so far on this list, and on a quick
discussion with colleagues, I'm inclined toward the third approach
(int_of_string fails in case of overflow).  Does anyone know of a use
scenario where this new behavior of int_of_string would be a problem?

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Marcin 'Qrczak' Kowalczyk]

W liście z pią, 31-10-2003, godz. 17:42, Xavier Leroy pisze:

> Based on the comments posted so far on this list, and on a quick
> discussion with colleagues, I'm inclined toward the third approach
> (int_of_string fails in case of overflow).

Great!

> Does anyone know of a use scenario where this new behavior of
> int_of_string would be a problem?

I know one, assuming literals in source would be parsed by
int_of_string: writing the -0x40000000 literal.

Some languages explicitly say that it's permissible to write their
maxint+1 value as a literal only as a direct application of negation.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Oleg Trott]

On Friday 31 October 2003 11:42 am, Xavier Leroy wrote:
> > I understand that int overflow is not checked on arithmetic for
> > efficiency reasons, but IMHO it would be better if it was checked
> > at least in literals. When someone writes 10000000000, he certainly
> > does not mean -737418240.
> > It caused confusion in a class when someone was interactively testing
> > a function with larger and larger inputs.
>
> This is a very good suggestion.  There are several ways to go about this:
>
> - The lexer emits a warning in case of overflow, and proceeds with
>   the value modulo the size of the type.
> - The lexer emits an error on overflow.
> - The int_of_string functions raise an exception on overflow.
>
> Based on the comments posted so far on this list, and on a quick
> discussion with colleagues, I'm inclined toward the third approach
> (int_of_string fails in case of overflow).  Does anyone know of a use
> scenario where this new behavior of int_of_string would be a problem?

How much do you think the performance of "typical programs" would be affected
(in percent) if +, - and * were also made to raise an exception on overflow?

-- 
Oleg Trott <oleg_trott@columbia.edu>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[skaller]

On Sat, 2003-11-01 at 03:42, Xavier Leroy wrote:

> - The int_of_string functions raise an exception on overflow.
> 
> Based on the comments posted so far on this list, and on a quick
> discussion with colleagues, I'm inclined toward the third approach
> (int_of_string fails in case of overflow).  Does anyone know of a use
> scenario where this new behavior of int_of_string would be a problem?

I cannot give a specific example, but I can paint a general
scenario: some data used with C is copied to ocaml (as text).

For example 0xFFFFFFFF is too large .. but in C it converts to -1
which is not too large :-)

Having given this example .. I have to say 'too bad', I think
your inclination is correct. [Whatever program extracted the
data from C should have handled this]



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Int overflow in literals

[Marcin 'Qrczak' Kowalczyk]

W liście z nie, 02-11-2003, godz. 16:23, skaller pisze:

> I cannot give a specific example, but I can paint a general
> scenario: some data used with C is copied to ocaml (as text).
> 
> For example 0xFFFFFFFF is too large .. but in C it converts to -1
> which is not too large :-)

Given that OCaml ints are one bit smaller than in C, I think this is
more likely to mask an error than to be correct.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners