Line number for index out of bounds

Line number for index out of bounds

[Andries Hekstra]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

Dear OCaml-List,

Thanks for replies to my previous email about a crashing Objective Caml 
program which simulates a next generation optical storage system that 
crashed due to an NFS error after 2 days of running. Special thanks to 
Gerd Stolpmann. In the mean time the program has computed for one and a 
half day more, ca. 3.5 days, and gives an index out of bound error, see 
below.

===================================================================================================

                 BestDelayLeadingPitEdgeInPerc = 23.0, 
bestAsymmetryInPerc= 10.1 
                 1-Track ScalarDiffr. for 38.8 rawGb with 6.1 pixels/bit 
and 83.3 bits/spot ................Fatal error: exception 
Invalid_argument("index out of bounds")
===================================================================================================

This is better than a C++ program giving a segmentation fault, as one now 
knows the reason of the crash. 
Of course, I am very curious in which line number of the program this 
exception occurs. 
Is there any way to get hold of this line number?

Best regards,

=Andries

------------------------------------------------------------------------
Dr. Ir. Andries P. Hekstra
Philips Research 
High Tech Campus 27  (WL-1-4.15)
5656 AG Eindhoven
Tel./Fax/Secr. +31 40 27 42048/42566/44051 
   *  Good open source break software for computer users : 
http://www.workrave.org 

[-- Attachment #2: Type: text/html, Size: 1962 bytes --]

Re: [Caml-list] Line number for index out of bounds

[Richard Jones]

On Mon, Mar 06, 2006 at 11:44:31AM +0100, Andries Hekstra wrote:
> Invalid_argument("index out of bounds")
[...]
> Of course, I am very curious in which line number of the program this 
> exception occurs. 
> Is there any way to get hold of this line number?

This is a real problem with OCaml - it's impossible to get stack
traces of where an exception happens with native code.  I'm assuming
you're using native code.  I commonly have cases where a program dies
with "exception: Not_found" because I forgot to enclose some List.find
with an appropriate try ... with clause, or made some wrong
assumption.  Tracking these down is time-consuming.

Possible workarounds:

* Use bytecode, and before running the program set the environment
variable OCAMLRUNPARAM=b which will print a stack trace.

* Surround every possible array index with a try ... with expression
like this:

  try
    (* code which accesses the array *)
  with
    Invalid_argument "index out of bounds" -> assert false

The "assert false" will print the line and character number of the
assertion.

* Hack ocamlopt to be able to print exceptions properly :-)

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Line number for index out of bounds

[Markus Mottl]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

On 3/6/06, Richard Jones <rich@annexia.org> wrote:
>
> * Hack ocamlopt to be able to print exceptions properly :-)
>

We have already developed a patch for the OCaml-compiler to generate
function call backtraces for native code programs and submitted it to the
OCaml-bugtracker in the hope to get it included in a future release.
Function call backtraces are different from stack backtraces, sometimes
more, sometimes less useful.  They are usually sufficient for tracking down
the source of exceptions.

In case you feel like running a patched runtime, look up issue 0003885 on
the bugtracker, which contains an attachment with the patch for OCaml 3.09:
http://caml.inria.fr/mantis/view.php?id=3885

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

[-- Attachment #2: Type: text/html, Size: 1333 bytes --]

Re: [Caml-list] Line number for index out of bounds

[Andries Hekstra]

[-- Attachment #1: Type: text/plain, Size: 2661 bytes --]

Dear Richard,

Thanks for your email. I indeed use native code as I need the speed. My 
program is 3500 lines, and includes multi-dimensional arrays, to putting 
try's everywhere by hand is out of the question. I would then have to 
write a metaprogram that adds such try commands to an existing OCaml 
program and outputs a longer program with the try's with the asserts. If 
possible I would like to postpone that and try your other option. 

Due to this crashing business I go on a business trip to Asia without any 
ready simulation results for one week.

> * Use bytecode, and before running the program set the environment
> variable OCAMLRUNPARAM=b which will print a stack trace.

If I would use this week of the trip to try this suggestion you made, how 
will the stack trace give me the line number?

Best regards,

Andries

------------------------------------------------------------------------
Dr. Ir. Andries P. Hekstra
Philips Research 
High Tech Campus 27  (WL-1-4.15)
5656 AG Eindhoven
Tel./Fax/Secr. +31 40 27 42048/42566/44051 
   *  Good open source break software for computer users : 
http://www.workrave.org 








Richard Jones <rich@annexia.org> 
06-03-2006 12:14

To
Andries Hekstra/EHV/RESEARCH/PHILIPS@PHILIPS
cc
caml-list@yquem.inria.fr
Subject
Re: [Caml-list] Line number for index out of bounds
Classification







On Mon, Mar 06, 2006 at 11:44:31AM +0100, Andries Hekstra wrote:
> Invalid_argument("index out of bounds")
[...]
> Of course, I am very curious in which line number of the program this 
> exception occurs. 
> Is there any way to get hold of this line number?

This is a real problem with OCaml - it's impossible to get stack
traces of where an exception happens with native code.  I'm assuming
you're using native code.  I commonly have cases where a program dies
with "exception: Not_found" because I forgot to enclose some List.find
with an appropriate try ... with clause, or made some wrong
assumption.  Tracking these down is time-consuming.

Possible workarounds:

* Use bytecode, and before running the program set the environment
variable OCAMLRUNPARAM=b which will print a stack trace.

* Surround every possible array index with a try ... with expression
like this:

  try
    (* code which accesses the array *)
  with
    Invalid_argument "index out of bounds" -> assert false

The "assert false" will print the line and character number of the
assertion.

* Hack ocamlopt to be able to print exceptions properly :-)

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - 
http://team-notepad.com


[-- Attachment #2: Type: text/html, Size: 4102 bytes --]

Re: [Caml-list] Line number for index out of bounds

[Richard Jones]

On Mon, Mar 06, 2006 at 08:08:02PM +0100, Andries Hekstra wrote:
> If I would use this week of the trip to try this suggestion you made, how 
> will the stack trace give me the line number?

I forgot to say that not only must you run your program in bytecode,
but you must compile your program with the '-g' option.

It will do something like this:

$ cat test.ml 
let a = Array.make 10 0
let f () = a.(11)
let g = f
let h = g
let main = h ()

$ ocamlc -g test.ml -o test

$ ./test 
Fatal error: exception Invalid_argument("index out of bounds")

$ OCAMLRUNPARAM=b ./test
Fatal error: exception Invalid_argument("index out of bounds")
Raised by primitive operation at unknown location
Called from file "test.ml", line 5, character 15

As you can see, often the stack backtraces aren't very accurate either :-(

You might want to look at Marcus Mottl's patch instead ...

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Line number for index out of bounds

[Alan Falloon]

Andries Hekstra wrote:
> This is better than a C++ program giving a segmentation fault, as one 
> now knows the reason of the crash.
> Of course, I am very curious in which line number of the program this 
> exception occurs.
> Is there any way to get hold of this line number?
Maybe ocamlexc can help:
http://caml.inria.fr/pub/old_caml_site/ocamlexc/ocamlexc.htm

It statically finds uncaught exceptions in your code.

--
Alan Falloon

Re: [Caml-list] Line number for index out of bounds / Exceptions for left hand side of assignments

[Andries Hekstra]

[-- Attachment #1: Type: text/plain, Size: 2704 bytes --]

Richard Jones wrote :
> Possible workarounds:
> ...
> * Surround every possible array index with a try ... with expression
> like this:
> 
>   try
>     (* code which accesses the array *)
>   with
>     Invalid_argument "index out of bounds" -> assert false
> 
> The "assert false" will print the line and character number of the
> assertion.

Dear Richard,

I agree with you that this is feasible whenever array elements feature in 
the right hand side of assignments. But how about the left hand sides of 
assignments ? E.g. the following obviously is not correct OCaml syntax.

        let a = Array.init 2 0;;

        try a.(2) <- 1
          with
            Invalid_argument "index out of bounds" -> assert false;;

        let h = try a.(2)
           with
            Invalid_argument "index out of bounds" -> assert false in
            h <- 1;;

Regards,

=Andries

------------------------------------------------------------------------
Dr. Ir. Andries P. Hekstra
Philips Research 
High Tech Campus 27  (WL-1-4.15)
5656 AG Eindhoven
Tel./Fax/Secr. +31 40 27 42048/42566/44051 
   *  Good open source break software for computer users : 
http://www.workrave.org 








Richard Jones <rich@annexia.org> 
06-03-2006 12:14

To
Andries Hekstra/EHV/RESEARCH/PHILIPS@PHILIPS
cc
caml-list@yquem.inria.fr
Subject
Re: [Caml-list] Line number for index out of bounds
Classification







On Mon, Mar 06, 2006 at 11:44:31AM +0100, Andries Hekstra wrote:
> Invalid_argument("index out of bounds")
[...]
> Of course, I am very curious in which line number of the program this 
> exception occurs. 
> Is there any way to get hold of this line number?

This is a real problem with OCaml - it's impossible to get stack
traces of where an exception happens with native code.  I'm assuming
you're using native code.  I commonly have cases where a program dies
with "exception: Not_found" because I forgot to enclose some List.find
with an appropriate try ... with clause, or made some wrong
assumption.  Tracking these down is time-consuming.

Possible workarounds:

* Use bytecode, and before running the program set the environment
variable OCAMLRUNPARAM=b which will print a stack trace.

* Surround every possible array index with a try ... with expression
like this:

  try
    (* code which accesses the array *)
  with
    Invalid_argument "index out of bounds" -> assert false

The "assert false" will print the line and character number of the
assertion.

* Hack ocamlopt to be able to print exceptions properly :-)

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - 
http://team-notepad.com


[-- Attachment #2: Type: text/html, Size: 4830 bytes --]

Re: [Caml-list] Line number for index out of bounds

[Martin Jambon]

On Mon, 6 Mar 2006, Andries Hekstra wrote:

> Dear Richard,
>
> Thanks for your email. I indeed use native code as I need the speed. My
> program is 3500 lines, and includes multi-dimensional arrays, to putting
> try's everywhere by hand is out of the question. I would then have to
> write a metaprogram that adds such try commands to an existing OCaml
> program and outputs a longer program with the try's with the asserts. If
> possible I would like to postpone that and try your other option.

I finally finished to implement a syntax extension that I once started to 
write, and gave up because of the difficulty. The result is a horribly 
long syntax extension, but it should work well with strings, arrays and 
bigarrays for both read and write accesses.

The syntax uses "#" instead of ".", but you can change this, if you don't 
want to change your program. There is also an option which allows you to 
restore the native mode. It's all there:

   http://martin.jambon.free.fr/ocaml.html#bounds

I tried it on a real program of my own which uses lots of arrays: my 
program runs 2-3 times slower than before, but this is due to a 
short piece of code. If I use regular array accesses in 
this portion of code, it works just as fast as before.

Well, that was not an easy syntax extension, but I'm satisfied with the 
result.
I hope you'll like it!


Martin

> Due to this crashing business I go on a business trip to Asia without any
> ready simulation results for one week.
>
>> * Use bytecode, and before running the program set the environment
>> variable OCAMLRUNPARAM=b which will print a stack trace.
>
> If I would use this week of the trip to try this suggestion you made, how
> will the stack trace give me the line number?
>
> Best regards,
>
> Andries
>
> ------------------------------------------------------------------------
> Dr. Ir. Andries P. Hekstra
> Philips Research
> High Tech Campus 27  (WL-1-4.15)
> 5656 AG Eindhoven
> Tel./Fax/Secr. +31 40 27 42048/42566/44051
>   *  Good open source break software for computer users :
> http://www.workrave.org
>
>
>
>
>
>
>
>
> Richard Jones <rich@annexia.org>
> 06-03-2006 12:14
>
> To
> Andries Hekstra/EHV/RESEARCH/PHILIPS@PHILIPS
> cc
> caml-list@yquem.inria.fr
> Subject
> Re: [Caml-list] Line number for index out of bounds
> Classification
>
>
>
>
>
>
>
> On Mon, Mar 06, 2006 at 11:44:31AM +0100, Andries Hekstra wrote:
>> Invalid_argument("index out of bounds")
> [...]
>> Of course, I am very curious in which line number of the program this
>> exception occurs.
>> Is there any way to get hold of this line number?
>
> This is a real problem with OCaml - it's impossible to get stack
> traces of where an exception happens with native code.  I'm assuming
> you're using native code.  I commonly have cases where a program dies
> with "exception: Not_found" because I forgot to enclose some List.find
> with an appropriate try ... with clause, or made some wrong
> assumption.  Tracking these down is time-consuming.
>
> Possible workarounds:
>
> * Use bytecode, and before running the program set the environment
> variable OCAMLRUNPARAM=b which will print a stack trace.
>
> * Surround every possible array index with a try ... with expression
> like this:
>
>  try
>    (* code which accesses the array *)
>  with
>    Invalid_argument "index out of bounds" -> assert false
>
> The "assert false" will print the line and character number of the
> assertion.
>
> * Hack ocamlopt to be able to print exceptions properly :-)
>
> Rich.
>
> -- 
> Richard Jones, CTO Merjis Ltd.
> Merjis - web marketing and technology - http://merjis.com
> Team Notepad - intranets and extranets for business -
> http://team-notepad.com
>
>

--
Martin Jambon, PhD
http://martin.jambon.free.fr

Edit http://wikiomics.org, bioinformatics wiki

Re: [Caml-list] Line number for index out of bounds

[Richard Jones]

On Sun, Mar 12, 2006 at 02:17:37AM -0800, Martin Jambon wrote:
> The syntax uses "#" instead of ".", but you can change this, if you don't 
> want to change your program. There is also an option which allows you to 
> restore the native mode. It's all there:
> 
>   http://martin.jambon.free.fr/ocaml.html#bounds

Slightly clearer description here :-)

http://martin.jambon.free.fr/pa_bounds/README

It'd be really nice to have this as the default behaviour of array and
string accesses ...  Since these are inlined, it seems like the only
penalty would be space for storing the location numbers.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com