[Caml-list] Segmentation faults in unexpected places

[Peter Frey <pjfrey@sympatico.ca>]

For some time I have been concerned about encountering "Segmentation
fault".  Since I am writing an interpreter, making heavy use of
continuations and streams, encountering these faults made be very
uneasy.  These errors always depended on the (order of) size of the
input and typically occurred for files larger then about 200 thousand,
when the program was compiled with ocamlopt.  With ocamlc, oddly enough,
errors rarely occur, and when they do, I usually get a stack error,
rather them a Segmentation fault.
After some serious digging I was able to construct a simple example that
seems to reliably demonstrate the problem:

open Unix
open Printf 

let stol len =          (* create some bogus int list of length len *)
  let head = ref [] in for i = 0 to pred len do head := i :: !head
done; !head

let _ =    (* read some file via io channel as stream; creating list *)
  let limit = (try (int_of_string (Sys.argv.(1))) with _ -> 300000 ) in
  let str_list = stol limit in
  let stupid_copy = List.fold_right (fun x l -> x :: l) str_list [] in
  printf"List copied with fold_right; len:%i\n" (List.lengthstupid_copy)

COMPILED WITH: ocamlfind ocamlopt -package unix -linkpkg dink.ml
./a.out
List copied with fold_right; len:300000
real 0m0.228s
user 0m0.150s
sys 0m0.070s

knoppix@ubuntu:~/ocaml/dink$ time ./a.out 1000000

Segmentation fault                  ************************************

real 0m0.216s
user 0m0.150s
sys 0m0.070s
knoppix@ubuntu:~/ocaml/dink$ 

when COMPILED with ocamlc
knoppix@ubuntu:~/ocaml/dink$ time ./a.out 100000000
Fatal error: exception Stack_overflow
Raised by primitive operation at file "list.ml", line 79, characters
16-37
Called from file "list.ml", line 79, characters 16-37
Called from file "list.ml", line 79, characters 16-37
...

Thus it seems that stack overflow is not always caught (in ocalopt).
Usually it is possible to program around it; by ensuring most procedures
are tail recursive; but when using continuations, this is not feasible.

The reason for stumbling upon this sample is the module Stream.  It has
a constructor:

let of_list l =
  {count = 0; data = List.fold_right (fun x l -> Scons (x, l)) l Sempty}

This is not tail recursive; but I can get around it:

  let streamed_list = ref str_list in
  let stream = Stream.from (fun pos -> match !streamed_list with
    | [] -> None
    | h :: t -> streamed_list := t; Some h) in
  
  let rec all len accu = 
    match Stream.peek stream with
    | Some c -> Stream.junk stream;
                all (succ len) ( c :: accu )
    | None -> len, accu     
  in let len, ul = all 0 [] in
  printf"\ndone ul len:%i\n" (List.length ul)   

Stream does not patch (append to) an empty stream; i.e.: to the end of
the list after the constructor returns None; so this should be safe.
(To be sure I will have to append while consuming ... )

Peter Frey