Trouble with Scanf and files

Trouble with Scanf and files

[Christopher Kauffman]

I have encountered some difficulty using Scanf with a large number of files. As 
suggested in the the Ocaml manual, my strategy so far when reading a number of 
files has been to use Scanf.Scanning.from_file to create a scanbuf for each 
file. I had always wondered if this would cause problems with large numbers of 
files because I could not figure out a way to close a file after completing 
operations on it. Today I attempted to process too many and received a fatal 
system error. If one allocates a scanbuf via

let b = Scanf.Scanning.from_file "somefile" in ...

is there a way to subsequently close the file when the buffer is no longer 
needed? Or is this an issue that should be taken care of by the garbage 
collector in some strange way?

Cheers,
Chris Kauffman

Re: [Caml-list] Trouble with Scanf and files

[Karl Zilles]

Christopher Kauffman wrote:
> I have encountered some difficulty using Scanf with a large number of
> files. As suggested in the the Ocaml manual, my strategy so far when
> reading a number of files has been to use Scanf.Scanning.from_file to
> create a scanbuf for each file. I had always wondered if this would
> cause problems with large numbers of files because I could not figure
> out a way to close a file after completing operations on it. Today I
> attempted to process too many and received a fatal system error. If one
> allocates a scanbuf via
 >
 > let b = Scanf.Scanning.from_file "somefile" in ...
 >
> is there a way to subsequently close the file when the buffer is no
> longer needed? Or is this an issue that should be taken care of by the
> garbage collector in some strange way?

The documentation doesn't seem to be very clear.  But, it would be 
unusual for Ocaml code to rely on the garbage collector to close file 
handles.

Perhaps this is obvious, but maybe

let ic = open_in "somefile" in
let b = Scanf.Scanning.from_channel ic in
...
close_in ic

would solve your problem.

Re: [Caml-list] Trouble with Scanf and files

[Olivier Andrieu]

Hi,

On Jan 18, 2008 1:51 AM, Christopher Kauffman <kauffman@cs.umn.edu> wrote:
> I have encountered some difficulty using Scanf with a large number of files. As
> suggested in the the Ocaml manual, my strategy so far when reading a number of
> files has been to use Scanf.Scanning.from_file to create a scanbuf for each
> file. I had always wondered if this would cause problems with large numbers of
> files because I could not figure out a way to close a file after completing
> operations on it. Today I attempted to process too many and received a fatal
> system error. If one allocates a scanbuf via
>
> let b = Scanf.Scanning.from_file "somefile" in ...
>
> is there a way to subsequently close the file when the buffer is no longer
> needed? Or is this an issue that should be taken care of by the garbage
> collector in some strange way?

The garbage collector does not close I/O channels.

Have a look at the implementation of Scanf:
http://camlcvs.inria.fr/cgi-bin/cvsweb/ocaml/stdlib/scanf.ml?rev=1.77
Apparently close_in is called when the end of file is encountered.
So if you stop scanning before the end of file, you'd better use
Scanning.from_channel and close the channel yourself.

-- 
  Olivier

Re: [Caml-list] Trouble with Scanf and files

[Chris Kauffman]

Most interesting - I used Scanf.Scanning.from_file as the
documentation states that it uses block reads that are easier on the
I/O system than Scanning.from_channel. I rewrote using instead the
standard Pervasives.open_in and the associated input_line and
Scanf.sscanf; this seems to work fine for my purposes.

Olivier seems to have hit the nail on the head: if the end of a file
is not reached with Scanning.from_file, it stays open with essentially
no way to close it (other than artificially scanning to the end). I
think this issue is worth a bug report so I'll see if I can submit
one.

Thanks for helping to resolve this.
-Chris