[Caml-list] Destructive use of file descriptors

[Caml-list] Destructive use of file descriptors

[Goswin von Brederlow]

Hi,

in Python one can write:

    with open("foo") as fd:
        fd.write(str)

This involves some language magic that will open the file for the
duration of the block and close it at the end. The file descriptor is
automatically closed at a know time and not leaked or left until the
GC gets around to cleaning it up.

Has anyone constructed something like that for ocaml? Maybe with a ppx
extension?

MfG
	Goswin

Re: [Caml-list] Destructive use of file descriptors

[Nicolas Ojeda Bar]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

Hi,

No magic needed in OCaml:

  let with_open path f =
    let oc = open_out path in
    match f oc with
    | x -> close_out_noerr oc; x
    | exception e -> close_out_noerr oc; raise e

and then use it as

  with_open "foo" (fun oc -> output_string oc str)

Cheers

Nicolas

On Thu, Aug 6, 2015 at 11:40 AM, Goswin von Brederlow <goswin-v-b@web.de>
wrote:

> Hi,
>
> in Python one can write:
>
>     with open("foo") as fd:
>         fd.write(str)
>
> This involves some language magic that will open the file for the
> duration of the block and close it at the end. The file descriptor is
> automatically closed at a know time and not leaked or left until the
> GC gets around to cleaning it up.
>
> Has anyone constructed something like that for ocaml? Maybe with a ppx
> extension?
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 1984 bytes --]

Re: [Caml-list] Destructive use of file descriptors

[Gabriel Scherer]

Many libraries provide a with_file_{in,out} function capturing this
pattern, without need for specific language support.

  BatFile.with_file_out "foo" (fun fd ->
    print_endline str
  )

On Thu, Aug 6, 2015 at 11:40 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Hi,
>
> in Python one can write:
>
>     with open("foo") as fd:
>         fd.write(str)
>
> This involves some language magic that will open the file for the
> duration of the block and close it at the end. The file descriptor is
> automatically closed at a know time and not leaked or left until the
> GC gets around to cleaning it up.
>
> Has anyone constructed something like that for ocaml? Maybe with a ppx
> extension?
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Destructive use of file descriptors

[Pierrick Couderc]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

Using higher order functions should be enough, I think?

I personnaly use this one when opening files (input in that case):

type 'a res = Ok of 'a | Error of exn
let safe_open_in f file =
  try
    let ic = open_in file in
    let r = try Ok (f ic) with e -> Error e in
    close_in ic;
    r
  with e -> Error e



2015-08-06 11:40 GMT+02:00 Goswin von Brederlow <goswin-v-b@web.de>:

> Hi,
>
> in Python one can write:
>
>     with open("foo") as fd:
>         fd.write(str)
>
> This involves some language magic that will open the file for the
> duration of the block and close it at the end. The file descriptor is
> automatically closed at a know time and not leaked or left until the
> GC gets around to cleaning it up.
>
> Has anyone constructed something like that for ocaml? Maybe with a ppx
> extension?
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 1994 bytes --]

Re: [Caml-list] Destructive use of file descriptors

[Malcolm Matalka]

Nicolas Ojeda Bar <n.oje.bar@gmail.com> writes:

> Hi,
>
> No magic needed in OCaml:
>
>   let with_open path f =
>     let oc = open_out path in
>     match f oc with
>     | x -> close_out_noerr oc; x
>     | exception e -> close_out_noerr oc; raise e
>
> and then use it as
>
>   with_open "foo" (fun oc -> output_string oc str)
>
> Cheers
>
> Nicolas
>

The weakness with this (which I'm not sure if Python offers protection)
is returning something that references the file created.  It can lead to
some confusing error messages.

Re: [Caml-list] Destructive use of file descriptors

[Goswin von Brederlow]

On Thu, Aug 06, 2015 at 07:50:38PM +0000, Malcolm Matalka wrote:
> Nicolas Ojeda Bar <n.oje.bar@gmail.com> writes:
> 
> > Hi,
> >
> > No magic needed in OCaml:
> >
> >   let with_open path f =
> >     let oc = open_out path in
> >     match f oc with
> >     | x -> close_out_noerr oc; x
> >     | exception e -> close_out_noerr oc; raise e
> >
> > and then use it as
> >
> >   with_open "foo" (fun oc -> output_string oc str)
> >
> > Cheers
> >
> > Nicolas
> >
> 
> The weakness with this (which I'm not sure if Python offers protection)
> is returning something that references the file created.  It can lead to
> some confusing error messages.

Yeah. I was thinking that that might be avoided. Something on the line
of putting the file descriptor into a local module so ocaml would
complain about it escaping its scope or something.

At least unlike the other suggestion this one closes the FD on exception.

MfG
	Goswin

Re: [Caml-list] Destructive use of file descriptors

[Gabriel Scherer]

> Yeah. I was thinking that that might be avoided. Something on the line
> of putting the file descriptor into a local module so ocaml would
> complain about it escaping its scope or something.

You can use the "ST monad" trick to track escape of values or effects:
  "Region-based resource management", Oleg
  http://okmij.org/ftp/Haskell/regions.html

However, this requires writing the resource-manipulating operations in
monadic style.

module ST : sig
  type ('s, 'a) st
  val return : 'a -> ('s, 'a) st
  val bind : ('a -> ('s, 'b) st) -> ('s, 'a) st -> ('s, 'b) st

  type ('s, 'a) st_ref
  val st_ref : 'a -> ('s, ('s, 'a) st_ref) st
  val get : ('s, 'a) st_ref -> ('s, 'a) st
  val set : ('s, 'a) st_ref -> 'a -> ('s, unit) st

  type 'a secretive = { run : 's . unit -> ('s, 'a) st }
  val run_st : 'a secretive -> 'a
end = struct
  type ('s, 'a) st = 'a
  let return x = x
  let bind f m = f m

  type ('s, 'a) st_ref = 'a ref
  let st_ref, get, set = ref, (!), (:=)

  type 'a secretive = { run : 's . unit -> ('s, 'a) st }
  let run_st st = st.run ()
end

let (>>=) m f = ST.bind f m



# let test =
    ST.run_st { ST.run = fun () ->
      ST.st_ref 1 >>= fun r ->
      ST.get r >>= fun before ->
      ST.set r 2 >>= fun () ->
      ST.get r >>= fun after ->
      ST.return (before + after)
    };;
val test : int = 3

# let test_escape_1 =
    ST.run_st { ST.run = fun () ->
      ST.st_ref 1 >>= fun r ->
      ST.return r
    };;
Error: This field value has type unit -> ('a, ('a, int) ST.st_ref) ST.st
  which is less general than 'b. unit -> ('b, 'c) ST.st

# let test_escape_2 =
    let hole = ref None in
    ST.run_st { ST.run = fun () ->
      ST.st_ref 1 >>= fun r ->
      hole := Some r;
      ST.return ()
    };
    !hole;;
Error: This field value has type unit -> ('a, unit) ST.st
  which is less general than 'b. unit -> ('b, 'c) ST.st


On Tue, Aug 11, 2015 at 11:36 AM, Goswin von Brederlow
<goswin-v-b@web.de> wrote:
> On Thu, Aug 06, 2015 at 07:50:38PM +0000, Malcolm Matalka wrote:
>> Nicolas Ojeda Bar <n.oje.bar@gmail.com> writes:
>>
>> > Hi,
>> >
>> > No magic needed in OCaml:
>> >
>> >   let with_open path f =
>> >     let oc = open_out path in
>> >     match f oc with
>> >     | x -> close_out_noerr oc; x
>> >     | exception e -> close_out_noerr oc; raise e
>> >
>> > and then use it as
>> >
>> >   with_open "foo" (fun oc -> output_string oc str)
>> >
>> > Cheers
>> >
>> > Nicolas
>> >
>>
>> The weakness with this (which I'm not sure if Python offers protection)
>> is returning something that references the file created.  It can lead to
>> some confusing error messages.
>
> Yeah. I was thinking that that might be avoided. Something on the line
> of putting the file descriptor into a local module so ocaml would
> complain about it escaping its scope or something.
>
> At least unlike the other suggestion this one closes the FD on exception.
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Destructive use of file descriptors

[Ben Millwood]

[-- Attachment #1: Type: text/plain, Size: 3907 bytes --]

See also the following thread last year, "Not letting channels escape",
which (after some incomplete attempts from me) illustrates a fully safe
with_file function.

https://sympa.inria.fr/sympa/arc/caml-list/2014-08/msg00024.html

On 11 August 2015 at 12:14, Gabriel Scherer <gabriel.scherer@gmail.com>
wrote:

> > Yeah. I was thinking that that might be avoided. Something on the line
> > of putting the file descriptor into a local module so ocaml would
> > complain about it escaping its scope or something.
>
> You can use the "ST monad" trick to track escape of values or effects:
>   "Region-based resource management", Oleg
>   http://okmij.org/ftp/Haskell/regions.html
>
> However, this requires writing the resource-manipulating operations in
> monadic style.
>
> module ST : sig
>   type ('s, 'a) st
>   val return : 'a -> ('s, 'a) st
>   val bind : ('a -> ('s, 'b) st) -> ('s, 'a) st -> ('s, 'b) st
>
>   type ('s, 'a) st_ref
>   val st_ref : 'a -> ('s, ('s, 'a) st_ref) st
>   val get : ('s, 'a) st_ref -> ('s, 'a) st
>   val set : ('s, 'a) st_ref -> 'a -> ('s, unit) st
>
>   type 'a secretive = { run : 's . unit -> ('s, 'a) st }
>   val run_st : 'a secretive -> 'a
> end = struct
>   type ('s, 'a) st = 'a
>   let return x = x
>   let bind f m = f m
>
>   type ('s, 'a) st_ref = 'a ref
>   let st_ref, get, set = ref, (!), (:=)
>
>   type 'a secretive = { run : 's . unit -> ('s, 'a) st }
>   let run_st st = st.run ()
> end
>
> let (>>=) m f = ST.bind f m
>
>
>
> # let test =
>     ST.run_st { ST.run = fun () ->
>       ST.st_ref 1 >>= fun r ->
>       ST.get r >>= fun before ->
>       ST.set r 2 >>= fun () ->
>       ST.get r >>= fun after ->
>       ST.return (before + after)
>     };;
> val test : int = 3
>
> # let test_escape_1 =
>     ST.run_st { ST.run = fun () ->
>       ST.st_ref 1 >>= fun r ->
>       ST.return r
>     };;
> Error: This field value has type unit -> ('a, ('a, int) ST.st_ref) ST.st
>   which is less general than 'b. unit -> ('b, 'c) ST.st
>
> # let test_escape_2 =
>     let hole = ref None in
>     ST.run_st { ST.run = fun () ->
>       ST.st_ref 1 >>= fun r ->
>       hole := Some r;
>       ST.return ()
>     };
>     !hole;;
> Error: This field value has type unit -> ('a, unit) ST.st
>   which is less general than 'b. unit -> ('b, 'c) ST.st
>
>
> On Tue, Aug 11, 2015 at 11:36 AM, Goswin von Brederlow
> <goswin-v-b@web.de> wrote:
> > On Thu, Aug 06, 2015 at 07:50:38PM +0000, Malcolm Matalka wrote:
> >> Nicolas Ojeda Bar <n.oje.bar@gmail.com> writes:
> >>
> >> > Hi,
> >> >
> >> > No magic needed in OCaml:
> >> >
> >> >   let with_open path f =
> >> >     let oc = open_out path in
> >> >     match f oc with
> >> >     | x -> close_out_noerr oc; x
> >> >     | exception e -> close_out_noerr oc; raise e
> >> >
> >> > and then use it as
> >> >
> >> >   with_open "foo" (fun oc -> output_string oc str)
> >> >
> >> > Cheers
> >> >
> >> > Nicolas
> >> >
> >>
> >> The weakness with this (which I'm not sure if Python offers protection)
> >> is returning something that references the file created.  It can lead to
> >> some confusing error messages.
> >
> > Yeah. I was thinking that that might be avoided. Something on the line
> > of putting the file descriptor into a local module so ocaml would
> > complain about it escaping its scope or something.
> >
> > At least unlike the other suggestion this one closes the FD on exception.
> >
> > MfG
> >         Goswin
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 6184 bytes --]