[Caml-list] Lwt and exceptions

[Caml-list] Lwt and exceptions

[rixed]

The Lwt doc states that you should not use "raise" when using Lwt
but use Lwt.fail instead.

So, is it still OK to call functions (for instance from the stdlib)
that may raise an exception, provided we catch it soon enough ?
And by "soon enough" I mean: before an Lwt call that could schedule
another thread.

Re: [Caml-list] Lwt and exceptions

[Anil Madhavapeddy]

Yes, it's essential to do this for many common functions such as Hashtbl.find. The nice thing about Lwt is that you control yielding, so as long as you catch the exception locally and "convert" it into the Lwt monad, everything works great.

The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)

Anil

On 13 Sep 2011, at 19:37, rixed@happyleptic.org wrote:

> The Lwt doc states that you should not use "raise" when using Lwt
> but use Lwt.fail instead.
> 
> So, is it still OK to call functions (for instance from the stdlib)
> that may raise an exception, provided we catch it soon enough ?
> And by "soon enough" I mean: before an Lwt call that could schedule
> another thread.
> 
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 

Re: [Caml-list] Lwt and exceptions

[rixed]

-[ Tue, Sep 13, 2011 at 08:17:24PM +0100, Anil Madhavapeddy ]----
> The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)

Yes. Also, I'd like a Lwt.assert syntax extension to replace the
assert keyword. :-(

Re: [Caml-list] Lwt and exceptions

[Jérémie Dimino]

Le mercredi 14 septembre 2011 à 18:15 +0200, rixed@happyleptic.org a
écrit : 
> -[ Tue, Sep 13, 2011 at 08:17:24PM +0100, Anil Madhavapeddy ]----
> > The occasional exception leak is really hard to track down though, if you let one propagate by mistake. Ban OCaml exceptions! :)
> 
> Yes. Also, I'd like a Lwt.assert syntax extension to replace the
> assert keyword. :-(

I have added the assert_lwt keyword in pa_lwt.

-- 
Jérémie

Re: [Caml-list] Lwt and exceptions

[Mehdi Dogguy]

On 13/09/2011 20:37, rixed@happyleptic.org wrote:
> The Lwt doc states that you should not use "raise" when using Lwt but 
> use Lwt.fail instead.
> 
> So, is it still OK to call functions (for instance from the stdlib) 
> that may raise an exception, provided we catch it soon enough ? And by 
> "soon enough" I mean: before an Lwt call that could schedule another 
> thread.
> 

I guess, not (and it has been answered already). In fact, I was wondering
if Lwt's authors would be against adding a function like:

	let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e

It is stupid, trivial, etc… but looks what we need most of the time, no?
Instead of doing it in our own code, it could land in Lwt directly.
But, if it gets integrated into Lwt proper, users should be warned about
its behaviour. (especially with impure functions).

Regards,

-- 
Mehdi Dogguy مهدي الدڤي
http://dogguy.org/

Re: [Caml-list] Lwt and exceptions

[Anil Madhavapeddy]

On 15 Sep 2011, at 11:04, Mehdi Dogguy wrote:

> On 13/09/2011 20:37, rixed@happyleptic.org wrote:
>> The Lwt doc states that you should not use "raise" when using Lwt but 
>> use Lwt.fail instead.
>> 
>> So, is it still OK to call functions (for instance from the stdlib) 
>> that may raise an exception, provided we catch it soon enough ? And by 
>> "soon enough" I mean: before an Lwt call that could schedule another 
>> thread.
>> 
> 
> I guess, not (and it has been answered already). In fact, I was wondering
> if Lwt's authors would be against adding a function like:
> 
> 	let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
> 
> It is stupid, trivial, etc… but looks what we need most of the time, no?
> Instead of doing it in our own code, it could land in Lwt directly.
> But, if it gets integrated into Lwt proper, users should be warned about
> its behaviour. (especially with impure functions).

The try_lwt construct (in pa_lwt) or try_bind already converts normal exceptions into Lwt ones:

  try_lwt
    raise (Failure "")
  with Failure _ ->
    print_endline "Fail"; Lwt.return ()

...will print "Fail".

You just need to careful about raising exceptions across yield points when a try_lwt isn't present. The slightly painful thing about converting existing code to Lwt is that any function that raises an exception will gain the Lwt type if you convert it to use try_lwt/catch, even if the rest of the code doesn't otherwise block.

Anil

Re: [Caml-list] Lwt and exceptions

[Jérémie Dimino]

Le jeudi 15 septembre 2011 à 12:04 +0200, Mehdi Dogguy a écrit : 
> I guess, not (and it has been answered already). In fact, I was wondering
> if Lwt's authors would be against adding a function like:
> 
> 	let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
> 
> It is stupid, trivial, etc… but looks what we need most of the time, no?

Yes, it seems useful. But should it be:

  val wrap : ('a -> 'b) -> 'a -> 'b t

or:

  val wrap : (unit -> 'a) -> 'a t

?

I would tend for the second solution because if you are wrapping a
function that takes multiple arguments you are going to write:

  wrap (fun () -> f x y z) ()

anyway. Plus maybe wrap1, wrap2, ..., wrapn for a reasonable value of n.

-- 
Jérémie

Re: [Caml-list] Lwt and exceptions

[Mehdi Dogguy]

On 15/09/2011 14:09, Jérémie Dimino wrote:
> Le jeudi 15 septembre 2011 à 12:04 +0200, Mehdi Dogguy a écrit : 
>> I guess, not (and it has been answered already). In fact, I was wondering
>> if Lwt's authors would be against adding a function like:
>>
>> 	let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e
>>
>> It is stupid, trivial, etc… but looks what we need most of the time, no?
> 
> Yes, it seems useful. But should it be:
> 
>   val wrap : ('a -> 'b) -> 'a -> 'b t
> 
> or:
> 
>   val wrap : (unit -> 'a) -> 'a t
> 
> ?
> 
> I would tend for the second solution because if you are wrapping a
> function that takes multiple arguments you are going to write:
> 
>   wrap (fun () -> f x y z) ()
> 
> anyway. Plus maybe wrap1, wrap2, ..., wrapn for a reasonable value of n.
> 

Sure. Thanks for considering. Any of the above proposals would be fine for
me :)

-- 
Mehdi Dogguy مهدي الدڤي
http://dogguy.org/

Re: [Caml-list] Lwt and exceptions

[Dmitry Grebeniuk]

Hello.

> In fact, I was wondering
> if Lwt's authors would be against adding a function like:
>
>        let wrap f x = try Lwt.return (f x) with e -> Lwt.fail e

  Not a very good idea, since evaluation of wrap'ped
function's arguments can raise an exception that
won't be caught by wrap, suppose:
    let lwt_string_of_int = wrap string_of_int
    let () = Lwt_main.run (
       lwt_string_of_int (raise Exit) >>= fun s ->
       Lwt_io.printf "everything's okay buddies, it's %s\n" s
      )

  So it's up to the caller to guarantee the absence of
native exceptions.  It's bad.  If I will ever need to write
such wrapper, I will make it to have type
    wrap : (unit -> ('a -> 'b)) -> (unit -> 'a) -> 'b Lwt.t
maybe with the name of function too, and with
early evaluation of first argument, like
    let wrap ?(funcname="") u_f =
      let r_f =
        try
          `Ok (u_f ())
        with e_f ->
          `Error e_f
      in
      match r_f with
      | `Ok f -> fun x ->
        try
          Lwt.return (f x)
        with e_x ->
          (* 1.either *)
            Lwt.fail (Failure (Printf.sprintf
              "error evaluating function %S: %s"
              funcname (Printexc.to_string e_x)))
          (* 1.or *)
            Lwt.fail e_x
      | `Error e_f ->
         (* 2.either *)
           failwith (Printf.sprintf
              "the wrapped function %S itself \
               raises error: %s"
                  funcname (Printexc.to_string e_f))
         (* 2.or *)
          fun _u_x ->
            Lwt.fail e_f

  (I haven't checked this code!  I don't remember
where should I set begin-end or parenthesis in
nested matches in the original syntax, and I don't
want to remember it now.)

  I like the uniform error handling.  So I have
wrapped the common IO-specific functions
to a library that gives IO_lwt for lwt, IO_direct
for direct OCaml I/O (stdlib), and... functor
functor functor it sometimes.
  It uses "bind" and "return", but I'm very unsure
whether the code can be called "monadic IO",
since lwt itself does not respect the monad laws.
But it doesn't matter, since I'm not a academician,
I'm an engineer.
  So I'm using
    IO.catch : (unit -> 'a m) -> (exn -> 'a m) -> 'a m
even for direct IO computations.

Re: [Caml-list] Lwt and exceptions

[Jérémie Dimino]

Le jeudi 15 septembre 2011 à 15:22 +0300, Dmitry Grebeniuk a écrit : 
> It uses "bind" and "return", but I'm very unsure
> whether the code can be called "monadic IO",
> since lwt itself does not respect the monad laws.

What monadic laws lwt does not respect ?

-- 
Jérémie

Re: [Caml-list] Lwt and exceptions

[Dmitry Grebeniuk]

Hello.

>> It uses "bind" and "return", but I'm very unsure
>> whether the code can be called "monadic IO",
>> since lwt itself does not respect the monad laws.
>
> What monadic laws lwt does not respect ?

  Lwt sees values of type Lwt.t not as IO actions,
but sometimes as the values of these actions.

  It does not respect the monad associativity:

    (m >>= f) >>= g   ==   m >>= (fun x -> f x >>= g)

  Look at the source, see the left and the right
parts of this law as the "res" values, uncomment
either block, run the program, enter two lines
of text to program's stdin and see the output.
The output should be the same in both cases
if the law is respected.

==============
open Lwt;

value () = Printf.printf "case 1, enter two lines:\n%!";
value m = Lwt_io.read_line Lwt_io.stdin;
value f_v = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = (m >>= f) >>= g;

(*
value () = Printf.printf "case 2, enter two lines:\n%!";
value f_v = Lwt_io.read_line Lwt_io.stdin;
value m = Lwt_io.read_line Lwt_io.stdin;
value f = fun _ -> f_v;
value g = Lwt.return;
value res = m >>= (fun x -> f x >>= g);
*)

value () = Lwt_main.run (res >>= fun s ->
  Lwt_io.write_line Lwt_io.stdout ("res = " ^ s));
==============

  In the first case it's:

case 1, enter two lines:
a
b
res = b

  In the second case it's:

case 2, enter two lines:
a
b
res = a

  When I began to use Lwt, I was badly surprised
by the fact that Lwt.t values are not real I/O actions.
So I had to get a new habit: if the top-level module
value has type Lwt.t 'a, I have to defer its
computation by wrapping it under "fun () -> <expr>".
And I have to remember which of my functions return
Lwt.t values, which of my abstract types really have
type Lwt.t 'a or contain the Lwt.t 'a values, to defer
their computations/sideeffects too.

Re: [Caml-list] Lwt and exceptions

[Stéphane Glondu]

Le 17/09/2011 08:38, Dmitry Grebeniuk a écrit :
>   It does not respect the monad associativity:
> 
>     (m >>= f) >>= g   ==   m >>= (fun x -> f x >>= g)
> [...]
> ==============
> open Lwt;
> 
> value () = Printf.printf "case 1, enter two lines:\n%!";
> value m = Lwt_io.read_line Lwt_io.stdin;
> value f_v = Lwt_io.read_line Lwt_io.stdin;
> value f = fun _ -> f_v;
> value g = Lwt.return;
> value res = (m >>= f) >>= g;
> 
> (*
> value () = Printf.printf "case 2, enter two lines:\n%!";
> value f_v = Lwt_io.read_line Lwt_io.stdin;
> value m = Lwt_io.read_line Lwt_io.stdin;
> value f = fun _ -> f_v;
> value g = Lwt.return;
> value res = m >>= (fun x -> f x >>= g);
> *)
> 
> value () = Lwt_main.run (res >>= fun s ->
>   Lwt_io.write_line Lwt_io.stdout ("res = " ^ s));
> ==============

Using Lwt doesn't automatigally make OCaml purely functional... in your
example, you change the order of m and f_v, which both do side-effects,
so they are different in the two instances of res, so they cannot be
compared. Can you give an example that involves only pure functions?

>   When I began to use Lwt, I was badly surprised
> by the fact that Lwt.t values are not real I/O actions.
> So I had to get a new habit: if the top-level module
> value has type Lwt.t 'a, I have to defer its
> computation by wrapping it under "fun () -> <expr>".
> And I have to remember which of my functions return
> Lwt.t values, which of my abstract types really have
> type Lwt.t 'a or contain the Lwt.t 'a values, to defer
> their computations/sideeffects too.

Maybe what you expected was something that encapsulates the IO monad
too. Lwt doesn't do that: as with raw OCaml, IOs are direct with Lwt.


Cheers,

-- 
Stéphane

Re: [Caml-list] Lwt and exceptions

[Jérémie Dimino]

Le samedi 17 septembre 2011 à 09:38 +0300, Dmitry Grebeniuk a écrit : 
> When I began to use Lwt, I was badly surprised
> by the fact that Lwt.t values are not real I/O actions.
> So I had to get a new habit: if the top-level module
> value has type Lwt.t 'a, I have to defer its
> computation by wrapping it under "fun () -> <expr>".
> And I have to remember which of my functions return
> Lwt.t values, which of my abstract types really have
> type Lwt.t 'a or contain the Lwt.t 'a values, to defer
> their computations/sideeffects too.

You are comparing Lwt with the IO monad. They are two different monads:
IO deals with actions while Lwt deals with threads. If you write:

  let m = IO.read_char IO.stdin

then [m] is just the description of the action of reading a character
from [stdin]. On the contrary, if you write:

  let m = Lwt_io.read_char Lwt_io.stdin

then [m] is really a thread waiting for a character from [stdin]. If you
use "fun () -> ..." everywhere then this becomes actions but it is not
threads anymore and you loose most of the advantages of Lwt. For example
with Lwt you can write:

  let t1 = f1 ()
  and t2 = f2 ()
  ...
  and tn = fn () in 
  Lwt.bind t1 (fun x1 -> Lwt.bind t2 (fun x2 -> ... Lwt.bind tn (fun xn -> return (x1, x2, ..., xn)) ...))

and this will let [t1], ..., [tn] run concurrently. With the IO monad
this is just exactly the same as:

  let x1 = f1 () in
  let x2 = f2 () in
  ...
  let xn = fn () in
  (x1, x2, ..., xn)

in a non-monadic world. There is no concurrency at all.

Cheers,

-- 
Jérémie