[Caml-list] Uncaught exceptions in function type.

[Caml-list] Uncaught exceptions in function type.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

Hi everyone,

Out of curiosity, I was wondering how difficult it would be in theory to
extend the type system so that exceptions that can pop out of a function
when it is called would be included in the type of the function. Could this
type information be infered automatically? Could this be used to have an
exhaustivity check in the "with" part of a try ... with expression?

I guess that if it was so easy, we would already be enjoying it within our
favorite compiler, but I fail to see how hairy is the question.

Cheers,
  Philippe.

[-- Attachment #2: Type: text/html, Size: 609 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Romain Bardou]

On 26/05/2014 16:23, Philippe Veber wrote:
> Hi everyone,
> 
> Out of curiosity, I was wondering how difficult it would be in theory to
> extend the type system so that exceptions that can pop out of a function
> when it is called would be included in the type of the function. Could
> this type information be infered automatically? Could this be used to
> have an exhaustivity check in the "with" part of a try ... with expression?
> 
> I guess that if it was so easy, we would already be enjoying it within
> our favorite compiler, but I fail to see how hairy is the question.
> 
> Cheers,
>   Philippe.
> 
> 

Some issues:

- One needs to separate exceptions into two groups, the ones that you
are actually interested in typing (their purpose is to kill the program,
so to speak) and the ones that you are not (they are actually used for
control flow).

- I'm not sure it is easy to infer. For instance, in:

let f g =
  g 1

should we just assume that g raises nothing? Or should we use some kind
of row variable, like:

let f (g: 'a raise ([< ] as 'raises): 'b raise 'raises =
  g 1

But then what about:

let f
    (g: 'a raise ([< ] as 'raises_h)
    (h: 'a raise ([< ] as 'raises_g): 'b raise [ 'raises_g | 'raises_h ] =
  g 1 + h 2

is it sound to have those abstract union types [ 'raises_g | 'raises_h ]?

Do we want all functions to have these convoluted types?

-- 
Romain Bardou

Re: [Caml-list] Uncaught exceptions in function type.

[Ben Millwood]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

First of all, it seems to be universal among all languages I've much
experience with that there are unchecked exceptions. Your program might be
interrupted at any point by an out of memory error or a signal or
something, and there's just not much you can do about that. You can either
model it in your code or not, but you can't stop it from happening. So it
seems like the best we can hope for with typed exceptions is in addition to
unchecked ones, to make it possible (but not required) that a function
might declare some of the exceptions it can throw.

But after all exceptions are just "things I can return instead of a
result", and lightweight sum types are already pretty good at that. E.g.
(to use Romain's syntax)

    val lookup : map -> key -> value raise Not_found

is pretty much just the same as:

    val lookup : map -> key -> value option

True, exceptions get automatic propagation, but the option monad interface
makes that pretty lightweight, and you can do a similar thing with a less
trivial sum type if you need richer type information.

See also:
https://blogs.janestreet.com/how-to-fail-introducing-or-error-dot-t/ for
discussions of more ways you can make error handling both explicit and
concise.

[-- Attachment #2: Type: text/html, Size: 1499 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1848 bytes --]

Hi Romain!


> Some issues:
>
> - One needs to separate exceptions into two groups, the ones that you
> are actually interested in typing (their purpose is to kill the program,
> so to speak) and the ones that you are not (they are actually used for
> control flow).
>
I would not like such a separation, unless the user can decide the kind of
each exception (did you say burdensome? ;o). This kind of separation is
pretty much the reason why there are errors and exceptions in Java, IIRC.


>
> - I'm not sure it is easy to infer. For instance, in:
>
> let f g =
>   g 1
>
> should we just assume that g raises nothing?

No, we should assume g raises something, and that f raises the same.


> Or should we use some kind
> of row variable, like:
>
> let f (g: 'a raise ([< ] as 'raises): 'b raise 'raises =
>   g 1
>

Yes more like that


>
> But then what about:
>
> let f
>     (g: 'a raise ([< ] as 'raises_h)
>     (h: 'a raise ([< ] as 'raises_g): 'b raise [ 'raises_g | 'raises_h ] =
>   g 1 + h 2
>
> is it sound to have those abstract union types [ 'raises_g | 'raises_h ]?
>

I guess it is not easy because 'raises_g and 'raises_h may have
incompatibilities (same constructors with different arguments). Since you
worked on union of abstract polymorphic variant types, I guess you know
pretty well how difficult that would be ;o).


>
> Do we want all functions to have these convoluted types?
>
Representing those types could be optional most of the time. The only place
where they would be important would when catching exceptions, to have the
compiler check for exhaustivity.


>
> --
> Romain Bardou
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3255 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Thomas Blanc]

Le 26/05/2014 16:23, Philippe Veber a écrit :
> Hi everyone,
>
> Out of curiosity, I was wondering how difficult it would be in theory 
> to extend the type system so that exceptions that can pop out of a 
> function when it is called would be included in the type of the 
> function. Could this type information be infered automatically? Could 
> this be used to have an exhaustivity check in the "with" part of a try 
> ... with expression?
>
> I guess that if it was so easy, we would already be enjoying it within 
> our favorite compiler, but I fail to see how hairy is the question.
>
> Cheers,
>   Philippe.
>
>

This as already been analyzed by François Pessaux and Xavier Leroy, see 
[1] that does a very good survey on the matter.

The main problem for it is (as Romain pointed out) the problem of 
higher-order functions:
you would have to reannotate all of your .mli to add the exception 
annotations.
([1] solved it by doing a separate type analysis)

This would lead to a lot of problems (along with breaking existing code):
* "exceptions we know won't be throwed":
Typing analysis can't know that "x/2" or a.((Array.length a) - 1) 
wouldn't raise anything, so you'd have a lot of unexpected
exceptions that would come up.
* An arbitrary big number of exceptions
At the time [1] was wrote, there was necessarily a finite number of 
exceptions in your program, with nifty stuff like first class modules 
and generative functors it is no longer the case.

[1] http://pauillac.inria.fr/~xleroy/publi/exceptions-popl.ps.gz

Re: [Caml-list] Uncaught exceptions in function type.

[Gabriel Scherer]

The problem is with functions taken as parameter. What would be the
exception signature of List.map?

This work has already been done by François Pessaux in the "ocamlexc"
project, using row variables to infer convenient exception typings.
The problem is that he basically had to re-implement a full
type-checker for OCaml to do this, which grew unmaintained (it was a
PhD work) and is probably unusable today. There is of course interest
in reviving it (looking for simpler or integrated-upstream approaches
to reduce the maintenance burden), and he recently made the ocamlexc
sources available online for this purpose. If anyone is tempted to
contribute...

On Mon, May 26, 2014 at 4:23 PM, Philippe Veber
<philippe.veber@gmail.com> wrote:
> Hi everyone,
>
> Out of curiosity, I was wondering how difficult it would be in theory to
> extend the type system so that exceptions that can pop out of a function
> when it is called would be included in the type of the function. Could this
> type information be infered automatically? Could this be used to have an
> exhaustivity check in the "with" part of a try ... with expression?
>
> I guess that if it was so easy, we would already be enjoying it within our
> favorite compiler, but I fail to see how hairy is the question.
>
> Cheers,
>   Philippe.
>
>

Re: [Caml-list] Uncaught exceptions in function type.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

Hi Ben,

2014-05-26 17:13 GMT+02:00 Ben Millwood <bmillwood@janestreet.com>:

> First of all, it seems to be universal among all languages I've much
> experience with that there are unchecked exceptions. Your program might be
> interrupted at any point by an out of memory error or a signal or
> something, and there's just not much you can do about that. You can either
> model it in your code or not, but you can't stop it from happening. So it
> seems like the best we can hope for with typed exceptions is in addition to
> unchecked ones, to make it possible (but not required) that a function
> might declare some of the exceptions it can throw.
>

That's perfectly right.


>
> But after all exceptions are just "things I can return instead of a
> result", and lightweight sum types are already pretty good at that. E.g.
> (to use Romain's syntax)
>
>     val lookup : map -> key -> value raise Not_found
>
> is pretty much just the same as:
>
>     val lookup : map -> key -> value option
>

Right. This is a very good use case that I adopted thanks to core.


>
> True, exceptions get automatic propagation, but the option monad interface
> makes that pretty lightweight, and you can do a similar thing with a less
> trivial sum type if you need richer type information.
>

Using options or Result from time to time is certainly a nice habit.
However to be honest, using option/result monads extensively does not seem
right to me, like pulling the language in a direction it was not really
designed for. Monad combinators do a great job at propagating errors, but
still there is a serious addition of plumbing in function body's and Result
returning functions have a significantly less readable type. But I guess
this is pretty much a question of taste/habit to see if the added verbosity
is worth it.


>
> See also:
> https://blogs.janestreet.com/how-to-fail-introducing-or-error-dot-t/ for
> discussions of more ways you can make error handling both explicit and
> concise.
>

Thanks! BTW core still uses exceptions. Is there an explicit rule as to how
to decide between Result type or exceptions. For instance, why not write
the Array.create function like this:

val create : int -> 'a -> 'a array Or_error.t

where create fails for a negative integer?

[-- Attachment #2: Type: text/html, Size: 3469 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 2010 bytes --]

Thanks a lot Thomas, this is certainly enough to satisfy my curiosity!


2014-05-26 17:33 GMT+02:00 Thomas Blanc <thomas.blanc@crans.org>:

> Le 26/05/2014 16:23, Philippe Veber a écrit :
>
>  Hi everyone,
>>
>> Out of curiosity, I was wondering how difficult it would be in theory to
>> extend the type system so that exceptions that can pop out of a function
>> when it is called would be included in the type of the function. Could this
>> type information be infered automatically? Could this be used to have an
>> exhaustivity check in the "with" part of a try ... with expression?
>>
>> I guess that if it was so easy, we would already be enjoying it within
>> our favorite compiler, but I fail to see how hairy is the question.
>>
>> Cheers,
>>   Philippe.
>>
>>
>>
> This as already been analyzed by François Pessaux and Xavier Leroy, see
> [1] that does a very good survey on the matter.
>
> The main problem for it is (as Romain pointed out) the problem of
> higher-order functions:
> you would have to reannotate all of your .mli to add the exception
> annotations.
> ([1] solved it by doing a separate type analysis)
>
> This would lead to a lot of problems (along with breaking existing code):
> * "exceptions we know won't be throwed":
> Typing analysis can't know that "x/2" or a.((Array.length a) - 1) wouldn't
> raise anything, so you'd have a lot of unexpected
> exceptions that would come up.
> * An arbitrary big number of exceptions
> At the time [1] was wrote, there was necessarily a finite number of
> exceptions in your program, with nifty stuff like first class modules and
> generative functors it is no longer the case.
>
> [1] http://pauillac.inria.fr/~xleroy/publi/exceptions-popl.ps.gz
>
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 2930 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le lundi, 26 mai 2014 à 18:02, Philippe Veber a écrit :
> Thanks! BTW core still uses exceptions. Is there an explicit rule as to how to decide between Result type or exceptions. For instance, why not write the Array.create function like this:
>  
> val create : int -> 'a -> 'a array Or_error.t
>  
> where create fails for a negative integer?
Because that would be utterly annoying. You need to make the following distinctions:

* Programming errors, for contracts with the programmer that cannot be enforced through types. For that raises Invalid_argument if the contract is violated. Invalid_argument is not supposed to be handled, it denotes an API misuse, like calling Array.create with a negative integer.

* Exceptional errors, for errors that the programmer is unlikely to handle at all (e.g. out of memory). For that raise a custom exception. This should occur very rarely, you are unlikely to ever define one such exception.

* Non-exceptional errors, errors that the programmer will have to handle (e.g. failing to connect a socket), for that do not use a custom exception but use variants or options types.  

In general if you write libraries it’s better to err on the side of exceptionless design: never use exceptions beyond Invalid_argument (and especially never use Not_found or Failure). Leave exception definition/usage at the discretion of the user (if he wishes to shoot himself in the foot).

Best,

Daniel

Re: [Caml-list] Uncaught exceptions in function type.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 3829 bytes --]

2014-05-26 18:34 GMT+02:00 Daniel Bünzli <daniel.buenzli@erratique.ch>:

> Le lundi, 26 mai 2014 à 18:02, Philippe Veber a écrit :
> > Thanks! BTW core still uses exceptions. Is there an explicit rule as to
> how to decide between Result type or exceptions. For instance, why not
> write the Array.create function like this:
> >
> > val create : int -> 'a -> 'a array Or_error.t
> >
> > where create fails for a negative integer?
> Because that would be utterly annoying. You need to make the following
> distinctions:
>
> * Programming errors, for contracts with the programmer that cannot be
> enforced through types. For that raises Invalid_argument if the contract is
> violated. Invalid_argument is not supposed to be handled, it denotes an API
> misuse, like calling Array.create with a negative integer.
>
> * Exceptional errors, for errors that the programmer is unlikely to handle
> at all (e.g. out of memory). For that raise a custom exception. This should
> occur very rarely, you are unlikely to ever define one such exception.
>
> * Non-exceptional errors, errors that the programmer will have to handle
> (e.g. failing to connect a socket), for that do not use a custom exception
> but use variants or options types.
>
> In general if you write libraries it's better to err on the side of
> exceptionless design: never use exceptions beyond Invalid_argument (and
> especially never use Not_found or Failure).


Thanks Daniel, this is a useful summary. A couple of questions/remarks:
- I observed in many libraries (including Xmlm) that people tend to define
custom exceptions to signal parsing errors. According to the rules you
provide, parsers should return variants, right?
- as Ben noted earlier, I think point 2 would better be formulated as
errors that can virtually appear anywhere and/or a programmer cannot do
much about (as you say)
- The adjective "exceptional" is a bit misleading IMO, in particular I
guess it should not be understood as "not frequent". Rare failures can be
very harmful and should be given extra-care since they are not easy to
provoke and debug. Did you mean more like "very abnormal" and so difficult
to recover?
- Another criterion I like is that if checking the precondition of a
function f basically amounts (in terms of complexity/algorithm) to calling
f, then f shouldn't raise an exception for this precondition. If the
precondition is difficult to check, a user is likely not to check it:
    - List.find assumes the searched element is in the list, but checking
that is precisely what the function is about.
    - Connecting to a socket assumes it is up and available, but checking
that without doing the connection is meaningless because you are not sure
the socket will be available just after the check
    - in a parser you won't know if the input is syntactically correct
before actually performing the parsing.



> Leave exception definition/usage at the discretion of the user (if he
> wishes to shoot himself in the foot).
>

Having no experience on large projects, it is not clear to me why
exceptions are considered so dangerous. One point I see is that when you
decide to handle errors with a try ... with expression, there is no
exhaustivity check to ensure you handled all possible cases, and that you
have to rely on (possibly incomplete) documentation and reasonning to
convince yourself you forgot nothing. Another related one is that there is
no automatic analysis to show exceptions thrown by a function in the
documentation. These are of course already strong reasons why not to use
too many exceptions (and were the motivation for my initial question). Do
you see other reasons?

 Anyway, thanks everyone, this has been very informative to me!


> Best,
>
> Daniel
>
>
>

[-- Attachment #2: Type: text/html, Size: 4647 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Ben Millwood]

[-- Attachment #1: Type: text/plain, Size: 3770 bytes --]

On 26 May 2014 17:02, Philippe Veber <philippe.veber@gmail.com> wrote:
>
> Using options or Result from time to time is certainly a nice habit.
> However to be honest, using option/result monads extensively does not seem
> right to me, like pulling the language in a direction it was not really
> designed for. Monad combinators do a great job at propagating errors, but
> still there is a serious addition of plumbing in function body's and Result
> returning functions have a significantly less readable type. But I guess
> this is pretty much a question of taste/habit to see if the added verbosity
> is worth it.
>

It's true that using a monadic style does introduce *some* noise, but I
don't think it's so bad, especially if you design your API to fit with the
style. In a way I almost prefer having the binds there so that I can easily
see where the possible interruption points in the flow are, and in
particular it's /syntactically/ obvious where errors /can't/ occur.

BTW core still uses exceptions. Is there an explicit rule as to how to
> decide between Result type or exceptions. For instance, why not write the
> Array.create function like this:
>
> val create : int -> 'a -> 'a array Or_error.t
>
> where create fails for a negative integer?
>

Daniel's summary is very good, although we do use Failure in Core, often
for assertion-failure like scenarios or just any case where the error isn't
meant to be handled specifically. In many cases Invalid_argument would
arguably be more appropriate, but in other cases that isn't obvious (say if
you mutate a Queue.t while you are folding over it, that will be detected
and an error will be thrown, but it isn't /really/ an invalid argument
error), and in any case since we often don't intend to catch them except to
print a stack trace and die, it doesn't matter too much.

We also do just break our own rules sometimes. Possible reasons why are:
- some aspect of usability takes priority over dogmatics (although this is
much rarer than you'd think)
- legacy exceptions (see below)
- sometimes we make bad decisions :(

Having no experience on large projects, it is not clear to me why
> exceptions are considered so dangerous. One point I see is that when you
> decide to handle errors with a try ... with expression, there is no
> exhaustivity check to ensure you handled all possible cases, and that you
> have to rely on (possibly incomplete) documentation and reasonning to
> convince yourself you forgot nothing. Another related one is that there is
> no automatic analysis to show exceptions thrown by a function in the
> documentation. These are of course already strong reasons why not to use
> too many exceptions (and were the motivation for my initial question). Do
> you see other reasons?
>

One particularly galling aspect of exceptions not showing up in types is
that they can't easily be refactored. If you decide to change which
exceptions a given function throws, you can't rely on the typechecker to
flag up where code was written to the old specification and needs changing.
You can't even just eyeball the code to see where the function was used and
see if the exception was caught, because who knows where in the call stack
it might have been handled? Explicit error types, on the other hand, will
refuse to compile unless you have changed your code to deal with it, so you
can make your errors richer or more precise with a high level of confidence
that you haven't introduced any "holes" for exceptional conditions to sneak
out where previously they couldn't.

This is essentially why we still have a few Not_found exceptions in Core,
because it's really pretty hard to know where they might be relied upon, so
it's easier to leave them in than purge them and risk silent breakage.

[-- Attachment #2: Type: text/html, Size: 4865 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

On Mon, May 26, 2014 at 06:34:33PM +0200, Daniel Bünzli wrote:
> Le lundi, 26 mai 2014 à 18:02, Philippe Veber a écrit :
> > Thanks! BTW core still uses exceptions. Is there an explicit rule as to how to decide between Result type or exceptions. For instance, why not write the Array.create function like this:
> >  
> > val create : int -> 'a -> 'a array Or_error.t
> >  
> > where create fails for a negative integer?
> Because that would be utterly annoying. You need to make the
> following distinctions:
> 
> * Programming errors, for contracts with the programmer that cannot
> be enforced through types. For that raises Invalid_argument if the
> contract is violated. Invalid_argument is not supposed to be handled,
> it denotes an API misuse, like calling Array.create with a negative
> integer.

Sometimes arguments are computed and suddenly exceed aceptable
parameters. E.g. the user specifies a data file that needs to be read
into a string. If the file is too large you get Invalid_argument.
Programms can catch and handle such cases, at least to the point of
giving a good error message.

Idealy a program should catch all exceptions and explain in a human
readable way why they happened.
 
> * Exceptional errors, for errors that the programmer is unlikely to
> handle at all (e.g. out of memory). For that raise a custom exception.
> This should occur very rarely, you are unlikely to ever define one
> such exception.

Out of memory is probably the only exception you can't do anything
about in ocaml. Catching and handling it would nearly always need more
ram causing just another Ouf_of_memory exception. In ocaml you are
pretty much screwed.
 
> * Non-exceptional errors, errors that the programmer will have to
> handle (e.g. failing to connect a socket), for that do not use a
> custom exception but use variants or options types.  

That is a matter of taste and with recursions in the mix exceptions
can be a great way to abort the recursion while keeping the code flow
simple. They also allow you to use things like List.fold_left and
abort early.

Variants or option types are not always the best solution.
 
> In general if you write libraries it???s better to err on the side
> of exceptionless design: never use exceptions beyond Invalid_argument
> (and especially never use Not_found or Failure). Leave exception
> definition/usage at the discretion of the user (if he wishes to shoot
> himself in the foot).
> 
> Best,
> 
> Daniel

MfG
	Goswin

Re: [Caml-list] Uncaught exceptions in function type.

[David House]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

On 27 May 2014 09:49, Goswin von Brederlow <goswin-v-b@web.de> wrote:

> > * Non-exceptional errors, errors that the programmer will have to
> > handle (e.g. failing to connect a socket), for that do not use a
> > custom exception but use variants or options types.
>
> That is a matter of taste and with recursions in the mix exceptions
> can be a great way to abort the recursion while keeping the code flow
> simple. They also allow you to use things like List.fold_left and
> abort early.
>

See https://blogs.janestreet.com/core-gems-many-happy-returns/ for how we
solve this problem in core. The interface is very natural. It uses
exceptions under the hood, although that is not exposed.

However, I think this is entirely separate from the rest of the discussion.
Here we are merely using exceptions for their control flow properties --
there is no error condition.


> Variants or option types are not always the best solution.
>

That's certainly true, although I would argue that exceptions are certainly
overused. We use them quite a bit in core and in the rest of JS's codebase
-- but only for truly "exceptional" circumstances.


> > In general if you write libraries it???s better to err on the side
> > of exceptionless design: never use exceptions beyond Invalid_argument
> > (and especially never use Not_found or Failure). Leave exception
> > definition/usage at the discretion of the user (if he wishes to shoot
> > himself in the foot).
> >
> > Best,
> >
> > Daniel
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3134 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Nicolas Boulay]

[-- Attachment #1: Type: text/plain, Size: 2456 bytes --]

2014-05-26 18:34 GMT+02:00 Daniel Bünzli <daniel.buenzli@erratique.ch>:

> Le lundi, 26 mai 2014 à 18:02, Philippe Veber a écrit :
> > Thanks! BTW core still uses exceptions. Is there an explicit rule as to
> how to decide between Result type or exceptions. For instance, why not
> write the Array.create function like this:
> >
> > val create : int -> 'a -> 'a array Or_error.t
> >
> > where create fails for a negative integer?
> Because that would be utterly annoying. You need to make the following
> distinctions:
>
>
Yes it could be annoying, but very high quality software become much harder
to write. Refactoring is harder. Missing exception handler are harder to
find.


> * Programming errors, for contracts with the programmer that cannot be
> enforced through types. For that raises Invalid_argument if the contract is
> violated. Invalid_argument is not supposed to be handled, it denotes an API
> misuse, like calling Array.create with a negative integer.
>

> * Exceptional errors, for errors that the programmer is unlikely to handle
> at all (e.g. out of memory). For that raise a custom exception. This should
> occur very rarely, you are unlikely to ever define one such exception.
>
> That means intensive testing to be sur to avoid such failure for normal
user input. For most long running programme (server, gui), that's could a
problem.

For example, an undo/redo use lot of memory after some copy/paste on big
data, then the save command have not enough memory to work, and the file
are troncated. That's not acceptable, and can be see only with big data,
after few high level command run.

Regards,
Nicolas


> * Non-exceptional errors, errors that the programmer will have to handle
> (e.g. failing to connect a socket), for that do not use a custom exception
> but use variants or options types.
>
>
In general if you write libraries it’s better to err on the side of
> exceptionless design: never use exceptions beyond Invalid_argument (and
> especially never use Not_found or Failure). Leave exception
> definition/usage at the discretion of the user (if he wishes to shoot
> himself in the foot).
>
> Best,
>
> Daniel
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3741 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

On Mon, May 26, 2014 at 05:25:21PM +0200, Philippe Veber wrote:
> Hi Romain!
> 
> 
> > Some issues:
> >
> > - One needs to separate exceptions into two groups, the ones that you
> > are actually interested in typing (their purpose is to kill the program,
> > so to speak) and the ones that you are not (they are actually used for
> > control flow).
> >
> I would not like such a separation, unless the user can decide the kind of
> each exception (did you say burdensome? ;o). This kind of separation is
> pretty much the reason why there are errors and exceptions in Java, IIRC.

An exception that is used for control flow will be thrown and quickly
caught. It would not appear in a functions signature if the control
flow is done right. But if you made a mistake and didn't catch it then
it should appear in the signature and give you cause to recheck your
code.

I would say that exceptions used for control flow are exactly the
ones that you want typed.

On the other hand exceptions that kill the programm, like out of
memory, would be completly useless to type. Basically anything can
throw out of memory and nobody can catch it. It would simply appear in
every single (non trivial) function and annoy the programmer.

> > - I'm not sure it is easy to infer. For instance, in:
> >
> > let f g =
> >   g 1
> >
> > should we just assume that g raises nothing?
> 
> No, we should assume g raises something, and that f raises the same.
> 
> 
> > Or should we use some kind
> > of row variable, like:
> >
> > let f (g: 'a raise ([< ] as 'raises): 'b raise 'raises =
> >   g 1
> >
> 
> Yes more like that
> 
> 
> >
> > But then what about:
> >
> > let f
> >     (g: 'a raise ([< ] as 'raises_h)
> >     (h: 'a raise ([< ] as 'raises_g): 'b raise [ 'raises_g | 'raises_h ] =
> >   g 1 + h 2
> >
> > is it sound to have those abstract union types [ 'raises_g | 'raises_h ]?
> >
> 
> I guess it is not easy because 'raises_g and 'raises_h may have
> incompatibilities (same constructors with different arguments). Since you
> worked on union of abstract polymorphic variant types, I guess you know
> pretty well how difficult that would be ;o).

# exception Foo of int;;
exception Foo of int
# let f () = raise (Foo 1);;
val f : unit -> 'a = <fun>
# exception Foo of float;;
exception Foo of float
# let g () = raise (Foo 1.0);;
val g : unit -> 'a = <fun>
# let h () = f () + g ();;
# let h () = let t = f () in g ();;
Warning 26: unused variable t.
val h : unit -> 'a = <fun>
# f ();;
Exception: Foo 1.
# g ();;
Exception: Foo 1..
# h ();;
Exception: Foo 1.
# try h (); 1.0 with Foo x -> x;;
Warning 21: this statement never returns (or has an unsound type.)
Exception: Foo 1.

Nothing wrong with h () raising [ Foo of int | float ] other than that
it makes it impossible to catch because Foo of int is shadowed by Foo
of float. The user couldn't catch both, the compiler would infer that
anything using h throws at least Foo of int and the user would quickly
realize that it wasn't a good idea to have to constructors with
different types. Problem solved.

Actualy infering exceptions would be a great help there. The user
might think that he caught Foo of int in the above code. The compiler
would proof him wrong highlighting the problem.



But consider the following:

let h f = let t = f () in function g -> g ()

val h : (unit -> 'a raise ([< ] as 'rf))
        -> (((unit -> 'a raise ([< ] as 'rg)) -> unit raise 'rg)) raise 'rf

Here h raises both 'rf and 'rg, just like in the above example. BUT
'rf is raised on and only on the partial application of f. This is
important when you have:

let h' f g =
  let t = try Some (f ()) with Foo x -> None in
  match t with
  | None -> None
  | Some x -> g x

The compiler has to see that (at least some of) the execptions of f
are cought and only the exceptions of g can be raised (and what isn't
cought of f).

Types can get rather comvoluted.

But hey, if they type gets to complex then maybe you should try using
less exceptions and more variant types and options. :)

> > Do we want all functions to have these convoluted types?
> >
> Representing those types could be optional most of the time. The only place
> where they would be important would when catching exceptions, to have the
> compiler check for exhaustivity.

And .mli files and signatures.

But I agree that the point of infering exceptions would be to have the
compiler check that there are none (or only the short list of allowed
exceptions).

That just leaves higher level functions. They would all have to be
convoluted.

Also would

# let f g x = g x
val f : ('a -> 'b) -> 'a -> 'b = <fun>

mean that g must not throw any exceptions? Or can the type system
infer that f will throw the same exceptions as g because no complex
exception type was specified? Can we make the "raise ([< ] as
'raises_g)" part implied? That would greatly help the transition and
cut down on convoluted types. The case of higher level functions just
passing through any exceptions would be the most common case and
should have a simple type.

MfG
	Goswin

Re: [Caml-list] Uncaught exceptions in function type.

[Romain Bardou]

On 27/05/2014 11:28, Goswin von Brederlow wrote:
> 
> On Mon, May 26, 2014 at 05:25:21PM +0200, Philippe Veber wrote:
>> Hi Romain!
>>
>>
>>> Some issues:
>>>
>>> - One needs to separate exceptions into two groups, the ones that you
>>> are actually interested in typing (their purpose is to kill the program,
>>> so to speak) and the ones that you are not (they are actually used for
>>> control flow).
>>>
>> I would not like such a separation, unless the user can decide the kind of
>> each exception (did you say burdensome? ;o). This kind of separation is
>> pretty much the reason why there are errors and exceptions in Java, IIRC.
> 
> An exception that is used for control flow will be thrown and quickly
> caught. It would not appear in a functions signature if the control
> flow is done right. But if you made a mistake and didn't catch it then
> it should appear in the signature and give you cause to recheck your
> code.
> 
> I would say that exceptions used for control flow are exactly the
> ones that you want typed.
> 
> On the other hand exceptions that kill the programm, like out of
> memory, would be completly useless to type. Basically anything can
> throw out of memory and nobody can catch it. It would simply appear in
> every single (non trivial) function and annoy the programmer.

Right sorry I was apparently not typing what was actually in my head :D

-- 
Romain

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

On Tue, May 27, 2014 at 09:42:35AM +0100, Ben Millwood wrote:
> One particularly galling aspect of exceptions not showing up in types is
> that they can't easily be refactored. If you decide to change which
> exceptions a given function throws, you can't rely on the typechecker to
> flag up where code was written to the old specification and needs changing.
> You can't even just eyeball the code to see where the function was used and
> see if the exception was caught, because who knows where in the call stack
> it might have been handled? Explicit error types, on the other hand, will
> refuse to compile unless you have changed your code to deal with it, so you
> can make your errors richer or more precise with a high level of confidence
> that you haven't introduced any "holes" for exceptional conditions to sneak
> out where previously they couldn't.
> 
> This is essentially why we still have a few Not_found exceptions in Core,
> because it's really pretty hard to know where they might be relied upon, so
> it's easier to leave them in than purge them and risk silent breakage.

HUH?

Say you have a function

val f : ('a, 'b) t -> 'a -> 'b   (raise [Not_found])

then eliminating Not_found that function would become

val f : ('a, 'b) t -> 'a -> 'b option

That should fail to compile if the return type is used or a signature
exists. Where do you think a change would go unnoticed?

MfG
	Goswin

Re: [Caml-list] Uncaught exceptions in function type.

[Ben Millwood]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

On 27 May 2014 11:05, Goswin von Brederlow <goswin-v-b@web.de> wrote:

> On Tue, May 27, 2014 at 09:42:35AM +0100, Ben Millwood wrote:
> > This is essentially why we still have a few Not_found exceptions in Core,
> > because it's really pretty hard to know where they might be relied upon,
> so
> > it's easier to leave them in than purge them and risk silent breakage.
>
> HUH?
>
> Say you have a function
>
> val f : ('a, 'b) t -> 'a -> 'b   (raise [Not_found])
>
> then eliminating Not_found that function would become
>
> val f : ('a, 'b) t -> 'a -> 'b option
>
> That should fail to compile if the return type is used or a signature
> exists. Where do you think a change would go unnoticed?
>
> MfG
>         Goswin
>

I really mean where we'd like to change Not_found to a more informative or
appropriate exception. Sorry, I forgot about one case where we are
completely fine with using exceptions: in (usually short-lived) programs
where the correct behaviour on error is always to explode apologetically
right there and then. The choice of exception is still relevant because it
affects the quality of the backtrace and message.

[-- Attachment #2: Type: text/html, Size: 1665 bytes --]

Re: [Caml-list] Uncaught exceptions in function type.

[Yaron Minsky]

You shouldn't take Ben's comments to indicate that we don't ever throw
exceptions.  We have lots of functions whose names are explicit about
the fact that they throw exceptions, like Map.find_exn or
List.last_exn.  They have legitimate uses, for example, in contexts
where you've checked that the precondition for that function to not
throw an exception, and so it would simply be a bug if it were thrown.
 In those cases, it is indeed hard to change the specific exception
that is thrown without potentially breaking external code that depends
on it.

That's why in new modules, are usual behavior is to not expose the
concrete exceptions we declare in the mli of our modules, on the
theory that we don't want users of the module to depend on the
identity of an exception.

y

On Tue, May 27, 2014 at 6:36 AM, Ben Millwood <bmillwood@janestreet.com> wrote:
> On 27 May 2014 11:05, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>>
>> On Tue, May 27, 2014 at 09:42:35AM +0100, Ben Millwood wrote:
>> > This is essentially why we still have a few Not_found exceptions in
>> > Core,
>> > because it's really pretty hard to know where they might be relied upon,
>> > so
>> > it's easier to leave them in than purge them and risk silent breakage.
>>
>> HUH?
>>
>> Say you have a function
>>
>> val f : ('a, 'b) t -> 'a -> 'b   (raise [Not_found])
>>
>> then eliminating Not_found that function would become
>>
>> val f : ('a, 'b) t -> 'a -> 'b option
>>
>> That should fail to compile if the return type is used or a signature
>> exists. Where do you think a change would go unnoticed?
>>
>> MfG
>>         Goswin
>
>
> I really mean where we'd like to change Not_found to a more informative or
> appropriate exception. Sorry, I forgot about one case where we are
> completely fine with using exceptions: in (usually short-lived) programs
> where the correct behaviour on error is always to explode apologetically
> right there and then. The choice of exception is still relevant because it
> affects the quality of the backtrace and message.

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le mardi, 27 mai 2014 à 08:52, Philippe Veber a écrit :

> - I observed in many libraries (including Xmlm) that people tend to define custom exceptions to signal parsing errors. According to the rules you provide, parsers should return variants, right?

Yes, I was unexeperienced.  Also I didn’t understand that the world was asynchronous at that time. See jsonm for a more sensitive interface.

> - as Ben noted earlier, I think point 2 would better be formulated as errors that can virtually appear anywhere and/or a programmer cannot do much about (as you say)
> - The adjective "exceptional" is a bit misleading IMO, in particular I guess it should not be understood as "not frequent". Rare failures can be very harmful and should be given extra-care since they are not easy to provoke and debug. Did you mean more like "very abnormal" and so difficult to recover?

Well all what you mention seems exceptional to me (at least I wouldn’t like to work with an API that allows errors to virtually happen everywhere on a regular basis)… But feel free to call that differently.  
  
> - Another criterion I like is that if checking the precondition of a function f basically amounts (in terms of complexity/algorithm) to calling f, then f shouldn't raise an exception for this precondition. If the precondition is difficult to check, a user is likely not to check it:
> - List.find assumes the searched element is in the list, but checking that is precisely what the function is about.

Yes, note that finding things in datastructures is pretty clear API wise. Usually there are two cases 1) the programmer doesn’t if the item is in the datastructure 2) the programmer knows it’s in there and doesn’t want to be annoyed. This leads to

1) val find : t -> key -> value option
2) val get : t -> key -> value (and raises Invalid_argument if the thing is not in there).  
  
> Having no experience on large projects, it is not clear to me why exceptions are considered so dangerous.  

If you don’t/forget to catch the exception at the right place it disrupts your whole call stack, which means that it breaks any invariants a correct excecution of your call stack was supposed to maintain. Which means that your program state is completely broken and you have to exit 1, hoping that you didn’t partially (invariant wise) persist anything to disk/network meanwhile.

Best,

Daniel

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le mardi, 27 mai 2014 à 10:49, Goswin von Brederlow a écrit :
> Sometimes arguments are computed and suddenly exceed aceptable
> parameters. E.g. the user specifies a data file that needs to be read
> into a string. If the file is too large you get Invalid_argument.
> Programms can catch and handle such cases, at least to the point of
> giving a good error message.


Bad api. Neither exceptional error, neither programming error, *should* not raise Invalid_argument, of course you’ll have to handle it but don't forget to send hate emails to the API designer.  

> That is a matter of taste and with recursions in the mix exceptions
> can be a great way to abort the recursion while keeping the code flow
> simple. They also allow you to use things like List.fold_left and
> abort early.

It’s not a matter of taste. What you are describing here is usage of exceptions by a programmer that wants to shoot himself in the foot. Libraries themselves should never disrupt the control flow of clients.  

Best,

Daniel

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le mardi, 27 mai 2014 à 10:42, Ben Millwood a écrit :

> but in other cases that isn't obvious (say if you mutate a Queue.t while you are folding over it, that will be detected and an error will be thrown, but it isn't /really/ an invalid argument error),  

Why not ? If you document (but yes, you have to write documentation…) that the folding function must not mutate the queue it is folding over, a function that does that is an invalid argument.  

Best,

Daniel

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le mardi, 27 mai 2014 à 11:25, Nicolas Boulay a écrit :

> > Because that would be utterly annoying. You need to make the following distinctions:
>  
> Yes it could be annoying, but very high quality software become much harder to write. Refactoring is harder. Missing exception handler are harder to find.

That’s non sense. What would the programmer do if Array.create wouldn’t raise in case of invalid argument 99.9999999% of the code would read like this:  

let a = match Array.create n with  
| None -> assert false  
| Some a -> a  
in

You solved absolutely nothing by having an option type here. But you did manage to be annoying.
  
>  
> > * Exceptional errors, for errors that the programmer is unlikely to handle at all (e.g. out of memory). For that raise a custom exception. This should occur very rarely, you are unlikely to ever define one such exception.
>  
> That means intensive testing to be sur to avoid such failure for normal user input. For most long running programme (server, gui), that's could a problem.
>  
> For example, an undo/redo use lot of memory after some copy/paste on big data, then the save command have not enough memory to work, and the file are troncated. That's not acceptable, and can be see only with big data, after few high level command run.
That is certainly not something you want to solve by handling exceptions but by making sure/designing your program is able to function under a given fixed memory budget (if only to be able to test the error condition).  

Best,

Daniel

Re: [Caml-list] Uncaught exceptions in function type.

[Florian Weimer]

* Daniel Bünzli:

> * Non-exceptional errors, errors that the programmer will have to
> handle (e.g. failing to connect a socket), for that do not use a
> custom exception but use variants or options types.

In some cases this introduces allocations even for the success case,
and exception-based design avoids that.

Re: [Caml-list] Uncaught exceptions in function type.

[Daniel Bünzli]

Le vendredi, 30 mai 2014 à 20:03, Florian Weimer a écrit :

> * Daniel Bünzli:
>  
> > * Non-exceptional errors, errors that the programmer will have to
> > handle (e.g. failing to connect a socket), for that do not use a
> > custom exception but use variants or options types.
>  
> In some cases this introduces allocations even for the success case,
> and exception-based design avoids that.

Exception handlers have their cost too.

Daniel
  

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

On Tue, May 27, 2014 at 11:39:20PM +0200, Daniel Bünzli wrote:
> Le mardi, 27 mai 2014 à 10:49, Goswin von Brederlow a écrit :
> > Sometimes arguments are computed and suddenly exceed aceptable
> > parameters. E.g. the user specifies a data file that needs to be read
> > into a string. If the file is too large you get Invalid_argument.
> > Programms can catch and handle such cases, at least to the point of
> > giving a good error message.
> 
> 
> Bad api. Neither exceptional error, neither programming error, *should* not raise Invalid_argument, of course you???ll have to handle it but don't forget to send hate emails to the API designer.  
> 
> > That is a matter of taste and with recursions in the mix exceptions
> > can be a great way to abort the recursion while keeping the code flow
> > simple. They also allow you to use things like List.fold_left and
> > abort early.
> 
> It???s not a matter of taste. What you are describing here is usage of exceptions by a programmer that wants to shoot himself in the foot. Libraries themselves should never disrupt the control flow of clients.  
> 
> Best,
> 
> Daniel

Who said anything about disrupting the codeflow of clients?

The use case is like

exception Result of int

let find_less_than x list =
  try
    List.iter (fun y -> if x > y then raise (Result y)) list;
    None
  with Result y -> Some y


# find_less_than 10 [100; 23; 3; 42; 17];;
- : int option = Some 3

You hrow the exception internally and catch it internally. Nothing of
the clients gets disruppted.

Obviously this is a constructed simplistic example. But in more
complex code an exception can be far more readable than 50 ifs and
checks of None/Some all over the place.

MfG
	Goswin

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

On Tue, May 27, 2014 at 11:16:18PM +0200, Daniel Bünzli wrote:
> Le mardi, 27 mai 2014 à 08:52, Philippe Veber a écrit :
> > Having no experience on large projects, it is not clear to me why
> > exceptions are considered so dangerous.  
> 
> If you don???t/forget to catch the exception at the right place it
> disrupts your whole call stack, which means that it breaks any
> invariants a correct excecution of your call stack was supposed to
> maintain. Which means that your program state is completely broken and
> you have to exit 1, hoping that you didn???t partially (invariant
> wise) persist anything to disk/network meanwhile.
> 
> Best,
> 
> Daniel

Which is where tracking exceptions in the type of a functions would
come in handy. If you write code that must not be aborted then you
annotate the type to throw no exceptions. The type inference would
then check that that is the case and give you  a compile error
otherwise. That way there couldn't be any surprises of an exception
breaking your call stack.

Also note, even though YOU don't use exceptions doesn't mean some
module you use doesn't. With exception part of a functions type this
would be exposed in the interface and you can deal with it accordingly
(worst case rewrite the module :).

So I think you should be all for having exceptions in the type
especially because you don't like them.

MfG
	Goswin

Re: [Caml-list] Uncaught exceptions in function type.

[Goswin von Brederlow]

> Le vendredi, 30 mai 2014 à 20:03, Florian Weimer a écrit :
> 
> > * Daniel Bünzli:
> >  
> > > * Non-exceptional errors, errors that the programmer will have to
> > > handle (e.g. failing to connect a socket), for that do not use a
> > > custom exception but use variants or options types.
> >  
> > In some cases this introduces allocations even for the success case,
> > and exception-based design avoids that.


Doesn't success with option types always introduce an allocation? Some
x is a block and wants to be created. Worse if you constantly unpack
and repack the option.