Re: [Caml-list] OCaml troll on Slashdot

[brogoff <brogoff@speakeasy.net>]

On Thu, 17 Mar 2005, Jon Harrop wrote:
> On Thursday 17 March 2005 03:48, Yaron Minsky wrote:
> > On Wed, 16 Mar 2005 19:35:17 -0800 (PST), brogoff
> > <brogoff@speakeasy.net> wrote:
> > > Anyways, long lists do occur, and Stack_overflow at the customer site
> > > sucketh bigtime.
> >
> > I completely agree.  As someone who makes extensive use of OCaml in a
> > business setting, the fact that the standard libraries throw
> > exceptions on large data structures is a real pain.
>
> I believe the Stack_overflow exception is not necessarily thrown - the process
> may simply die. IIRC, I found this on x86 Debian Linux when swap was turned
> off.
>
> > It's a violation
> > of the principle of least surprise, and I've run into it in practice
> > quite a few times.
>
> I understand your (and Brian's) concern but I think the solution is to be more
> careful when programming, rather than to replace all of the library
> functions. That seems like a sledgehammer to crack a nut to me...

There are a few problems with that suggestion. The cases where I've been bitten
didn't have lists of length 100_000 (or whatever the rather arbitrary number
above which we're not supposed to use lists) but rather had quite a few lists
that were close enough to that number, and the maps (appends, etc.) were being
called inside other mapping functions, on other lists.

Also, as I state above, the number is arbitrary, and having OCaml choke at some
particular size rather than letting me use large lists violates that least
surprise principle. I had an offline discussion recently with another caml-list
person in which he told me that he wished OCaml used Bignums instead of int's
by default. I disagree, but I don't think it's a dumb idea. The behavior of
the standard List functions is worse IMO. Maybe the standard Lisp.map should be
named List.unsafe_map (1/2 :-))?

I realize that this problem can be coded around, sometimes with better data
structures, or by the double reversing approach (which is what I used to use)
but my own sense of programming language aesthetics is that this is a flaw, or
at least a hole in the language that should be filled one day.

> > We're happy to make the performance
> > tradeoff.  If we really want things to be fast, List.map is no good
> > anyway, due to the lack of inlining of the function parameter.

That's quite true. However, Yaron, I bet you usually write the program with map
first, and then optimize later iff it isn't fast enough, right? Manual inlining
makes code uglier.

> Yes, if your program uses non-tail-recursive functions with very deep
> recursion then it will be much slower anyway. Consequently, you are likely to
> fix this whilst optimising anyway.
>
> As Brian has said before, users can throw you a sideball by giving input which
> requires deep recursion by a non-tail-recursive function which had not been
> expected by the programmer. Provided you are thorough, this shouldn't happen
> though.

As I said above, you can have "not so deep recursions" nested inside each other
many times, leading to a deep recursion. Besides, "deep" is arbitrary.

> I must say that I've had fewer stack problems with my OCaml code than with
> code I've used in other languages...

Absolutely correct, and my whining should in no way be construed as saying
that OCaml sucks. However, when the language is so good, it's users begin to
desire perfection. When a language is like, say C++ or Perl, then Ada and
Python look really good.

>
> > I do think that it would be a great thing if the tail-recursive list
> > functions were moved into the standard library.
>
> I would not like to see trivial tail-recursive alternatives put into the
> library (e.g. let map f l = rev_map f (rev l)).


I agree with you here, and I'm only beating this dead dromedary to keep up the
pressure for a better solution. It's known in theory how to better than
the double rev solution, but as Jacques pointed out (and I accept his
counterexample for now!) there may be implications on GC which still give a
slight edge to the non tail recursive versions for smaller lists.

-- Brian