[Caml-list] Ocamlc stack overflow

[Caml-list] Ocamlc stack overflow

[Alex Baretta]

I am experiencing stack overflows while compiling a source file with 
ocamlc. Why in world should ocamlc overflow? How can I diagnose the problem?

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow

[Richard Jones]

On Mon, Nov 24, 2003 at 06:21:07PM +0100, Alex Baretta wrote:
> I am experiencing stack overflows while compiling a source file with 
> ocamlc. Why in world should ocamlc overflow? How can I diagnose the problem?

We had this when compiling some OLE code (auto-generated ML). The
workaround was to increase the stack size:

export OCAMLRUNPARAM=l=16M

As for diagnosing the problem: I imagine the following would work:

(1) Chop your file in half around the 50% mark. (Don't chop in the
middle of a statement, or somewhere which would cause an error).

(2) If that compiles successfully, then repeat, but chopping at the
75% mark. Otherwise if it fails, repeat chopping at the 25% mark.

(3) Continue like so doing a binary search until you find out where it
crashes.

In the OLE case it appeared to crash simply because of the size of the
file, rather than any specific OCaml statement.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
"One serious obstacle to the adoption of good programming languages is
the notion that everything has to be sacrificed for speed. In computer
languages as in life, speed kills." -- Mike Vanier

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow

[Alex Baretta]

Thanks for answering.

I finally managed to fix my mail client. I haven't been able to get 
anything from the list yesterday..

Richard Jones wrote:

> On Mon, Nov 24, 2003 at 06:21:07PM +0100, Alex Baretta wrote:
> 
>>I am experiencing stack overflows while compiling a source file with 
>>ocamlc. Why in world should ocamlc overflow? How can I diagnose the problem?
> 
> 
> We had this when compiling some OLE code (auto-generated ML). The
> workaround was to increase the stack size:

I am also trying to compile an auto-generated file, but it is rather 
small (140 lines of code). The binary search is probably not the way to 
go, but let me see what happens if I try to increase the stack size...

> export OCAMLRUNPARAM=l=16M

OK, done that...
ocamlc is presently running with 98% of the cpu and some 70 MB of memory 
footprint. It's almost certainly looping somewhere, allocating it's head 
off...

Damien Doligez wrote:
 > On Monday, November 24, 2003, at 06:21 PM, Alex Baretta wrote:
 >
 > If you are using the CVS version, you should try to update to the
 > latest one.  One version was committed last week with a stack
 > overflow problem.
 >
 > -- Damien

I am using the Xavier-patch 2 to ocaml-3.07. I'm not ehntusiastic about 
using CVS. Since my company uses ocaml as it's main language for 
developing commercial applications, I'd prefer to have a stable stock 
version. After all the turmoil around 3.07, a stable bugfix release is 
overdue now.

***

Let try to produce a backtrace for the list...

OK, so the standard ocamlc is compiled without debugging info. Let me 
tweak the Makefile, so that I can build a copy of ocamlc with the -g option.

Done that. Here's the command I'm using to run the debugger:
[alex@flyingtuxman anagrafiche]$ ocamldebug 
/opt/ocaml/3.07+2g/bin/ocamlc -I 
/opt/ocaml/3.07/lib/ocaml/site-lib/postgres -I 
/opt/ocaml/3.07/lib/ocaml/site-lib/cgi -I 
/home/alex/cvs/sw2/ocamllib-addons  -I /home/alex/cvs/sw2/unixlib-addons 
  -I /home/alex/cvs/sw2/rules-engine  -I 
/home/alex/cvs/sw2/xcaml/xcaml-lib  -I /home/alex/cvs/sw2/dbinterface 
-I /home/alex/cvs/sw2/freerp/db/db_access  -I 
/home/alex/cvs/sw2/freerp/lib  -I 
/home/alex/cvs/sw2/freerp/business_rules  -I /home/alex/cvs/sw2/dbschema 
  -I /home/alex/cvs/sw2/freerp/db/xcaml_db_schema  -I 
/home/alex/cvs/sw2/xcaml/xcaml-lib -c inserimento_enti_sql.ml
	Objective Caml Debugger version 3.07+2

(ocd) run
Loading program... done.
Time : 2766617
Program end.
Uncaught exception: Stack_overflow
(ocd) prev
Time : 2766616 - pc : 824648 - module Errors
No source file for Errors.
(ocd) bt
#0  Pc : 824648  Errors char 2807
#1  Pc : 68568  Format char 35089
#2  Pc : 830376  Main char 5649
#3  Pc : 834284  Main char 5678

************

Now what?

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow

[Richard Jones]

It's only 140 lines of code? Try chopping lines off the end of the
file until the thing compiles. Then you should be able to isolate
which statement causes the stack overflow. From there it should be a
simple enough job to either understand the problem and work around it,
or else come up with a minimal example which exhibits the bug.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
MONOLITH is an advanced framework for writing web applications in C, easier
than using Perl & Java, much faster and smaller, reusable widget-based arch,
database-backed, discussion, chat, calendaring:
http://www.annexia.org/freeware/monolith/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow (Probably a typechecking bug)

[Alex Baretta]

Richard Jones wrote:
> It's only 140 lines of code? Try chopping lines off the end of the
> file until the thing compiles. Then you should be able to isolate
> which statement causes the stack overflow. From there it should be a
> simple enough job to either understand the problem and work around it,
> or else come up with a minimal example which exhibits the bug.
> 
> Rich.
> 

I don't need to chop off lines of code. I know exactly what line of code 
causes the problem. Let me quote it to you:

 > let cont f init res execute =
 >   Local_rules.Rewrite.rewrite_continuation rule_base (cont f)

Let me try to explain the dependency relations in my libraries. You will 
see that it is impossible to come up with a "minimal example". I have a 
rule-based abstract processor (à la CLIPS) written as functor whose 
parameter module defines types "fact" and "fact_base" as well the 
operations which can be performed on them. I also have an SQL library 
which gives provides a statically typechecked abstract syntax for SQL 
and facilities to connect to databases through a doubly generic 
interface: the first parameter module provides a database connection 
layer, the second parameter (we call it the Access module) provides the 
type information to the ocaml typechecking system to achieve static 
typechecking of SQL in ocaml. This same generic library has a query 
rewriting rule engine à la PostgreSQL which defines a fact base module 
for the rule-based processor. This rewrite-rule functor takes an Access 
parameter, which must match the Access parameter passed to the DB 
functor. The actual rewrite rules are also defined in a functor which 
takes an Access module as a parameter.

Finally, I have an embedded-sql syntax extension which provides the 
syntactic sugar to make everything nice and easy in an ocaml source 
file. This syntax extension transforms SQL queries in concrete syntax 
into an ocaml module which instantiates the DB functor and the 
rewrite-rule engine with the same Access module. Since no module has an 
explicit signature hiding the type representations, I would expect the 
compiler to be able to figure things out correctly. Instead, I used to 
get error messages of the following kind.

 > File "inserimento_enti.xcaml", line 95, characters 56-65:
 > This expression has type
 >   Local_rules.Rewrite.Rules.rule_base =
 >     Local_rules.Rewrite.Rules.rule list
 >     Map.Make(Local_rules.Rewrite.Rules.Fact_class_order).t
 > but is here used with type
 >   RW.Rules.rule list RW.Rules.FCM.t =
 >     RW.Rules.rule list Map.Make(RW.Rules.Fact_class_order).t


Which points to the following line:

 > let cont f init res execute =
 >   RW.rewrite_continuation rule_base (cont f)

It is worth noting that the following module definitions imply that the 
error message is actually wrong.
 > (* in Inserimento_enti *)
 > module Local_rules = Generic_rules (Anagrafiche_logical)
 > module RW = Local_rules.Rewrite


Intuitively, I'd say the type checker is having some real trouble with 
the complex module operations I use. But, then again, there might be a 
problem in my code. So, I removed the RW definition and changed the 
troublesome line to the one originally mentioned in this post:
 > let cont f init res execute =
 >   Local_rules.Rewrite.rewrite_continuation rule_base (cont f)

Now, this really turns the typechecker nuts. It simply cannot state that 
there is a type error for the inferred type is identical to the actual 
type. Here comes the stack overflow problem.

So, dear caml riders and caml breeders, how do we get out of this 
impasse? I'm willing to submit my entire source tree. (I'd release it 
GPLed if only I had time to write a minimum of documentation and got it 
to compile properly...).

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow

[Xavier Leroy]

> Since my company uses ocaml as it's main language for 
> developing commercial applications, I'd prefer to have a stable stock 
> version. After all the turmoil around 3.07, a stable bugfix release is 
> overdue now.

It's called 3.07pl2 and has been available since Oct 20 at the usual
place (http://caml.inria.fr/).  (It is identical to 3.07 + patch 2,
for those who have applied the patch, but the tarball and binary
distributions were updated.)

> Let try to produce a backtrace for the list...
> #0  Pc : 824648  Errors char 2807
> #1  Pc : 68568  Format char 35089
> #2  Pc : 830376  Main char 5649
> #3  Pc : 834284  Main char 5678

That exception is caught and re-raised (for finalization purposes), so
you're just seeing the finalization point.  If you really want to
pinpoint the location of the error, either go back in time using
ocamldebug, or just run ocamlc outside ocamldebug with the "stack
backtrace" option set (OCAMLRUNPARAM=b).

> Now what?

What about sending some code to reproduce the problem to caml@inria.fr?
Or even better submit a bug report to caml-bugs@inria.fr with a URL to
the repro case?  (Remember: shouting "I have a bug!" on this list
accomplishes nothing beyond venting some steam.)
The repro case doesn't have to be small, all what is needed is that
it reproduces the problem.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Ocamlc stack overflow

[Damien Doligez]

On Monday, November 24, 2003, at 06:21 PM, Alex Baretta wrote:

> I am experiencing stack overflows while compiling a source file with 
> ocamlc. Why in world should ocamlc overflow? How can I diagnose the 
> problem?

If you are using the CVS version, you should try to update to the
latest one.  One version was committed last week with a stack
overflow problem.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

ocamlc Stack_overflow

[Denis Bueno]

[-- Attachment #1: Type: text/plain, Size: 680 bytes --]

All--

The attached file gives the following output (when uncompressed, of course):

,----
| $ ocamlc -g -c spit_etest.ml
| Fatal error: exception Stack_overflow
`----

I really have no idea where to start, whether it's my fault or
ocamlc's. I have included some relevant system information [1].

Could someone point my in the right direction?

-Denis

[1]
,----
| $ uname -a
| Darwin ford.local 8.7.0 Darwin Kernel Version 8.7.0: Fri May 26
15:20:53 PDT 2006; root:xnu-792.6.76.obj~1/RELEASE_PPC Power Macintosh
powerpc
`----

,----
| $ ocamlc -version
| 3.09.2
`----

The ocaml I'm using was installed by Fink; the fink version (as
reported by `fink list ocaml` is 3.09.2-1000.

[-- Attachment #2: spit_etest.ml.tar.gz --]
[-- Type: application/x-gzip, Size: 66078 bytes --]

ocamlc Stack_overflow

[Denis Bueno]

[I tried sending this a few days ago with a rather large attachment.
Since it hasn't been approved since then, I'm sending it again, sans
attachment.]

All--

The source file available on my website [2] gives the following output
(when uncompressed, of course):

,----
| $ ocamlc -g -c spit_etest.ml
| Fatal error: exception Stack_overflow
`----

I really have no idea where to start, whether it's my fault or
ocamlc's. I have included some relevant system information [1].

Could someone point my in the right direction?

-Denis

[1]
,----
| $ uname -a
| Darwin ford.local 8.7.0 Darwin Kernel Version 8.7.0: Fri May 26
15:20:53 PDT 2006; root:xnu-792.6.76.obj~1/RELEASE_PPC Power Macintosh
powerpc
`----

,----
| $ ocamlc -version
| 3.09.2
`----

The ocaml I'm using was installed by Fink; the fink version (as
reported by `fink list ocaml` is 3.09.2-1000.

[2]
http://churn.ath.cx/spit_etest.ml.tar.bz2

Re: [Caml-list] ocamlc Stack_overflow

[Jon Harrop]

On Wednesday 09 August 2006 03:00, Denis Bueno wrote:
> | $ ocamlc -g -c spit_etest.ml
> | Fatal error: exception Stack_overflow
>
> `----
>
> I really have no idea where to start, whether it's my fault or
> ocamlc's. I have included some relevant system information [1].
>
> Could someone point my in the right direction?

The OCaml compilers aren't entirely tail recursive so they sometimes overflow 
the stack on large input (especially autogenerated code). Easiest fixes:

1. Use ocamlc.opt because native code gets a bigger stack.
2. Use a bigger bytecode stack, set via the environment variable CAMLRUNPARAM.

You can also get stack overflows by compiling with -rectypes and linking 
without it. I assume the type checker goes into infinite non-tail 
recursion...

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] ocamlc Stack_overflow

[Martin Jambon]

On Tue, 8 Aug 2006, Denis Bueno wrote:

> [I tried sending this a few days ago with a rather large attachment.
> Since it hasn't been approved since then, I'm sending it again, sans
> attachment.]
>
> All--
>
> The source file available on my website [2] gives the following output
> (when uncompressed, of course):
>
> ,----
> | $ ocamlc -g -c spit_etest.ml
> | Fatal error: exception Stack_overflow
> `----
>
> I really have no idea where to start, whether it's my fault or
> ocamlc's. I have included some relevant system information [1].
>
> Could someone point my in the right direction?

You are using a huge list, and it is likely that the compiler uses some 
functions which will use stack space proportionally to the size of your list.
You can use one of the following options:
- use ocamlc.opt instead of ocamlc
- increase the maximum size of the stack:
     env OCAMLRUNPARAM=l=10M ocamlc -c spit_etest.ml
- store your data externally

Note that your program itself might have the same problem at runtime if 
you use non-tail-call optimized functions like List.map (see 
documentation for module List).


Martin

--
Martin Jambon, PhD
http://martin.jambon.free.fr

Re: [Caml-list] ocamlc Stack_overflow

[Denis Bueno]

Martin Jambon wrote:
> You are using a huge list, and it is likely that the compiler uses some 
> functions which will use stack space proportionally to the size of your 
> list.
> You can use one of the following options:
> - use ocamlc.opt instead of ocamlc
> - increase the maximum size of the stack:
>     env OCAMLRUNPARAM=l=10M ocamlc -c spit_etest.ml

To anyone who may be interested: both of these methods worked for me.

-Denis