cywgin stack overflow

cywgin stack overflow

[skaller]

Hi, we've got a problem with an Ocaml native code program running under
Cygwin exiting without producing output or diagnostics. I think
Cygwin is running Ocaml 3.08. It was suggested
this was probably a stack overflow. The program is Dypgen GLR
parser generator, which is probably heavily recursive. It works
fine on my AMD64/Ubuntu Linux box, Ocaml 3.10-dev+25 or something.

Does anyone know how to increase the stack size, or have any other
idea how to work around this problem? Or any idea if it could
be any other problem?

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] cywgin stack overflow

[Igor Peshansky]

On Wed, 13 Jun 2007, skaller wrote:

> Hi, we've got a problem with an Ocaml native code program running under
> Cygwin exiting without producing output or diagnostics. I think
> Cygwin is running Ocaml 3.08. It was suggested
> this was probably a stack overflow. The program is Dypgen GLR
> parser generator, which is probably heavily recursive. It works
> fine on my AMD64/Ubuntu Linux box, Ocaml 3.10-dev+25 or something.
>
> Does anyone know how to increase the stack size, or have any other
> idea how to work around this problem? Or any idea if it could
> be any other problem?

John,

It may be running out of memory.  See if
<http://cygwin.com/cygwin-ug-net/setup-maxmem.html> helps...
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Freedom is just another word for "nothing left to lose"...  -- Janis Joplin

Re: [Caml-list] cywgin stack overflow

[skaller]

On Wed, 2007-06-13 at 07:29 -0400, Igor Peshansky wrote:

> It may be running out of memory.  See if
> <http://cygwin.com/cygwin-ug-net/setup-maxmem.html> helps...
> 	Igor

Ok, thanks for info!

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] cywgin stack overflow

[skaller]

On Wed, 2007-06-13 at 17:34 +1000, skaller wrote:
> Hi, we've got a problem with an Ocaml native code program running under
> Cygwin exiting without producing output or diagnostics. I think
> Cygwin is running Ocaml 3.08. It was suggested
> this was probably a stack overflow. The program is Dypgen GLR
> parser generator, which is probably heavily recursive. It works
> fine on my AMD64/Ubuntu Linux box, Ocaml 3.10-dev+25 or something.
> 
> Does anyone know how to increase the stack size, or have any other
> idea how to work around this problem? Or any idea if it could
> be any other problem?

Woops! I misdiagnosed the problem I think. I put -ccopts and
dypgen runs fine .. however Ocamlopt.opt does not compile it
the generated file ;(

The file src/flx_parser.ml is generated by Dypgen, but ocamlopt.opt
doesn't compile it, nor issue any diagnostic (not even returning
with an error code). It just completes without generating a
src/flx_parse.cmx file.

Unfortunately, recompiling Ocaml on Cygwin isn't an option.
Is there someone here maintaining the Cygwin port? 
Any chance of 3.09 series Ocaml, and, with the native
code compiler built with a larger stack?

I've yet to try running with larger memory for Cygwin as
suggested by Igor -- the ml file is quite large:

 20555  92221 804093 src/flx_parse.ml

src/flx_parse.cmx on amd64/Ubuntu is 83569 bytes.

I also suppose I can try ocamlopt (sans .opt) or even
ocamlc (ouch!).


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] cywgin stack overflow

[Igor Peshansky]

On Wed, 13 Jun 2007, skaller wrote:

> On Wed, 2007-06-13 at 17:34 +1000, skaller wrote:
> > Hi, we've got a problem with an Ocaml native code program running under
> > Cygwin exiting without producing output or diagnostics. I think
> > Cygwin is running Ocaml 3.08. It was suggested
> > this was probably a stack overflow. The program is Dypgen GLR
> > parser generator, which is probably heavily recursive. It works
> > fine on my AMD64/Ubuntu Linux box, Ocaml 3.10-dev+25 or something.
> >
> > Does anyone know how to increase the stack size, or have any other
> > idea how to work around this problem? Or any idea if it could
> > be any other problem?
>
> Woops! I misdiagnosed the problem I think. I put -ccopts and
> dypgen runs fine .. however Ocamlopt.opt does not compile it
> the generated file ;(
>
> The file src/flx_parser.ml is generated by Dypgen, but ocamlopt.opt
> doesn't compile it, nor issue any diagnostic (not even returning
> with an error code). It just completes without generating a
> src/flx_parse.cmx file.
>
> Unfortunately, recompiling Ocaml on Cygwin isn't an option.
> Is there someone here maintaining the Cygwin port?

Yes.  I am.  But I haven't yet had a chance to build anything newer than
3.08.

> Any chance of 3.09 series Ocaml, and, with the native
> code compiler built with a larger stack?

I thought Ocaml had growable stacks...  I must've been confused.

Besides, I'm planning to build a 3.10 series shortly -- is there really a
pressing need for 3.09?

> I've yet to try running with larger memory for Cygwin as
> suggested by Igor -- the ml file is quite large:
>
>  20555  92221 804093 src/flx_parse.ml
>
> src/flx_parse.cmx on amd64/Ubuntu is 83569 bytes.
>
> I also suppose I can try ocamlopt (sans .opt) or even
> ocamlc (ouch!).

Do try to increase the available memory -- the amount Cygwin allocates by
default is rather small (256M, I believe).  Another possibility is to run
the compilation under strace to see why it exits.
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Freedom is just another word for "nothing left to lose"...  -- Janis Joplin

Re: [Caml-list] cywgin stack overflow

[skaller]

On Wed, 2007-06-13 at 09:44 -0400, Igor Peshansky wrote:

> > Unfortunately, recompiling Ocaml on Cygwin isn't an option.
> > Is there someone here maintaining the Cygwin port?
> 
> Yes.  I am.  

Ah, good! Thanks!

> > Any chance of 3.09 series Ocaml, and, with the native
> > code compiler built with a larger stack?

> I thought Ocaml had growable stacks...  I must've been confused.

I have no idea how it works.

> Besides, I'm planning to build a 3.10 series shortly -- is there really a
> pressing need for 3.09?

I think my project should build with 3.08 up, so no 
pressing need for 3.09 here.


> Do try to increase the available memory -- the amount Cygwin allocates by
> default is rather small (256M, I believe).  Another possibility is to run
> the compilation under strace to see why it exits.

Yes, I can try that .. a bit messy (dual booting, I lose all
the email with nice info like this when running Cygwin).


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] cywgin stack overflow

[skaller]

On Thu, 2007-06-14 at 00:00 +1000, skaller wrote:
> On Wed, 2007-06-13 at 09:44 -0400, Igor Peshansky wrote:
[]
> > Do try to increase the available memory -- the amount Cygwin allocates by
> > default is rather small (256M, I believe).  Another possibility is to run
> > the compilation under strace to see why it exits.
> 
> Yes, I can try that .. a bit messy (dual booting, I lose all
> the email with nice info like this when running Cygwin).
> 

OK, the full story is:

ocamlopt.opt: fails without message
ocamlopt: fails with Stack overflow message
ocamlc: compiles the file to *.cmo (correctly, I assume)

this is with Cygwin set to 1024M memory using regtool.
I didn't try ocamlc.opt.



-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] cywgin stack overflow

[Igor Peshansky]

On Thu, 14 Jun 2007, skaller wrote:

> On Thu, 2007-06-14 at 00:00 +1000, skaller wrote:
> > On Wed, 2007-06-13 at 09:44 -0400, Igor Peshansky wrote:
> []
> > > Do try to increase the available memory -- the amount Cygwin
> > > allocates by default is rather small (256M, I believe).  Another
> > > possibility is to run the compilation under strace to see why it
> > > exits.
> >
> > Yes, I can try that .. a bit messy (dual booting, I lose all
> > the email with nice info like this when running Cygwin).
>
> OK, the full story is:
>
> ocamlopt.opt: fails without message
> ocamlopt: fails with Stack overflow message
> ocamlc: compiles the file to *.cmo (correctly, I assume)
>
> this is with Cygwin set to 1024M memory using regtool.
> I didn't try ocamlc.opt.

Ouch.  Is there a place I can find information on how stacks are
implemented in Ocaml without having to wade through source?
Particularly, I imagine the options to increase stack size of the
executable compiled with ocamlopt would not be the same as for gcc...
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Freedom is just another word for "nothing left to lose"...  -- Janis Joplin

Re: [Caml-list] cywgin stack overflow

[Xavier Leroy]

> Ouch.  Is there a place I can find information on how stacks are
> implemented in Ocaml without having to wade through source?
> Particularly, I imagine the options to increase stack size of the
> executable compiled with ocamlopt would not be the same as for gcc...

Actually, they are the same.  Let me try to clarify:

For bytecode executables produced by ocamlc or ocamlc.opt
(this includes ocamlc, ocamlopt, etc):
the executables do not not contain any limit on the stack size.  The
bytecode interpreter will resize its stack (allocated with malloc())
as needed, up to a limit that can be set using the 'l' parameter in
the OCAMLRUNPARAM environment variable.

For native-code executables produced by ocamlopt or ocamlopt.opt
(this includes ocamlc.opt, ocamlopt.opt, etc):
these are produced by calling the C compiler with a bunch of .o and .a
files, some generated from Caml via the assembler, others (the
run-time system) being compiled from C.  So, if your C compiler has
flags to set a stack limit in the generated executable, just give
these flags to ocamlopt in a -ccopt option.  (See Matthieu Wipliez's
reply on this list.)

Actually, it might be sufficient to build the Cygwin OCaml package
by configuring with
   ./configure -cc "gcc -Wl,--stack -Wl,10485760"
so that these options will be added automatically, both when building
ocamlrun and when ocamlopt calls gcc to produce native executables.

Hope this helps,

- Xavier Leroy

Re: [Caml-list] cywgin stack overflow

[Alain Frisch]

skaller wrote:
> ocamlopt.opt: fails without message
> ocamlopt: fails with Stack overflow message

FYI, OCaml does not implement stack detection under Cygwin in native
code (contrary to MSVC and Mingw ports), so the difference of behavior
between ocamlopt and ocamlopt.opt is not suprising.  (I guess you get
the equivalent of a segfault with ocamlopt.opt, but maybe nothing is
printed.)

-- Alain

Re: [Caml-list] cywgin stack overflow

[Igor Peshansky]

On Wed, 13 Jun 2007, Alain Frisch wrote:

> skaller wrote:
> > ocamlopt.opt: fails without message
> > ocamlopt: fails with Stack overflow message
>
> FYI, OCaml does not implement stack detection under Cygwin in native
> code (contrary to MSVC and Mingw ports), so the difference of behavior
> between ocamlopt and ocamlopt.opt is not suprising.  (I guess you get
> the equivalent of a segfault with ocamlopt.opt, but maybe nothing is
> printed.)

In that case there should be a ocamlopt.opt.exe.stackdump somewhere...

On a different note, why doesn't it?  Is there a special Cygwin flag that
disables that code?  Is the mechanism any different than on Linux?
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Freedom is just another word for "nothing left to lose"...  -- Janis Joplin

Re: [Caml-list] cywgin stack overflow

[Alain Frisch]

Igor Peshansky wrote:
> On Wed, 13 Jun 2007, Alain Frisch wrote:
>> FYI, OCaml does not implement stack detection under Cygwin in native
>> code (contrary to MSVC and Mingw ports), so the difference of behavior
>> between ocamlopt and ocamlopt.opt is not suprising.  (I guess you get
>> the equivalent of a segfault with ocamlopt.opt, but maybe nothing is
>> printed.)
> 
> In that case there should be a ocamlopt.opt.exe.stackdump somewhere...
> 
> On a different note, why doesn't it?  Is there a special Cygwin flag that
> disables that code?  Is the mechanism any different than on Linux?

The technique to catch stack overflow is specific to the system. For
Win32, it is implemented in byterun/win32.c (for unices, stack detection
is implemented in asmrun/signals_asm.c). As far as I understand, win32.c
is not used by the Cygwin port (which is considered as a Unix for the
build process). I don't know whether the stack detection code in win32.c
would work for Cygwin, but I don't see any reason why I wouldn't.

-- Alain

Re: [Caml-list] cywgin stack overflow

[Olivier Andrieu]

On 6/13/07, Alain Frisch <Alain.Frisch@inria.fr> wrote:
> Igor Peshansky wrote:
> > On Wed, 13 Jun 2007, Alain Frisch wrote:
> >> FYI, OCaml does not implement stack detection under Cygwin in native
> >> code (contrary to MSVC and Mingw ports), so the difference of behavior
> >> between ocamlopt and ocamlopt.opt is not suprising.  (I guess you get
> >> the equivalent of a segfault with ocamlopt.opt, but maybe nothing is
> >> printed.)
> >
> > In that case there should be a ocamlopt.opt.exe.stackdump somewhere...
> >
> > On a different note, why doesn't it?  Is there a special Cygwin flag that
> > disables that code?  Is the mechanism any different than on Linux?
>
> The technique to catch stack overflow is specific to the system. For
> Win32, it is implemented in byterun/win32.c (for unices, stack detection
> is implemented in asmrun/signals_asm.c). As far as I understand, win32.c
> is not used by the Cygwin port (which is considered as a Unix for the
> build process).

Indeed.

> I don't know whether the stack detection code in win32.c
> would work for Cygwin, but I don't see any reason why I wouldn't.

actually it doesn't work because Cygwin installs its own top-level
exception handler and it does mysterious things. You'd have to work
around it so that the ocaml one can do its job.

-- 
  Olivier