[Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Tomasz Zielonka]

Hi

Situation: 

I'm doing some integration of OCaml and PostgreSQL. PostgreSQL loads a
DSO which implements so called "procedural language call handler". 

I've built this DSO by ld-shared'ing libcamlrun.a with some additional C
.o files, .o with wrappers for callback'ed ocaml funs, .o with new
primitives and .o with ocaml bytecode. I also removed main.o from
libcamlrun.

I linked bytecode files into .o with following command:

  ocamlc -g -custom -output-obj -o plocamlb_bytecode.o plocamlb_spi.cmo \
  plocamlb_mlfuns.cmo

Problem: Everything worked fine until I wanted to define a class.

When I define a class with _no methods_, it works, but there is little
use of such class.

When I define any method in this class (but still don't use a single
instance of the class) it stops working. After loading the DSO, ocaml
raises Not_found exception and quits.

I wanted to use CAMLRUNPARAM b (stack backtrace) option, but apparently
it doesn't work with such a library/DSO (Program not linked with -g,
cannot print stack backtrace).

I couldn't force ocamlc to keep DBUG section. I'm not sure it would
help, because read_debug_info tries to open executable file, not the
loaded DSO. I tried several caml_main_param[0] (different files), no
luck.

When I make native/opt library _and_ define a method, I get corrupted
memory in the postgresql backend.

So:
1) There is a bug, but where? In my program, in OCaml?
2) How to get stack backtrace in this situation?

thank you in advance
tom

-- 
   .-.   Tomasz Zielonka                           CYBER SERVICE
   oo|   programista                        http://www.cs.net.pl
  /`'\   zielony@cs.net.pl
 (\_;/)  tel: [48] (22) 723-06-79 | tel/fax: [48] (22) 723-01-75
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Tomasz Zielonka]

On Sun, Jan 27, 2002 at 02:35:19AM +0100, Tomasz Zielonka wrote:
> I linked bytecode files into .o with following command:
> 
>   ocamlc -g -custom -output-obj -o plocamlb_bytecode.o plocamlb_spi.cmo \
>   plocamlb_mlfuns.cmo

I forgot to write:

System is Linux 2.4.17 with gcc 2.95.3, glibc 2.2.3 (slackware 8.0)
Objective Caml version: 3.04

tom

-- 
   .-.   Tomasz Zielonka                           CYBER SERVICE
   oo|   programista                        http://www.cs.net.pl
  /`'\   zielony@cs.net.pl
 (\_;/)  tel: [48] (22) 723-06-79 | tel/fax: [48] (22) 723-01-75
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Gerd Stolpmann]

On 2002.01.27 02:35 Tomasz Zielonka wrote:
> Hi
> 
> Situation: 
> 
> I'm doing some integration of OCaml and PostgreSQL. PostgreSQL loads a
> DSO which implements so called "procedural language call handler". 
> 
> I've built this DSO by ld-shared'ing libcamlrun.a with some additional C
> .o files, .o with wrappers for callback'ed ocaml funs, .o with new
> primitives and .o with ocaml bytecode. I also removed main.o from
> libcamlrun.
> 
> I linked bytecode files into .o with following command:
> 
>   ocamlc -g -custom -output-obj -o plocamlb_bytecode.o plocamlb_spi.cmo \
>   plocamlb_mlfuns.cmo
> 
> Problem: Everything worked fine until I wanted to define a class.

I did this kind of linking, too, but I did not have any problems. (I am
not loading into Postgresql, but into a Perl interpreter, but I think
this does not make any difference.) I have used -output-obj for a long
time, but recently (with the availability of O'Caml 3.04) I switched
to loading the bytecode directly. The debug sections are not lost, 
and the Dynlink library works.

I can quickly describe what I am doing:

- build libocamlrun.so by compiling the code in the "byterun"
  directory with -fPIC, and linking everything with ld -shared
  (but not main.o)

- build some special libraries as DSOs. With O'Caml 3.04, you
  can load them dynamically. 

- build stubs.so. This is the entry point, i.e. the Perl interpreter
  does ldopen("stubs.so"), and it depends on some libraries:
  libocamlrun.so, and the libraries of the embedding application
  so their symbols are visible here.

  The entry point creates the argv array, and puts into argv[0] the
  absolute path of the bytecode file. (argv[1] is NULL.) Finally,
  I am calling caml_main(argv). This will load the bytecode file,
  and start executing it.

  In O'Caml I only initialize everything, and register a lot of
  callbacks, so that caml_main will quickly return to its caller.

- Compiling the ocaml part is tricky:
  * Create a bytecode executable
  * Name all required libraries
  ==> ocamlc -o bytecode lib1.cma lib2.cma ... main.cma

  This will fail, because ocamlc looks up all primitives of the
  libraries, but the symbols of the embedding application are missing!
  (IMHO, there should be a command-line switch for ocamlc turning
  this off.) As a workaround, I create a fake library libfake.so
  containing all missing symbols, and I call ocamlc this way:
  LD_PRELOAD=libfake.so ocamlc ...

> When I define a class with _no methods_, it works, but there is little
> use of such class.
> 
> When I define any method in this class (but still don't use a single
> instance of the class) it stops working. After loading the DSO, ocaml
> raises Not_found exception and quits.
> 
> I wanted to use CAMLRUNPARAM b (stack backtrace) option, but apparently
> it doesn't work with such a library/DSO (Program not linked with -g,
> cannot print stack backtrace).
> 
> I couldn't force ocamlc to keep DBUG section. I'm not sure it would
> help, because read_debug_info tries to open executable file, not the
> loaded DSO. I tried several caml_main_param[0] (different files), no
> luck.
> 
> When I make native/opt library _and_ define a method, I get corrupted
> memory in the postgresql backend.

The generated code is not PIC, so it won't work.
 
> So:
> 1) There is a bug, but where? In my program, in OCaml?

It is more likely that the bug is in your program.

> 2) How to get stack backtrace in this situation?

See my method. Stack backtraces work, even the debugger works if
you name the socket explicitly.

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 45             
64293 Darmstadt     EMail:   gerd@gerd-stolpmann.de
Germany                     
----------------------------------------------------------------------------
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Tomasz Zielonka]

On Sun, Jan 27, 2002 at 02:35:19AM +0100, Tomasz Zielonka wrote:
> When I define any method in this class (but still don't use a single
> instance of the class) it stops working. After loading the DSO, ocaml
> raises Not_found exception and quits.

After some thinking I came to conclusion that module approach will suit
my needs better.

However, the problem with classes remains. I'm still curious, what is
the cause.

greetings,
tom

-- 
   .-.   Tomasz Zielonka                           CYBER SERVICE
   oo|   programista                        http://www.cs.net.pl
  /`'\   zielony@cs.net.pl
 (\_;/)  tel: [48] (22) 723-06-79 | tel/fax: [48] (22) 723-01-75
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] Problem with using oo in dynamically loaded bytecode ocaml runtime

[Tomasz Zielonka]

On Sun, Jan 27, 2002 at 01:36:48PM +0100, Gerd Stolpmann wrote:
> On 2002.01.27 02:35 Tomasz Zielonka wrote:
> 
> I did this kind of linking, too, but I did not have any problems. (I am
> not loading into Postgresql, but into a Perl interpreter, but I think
> this does not make any difference.) I have used -output-obj for a long
> time, but recently (with the availability of O'Caml 3.04) I switched
> to loading the bytecode directly. The debug sections are not lost, 
> and the Dynlink library works.

[...]

Thank you very much. I will try this method.

> > When I make native/opt library _and_ define a method, I get corrupted
> > memory in the postgresql backend.
> 
> The generated code is not PIC, so it won't work.

Yes. At some point it came to me and I was puzzled: why the hell it
works (except for classes) when it shouldn't?

> > So:
> > 1) There is a bug, but where? In my program, in OCaml?
> 
> It is more likely that the bug is in your program.

Probably.

But it happens inside caml_startup(), during initialization.

> > 2) How to get stack backtrace in this situation?
> 
> See my method. Stack backtraces work, even the debugger works if
> you name the socket explicitly.

Wow, that's cool. Will try.

thank you,
tom

-- 
   .-.   Tomasz Zielonka                           CYBER SERVICE
   oo|   programista                        http://www.cs.net.pl
  /`'\   zielony@cs.net.pl
 (\_;/)  tel: [48] (22) 723-06-79 | tel/fax: [48] (22) 723-01-75
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr