Building with OCamlMkLib

Building with OCamlMkLib

[John Whitington]

Hi Folks,

I'm building a Plain C interface to our PDF libraries, but am stuck.  
The idea is to build a library with Ocamlmklib containing the C  
wrapper around the ocaml code.

I've used some test files (included below) in place of the real ones.

I can successfully build the library (I'm using OS x / intel):

ocamlc -c -cc "cc" -ccopt " -DNATIVE_CODE -o cpdflibwrapper.o"  
cpdflibwrapper.c
ocamlc cpdflibc.mli
ocamlc cpdflibc.ml
ocamlmklib -o camlpdfc cpdflibc.ml cpdflibwrapper.a

(builds dllcamlpdfc.so, camlpdfc.a, camlpdfc.cma, camlpdfc.cmxa)

But trying to link a C program which uses the new library fails. Do I  
need to include something else, or have I got the ocamlmklib stage  
wrong?

feast:trunk john$ gcc test.c -o test_executable -L. -L/usr/local/lib/ 
ocaml -lcamlpdfc -lasmrun
Undefined symbols:
   "_caml_code_area_end", referenced from:
       _caml_code_area_end$non_lazy_ptr in libasmrun.a(signals_asm.o)
       _caml_code_area_end$non_lazy_ptr in libasmrun.a(intern.o)
       _caml_code_area_end$non_lazy_ptr in libasmrun.a(extern.o)
   "_caml_program", referenced from:
       _caml_start_program in libasmrun.a(i386.o)
   "_caml_bucket_Out_of_memory", referenced from:
       _caml_bucket_Out_of_memory$non_lazy_ptr in libasmrun.a(fail.o)

(many more lines)

Here are the input files forming the mixed ocaml/c library:

(* cpdflibc.mli *)
val twice : int -> int

(* cpdflibc.ml *)
let twice x = x * 2

/* cpdflibwrapper.h */
int twice (int);

/* cpdflibwrapper.c */
#include <stdio.h>
#include <caml/memory.h>
#include <caml/callback.h>

int twice_c (int i)
{
   CAMLlocal1 (result);
   result = caml_callback(*caml_named_value("twice"), Val_int(i));
   return(0);
}

And here's the C program which will be using the libary:

/* test.c */
#include <stdio.h>
#include "cpdflibwrapper.h"

int main ()
{
   printf("Twice two is %i\n", twice_c(2));
   return(0);
}

Here's a zip containing those files: http://www.coherentpdf.com/files.zip

Any ideas?

-- 
John Whitington
Coherent Graphics Ltd
http://www.coherentpdf.com/

Re: [Caml-list] Building with OCamlMkLib

[Alain Frisch]

John Whitington wrote:
> Hi Folks,
> 
> I'm building a Plain C interface to our PDF libraries, but am stuck. The 
> idea is to build a library with Ocamlmklib containing the C wrapper 
> around the ocaml code.
> 
> I've used some test files (included below) in place of the real ones.
> 
> I can successfully build the library (I'm using OS x / intel):
> 
> ocamlc -c -cc "cc" -ccopt " -DNATIVE_CODE -o cpdflibwrapper.o" 
> cpdflibwrapper.c
> ocamlc cpdflibc.mli
> ocamlc cpdflibc.ml
> ocamlmklib -o camlpdfc cpdflibc.ml cpdflibwrapper.a
> 
> (builds dllcamlpdfc.so, camlpdfc.a, camlpdfc.cma, camlpdfc.cmxa)
> 
> But trying to link a C program which uses the new library fails. Do I 
> need to include something else, or have I got the ocamlmklib stage wrong?

You need to include an OCaml "main program" into your library. The 
easiest way to build a library that include the OCaml runtime, some 
OCaml code and custom C code is the -output-obj option:

ocamlc -output-obj -o camlpdfc.so cpdlibc.cmo cpdflibwrapper.o

Note that cpdflibrwrapper should define a function that calls 
caml_startup in order to start the OCaml runtime and run the OCaml code.

If you want to build a static library, you can use "-output-obj -o 
camlpdf.o" and then create the library yourself.

-- Alain

Re: [Caml-list] Building with OCamlMkLib

[Matthieu Dubuget]

[-- Attachment #1: Type: text/plain, Size: 38 bytes --]

My last patch was not usable…
Sorry

[-- Attachment #2: files-patch1 --]
[-- Type: text/plain, Size: 5578 bytes --]

diff -urN ./files/camlpdfc.mlcdll ./files-modified/camlpdfc.mlcdll
--- ./files/camlpdfc.mlcdll	1970-01-01 01:00:00.000000000 +0100
+++ ./files-modified/camlpdfc.mlcdll	2009-01-20 22:16:52.000000000 +0100
@@ -0,0 +1,2 @@
+Cpdflibc
+cpdflibwrapper.o
diff -urN ./files/cpdflibc.ml ./files-modified/cpdflibc.ml
--- ./files/cpdflibc.ml	2009-01-20 18:00:51.000000000 +0100
+++ ./files-modified/cpdflibc.ml	2009-01-20 22:23:27.000000000 +0100
@@ -1,2 +1,4 @@
 (* cpdflibc.ml *)
 let twice x = x * 2
+
+let _ = Callback.register "twice" twice
diff -urN ./files/cpdflibwrapper.c ./files-modified/cpdflibwrapper.c
--- ./files/cpdflibwrapper.c	2009-01-20 18:00:51.000000000 +0100
+++ ./files-modified/cpdflibwrapper.c	2009-01-20 22:18:58.000000000 +0100
@@ -2,10 +2,33 @@
 #include <stdio.h>
 #include <caml/memory.h>
 #include <caml/callback.h>
+#include <assert.h>
 
+#define CAMLPDFLIB_CAMLCBK_INIT( callback, cbk_name) \
+        static value *callback = NULL; \
+        if (callback == NULL) callback = \
+              caml_named_value(cbk_name);\
+          assert(callback);
+
+static int cpdflibc_init_done = 0;
+
+void
+init_cpdflib ()
+{
+  char *vide[1];
+  vide[0] = NULL;
+  if (!cpdflibc_init_done)
+    {
+      caml_startup (vide);
+      cpdflibc_init_done = 1;
+    }
+}
+ 
 int twice_c (int i)
 {
-  CAMLlocal1 (result);
-  result = caml_callback(*caml_named_value("twice"), Val_int(i));
-  return(0);
+  CAMLparam0();
+  CAMLlocal1(res);
+  CAMLPDFLIB_CAMLCBK_INIT(cbk, "twice");
+  res = caml_callback(*cbk, Val_int(i));
+  CAMLreturnT(int, Int_val(res));
 }
diff -urN ./files/cpdflibwrapper.h ./files-modified/cpdflibwrapper.h
--- ./files/cpdflibwrapper.h	2009-01-20 18:00:51.000000000 +0100
+++ ./files-modified/cpdflibwrapper.h	2009-01-20 22:22:45.000000000 +0100
@@ -1,3 +1,4 @@
 /* cpdflibwrapper.h */
-int twice (int);
+void init_cpdflib();
+int twice_c (int);
 
diff -urN ./files/myocamlbuild.ml ./files-modified/myocamlbuild.ml
--- ./files/myocamlbuild.ml	1970-01-01 01:00:00.000000000 +0100
+++ ./files-modified/myocamlbuild.ml	2009-01-20 22:19:59.000000000 +0100
@@ -0,0 +1,103 @@
+open Ocamlbuild_plugin
+open Command
+open Outcome
+
+let (ext_o, ext_so, syst) =
+  let my_ocamlc_config =
+    let rec sc s h = 
+      Scanf.sscanf s "%s@: %s@\n%n" 
+        begin fun k v n -> 
+          let h' = (k, v) :: h in
+          let len = String.length s in  
+            if 
+              len - n <= 0 
+            then 
+              h'
+            else 
+              sc (String.sub s n (len - n)) h'
+        end in
+      sc (Ocamlbuild_pack.My_unix.run_and_read "ocamlc -config") [] in
+    match 
+      List.map 
+        (fun x -> List.assoc x my_ocamlc_config)
+        ["ext_obj"; "ext_dll"; "system";] 
+    with
+        [ext_o; ext_so; syst] -> ext_o, ext_so, syst
+      | _ -> assert false
+
+exception Found of string
+
+let pwd () =
+  let env = Array.to_list (Unix.environment ()) in
+  let rec search = function
+      [] -> ""
+    | h :: tl ->
+        try 
+          Scanf.sscanf h "PWD=%s" (fun x -> x) 
+        with 
+            _ -> search tl in
+    search env
+    
+let uncap_module_path p =
+  (Pathname.dirname p) / (String.uncapitalize (Pathname.basename p))
+
+let _ = dispatch begin function
+
+  | After_rules ->
+	  	  
+	  flag ["link";"mlcdll"] (S[A"-output-obj"]);
+	  	  
+      rule "Mixed C-Ocaml native DLL: mlcdll & o* & cmx* -> native DLL (.dll | .so)"
+        ~dep:"%.mlcdll"
+        ~prod:("%.native" ^ ext_so)
+        begin
+          fun env build ->
+            let output = env ("%.native" ^ ext_so)
+            and input = env "%.mlcdll" in
+            let dir = Pathname.dirname input in
+			  
+            (* TODO: use functions of Pathname module? *)
+            let ext_o_files, moduls_files = 
+              string_list_of_file input |>
+                  List.partition (fun fic -> Filename.check_suffix fic ".o") in
+            let objs = ext_o_files |>
+                List.map Filename.chop_extension |>
+                    List.map (fun fic -> fic ^ ext_o) in
+            let cmxs = moduls_files |>
+                List.map (fun modul ->
+                            (uncap_module_path modul) -.- "cmx") in
+            let deps = cmxs @ objs in
+			  
+              List.iter ignore_good 
+                (build (List.map (fun x -> [dir/x]) deps));
+			  
+              Cmd (S [!Options.ocamlopt;
+                      A"-o"; Px output;
+                      T (tags_of_pathname output++"ocaml"++"native"++"mlcdll"++"link");
+                      atomize_paths deps
+                     ])
+        end;
+	  
+      (* Allows to have .h copied in the _build directory *)
+      dep ["file:cpdflibwrapper.c"] ["cpdflibwrapper.h"];
+	  dep ["file:test.c"]["cpdflibwrapper.h"];
+	  
+      rule "Compile test.native"
+        ~deps:["test.o"; "camlpdfc.native.so"]
+        ~prod:"test.native"
+        begin fun _ _ ->
+          let target = "test.native" in
+          let spc = "-Wl,-rpath=" ^ ((pwd ()) / (!Options.build_dir))  in
+            Cmd(
+              S[
+                A"gcc";
+                A"-o";A target;
+                A"test.o";
+                A"-l:camlpdfc.native.so";
+                A"-L."; A spc;
+              ])
+        end;
+
+
+  | _ -> ()
+end
diff -urN ./files/test.c ./files-modified/test.c
--- ./files/test.c	2009-01-20 18:00:51.000000000 +0100
+++ ./files-modified/test.c	2009-01-20 22:20:29.000000000 +0100
@@ -4,6 +4,7 @@
 
 int main ()
 {
+  init_cpdflib();
   printf("Twice two is %i\n", twice_c(2));
   return(0);
 }

Re: [Caml-list] Building with OCamlMkLib

[Stefano Zacchiroli]

On Tue, Jan 20, 2009 at 05:03:55PM +0000, John Whitington wrote:
> I'm building a Plain C interface to our PDF libraries, but am
> stuck. The idea is to build a library with Ocamlmklib containing the
> C wrapper around the ocaml code.

I've been working on a similar task these days.

As Alain pointed out you were missing the -output-obj step. The actual
rules I'm using to build the (static) library which is my current
output are something like the following (Makefile excerpt):

  libcudf.a: cudf-caml.o cudf.o
	  cp $(OCAML_LIBDIR)/libcamlrun.a $@
	  ar r $@ $^

  cudf-caml.o: ../_build/cudf.cma ../_build/cudf_c.cmo
	  ocamlfind ocamlc -linkpkg -output-obj -o $@ $^

where the stuff coming from _build/ is the output of the OCaml-only
part of the library, and libcudf.a is the final C library to be
delivered to C programmers.

The real code from which this comes from is available at [1].

Note that the C programmers will then need to link in a peculiar way,
here are my current instructions for them:

  cc -o test test.c `pkg-config --cflags cudf` `pkg-config --cflags cudf`

Which expands to something schematic like:

  cc -o test test.c <INCLUDE PATHS> <LIB PATHS> -lcudf -lm -ldl -lunix -lncurses <OTHER -l FOR EXTERNAL LIBS>

The glitch here is that test.c, which contains the invocation to
caml_startup, must appear *before* -lcamlrun, as AFAICT it needs to
override the main() defined there. If you try to put it at the end of
the linking line, linking will fail.

Cheers.

[1] http://gforge.info.ucl.ac.be/plugins/scmsvn/viewcvs.php/trunk/updb/libcudf/c-lib/?root=mancoosi

-- 
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

Re: Building with OCamlMkLib

[John Whitington]

Hi.

On 20 Jan 2009, at 17:03, John Whitington wrote:
> I'm building a Plain C interface to our PDF libraries, but am stuck.  
> The idea is to build a library with Ocamlmklib containing the C  
> wrapper around the ocaml code.

Thanks to those who helped me here - I finally settled on a modified  
version of what's given in 18.8 in the manual.

Here are the new files:

http://www.coherentpdf.com/solution.tar.bz2

And here's what we ended up with...

ocamlc cpdflibc.mli
ocamlopt cpdflibc.ml
ocamlc cpdflibwrapper.c
ocamlopt -output-obj -o cpdflibc.o cpdflibc.cmx
cp /usr/local/lib/ocaml/libasmrun.a cpdflib.a
ar r cpdflib.a cpdflibc.o cpdflibwrapper.o

...producing the final cpdflib.a with which we can do...

cc -o test test.c cpdflib.a

...to use the library.

With Thanks,

-- 
John Whitington
Coherent Graphics Ltd
http://www.coherentpdf.com/