[Caml-list] Including a C library statically in an Ocaml library

[Caml-list] Including a C library statically in an Ocaml library

[Malcolm Matalka]

The core problem I am having is a C library I want to bind has a number
of macros which I need the value of.  Here is how I am trying to solve
it, but perhaps there is a better way:

I have a small C library which gets compiled to libfoo.a which provides
functions that return the macro values, like:

int macro1() { return MACRO1; }

I then have an ocaml library, called ofoo, that uses Ctypes to bind to macro1:

let macro1 = foreign "macro" (void @-> returning int)

Where I am having issues is this small library, I'd prefer it to not
have to be installed on the system but just compiled into the Ocaml
library so that a user just has to link against that library.  Right
now, none of the symbols (macro1) are being included in the library, I'm
guessing because the linker sees no direct use of them.  And I'm not
even sure if I can get it included in the ocaml library.  I'm also not
able to get the libfoo symbols linked into a final executable, I'm
guessing for similar reasons.

What are my options here?

If I've missed any useful information, let me know.  I haven't interoped
much with C directory in Ocaml so I'm not sure what information is
important.

/Malcolm

Re: [Caml-list] Including a C library statically in an Ocaml library

[Christopher Zimmermann]

On Fri, 08 Jan 2016 14:14:22 +0000 Malcolm Matalka <mmatalka@gmail.com>
wrote:

> The core problem I am having is a C library I want to bind has a
> number of macros which I need the value of.  Here is how I am trying
> to solve it, but perhaps there is a better way:
> 
> I have a small C library which gets compiled to libfoo.a which
> provides functions that return the macro values, like:
> 
> int macro1() { return MACRO1; }
> 
> I then have an ocaml library, called ofoo, that uses Ctypes to bind
> to macro1:
> 
> let macro1 = foreign "macro" (void @-> returning int)
> 
> Where I am having issues is this small library, I'd prefer it to not
> have to be installed on the system but just compiled into the Ocaml
> library so that a user just has to link against that library.  Right
> now, none of the symbols (macro1) are being included in the library,

That's no surprise. Macros are evaluated at compile time. You won't see
their name (only their value) in the compiled object file.

> I'm guessing because the linker sees no direct use of them.  And I'm
> not even sure if I can get it included in the ocaml library.  I'm
> also not able to get the libfoo symbols linked into a final
> executable, I'm guessing for similar reasons.
> 
> What are my options here?
> 
> If I've missed any useful information, let me know.  I haven't
> interoped much with C directory in Ocaml so I'm not sure what
> information is important.

You could hardcode the value of the macros into your ocaml code like
this:

let macro1 = 4096;;

But I don't see anything wrong with your approach. If you have really
many macros and don't want to add that many functions you could create 
a static record of all the macro values in your C stub code like this 
(not tested):

value get_constants(value) {
  static value constants[5] = {
    (4 << 10), /* header for block of size 4 */
     Val_long(Macro1),
     Val_long(Macro2),
     NULL,
     Val_long(Macro4)
  };

  if (constants.3 == NULL) {
    constants.3 = caml_copy_string(Macro3);
  }

  return (constants + 1);
}


Christopher



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

Re: [Caml-list] Including a C library statically in an Ocaml library

[Malcolm Matalka]

Christopher Zimmermann <christopher@gmerlin.de> writes:

> On Fri, 08 Jan 2016 14:14:22 +0000 Malcolm Matalka <mmatalka@gmail.com>
> wrote:
>
>> The core problem I am having is a C library I want to bind has a
>> number of macros which I need the value of.  Here is how I am trying
>> to solve it, but perhaps there is a better way:
>> 
>> I have a small C library which gets compiled to libfoo.a which
>> provides functions that return the macro values, like:
>> 
>> int macro1() { return MACRO1; }
>> 
>> I then have an ocaml library, called ofoo, that uses Ctypes to bind
>> to macro1:
>> 
>> let macro1 = foreign "macro" (void @-> returning int)
>> 
>> Where I am having issues is this small library, I'd prefer it to not
>> have to be installed on the system but just compiled into the Ocaml
>> library so that a user just has to link against that library.  Right
>> now, none of the symbols (macro1) are being included in the library,
>
> That's no surprise. Macros are evaluated at compile time. You won't see
> their name (only their value) in the compiled object file.

This is why I'm creating a function that returns the macro value.

>
>> I'm guessing because the linker sees no direct use of them.  And I'm
>> not even sure if I can get it included in the ocaml library.  I'm
>> also not able to get the libfoo symbols linked into a final
>> executable, I'm guessing for similar reasons.
>> 
>> What are my options here?
>> 
>> If I've missed any useful information, let me know.  I haven't
>> interoped much with C directory in Ocaml so I'm not sure what
>> information is important.
>
> You could hardcode the value of the macros into your ocaml code like
> this:
>
> let macro1 = 4096;;
>
> But I don't see anything wrong with your approach. If you have really
> many macros and don't want to add that many functions you could create 
> a static record of all the macro values in your C stub code like this 
> (not tested):
>
> value get_constants(value) {
>   static value constants[5] = {
>     (4 << 10), /* header for block of size 4 */
>      Val_long(Macro1),
>      Val_long(Macro2),
>      NULL,
>      Val_long(Macro4)
>   };
>
>   if (constants.3 == NULL) {
>     constants.3 = caml_copy_string(Macro3);
>   }
>
>   return (constants + 1);
> }
>
>
> Christopher

Re: [Caml-list] Including a C library statically in an Ocaml library

[David Sheets]

On Fri, Jan 8, 2016 at 2:14 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
> The core problem I am having is a C library I want to bind has a number
> of macros which I need the value of.  Here is how I am trying to solve
> it, but perhaps there is a better way:
>
> I have a small C library which gets compiled to libfoo.a which provides
> functions that return the macro values, like:
>
> int macro1() { return MACRO1; }
>
> I then have an ocaml library, called ofoo, that uses Ctypes to bind to macro1:
>
> let macro1 = foreign "macro" (void @-> returning int)
>
> Where I am having issues is this small library, I'd prefer it to not
> have to be installed on the system but just compiled into the Ocaml
> library so that a user just has to link against that library.  Right
> now, none of the symbols (macro1) are being included in the library, I'm
> guessing because the linker sees no direct use of them.  And I'm not
> even sure if I can get it included in the ocaml library.  I'm also not
> able to get the libfoo symbols linked into a final executable, I'm
> guessing for similar reasons.
>
> What are my options here?

I'd recommend using ctypes 0.4+ stub generation support which can bind
macro values and detect struct layout at an early compilation stage.
You can see it in use to do this in my ocaml-unix-errno library
<https://github.com/dsheets/ocaml-unix-errno>. One benefit with this
approach is greater static checking and c <-> ocaml type safety.

If you continue using dynamic bindings, two lonker flags may be of use to you:

--no-as-needed : on recent Ubuntu distributions, gcc automatically
includes the --as-needed flag which drops symbols that are not
referenced. Unfortunately, clang does not understand this flag so you
need to have a conditional in your build system to detect the compiler
in use if you want a cross-platform library.

-E : The Exports local symbols which are statically linked into a
binary into the dynamic symbol table so that they can be found with
dlsym.

You can use these with gcc like -Wl,-E for example.

Hope this helps,

David Sheets


> If I've missed any useful information, let me know.  I haven't interoped
> much with C directory in Ocaml so I'm not sure what information is
> important.
>
> /Malcolm
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Including a C library statically in an Ocaml library

[Malcolm Matalka]

David Sheets <sheets@alum.mit.edu> writes:

> On Fri, Jan 8, 2016 at 2:14 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>> The core problem I am having is a C library I want to bind has a number
>> of macros which I need the value of.  Here is how I am trying to solve
>> it, but perhaps there is a better way:
>>
>> I have a small C library which gets compiled to libfoo.a which provides
>> functions that return the macro values, like:
>>
>> int macro1() { return MACRO1; }
>>
>> I then have an ocaml library, called ofoo, that uses Ctypes to bind to macro1:
>>
>> let macro1 = foreign "macro" (void @-> returning int)
>>
>> Where I am having issues is this small library, I'd prefer it to not
>> have to be installed on the system but just compiled into the Ocaml
>> library so that a user just has to link against that library.  Right
>> now, none of the symbols (macro1) are being included in the library, I'm
>> guessing because the linker sees no direct use of them.  And I'm not
>> even sure if I can get it included in the ocaml library.  I'm also not
>> able to get the libfoo symbols linked into a final executable, I'm
>> guessing for similar reasons.
>>
>> What are my options here?
>
> I'd recommend using ctypes 0.4+ stub generation support which can bind
> macro values and detect struct layout at an early compilation stage.
> You can see it in use to do this in my ocaml-unix-errno library
> <https://github.com/dsheets/ocaml-unix-errno>. One benefit with this
> approach is greater static checking and c <-> ocaml type safety.
>
> If you continue using dynamic bindings, two lonker flags may be of use to you:
>
> --no-as-needed : on recent Ubuntu distributions, gcc automatically
> includes the --as-needed flag which drops symbols that are not
> referenced. Unfortunately, clang does not understand this flag so you
> need to have a conditional in your build system to detect the compiler
> in use if you want a cross-platform library.
>
> -E : The Exports local symbols which are statically linked into a
> binary into the dynamic symbol table so that they can be found with
> dlsym.
>
> You can use these with gcc like -Wl,-E for example.

Will these approaches require that I have the C library installed to
compile against any binary using my library or will the symbols be part
of the ocaml library?  In my current version I have a libfoo.a that gets
created in the project and then linked against the library.

>
> Hope this helps,
>
> David Sheets
>
>
>> If I've missed any useful information, let me know.  I haven't interoped
>> much with C directory in Ocaml so I'm not sure what information is
>> important.
>>
>> /Malcolm
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Including a C library statically in an Ocaml library

[David Sheets]

On Fri, Jan 8, 2016 at 3:19 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
> David Sheets <sheets@alum.mit.edu> writes:
>
>> On Fri, Jan 8, 2016 at 2:14 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>> The core problem I am having is a C library I want to bind has a number
>>> of macros which I need the value of.  Here is how I am trying to solve
>>> it, but perhaps there is a better way:
>>>
>>> I have a small C library which gets compiled to libfoo.a which provides
>>> functions that return the macro values, like:
>>>
>>> int macro1() { return MACRO1; }
>>>
>>> I then have an ocaml library, called ofoo, that uses Ctypes to bind to macro1:
>>>
>>> let macro1 = foreign "macro" (void @-> returning int)
>>>
>>> Where I am having issues is this small library, I'd prefer it to not
>>> have to be installed on the system but just compiled into the Ocaml
>>> library so that a user just has to link against that library.  Right
>>> now, none of the symbols (macro1) are being included in the library, I'm
>>> guessing because the linker sees no direct use of them.  And I'm not
>>> even sure if I can get it included in the ocaml library.  I'm also not
>>> able to get the libfoo symbols linked into a final executable, I'm
>>> guessing for similar reasons.
>>>
>>> What are my options here?
>>
>> I'd recommend using ctypes 0.4+ stub generation support which can bind
>> macro values and detect struct layout at an early compilation stage.
>> You can see it in use to do this in my ocaml-unix-errno library
>> <https://github.com/dsheets/ocaml-unix-errno>. One benefit with this
>> approach is greater static checking and c <-> ocaml type safety.
>>
>> If you continue using dynamic bindings, two lonker flags may be of use to you:
>>
>> --no-as-needed : on recent Ubuntu distributions, gcc automatically
>> includes the --as-needed flag which drops symbols that are not
>> referenced. Unfortunately, clang does not understand this flag so you
>> need to have a conditional in your build system to detect the compiler
>> in use if you want a cross-platform library.
>>
>> -E : The Exports local symbols which are statically linked into a
>> binary into the dynamic symbol table so that they can be found with
>> dlsym.
>>
>> You can use these with gcc like -Wl,-E for example.
>
> Will these approaches require that I have the C library installed to
> compile against any binary using my library or will the symbols be part
> of the ocaml library?  In my current version I have a libfoo.a that gets
> created in the project and then linked against the library.

In the stub generation case, you will not have symbols that represent
the macros. Instead, C code will be generated which generates OCaml
code which contains the macro constants.

In the dlsym static linking case, you would build regular cm(x)a files
and a .a file and install them all with ocamlfind. Your users don't
have to know about C libraries or have them installed globally on the
system.

David

>>
>> Hope this helps,
>>
>> David Sheets
>>
>>
>>> If I've missed any useful information, let me know.  I haven't interoped
>>> much with C directory in Ocaml so I'm not sure what information is
>>> important.
>>>
>>> /Malcolm
>>>
>>> --
>>> Caml-list mailing list.  Subscription management and archives:
>>> https://sympa.inria.fr/sympa/arc/caml-list
>>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>> Bug reports: http://caml.inria.fr/bin/caml-bugs