[Caml-list] ctypes - Advice for binding big structs?

[Caml-list] ctypes - Advice for binding big structs?

[Malcolm Matalka]

I have a large/complex struct I am trying to create bindings for
operations on it in Ocaml.  I have an API that tells me how many bytes
the struct is so I can allocate it just fine and pass it around to C
functions I've bound with ctypes.  But some data in it is accessed via
members.  I started implementing a structure in ctypes for it, but it's
getting large and awkward.  Are there any best practices around doing
this?

Some concerns I have:

- It seems fragile - a different version of the library might have
  different members in the struct so keeping my ocaml code in-synch
  seems error prone.

- It's annoying because the struct has a lot of members I don't care
  about in my case.  I only want access to a few members that have
  important details.

- The struct is large with lots of types that I don't necessarily want
  to create so creating the struct becomes somewhat awkward.  If I know
  the size of the types I might be able to pretend it's an array of N
  chars or something instead of trying to implement the type just to
  fill out this struct, but I don't know if that is valid.

- I thought about making C code that implements getters/setters for the
  struct elements I want.  The C code will always be correct, but that
  also feels like a lot of overhead if I'm using this struct in a tight
  loop.  I really just want to access that piece of memory, going in and
  out of a C call just seems like too much work.

Any suggestions?

Thanks,
/Malcolm

Re: [Caml-list] ctypes - Advice for binding big structs?

[Jeremy Yallop]

Dear Malcolm,

On 28/02/2016, Malcolm Matalka <mmatalka@gmail.com> wrote:
> I have a large/complex struct I am trying to create bindings for
> operations on it in Ocaml.  I have an API that tells me how many bytes
> the struct is so I can allocate it just fine and pass it around to C
> functions I've bound with ctypes.  But some data in it is accessed via
> members.  I started implementing a structure in ctypes for it, but it's
> getting large and awkward.  Are there any best practices around doing
> this?

The best approach is to use the Cstubs_structs module, which allows
you to declare just the parts of the structs that you need to access
in your program, and which generates code that uses the C struct
declarations to work out sizes, alignments, field offsets, etc.  The
basic API is the familiar set of functions "structure", "field" and
"seal" from the Ctypes module, but the build process is a little more
involved.  However, in return for the more complex build, all the
issues that you're concerned about are addressed.

The Cstubs_structs API is not yet very well documented, but there's a
brief guide with examples in the pull request that introduced it:

   https://github.com/ocamllabs/ocaml-ctypes/pull/62

> Some concerns I have:
>
> - It seems fragile - a different version of the library might have
>   different members in the struct so keeping my ocaml code in-synch
>   seems error prone.

The Cstubs_structs module addresses this by using generated C code to
determine the offsets of fields each time you build your library.

> - It's annoying because the struct has a lot of members I don't care
>   about in my case.  I only want access to a few members that have
>   important details.

Since Cstubs_structs retrieves layout rather than computing it you
only need to declare the members that you care about.

> - The struct is large with lots of types that I don't necessarily want
>   to create so creating the struct becomes somewhat awkward.  If I know
>   the size of the types I might be able to pretend it's an array of N
>   chars or something instead of trying to implement the type just to
>   fill out this struct, but I don't know if that is valid.

Again, since Cstubs_structs retrieves struct layout and alignment
information from C, you can use Ctypes.make to create struct values,
even if you haven't declared all the fields.

Kind regards,

Jeremy.

Re: [Caml-list] ctypes - Advice for binding big structs?

[Malcolm Matalka]

Jeremy Yallop <yallop@gmail.com> writes:

> Dear Malcolm,
>
> On 28/02/2016, Malcolm Matalka <mmatalka@gmail.com> wrote:
>> I have a large/complex struct I am trying to create bindings for
>> operations on it in Ocaml.  I have an API that tells me how many bytes
>> the struct is so I can allocate it just fine and pass it around to C
>> functions I've bound with ctypes.  But some data in it is accessed via
>> members.  I started implementing a structure in ctypes for it, but it's
>> getting large and awkward.  Are there any best practices around doing
>> this?
>
> The best approach is to use the Cstubs_structs module, which allows
> you to declare just the parts of the structs that you need to access
> in your program, and which generates code that uses the C struct
> declarations to work out sizes, alignments, field offsets, etc.  The
> basic API is the familiar set of functions "structure", "field" and
> "seal" from the Ctypes module, but the build process is a little more
> involved.  However, in return for the more complex build, all the
> issues that you're concerned about are addressed.
>
> The Cstubs_structs API is not yet very well documented, but there's a
> brief guide with examples in the pull request that introduced it:
>
>    https://github.com/ocamllabs/ocaml-ctypes/pull/62
>

Great, with some small modifications this worked like a charm.


>> Some concerns I have:
>>
>> - It seems fragile - a different version of the library might have
>>   different members in the struct so keeping my ocaml code in-synch
>>   seems error prone.
>
> The Cstubs_structs module addresses this by using generated C code to
> determine the offsets of fields each time you build your library.
>
>> - It's annoying because the struct has a lot of members I don't care
>>   about in my case.  I only want access to a few members that have
>>   important details.
>
> Since Cstubs_structs retrieves layout rather than computing it you
> only need to declare the members that you care about.
>
>> - The struct is large with lots of types that I don't necessarily want
>>   to create so creating the struct becomes somewhat awkward.  If I know
>>   the size of the types I might be able to pretend it's an array of N
>>   chars or something instead of trying to implement the type just to
>>   fill out this struct, but I don't know if that is valid.
>
> Again, since Cstubs_structs retrieves struct layout and alignment
> information from C, you can use Ctypes.make to create struct values,
> even if you haven't declared all the fields.
>
> Kind regards,
>
> Jeremy.