Re: [Caml-list] DSL for handling parsing binary memory structures?

["Markus Mottl" <markus.mottl@gmail.com>]

On 5/17/07, Richard Jones <rich@annexia.org> wrote:
> I wonder if anyone has thought about or developed a domain-specific
> language in OCaml for handling the processing of binary structures?
>
> I was thinking of writing something to make system calls into the Xen
> hypervisor.  The main problem is that the hypervisor takes & returns
> binary structures which have very precise size / alignment
> requirements, bitfields, etc. (and the requirements change with
> versions of the hypervisor).  It sounds like the ideal use for a DSL
> to describe the binary structure of each version of each call and
> automatically build accessors.

I'm currently developing a preprocessor that generates readers/writers
for binary protocols from OCaml type specifications.  It should be
very easy to add custom binary decoders by specifying an abstract type
for such values + custom readers/writer.  Or one can (ab)use the
existing ones to get the intended result (e.g. just use the int32 type
to read a 32bit integer from some memory loccation, etc.).

The protocol is very compact (even slightly more so than the
OCaml-marshal format), and performance is extremely high, usually even
higher than the standard marshal function, though it doesn't handle
shared/cyclic datastructures.  I also haven't looked into endianness
functionality yet (should be easy to add), but both 32bit and 64bit
platforms are supported.

In later stages there will also be a feature for a "protocol
handshake", which guarantees that e.g. the type specifications used by
two communicating machines are compatible.

We (Jane Street Capital) will eventually release this library as open
source so stay tuned...

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com