Marshalling of floats

Marshalling of floats

[Christopher Oliver]

In writing some signal processing software, I ran across a few
problems with Marshall.  I'd like ideas about how OCAML might
address these.

First, floats are simply written in increasing byte order OCAML
makes no attempt to rearrange based on endianness.  I think byte
order is discovered on configuration when building the system.
Should OCAML swap byte orders on some hardware to try to compen-
sate, or should we make no attempt to record floating point
portably between the major architectures?  The change in byterun/
intern.c and byterun/extern.c to handle byte swapping is fairly
trivial provided we include the config.h

Harder, does anyone have any good ideas besides conversion to
and from lists for marshalling floating arrays between bytecode
and native OCAML programs?  Without this sort of thing, one must
walk structures by hand if there are variant types with floating
arrays.

The reason for all this noise is that marshalling is ostensibly
to provide an external representation for non-code structured
data, but to be thorough, must we not try to be portable?

Ideas?

-- 
Christopher Oliver                     Traverse Internet
Systems Coordinator                    223 Grandview Pkwy, Suite 108
oliver@traverse.net                    Traverse City, Michigan, 49684
let magic f = fun x -> x and more_magic n f = fun x -> f ((n f) x);;

Re: Marshalling of floats

[Xavier Leroy]

> First, floats are simply written in increasing byte order OCAML
> makes no attempt to rearrange based on endianness.

Floats are written in "native" endianness (the natural endianness of
the machine writing the file), but along with a flag telling which
endianness is used.  input_value, then, swaps bytes if it runs on a
processor whose endianness does not match that used to create the
file.  This is more efficient than converting to and from a fixed
endianness in the frequent case where the data is read back on the
same machine that wrote it.

> Should OCAML swap byte orders on some hardware to try to compen-
> sate, or should we make no attempt to record floating point
> portably between the major architectures?  The change in byterun/
> intern.c and byterun/extern.c to handle byte swapping is fairly
> trivial provided we include the config.h

input_value/output_value is already portable across all platforms that
use IEEE floats.  The byte-swapping code is already in byterun/intern.c.

> Harder, does anyone have any good ideas besides conversion to
> and from lists for marshalling floating arrays between bytecode
> and native OCAML programs?  Without this sort of thing, one must
> walk structures by hand if there are variant types with floating
> arrays.

The next release of OCaml will use the same format for float arrays in
native-code and in bytecode, thus solving this problem.

Regards,

- Xavier Leroy

Re: Marshalling of floats

[Christopher Oliver]

> Floats are written in "native" endianness (the natural endianness of
> the machine writing the file), but along with a flag telling which
> endianness is used.

Ouch!  That will teach me to read only one piece of code.  My common
work usually has me dealing with standardized network representations,
so I missed this strategy completely.  [Chris sticks head in bucket.]

> > Harder, does anyone have any good ideas besides conversion to

> The next release of OCaml will use the same format for float arrays in
> native-code and in bytecode, thus solving this problem.

Both compact I hope.  Any idea on when this might happen?

-- 
Christopher Oliver                     Traverse Internet
Systems Coordinator                    223 Grandview Pkwy, Suite 108
oliver@traverse.net                    Traverse City, Michigan, 49684
let magic f = fun x -> x and more_magic n f = fun x -> f ((n f) x);;