bigarray & int24

bigarray & int24

[Christoph Bauer]

Hi,

my program should read unaligned 3 byte integers. (A real world
example for such a format would be an RGB image). What is
the best approach?

  - extend bigarray for the int24 format and send the patch to INRIA
  - write a C-function, which converts the data to an int32-array
  - read an 1-byte-bigarray and construct an int-bigarray
  - ???

It should be a bigarray solution, because my program reads other
(supported)
formats as well.

Speed matters, because there are a lot of data.

Thanks,

Christoph Bauer
--  
c++; /* this makes c bigger but returns the old value */ (Unknown)

Re: [Caml-list] bigarray & int24

[Berke Durak]

Christoph Bauer a écrit :
> Hi,
> 
> my program should read unaligned 3 byte integers. (A real world
> example for such a format would be an RGB image). What is
> the best approach?
> 
>   - extend bigarray for the int24 format and send the patch to INRIA
>   - write a C-function, which converts the data to an int32-array
>   - read an 1-byte-bigarray and construct an int-bigarray
>   - ???
> 
> It should be a bigarray solution, because my program reads other
> (supported)
> formats as well.
> 
> Speed matters, because there are a lot of data.
> 
> Thanks,
> 
> Christoph Bauer

Last time I did this, I used a (m x n x 3) byte bigarray, also known
pendantically as a rank 3 tensor, where m is the number of rows and
n the number of columns.  Should work reasonably well.

-- 
Berke DURAK

Re: bigarray & int24

[Sylvain Le Gall]

Hello,

On 31-03-2008, Christoph Bauer <Christoph.Bauer@lmsintl.com> wrote:
> Hi,
>
> my program should read unaligned 3 byte integers. (A real world
> example for such a format would be an RGB image). What is
> the best approach?
>
>   - extend bigarray for the int24 format and send the patch to INRIA
>   - write a C-function, which converts the data to an int32-array
>   - read an 1-byte-bigarray and construct an int-bigarray
>   - ???
>
> It should be a bigarray solution, because my program reads other
> (supported)
> formats as well.
>
> Speed matters, because there are a lot of data.
>

If speed matters, you could use an alternate approach that can give you
a very efficient way of processing:
* keep an int32/int64 bigarray (use native int format -- the one which is the
  more efficient)
* use a number of row that allow you to fit a x 24 bit data in b x
 (32/64) bit data (least common multiplicator / 32 or 64)
* create a layer that allow you to process your packed data 

This is a way to minimize the number of unaligned access, being fully
compatible with your raw format.

You will still have an issue with conversion to Nativeint when
extracting data. 

So i recommend you to write your abstract layer in C (gaining full speed
for extraction of packed-aligned data and conversion) and to return
OCaml int (which is possible since your data are only 24 bits long). 

This is not very OCaml friendly -- but allow you to get the best in term
of speed.

Regards,
Sylvain Le Gall