Re: How to read different ints from a Bigarray?

[Sylvain Le Gall <sylvain@le-gall.net>]

On 28-10-2009, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Sylvain Le Gall <sylvain@le-gall.net> writes:
>
>> Hello,
>>
>> On 28-10-2009, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>>> Hi,
>>>
>>
>> Well, we talk about this a little bit, but here is my opinion:
>> - calling a C function to add a single int will generate a big overhead
>> - OCaml string are quite fast to modify values
>>
>> So to my mind the best option is to have a buffer string (say 16/32
>> char) where you put data inside and flush it in a single C call to
>> Bigarray. 
>>
>> E.g.:
>> let append_char t c =
>>   if t.idx >= 64 then
>>     (
>>       flush t.bigarray t.buffer;
>>       t.idx <- 0
>>     );
>>   t.buffer.(t.idx) <- c;
>>   t.idx <- t.idx + 1
>>
>> let append_little_uint16 t i =
>>   append_char t ((i lsr 8) land 0xFF);
>>   append_char t ((i lsr 0) land 0xFF)
>>   
>>
>> I have used this kind of technique and it seems as fast as C, and a lot
>> less C coding.
>>
>> Regards,
>> Sylvain Le Gall
>
> This wont work so nicely:
>
> - Writes are not always in sequence. I want to do a stream access
>   too where this could be verry effective. But the plain buffer is
>   more for random / known offset access. At a minimum you would have
>   holes for alignment.
>
> - It makes read/write buffers complicated as you need to flush or peek
>   the string in case of uncommited changes. I can't do write-only
>   buffers as I want to be able to write a buffer and then add a
>   checksum to it in my application. The lib should not block that.
>

I was thinking to pure stream. It still stand with random access but you
don't get a lot less C function call. You just have to write less C
code.

>
> I also still wonder how bad a C function call really is. Consider the
> case of writing an int64.
>
> Directly: You get one C call that does range check, endian convert and
> write in one go.
>
> Bffered: With your code you have 7 Int64 shifts, 8 Int64 lands, 8
> conversions to int, at least one index check (more likely 8 to avoid
> handling unaligned access) and 1/8 C call to blit the 64 byte buffer
> string into the Bigarray.

Not at all, you begin to break your int64 into 3 int (24bit * 2 + 16bit)
and then 7 int shift, 8 int land. 

You can even manage to only break into 1 or 2 int.

And off course, you bypass index check. 

> PS: Is a.{i} <- x a C call?

Yes.

Regards,
Sylvain Le Gall