From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id B311CBC37 for ; Wed, 28 Oct 2009 16:17:38 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Al0GAEr950pQW+UMgGdsb2JhbACbPwEBFCZHAwEBvVqEPwSBYQ X-IronPort-AV: E=Sophos;i="4.44,640,1249250400"; d="scan'208";a="37147237" Received: from lo.gmane.org ([80.91.229.12]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES256-SHA; 28 Oct 2009 16:17:38 +0100 Received: from list by lo.gmane.org with local (Exim 4.50) id 1N3AHJ-0003wz-Fd for caml-list@inria.fr; Wed, 28 Oct 2009 16:17:37 +0100 Received: from ks300734.kimsufi.com ([91.121.65.225]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 Oct 2009 16:17:37 +0100 Received: from sylvain by ks300734.kimsufi.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 Oct 2009 16:17:37 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: caml-list@inria.fr From: Sylvain Le Gall Subject: Re: How to read different ints from a Bigarray? Date: Wed, 28 Oct 2009 15:17:18 +0000 (UTC) Message-ID: References: <87eiond3of.fsf@frosties.localdomain> <87639zd0m9.fsf@frosties.localdomain> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ks300734.kimsufi.com User-Agent: slrn/pre1.0.0-11 (Linux) Sender: news X-Spam: no; 0.00; le-gall:01 bigarray:01 le-gall:01 ocaml:01 buffer:01 bigarray:01 buffer:01 alignment:01 lib:01 endian:01 blit:01 byte:01 char:01 char:01 wrote:01 On 28-10-2009, Goswin von Brederlow wrote: > Sylvain Le Gall writes: > >> Hello, >> >> On 28-10-2009, Goswin von Brederlow wrote: >>> Hi, >>> >> >> Well, we talk about this a little bit, but here is my opinion: >> - calling a C function to add a single int will generate a big overhead >> - OCaml string are quite fast to modify values >> >> So to my mind the best option is to have a buffer string (say 16/32 >> char) where you put data inside and flush it in a single C call to >> Bigarray. >> >> E.g.: >> let append_char t c = >> if t.idx >= 64 then >> ( >> flush t.bigarray t.buffer; >> t.idx <- 0 >> ); >> t.buffer.(t.idx) <- c; >> t.idx <- t.idx + 1 >> >> let append_little_uint16 t i = >> append_char t ((i lsr 8) land 0xFF); >> append_char t ((i lsr 0) land 0xFF) >> >> >> I have used this kind of technique and it seems as fast as C, and a lot >> less C coding. >> >> Regards, >> Sylvain Le Gall > > This wont work so nicely: > > - Writes are not always in sequence. I want to do a stream access > too where this could be verry effective. But the plain buffer is > more for random / known offset access. At a minimum you would have > holes for alignment. > > - It makes read/write buffers complicated as you need to flush or peek > the string in case of uncommited changes. I can't do write-only > buffers as I want to be able to write a buffer and then add a > checksum to it in my application. The lib should not block that. > I was thinking to pure stream. It still stand with random access but you don't get a lot less C function call. You just have to write less C code. > > I also still wonder how bad a C function call really is. Consider the > case of writing an int64. > > Directly: You get one C call that does range check, endian convert and > write in one go. > > Bffered: With your code you have 7 Int64 shifts, 8 Int64 lands, 8 > conversions to int, at least one index check (more likely 8 to avoid > handling unaligned access) and 1/8 C call to blit the 64 byte buffer > string into the Bigarray. Not at all, you begin to break your int64 into 3 int (24bit * 2 + 16bit) and then 7 int shift, 8 int land. You can even manage to only break into 1 or 2 int. And off course, you bypass index check. > PS: Is a.{i} <- x a C call? Yes. Regards, Sylvain Le Gall