From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 1AC8DBC37 for ; Wed, 28 Oct 2009 15:17:44 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Al0GADrv50pQW+UMgGdsb2JhbACbPwEBFCZHAwEBvgGEPwSBYQ X-IronPort-AV: E=Sophos;i="4.44,640,1249250400"; d="scan'208";a="37143340" Received: from lo.gmane.org ([80.91.229.12]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES256-SHA; 28 Oct 2009 15:17:43 +0100 Received: from list by lo.gmane.org with local (Exim 4.50) id 1N39L6-0005Lt-8E for caml-list@inria.fr; Wed, 28 Oct 2009 15:17:28 +0100 Received: from ks300734.kimsufi.com ([91.121.65.225]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 Oct 2009 15:17:28 +0100 Received: from sylvain by ks300734.kimsufi.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 28 Oct 2009 15:17:28 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: caml-list@inria.fr From: Sylvain Le Gall Subject: Re: How to read different ints from a Bigarray? Date: Wed, 28 Oct 2009 14:16:52 +0000 (UTC) Message-ID: References: <87eiond3of.fsf@frosties.localdomain> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ks300734.kimsufi.com User-Agent: slrn/pre1.0.0-11 (Linux) Sender: news X-Spam: no; 0.00; le-gall:01 bigarray:01 buffer:01 elt:01 bigarray:01 buf:01 buf:01 elt:01 endian:01 byte:01 endian:01 elegantly:01 stubs:01 blit:01 memcpy:01 Hello, On 28-10-2009, Goswin von Brederlow wrote: > Hi, > > I'm working on binding s for linux libaio library (asynchron IO) with > a sharp eye on efficiency. That means no copying must be done on the > data, which in turn means I can not use string as buffer type. > > The best type for this seems to be a (int, int8_unsigned_elt, > c_layout) Bigarray.Array1.t. So far so good. > > Now I define helper functions: > > let get_uint8 buf off = buf.{off} > let set_uint8 buf off x = buf.{off} <- x > > But I want more: > > get/set_int8 - do I use Obj.magic to "convert" to int8_signed_elt? > > And endian correcting access for larger ints: > > get/set_big_uint16 > get/set_big_int16 > get/set_little_uint16 > get/set_little_int16 > get/set_big_uint24 > ... > get/set_little_int56 > get/set_big_int64 > get/set_little_int64 > > What is the best way there? For uintXX I can get_uint8 each byte and > shift and add them together. But that feels inefficient as each access > will range check and the shifting generates a lot of code while cpus > can usualy endian correct an int more elegantly. > > Is it worth the overhead of calling a C function to write optimized > stubs for this? > > And last: > > get/set_string, blit_from/to_string > > Do I create a string where needed and then loop over every char > calling s.(i) <- char_of_int buf.{off+i}? Or better a C function using > memcpy? > > What do you think? > Well, we talk about this a little bit, but here is my opinion: - calling a C function to add a single int will generate a big overhead - OCaml string are quite fast to modify values So to my mind the best option is to have a buffer string (say 16/32 char) where you put data inside and flush it in a single C call to Bigarray. E.g.: let append_char t c = if t.idx >= 64 then ( flush t.bigarray t.buffer; t.idx <- 0 ); t.buffer.(t.idx) <- c; t.idx <- t.idx + 1 let append_little_uint16 t i = append_char t ((i lsr 8) land 0xFF); append_char t ((i lsr 0) land 0xFF) I have used this kind of technique and it seems as fast as C, and a lot less C coding. Regards, Sylvain Le Gall