From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 3E235BC37 for ; Wed, 28 Oct 2009 19:37:06 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgcDACss6ErZSMDdi2dsb2JhbACBT5lxAQEBCgsKBxEFvwCEPwSBYQ X-IronPort-AV: E=Sophos;i="4.44,641,1249250400"; d="scan'208";a="37159396" Received: from fmmailgate01.web.de ([217.72.192.221]) by mail3-smtp-sop.national.inria.fr with ESMTP; 28 Oct 2009 19:37:05 +0100 Received: from smtp08.web.de (fmsmtp08.dlan.cinetic.de [172.20.5.216]) by fmmailgate01.web.de (Postfix) with ESMTP id 010A0133DD025; Wed, 28 Oct 2009 19:37:05 +0100 (CET) Received: from [95.208.117.111] (helo=frosties.localdomain) by smtp08.web.de with asmtp (TLSv1:AES256-SHA:256) (WEB.DE 4.110 #314) id 1N3DOC-0000iy-00; Wed, 28 Oct 2009 19:36:56 +0100 Received: from mrvn by frosties.localdomain with local (Exim 4.69) (envelope-from ) id 1N3Cxt-0007Zd-SG; Wed, 28 Oct 2009 19:09:45 +0100 From: Goswin von Brederlow To: Gerd Stolpmann Cc: Goswin von Brederlow , caml-list@inria.fr Subject: Re: [Caml-list] How to read different ints from a Bigarray? References: <87eiond3of.fsf@frosties.localdomain> <1256744580.4181.54.camel@flake.lan.gerd-stolpmann.de> Date: Wed, 28 Oct 2009 19:09:45 +0100 In-Reply-To: <1256744580.4181.54.camel@flake.lan.gerd-stolpmann.de> (Gerd Stolpmann's message of "Wed, 28 Oct 2009 16:43:00 +0100") Message-ID: <87my3b5r12.fsf@frosties.localdomain> User-Agent: Gnus/5.110006 (No Gnus v0.6) XEmacs/21.4.22 (linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: goswin-v-b@web.de X-Sender: goswin-v-b@web.de X-Provags-ID: V01U2FsdGVkX19udc9ZNJZIpVo2AFGbzh9/C0kfMMRHuYVKY4fA BZCsznhB72YFAzOcEgjjyuryE0QKTCrKT4XpRb9piL3XCk2W43 Fo6Rk+HaY= X-Spam: no; 0.00; bigarray:01 gerd:01 stolpmann:01 gerd:01 buffer:01 elt:01 bigarray:01 buf:01 buf:01 elt:01 endian:01 byte:01 endian:01 elegantly:01 stubs:01 Gerd Stolpmann writes: > Am Mittwoch, den 28.10.2009, 14:54 +0100 schrieb Goswin von Brederlow: >> Hi, >> >> I'm working on binding s for linux libaio library (asynchron IO) with >> a sharp eye on efficiency. That means no copying must be done on the >> data, which in turn means I can not use string as buffer type. >> >> The best type for this seems to be a (int, int8_unsigned_elt, >> c_layout) Bigarray.Array1.t. So far so good. >> >> Now I define helper functions: >> >> let get_uint8 buf off = buf.{off} >> let set_uint8 buf off x = buf.{off} <- x >> >> But I want more: >> >> get/set_int8 - do I use Obj.magic to "convert" to int8_signed_elt? >> >> And endian correcting access for larger ints: >> >> get/set_big_uint16 >> get/set_big_int16 >> get/set_little_uint16 >> get/set_little_int16 >> get/set_big_uint24 >> ... >> get/set_little_int56 >> get/set_big_int64 >> get/set_little_int64 >> >> What is the best way there? For uintXX I can get_uint8 each byte and >> shift and add them together. But that feels inefficient as each access >> will range check and the shifting generates a lot of code while cpus >> can usualy endian correct an int more elegantly. >> >> Is it worth the overhead of calling a C function to write optimized >> stubs for this? >> >> And last: >> >> get/set_string, blit_from/to_string >> >> Do I create a string where needed and then loop over every char >> calling s.(i) <- char_of_int buf.{off+i}? Or better a C function using >> memcpy? >> >> What do you think? > > A C call is too expensive for a single int (and ocamlopt). The runtime > needs to fix the stack and make it look C-compatible before it can do > the call. Maybe it's ok for an int64. > > Can you ensure that you only access the int's at word boundaries? If so, > it would be an option to wrap the same malloc'ed block of memory with > several bigarrays, e.g. you use an (int, int8_unsigned_elt, c_layout) > Bigarray.Array1.t when you access on byte level, but an (int32, > int32_unsigned_elt, c_layout) Bigarray.Array1.t when you access on int32 > level, but both bigarrays would point to the same block and share data. > This is trivial to do from C, just create several wrappers for the same > memory. I actualy need 512 byte aligned (better page aligned) data so that is definetly a possibility if only aligned access is required. > The nice thing about bigarrays is that the compiler can emit assembly > instructions for accessing them. Much faster than picking bytes and > reconstructing the int's on the caml side. However, if you cannot ensure > aligned int's the latter is probably unavoidable. So a.{i} <- x is not a C call. That is good to know. That leaves only the problem of endian conversion. I guess I could live with reading the int and shifting the bytes around for the rare cases of endianess of cpu and data differing. I might even not bother providing that since I don't need it at all. > Btw, I would be interested in your aio bindings if you do them as open > source project. See other mail. There is also an libfuse-ocaml that uses libaio-ocaml (althout that source is already in git instead of svn) if you want to see some more extensive use than the test.ml. > Gerd MfG Goswin