From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id p0FHP5kP009853 for ; Sat, 15 Jan 2011 18:25:05 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgcNAHZnMU1V2gB5ZGdsb2JhbAClCQsKBhIkvAANhUMEjl0 X-IronPort-AV: E=Sophos;i="4.60,327,1291590000"; d="scan'208";a="87002989" Received: from emailfrontal2.citycable.ch ([85.218.0.121]) by mail2-smtp-roc.national.inria.fr with SMTP; 15 Jan 2011 18:25:00 +0100 X-Alinto-smtpauth-localdomain: Yes Received: from seldon (unknown [85.218.92.99]) (Authenticated sender: guillaume.yziquel@citycable.ch) by emailfrontal2.citycable.ch (Postfix) with ESMTPA id 849C1834924; Sat, 15 Jan 2011 18:24:58 +0100 (CET) Received: from yziquel by seldon with local (Exim 4.72) (envelope-from ) id 1Pe9q8-0006gz-Op; Sat, 15 Jan 2011 18:23:00 +0100 Date: Sat, 15 Jan 2011 18:23:00 +0100 From: Guillaume Yziquel To: Eray Ozkural Cc: Caml List Message-ID: <20110115172300.GN4195@localhost> References: <20110115123838.GS4195@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by walapai.inria.fr id p0FHP5kP009853 Subject: Re: [Caml-list] Unboxing: how to do it best? Le Saturday 15 Jan 2011 à 16:00:21 (+0200), Eray Ozkural a écrit : > On Sat, Jan 15, 2011 at 2:38 PM, Guillaume Yziquel > <[1]guillaume.yziquel@citycable.ch> wrote: > > Then, for instance, given a datatype, you may wish to construct the > datatype of an array of such types. Such a function needs to know > details about the way OCaml boxes or unboxes different kinds of > arrays, > and it can be done (though rather awkwardly in my case). > > Hi Guillaume, Hi. > That's a good idea. > Theoretically a functor transforms programs. Radical program rewriting > would be just the thing to do with a functor, but I'd rather have it in > the compiler. I wasn't thinking of applying functors to rewrite / specialise (whatever you call it) some code to a datatype. I was more thinking of having a first-class module as a regular value that provides, when you unpack it, sufficient information to know how to cross the barriers from OCaml to C or Fortran or whatever, and then send it or receive it via an MPI implementation (since that's what I'm looking at). Which means all your HPC primitives must know how to read properly the datatype info enclosed in your first-class module. I'm saying first-class module, because it can be typed as 'value datatype. You only know what the 'value it is supposed to encode is, and have all the typing info of how to deal with it encapsulated in the first-class module and not leaking into the rest of your code. It's not program rewriting, and there is overhead, but I guess it can be kept quite low. > After all program-transformation is what an optimization pass is > essentially. There is absolutely nothing wrong with doing it in a > high-level way, as long as it doesn't introduce runtime overhead. > Has anyone designed a cool compiler like that? :) I'm not for the moment interested in the 'compiler' aspect of that. The solution I propose does cost some overhead reading the first-class module at runtime. Not really sure if functorising things fully would be a real performance benefit in my case. Functorising things too much makes things quite ugly to maintain. > As for 'annotating type definitions', where would you put the line > as to > what 'annotating' means? Using type-conv-like Camlp4 processing? > > I haven't thought much about the implementation, except verifying that > it's just an extension of the present kinds of unboxing in the > runtime. Not entirely following you here, but bubbling the runtime unboxing up into the syntax appears risky at best. > What I would like is something like (thinking of a typical simulation > datatype): > type cvector4 = ][ (complex * complex * complex * complex) > where ][ would be a "type operator" enforcing a flattened > representation of the type expression it is applied to. It would just > change the layout so it would be equivalent to the same type without > the unboxing op. If I'm not mistaking tuples or records of floats are already unboxed at runtime. Not seeing the great benefit here. But for an complex algebraic datatype you may think about the following: Instead of having, essentially, structured blocks with pointers to other blocks, you could define a "'value packed" type which would represent flattened OCaml 'values. You have locality there. The issue then is twofold: -1- You need information to pack and unpack, and as you do not want to pack and unpack too much, you will have problems dealing with packed values in your OCaml code. -2- The mentioned information to pack and unpack will have an uncomfortable representation to deal with, as you still need the GC to operate correctly on the packed values. So bit-twiddling headaches when recursively packing or unpacking values. And fun when encountering cycles of values. Not to mention polymorphic comparison. To sum up: A complete solution covering all OCaml types and values for packing doesn't seem realistic (at least I do not see how to pack in a same block, a custom block, and array of floats and a structured block, without uterly confusing the GC). Baby steps and dealing systematically with the corner cases as they pop seems a realistic approach however, but with limited scope of applicability. > Preprocessing might be one way to implement it, but i don't think it's > an easy implementation at any rate. > Just a small idea that I couldn't let slip from my mind. For what you propose, preprocessing seems less important than correct typing. > But as to avoiding pointer chasing, I think there's no workaround to > the > way OCaml handles memory. The only solution I can come of is the > obvious: use arrays or bigarrays and smart datatypes. > > Smart datatypes is OK, I think you could substitute many datatypes with > such a thing, but I'm not sure how easy that would be to do in > real-world programming? Pretty uncomfortable I guess. -- Guillaume Yziquel