From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id p0FIXap5011500 for ; Sat, 15 Jan 2011 19:33:36 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AucAAGd3MU3RVaG2kGdsb2JhbACGeo8nhjUBhzVVCBUBAQEBCQkMBxEEIKUTiXqCGIQMLoZaAQEDBYVLBIsf X-IronPort-AV: E=Sophos;i="4.60,327,1291590000"; d="scan'208";a="73137047" Received: from mail-gx0-f182.google.com ([209.85.161.182]) by mail3-smtp-sop.national.inria.fr with ESMTP; 15 Jan 2011 19:33:34 +0100 Received: by gxk8 with SMTP id 8so1557440gxk.27 for ; Sat, 15 Jan 2011 10:33:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=qUm9Dvug7XNYqFLCUHe8FVHn5HMJAEyZxAoG2D5tn6U=; b=WHszd+rMt83nWssKxqw5PW0UFee4jobltc1OeMQx0/An/kpM3CTOmscWX4CN4GC0JW 6TgjGhPB+pI5f/Dc3JyW+qmBTN3cbctvLnjlGSolWLUbQpeDmSKT34YnlkDqQN7gyk8f PDO/cX4gNCjwlfo2MqcJy+pmfp31jV1zY6rME= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=G/3P0Ubi12M9RJXHMGOGOGUD8N1Ii2vVxbeRdZCDpD/FNNv/Qt8zGCA8xYp9f0e5o+ OP27hanxu74dUjOOZenGPdoCFSTwSZhErGjiViSJqffwhJ5h9LtFbNbxGDKAA+NtwfWg bu5rkkKSzl8CFTGPQ1fMLc3Li+rbqMdnzaSuY= MIME-Version: 1.0 Received: by 10.91.21.33 with SMTP id y33mr2796412agi.92.1295116413577; Sat, 15 Jan 2011 10:33:33 -0800 (PST) Received: by 10.90.89.4 with HTTP; Sat, 15 Jan 2011 10:33:33 -0800 (PST) In-Reply-To: <20110115172300.GN4195@localhost> References: <20110115123838.GS4195@localhost> <20110115172300.GN4195@localhost> Date: Sat, 15 Jan 2011 20:33:33 +0200 Message-ID: From: Eray Ozkural To: Guillaume Yziquel Cc: Caml List Content-Type: multipart/alternative; boundary=001485f8617ac62afc0499e6c991 Subject: Re: [Caml-list] Unboxing: how to do it best? --001485f8617ac62afc0499e6c991 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Sat, Jan 15, 2011 at 7:23 PM, Guillaume Yziquel < guillaume.yziquel@citycable.ch> wrote: > Le Saturday 15 Jan 2011 =E0 16:00:21 (+0200), Eray Ozkural a =E9crit : > > On Sat, Jan 15, 2011 at 2:38 PM, Guillaume Yziquel > > <[1]guillaume.yziquel@citycable.ch> wrote: > > > > Then, for instance, given a datatype, you may wish to construct the > > datatype of an array of such types. Such a function needs to know > > details about the way OCaml boxes or unboxes different kinds of > > arrays, > > and it can be done (though rather awkwardly in my case). > > Awkwardly, but how? :) > > That's a good idea. > > Theoretically a functor transforms programs. Radical program rewriti= ng > > would be just the thing to do with a functor, but I'd rather have it > in > > the compiler. > > I wasn't thinking of applying functors to rewrite / specialise (whatever > you call it) some code to a datatype. > > Ok, you mean exactly like C++ type traits, where a static namespace provides further type information. In OCaml that'd be a module, right. > I was more thinking of having a first-class module as a regular value > that provides, when you unpack it, sufficient information to know how to > cross the barriers from OCaml to C or Fortran or whatever, and then send > it or receive it via an MPI implementation (since that's what I'm > looking at). Which means all your HPC primitives must know how to read > properly the datatype info enclosed in your first-class module. > I didn't really have MPI types on my mind, but it would surely be nice to be able to integrate nicely with MPI as well, though I think the Marshal module isn't costly (I made a small benchmark). What I had in mind was, say, I have this CA simulation or spiking neural net simulation code or a cell simulation, or a quantum chromodynamics simulation, maybe a visualization of an irregular mesh, or some other non-trivial scientific computing application where it's difficult to reduce everything to float arrays. Because usually you will have either vectors, or graphs of complex atomic structures and then this boxing is going to seriously hurt performance, as performance is hurt when you try to write any serious algorithm in Java in an OO fashion because everything is a pointer. When you have to start writing every algorithm in an awkward and bloated way to maintain some sense of performance, the benefit of the language quickly vanishes. (Main reason why Java should never be used except for toy web apps!) And then the HPC guy will have to turn to the portable assembly of C++, right? > I'm saying first-class module, because it can be typed as 'value datatype. > You only know what the 'value it is supposed to encode is, and have all > the typing info of how to deal with it encapsulated in the first-class > module and not leaking into the rest of your code. > Ok, care to give a minimal example? How do you pass and use the module value? This sounds interesting enough. You seem to be using the module to encapsulate encoding/decoding functions. Which is fine but how is it enough? How would that apply to changing the memory layout of a data type (or to provide an unboxed array of such values)? I thought you would be generating another module that represents the same type as an array of ints, and somehow convert the types transparently. How do you propose to do it? > What I would like is something like (thinking of a typical simulation > > datatype): > > type cvector4 =3D ][ (complex * complex * complex * complex) > > where ][ would be a "type operator" enforcing a flattened > > representation of the type expression it is applied to. It would just > > change the layout so it would be equivalent to the same type without > > the unboxing op. > > If I'm not mistaking tuples or records of floats are already unboxed at > runtime. Not seeing the great benefit here. > Yes, but the above is not a tuple of floats. Best, --=20 Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct --001485f8617ac62afc0499e6c991 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Sat, Jan 15, 2011 at 7:23 PM, Guillaume Yziqu= el <= guillaume.yziquel@citycable.ch> wrote:
Le Saturday 15 Jan 2011 =E0 16:00:21 (+0200), Eray Ozkural a =E9crit :
> =A0 =A0On Sat, Jan 15, 2011 at 2:38 PM, Guillaume Yz= iquel
> =A0 =A0<[1]guillaume.yziquel@citycable.ch> wrote:
>
> =A0 =A0 =A0Then, for instance, given a datatype, you may wish to const= ruct the
> =A0 =A0 =A0datatype of an array of such types. Such a function needs t= o know
> =A0 =A0 =A0details about the way OCaml boxes or unboxes different kind= s of
> =A0 =A0 =A0arrays,
> =A0 =A0 =A0and it can be done (though rather awkwardly in my case).


Awkwardly, but= how? :)

=A0
=
> =A0 =A0That's a good idea.
> =A0 =A0Theoretically a functor transforms programs. Radical program re= writing
> =A0 =A0would be just the thing to do with a functor, but I'd rathe= r have it in
> =A0 =A0the compiler.

I wasn't thinking of applying functors to rewrite / specialise (w= hatever
you call it) some code to a datatype.


Ok, you mean exactly like C++ type tra= its, where a static namespace provides further type information. In OCaml t= hat'd be a module, right.
=A0
I was more thinking of having a first-class module as a regular value
that provides, when you unpack it, sufficient information to know how to
cross the barriers from OCaml to C or Fortran or whatever, and then send
it or receive it via an MPI implementation (since that's what I'm looking at). Which means all your HPC primitives must know how to read
properly the datatype info enclosed in your first-class module.

I didn't really have MPI types on my mind, but= it would surely be nice to be able to integrate nicely with MPI as well, t= hough I think the Marshal module isn't costly (I made a small benchmark= ).=A0

What I had in mind was, say, I have this CA simulation = or spiking neural net simulation code or a cell simulation, or a quantum ch= romodynamics simulation, maybe a visualization of an irregular mesh, or som= e other non-trivial scientific computing application where it's difficu= lt to reduce everything to float arrays. Because usually you will have eith= er vectors, or graphs of complex atomic structures and then this boxing is = going to seriously hurt performance, as performance is hurt when you try to= write any serious algorithm in Java in an OO fashion because everything is= a pointer. When you have to start writing every algorithm in an awkward an= d bloated way to maintain some sense of performance, the benefit of the lan= guage quickly vanishes. (Main reason why Java should never be used except f= or toy web apps!) And then the HPC guy will have to turn to the portable as= sembly of C++, right?
=A0
I'm saying first-class module, because it can be typed as 'value da= tatype.
You only know what the 'value it is supposed to encode is, and have all=
the typing info of how to deal with it encapsulated in the first-class
module and not leaking into the rest of your code.
Ok, care to give a minimal example? How do you pass and use the= module value? This sounds interesting enough. You seem to be using the mod= ule to encapsulate encoding/decoding functions. Which is fine but how is it= enough? How would that apply to changing the memory layout of a data type = (or to provide an unboxed array of such values)? I thought you would be gen= erating another module that represents the same type as an array of ints, a= nd somehow convert the types transparently. How do you propose to do it?

> =A0 =A0What I would like is something like (thinking of a typical simu= lation
> =A0 =A0datatype):
> =A0 =A0type cvector4 =3D ][ (complex * complex * complex * complex)
> =A0 =A0where ][ would be a "type operator" enforcing a flatt= ened
> =A0 =A0representation of the type expression it is applied to. It woul= d just
> =A0 =A0change the layout so it would be equivalent to the same type wi= thout
> =A0 =A0the unboxing op.

If I'm not mistaking tuples or records of floats are already unbo= xed at
runtime. Not seeing the great benefit here.
=A0
<= div>Yes, but the above is not a tuple of floats.

<= /div>
Best,=A0

--
Eray Ozkural, PhD candidate.=A0 Co= mp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo= .com/group/ai-philosophy
h= ttp://myspace.com/arizanesil ht= tp://myspace.com/malfunct

--001485f8617ac62afc0499e6c991--