Metaprogramming features

Metaprogramming features

[Jacques Carette]

A number of you are already aware of the following feature request:

http://caml.inria.fr/mantis/view.php?id=4608

whose text reads
===================
I really wish that the basic features of metaocaml, namely .< >., .~ and .!
were included in OCaml proper.  The many other experimental features of
metaocaml could stay 'experimental', those are the 3 which are sorely
needed.

Note that OCaml could be really ahead of Haskell here (with its untyped
Template Haskell, which is closer to camlp4 than to metaocaml) by being
the first production language to have _typed_ metaprogramming facilities.
===================

This feature request is currently the entry in Mantis which has (by far!) the largest number of comments from separate people, as well as now being the entry with the most comments overall.  

Please add your voice to the chorus!

Jacques

Re: [Caml-list] Metaprogramming features

[Dario Teixeira]

Hi,

> I really wish that the basic features of metaocaml, namely
> .< >., .~ and .! were included in OCaml proper.  The many other
> experimental features of metaocaml could stay 'experimental',
> those are the 3 which are sorely needed.

To those of us not familiar with MetaOcaml, could you give us
an overview of what these features mean and why they are so
special?

Cheers,
Dario

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Friday 03 October 2008 16:09:40 Dario Teixeira wrote:
> Hi,
>
> > I really wish that the basic features of metaocaml, namely
> > .< >., .~ and .! were included in OCaml proper.  The many other
> > experimental features of metaocaml could stay 'experimental',
> > those are the 3 which are sorely needed.
>
> To those of us not familiar with MetaOcaml, could you give us
> an overview of what these features mean and why they are so
> special?

MetaOCaml extends OCaml with three constructs that allow programs to generate, 
compile and evaluate code at run time.

Code is represented by the form:

  .< 1 + 2 >.

This may be executed using:

  .! .< 1 + 2 >.

Values may be injected into code using:

  .< 1 + .~x >.

This makes it vastly easier to turn term-level interpreters into compilers to 
obtain substantial performance improvements.

However, MetaOCaml has problems:

1. Obvious applications are low-level compilers for regular expressions, 
parsers and bytecodes but MetaOCaml imposes the limitations of OCaml (e.g. 
slow char and int handling) which makes it unsuitable for most such 
applications.

2. Non-trivial features in the target language can require great complexity, 
e.g. writing a compiler in MetaOCaml for a language with exceptions imposes 
CPS everywhere which is obfuscated and slow without language support.

3. You cannot generate new pattern matches to leverage OCaml's optimizing 
pattern match compiler in your run-time generated code (but you can use 
static ones).

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[David Teller]

On Sat, 2008-10-04 at 03:00 +0100, Jon Harrop wrote:
> 1. Obvious applications are low-level compilers for regular expressions, 
> parsers and bytecodes but MetaOCaml imposes the limitations of OCaml (e.g. 
> slow char and int handling) which makes it unsuitable for most such 
> applications.

Oh, well, you answer some of your own question from your other post. I
was thinking along the lines of ulex. And I have the impression you can
get a nice "finally" with MetaOCaml, albeit perhaps with a weird
syntax. 

Now, another obvious application is writing efficient trampoline code
and other combinators, something which I believe may be very useful for
concurrency.

> 3. You cannot generate new pattern matches to leverage OCaml's optimizing 
> pattern match compiler in your run-time generated code (but you can use 
> static ones).

You can't? Well, that's unfortunate. We'll have to keep relying on
camlp4 for the moment.

Cheers,
 David
-- 
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act brings liquidations. 

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Saturday 04 October 2008 10:03:58 David Teller wrote:
> On Sat, 2008-10-04 at 03:00 +0100, Jon Harrop wrote:
> > 1. Obvious applications are low-level compilers for regular expressions,
> > parsers and bytecodes but MetaOCaml imposes the limitations of OCaml
> > (e.g. slow char and int handling) which makes it unsuitable for most such
> > applications.
>
> Oh, well, you answer some of your own question from your other post. I
> was thinking along the lines of ulex. And I have the impression you can
> get a nice "finally" with MetaOCaml, albeit perhaps with a weird
> syntax.

If you mean "try..finally" then that is a trivial syntax extension. I cannot 
see how MetaOCaml would help.

> Now, another obvious application is writing efficient trampoline code
> and other combinators, something which I believe may be very useful for
> concurrency.

Yes.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[Brian Hurt]

Jon Harrop wrote:
> On Friday 03 October 2008 16:09:40 Dario Teixeira wrote:
>   
>> Hi,
>>
>>     
>>> I really wish that the basic features of metaocaml, namely
>>> .< >., .~ and .! were included in OCaml proper.  The many other
>>> experimental features of metaocaml could stay 'experimental',
>>> those are the 3 which are sorely needed.
>>>       
>> To those of us not familiar with MetaOcaml, could you give us
>> an overview of what these features mean and why they are so
>> special?
>>     
>
> MetaOCaml extends OCaml with three constructs that allow programs to generate, 
> compile and evaluate code at run time.
>
> Code is represented by the form:
>
>   .< 1 + 2 >.
>
> This may be executed using:
>
>   .! .< 1 + 2 >.
>
> Values may be injected into code using:
>
>   .< 1 + .~x >.
>
>   
I'm definitely missing something here.  How are the above examples 
different from:
let f () = 1 + 2;;
f ();;
let f x = 1 + x;;

?

This is a non-trivial question, as there is no clear delineation between 
data structures and executable code.

Brian

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Monday 06 October 2008 15:06:12 you wrote:
> Jon Harrop wrote:
> > Code is represented by the form:
> >
> >   .< 1 + 2 >.
> >
> > This may be executed using:
> >
> >   .! .< 1 + 2 >.
> >
> > Values may be injected into code using:
> >
> >   .< 1 + .~x >.
>
> I'm definitely missing something here.  How are the above examples
> different from:
> let f () = 1 + 2;;
> f ();;
> let f x = 1 + x;;
>
> ?
>
> This is a non-trivial question, as there is no clear delineation between
> data structures and executable code.

Exactly. The difference is (only) the performance characteristics.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: Metaprogramming features

[Chung-chieh Shan]

Jon Harrop <jonathandeanharrop@googlemail.com> wrote in article <200810061656.42903.jon@ffconsultancy.com> in gmane.comp.lang.caml.inria:
> On Monday 06 October 2008 15:06:12 you wrote:
> > I'm definitely missing something here.  How are the above examples
> > different from:
> > let f () = 1 + 2;;
> > f ();;
> > let f x = 1 + x;;
> >
> > ?
> >
> > This is a non-trivial question, as there is no clear delineation between
> > data structures and executable code.

Indeed the question is not trivial.  Because MetaOCaml (like OCaml) has
higher-order functions (which may operate on code values or on functions
on code values), it is not trivial at all to `simulate' MetaOCaml in
OCaml.  Our PEPM paper this year (Kameyama, Kiselyov, and Shan) needed
the full power of System F just to express a subset of MetaOCaml without
polymorphism (even let-bound) or nested quotation (such as .<.<42>.>.).

    http://www.cs.rutgers.edu/~ccshan/metafx/pepm66-kameyama.pdf

Section 2 of this paper gives a series of examples to illustrate the
non-triviality of the simulation.  You can try to skip to Section 2.5
to see the most challenging example there.

> Exactly. The difference is (only) the performance characteristics.

To the contrary, that is not the only difference; the timing of
side effects (including non-termination and exceptions) can also be
different.  We give an example at the beginning of our Section 2.

-- 
Edit this signature at http://www.digitas.harvard.edu/cgi-bin/ken/sig
2008-10-01 International Day of Older Persons http://www.who.int/ageing/
2008-11-20 Universal Children's Day http://www.unicef.org/
2008-11-25 Day for the Elimination of Violence Against Women http://unifem.org/
1948-12-10 Universal Declaration of Human Rights http://everyhumanhasrights.org

Re: [Caml-list] Re: Metaprogramming features

[Jon Harrop]

On Monday 06 October 2008 17:46:49 Chung-chieh Shan wrote:
> Jon Harrop <jonathandeanharrop@googlemail.com> wrote in article 
<200810061656.42903.jon@ffconsultancy.com> in gmane.comp.lang.caml.inria:
> > Exactly. The difference is (only) the performance characteristics.
>
> To the contrary, that is not the only difference; the timing of
> side effects (including non-termination and exceptions) can also be
> different.  We give an example at the beginning of our Section 2.

I don't follow. Your paper translates the following staged function:

  let power7 : int -> int =
    .! .<fun x -> .~(Printf.printf "power\n"; power 7 .<x>.)>.;;

into this:

  let npower7 =
    (fun () -> fun x -> (printf "power\n"; npower 7 (fun () -> x)) ()) ();;

and notes that the result is different because the former calls printf 
immediately whereas the latter defers.

Surely the discrepancy is because the translation should be:

  let npower7 =
    let e = printf "power\n"; npower 7 (fun () -> x) in
    (fun () -> fun x -> e ()) ();;

because escaped code is always evaluated before the brackets that surround it, 
so it needs to be hoisted out of the "fun () -> ..."?

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Re: Metaprogramming features

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

Excerpts from jonathandeanharrop's message of Tue Oct 07 02:17:13 +0200 2008:
> On Monday 06 October 2008 17:46:49 Chung-chieh Shan wrote:
> > Jon Harrop <jonathandeanharrop@googlemail.com> wrote in article 
> <200810061656.42903.jon@ffconsultancy.com> in gmane.comp.lang.caml.inria:
> > > Exactly. The difference is (only) the performance characteristics.
> >
> > To the contrary, that is not the only difference; the timing of
> > side effects (including non-termination and exceptions) can also be
> > different.  We give an example at the beginning of our Section 2.
> 
> I don't follow. Your paper translates the following staged function:
> 
>   let power7 : int -> int =
>     .! .<fun x -> .~(Printf.printf "power\n"; power 7 .<x>.)>.;;
> 
> into this:
> 
>   let npower7 =
>     (fun () -> fun x -> (printf "power\n"; npower 7 (fun () -> x)) ()) ();;
> 
> and notes that the result is different because the former calls printf 
> immediately whereas the latter defers.
> 
> Surely the discrepancy is because the translation should be:
> 
>   let npower7 =
>     let e = printf "power\n"; npower 7 (fun () -> x) in
>     (fun () -> fun x -> e ()) ();;

Hum "unbound value x" :)

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [Caml-list] Re: Metaprogramming features

[Jon Harrop]

On Tuesday 07 October 2008 13:49:29 you wrote:
> Excerpts from jonathandeanharrop's message of Tue Oct 07 02:17:13 +0200 
2008:
> > Surely the discrepancy is because the translation should be:
> >
> >   let npower7 =
> >     let e = printf "power\n"; npower 7 (fun () -> x) in
> >     (fun () -> fun x -> e ()) ();;
>
> Hum "unbound value x" :)

Oh yeah. :-)

I hadn't noticed that you can do:

# .< fun x -> .~ .<x>. >.;;
- : ('a, 'b -> 'b) code = .<fun x_1 -> x_1>.

Which seems to be the source of my confusion. So "x" is bound inside the 
escaped code even though it was defined in the "fun x ->" of the bracketed 
code.

You are not allowed to do:

# .< fun x -> .~ x >.;;
Wrong level: variable bound at level 1 and used at level 0

That means a faithful translation into closures must capture such bound 
variables which will, in turn, defer the expression that does the 
constructing:

  let e x = printf "power\n"; npower 7 (fun () -> x)

and that now fails to incur side-effects at the same time as the MetaOCaml 
program.

So the translation from MetaOCaml to OCaml is not obvious at all.

This begs the question: could another formulation of MetaOCaml avoid these 
complications without losing expressiveness?

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Re: Metaprogramming features

[Jacques Carette]

Jon Harrop wrote:
> So the translation from MetaOCaml to OCaml is not obvious at all.
>
> This begs the question: could another formulation of MetaOCaml avoid these 
> complications without losing expressiveness?
>   
No.  See some of the early papers of Taha and papers from Sheard around 
the same time. 

BTW, F#'s metaprogramming features are much much less powerful than 
metaocaml's.  We tried to use them to generalize some previous work I 
(with some other colleagues) had done in metaocaml, but couldn't.  
Essentially, F#'s splices are first-order only.

Jacques

Re: [Caml-list] Metaprogramming features

[David Teller]

Gasp, lobbying on this list ? :)

I strongly agree with that feature request. One reason is  all the code
which is currently in Camlp4 but actually deserves the MetaOCaml
treatment. The other reason is that I believe that the Haskell vs. OCaml
race matters.

Cheers,
 David

On Fri, 2008-10-03 at 10:34 -0400, Jacques Carette wrote:

> Note that OCaml could be really ahead of Haskell here (with its untyped
> Template Haskell, which is closer to camlp4 than to metaocaml) by being
> the first production language to have _typed_ metaprogramming facilities.
> ===================
> 
> This feature request is currently the entry in Mantis which has (by far!) the largest number of comments from separate people, as well as now being the entry with the most comments overall.  
> 
> Please add your voice to the chorus!
> 
> Jacques

-- 
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act brings liquidations. 

Re: [Caml-list] Metaprogramming features

[Paolo Donadeo]

> I strongly agree with that feature request.

Me too!


-- 
Paolo
~
~
:wq

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Friday 03 October 2008 17:01:36 David Teller wrote:
> I strongly agree with that feature request. One reason is all the code
> which is currently in Camlp4 but actually deserves the MetaOCaml
> treatment.

Most Camlp4 code is macros providing syntax extensions to OCaml that would not 
benefit from run-time code generation. So what Camlp4 code are you referring 
to that "deserves MetaOCaml treatment"?

> The other reason is that I believe that the Haskell vs. OCaml 
> race matters.

If you want a race, forget Haskell and look at F# which already provides typed 
metaprogramming with quotations, unsafe high-performance metaprogramming via 
CIL and the two most valuable syntax extensions (try..finally and views) as 
well as most of the key benefits of OCaml plus decent libraries and a 
concurrent run-time.

IMHO, the most productive direction for OCaml right now is towards LLVM. An 
LLVM backend to OCaml would facilitate some productive improvements (e.g. 
free polymorphism, easy run-time code generation and dynamic linking, easy 
FFI, platform independent and performant intermediate representation, extra 
optimization passes for "free"). Moreover, this path can lead to completely 
independent compilers that could then be free to expose their lexers, 
parsers, type checkers and metaprogramming capabilities without licensing 
issues.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[David Teller]

On Sat, 2008-10-04 at 03:17 +0100, Jon Harrop wrote:
> If you want a race, forget Haskell and look at F# which already provides typed 
> metaprogramming with quotations, unsafe high-performance metaprogramming via 
> CIL and the two most valuable syntax extensions (try..finally and views) as 
> well as most of the key benefits of OCaml plus decent libraries and a 
> concurrent run-time.

Well, I'm not going to answer on the concurrency aspect, it's been amply
discussed in various other threads.

As for the decent libraries, that's our first objective with Batteries
Included and Community OCaml. Valuable  syntax extensions (including
boilerplate) is our secondary objective. We handle that part so that the
rest of the community can concentrate on the rest of the race :)

> IMHO, the most productive direction for OCaml right now is towards LLVM. An 
> LLVM backend to OCaml would facilitate some productive improvements (e.g. 
> free polymorphism, easy run-time code generation and dynamic linking, easy 
> FFI, platform independent and performant intermediate representation, extra 
> optimization passes for "free"). Moreover, this path can lead to completely 
> independent compilers that could then be free to expose their lexers, 
> parsers, type checkers and metaprogramming capabilities without licensing 
> issues.

I quite agree that a move towards LLVM would be an interesting
experiment.  Whether or not it proves useful remains to be seen. I'm
planning to put a few graduate students on writing a Caml bytecode =>
LLVM compiler later this year and if things go nicely on a whole OCaml
=> LLVM compiler next year. I believe that a Summer of Code on the
subject would be quite profitable (although I can't personally handle it
next year).

Cheers,
 David

-- 
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act brings liquidations. 

Re: [Caml-list] Metaprogramming features

[Stefano Zacchiroli]

On Fri, Oct 03, 2008 at 10:34:03AM -0400, Jacques Carette wrote:
> Please add your voice to the chorus!

No, please don't. It is pointless enough to have tons of AOL messages
in the bug log, we really don't want the same on list.

Sorry if I'm being too frank, but I don't see the usefulness of your
post. I would love to see basic meta-programming feature in vanilla
Caml, but the way to make it happen is for sure not lobbying / making
noise.  The way is provide patches which implement the requested
feature.

Cheers.

-- 
Stefano Zacchiroli -*- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
I'm still an SGML person,this newfangled /\ All one has to do is hit the
XML stuff is so ... simplistic  -- Manoj \/ right keys at the right time

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Saturday 04 October 2008 01:49:08 Stefano Zacchiroli wrote:
> Sorry if I'm being too frank, but I don't see the usefulness of your
> post. I would love to see basic meta-programming feature in vanilla
> Caml, but the way to make it happen is for sure not lobbying / making
> noise.  The way is provide patches which implement the requested
> feature.

If try..finally will not be accepted what makes you think that a MetaOCaml 
patch would be accepted?

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[Richard Jones]

On Sat, Oct 04, 2008 at 03:03:04AM +0100, Jon Harrop wrote:
> On Saturday 04 October 2008 01:49:08 Stefano Zacchiroli wrote:
> > Sorry if I'm being too frank, but I don't see the usefulness of your
> > post. I would love to see basic meta-programming feature in vanilla
> > Caml, but the way to make it happen is for sure not lobbying / making
> > noise.  The way is provide patches which implement the requested
> > feature.
> 
> If try..finally will not be accepted what makes you think that a MetaOCaml 
> patch would be accepted?

Where did you post the working compiler patch that implements
try/finally?  Where did you nurse the patch through many iterations,
as the language designers asked you to fix one thing and another,
before the final patch was rejected?

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Saturday 04 October 2008 09:23:46 Richard Jones wrote:
> On Sat, Oct 04, 2008 at 03:03:04AM +0100, Jon Harrop wrote:
> > On Saturday 04 October 2008 01:49:08 Stefano Zacchiroli wrote:
> > > Sorry if I'm being too frank, but I don't see the usefulness of your
> > > post. I would love to see basic meta-programming feature in vanilla
> > > Caml, but the way to make it happen is for sure not lobbying / making
> > > noise.  The way is provide patches which implement the requested
> > > feature.
> >
> > If try..finally will not be accepted what makes you think that a
> > MetaOCaml patch would be accepted?
>
> Where did you post the working compiler patch that implements
> try/finally?

I asked if it would be worth doing so before I even attempted it and was told 
that it would not be worth attempting by Pierre Weis. Xavier Leroy told me 
that copyright issues in French law essentially prohibit contributions from 
non-French programmers.

> Where did you nurse the patch through many iterations, 
> as the language designers asked you to fix one thing and another,
> before the final patch was rejected?

I would like to think that the OCaml community has try..finally pinned down 
now. It is the first example on every Camlp4 tutorial after all...

I'm not saying that there is anything wrong with having a language 
implementation written by language researchers for language research but 
almost all users would benefit enormously from a variety of simple 
improvements that the community could easily implement themselves were it 
feasible to get changes absorbed upstream more quickly. Now that OCaml is 
gaining traction in industry there are also a growing number of people 
willing to throw money around to get improvements made and we could all be 
benefitting from that.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[Richard Jones]

On Sat, Oct 04, 2008 at 03:31:18PM +0100, Jon Harrop wrote:
> I asked if it would be worth doing so before I even attempted it and was told 
> that it would not be worth attempting by Pierre Weis. Xavier Leroy told me 
> that copyright issues in French law essentially prohibit contributions from 
> non-French programmers.

I'm collaborating at the moment with a couple of French programmers on
a [non-OCaml] FSF project, so this seems unlikely to me (and yes, both
the FSF and Red Hat provide lawyers to check this sort of thing out).

> > Where did you nurse the patch through many iterations, 
> > as the language designers asked you to fix one thing and another,
> > before the final patch was rejected?
> 
> I would like to think that the OCaml community has try..finally pinned down 
> now. It is the first example on every Camlp4 tutorial after all...

Std.finally in extlib?  I've used it precisely twice, ever, so I'm not
convinced that try/finally is the one thing missing from OCaml
that would propel it to mass adoption.

> I'm not saying that there is anything wrong with having a language 
> implementation written by language researchers for language research but 
> almost all users would benefit enormously from a variety of simple 
> improvements that the community could easily implement themselves were it 
> feasible to get changes absorbed upstream more quickly. Now that OCaml is 
> gaining traction in industry there are also a growing number of people 
> willing to throw money around to get improvements made and we could all be 
> benefitting from that.

You still don't get the whole open source thing though.  You need to
make the patches yourself, or get FF Consultancy to pay someone to
develop them.  Deep compiler changes don't just happen because someone
demands them.  Whether we're talking about INRIA's busy researchers or
volunteers maintaining Perl & Ruby in their spare time, they don't
have time to just implement stuff because people shout loudly.
Instead they review patches submitted through the standard bug
tracking system.

Despite your claims, there are lots of features currently being
tracked through mantis.  I've submitted a couple of minor ones myself,
both of which will appear in 3.11.  But I didn't just demand they
happen - for both of them I submitted patches, listened to feedback,
modified things ...

  http://caml.inria.fr/mantis/
  http://et.redhat.com/~rjones/how-to-supply-code-to-open-source-projects/

So here's an idea:

(1) Start a project to list all the things you think are missing from
the compiler or language.  I'd join this project, because I've got a
few things of my own.

(2) Enumerate them on a wiki or editable web page somewhere, ask
others to join, argue about the features you think are priorities.

(3) WRITE PATCHES to implement them in the CVS version of the OCaml
compiler (or library, or wherever they need to be implemented).

(4) Submit the patches, and detailed rationales for the changes,
through the bugtracker.

(5) Listen to feedback, modify your patches as appropriate.  Some will
get rejected, some accepted straightaway, most will need to go through
many iterations.

When you've done all the above, in about a year I would say, and
nothing has been accepted, then you will be in a position to make wild
claims about OCaml upstream being too slow for your liking.  And you
can fork the project to make FlyingCaml++ the way you want it (ain't
open source great like that? -- try forking F# some time and find out
how hard the lawyers come down on you).

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Saturday 04 October 2008 14:57:16 Richard Jones wrote:
> On Sat, Oct 04, 2008 at 03:31:18PM +0100, Jon Harrop wrote:
> > I asked if it would be worth doing so before I even attempted it and was
> > told that it would not be worth attempting by Pierre Weis. Xavier Leroy
> > told me that copyright issues in French law essentially prohibit
> > contributions from non-French programmers.
>
> I'm collaborating at the moment with a couple of French programmers on
> a [non-OCaml] FSF project, so this seems unlikely to me (and yes, both
> the FSF and Red Hat provide lawyers to check this sort of thing out).

Did you transfer copyrights?

> > > Where did you nurse the patch through many iterations,
> > > as the language designers asked you to fix one thing and another,
> > > before the final patch was rejected?
> >
> > I would like to think that the OCaml community has try..finally pinned
> > down now. It is the first example on every Camlp4 tutorial after all...
>
> Std.finally in extlib?

Sure, many of OCaml's standard libraries contains a reimplementation.

> I've used it precisely twice, ever, so I'm not convinced that try/finally is
> the one thing missing from OCaml that would propel it to mass adoption.

Indeed, OCaml has many such issues that the community could easily fix and 
that would help adoption.

> > I'm not saying that there is anything wrong with having a language
> > implementation written by language researchers for language research but
> > almost all users would benefit enormously from a variety of simple
> > improvements that the community could easily implement themselves were it
> > feasible to get changes absorbed upstream more quickly. Now that OCaml is
> > gaining traction in industry there are also a growing number of people
> > willing to throw money around to get improvements made and we could all
> > be benefitting from that.
>
> You still don't get the whole open source thing though.  You need to
> make the patches yourself, or get FF Consultancy to pay someone to
> develop them.  Deep compiler changes don't just happen because someone
> demands them.  Whether we're talking about INRIA's busy researchers or
> volunteers maintaining Perl & Ruby in their spare time, they don't
> have time to just implement stuff because people shout loudly.
> Instead they review patches submitted through the standard bug
> tracking system.
>
> Despite your claims, there are lots of features currently being
> tracked through mantis.  I've submitted a couple of minor ones myself,
> both of which will appear in 3.11.  But I didn't just demand they
> happen - for both of them I submitted patches, listened to feedback,
> modified things ...
>
>   http://caml.inria.fr/mantis/
>   http://et.redhat.com/~rjones/how-to-supply-code-to-open-source-projects/
>
> So here's an idea:
>
> (1) Start a project to list all the things you think are missing from
> the compiler or language.  I'd join this project, because I've got a
> few things of my own.
>
> (2) Enumerate them on a wiki or editable web page somewhere, ask
> others to join, argue about the features you think are priorities.
>
> (3) WRITE PATCHES to implement them in the CVS version of the OCaml
> compiler (or library, or wherever they need to be implemented).
>
> (4) Submit the patches, and detailed rationales for the changes,
> through the bugtracker.
>
> (5) Listen to feedback, modify your patches as appropriate.  Some will
> get rejected, some accepted straightaway, most will need to go through
> many iterations.
>
> When you've done all the above, in about a year I would say, and
> nothing has been accepted, then you will be in a position to make wild
> claims about OCaml upstream being too slow for your liking.

I submitted the following trivial fix over a year ago:

  http://caml.inria.fr/mantis/view.php?id=4338

It was completely reimplemented by Xavier (thus avoiding copyright issues) and 
will be available in 3.11 well over a year after I fixed the problem.

In fact, I'm not sure there would have been any point in creating a patch 
rather than posting code given that the code gets reimplemented anyway.

I believe we can improve upon having Xavier reimplement trivial fixes himself 
and users having to wait for a year before they can benefit from such fixes.

> And you can fork the project to make FlyingCaml++ the way you want it (ain't
> open source great like that? -- try forking F# some time and find out how
> hard the lawyers come down on you). 

That will not help merge parts of MetaOCaml (which is already a fork of OCaml) 
into upstream OCaml.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] Metaprogramming features

[Richard Jones]

On Sat, Oct 04, 2008 at 08:41:35PM +0100, Jon Harrop wrote:
> I submitted the following trivial fix over a year ago:
> 
>   http://caml.inria.fr/mantis/view.php?id=4338

This makes the mistake of supplying a lump of code instead of a patch.
It's a very common mistake, so don't feel bad about it.  In fact it
happens so much with libvirt that I cowrote the following section to
cover it:

http://et.redhat.com/~rjones/how-to-supply-code-to-open-source-projects/#patches

Rich.

ObComment: People at Red Hat deal with patches almost as a form of
currency.  Typically I'll send and receive a dozen patches a day, and
the kernel developers probably deal with ten times as many.  The
interesting thing is that the actual diff / patch format is very
crude.  It's crying out for someone to invent a tool which actually
understands the syntax of the code in the files (not just as plain
text) and creates a more robust and easier to read format.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Metaprogramming features

[Jon Harrop]

On Saturday 04 October 2008 20:04:16 Richard Jones wrote:
> On Sat, Oct 04, 2008 at 08:41:35PM +0100, Jon Harrop wrote:
> > I submitted the following trivial fix over a year ago:
> >
> >   http://caml.inria.fr/mantis/view.php?id=4338
>
> This makes the mistake of supplying a lump of code instead of a patch.
> It's a very common mistake, so don't feel bad about it.

I don't think it is a mistake in this context because my patch could not have 
been applied.

> In fact it 
> happens so much with libvirt that I cowrote the following section to
> cover it:
>
> http://et.redhat.com/~rjones/how-to-supply-code-to-open-source-projects/#pa
>tches
>
> Rich.
>
> ObComment: People at Red Hat deal with patches almost as a form of
> currency.  Typically I'll send and receive a dozen patches a day, and
> the kernel developers probably deal with ten times as many.  The
> interesting thing is that the actual diff / patch format is very
> crude.  It's crying out for someone to invent a tool which actually
> understands the syntax of the code in the files (not just as plain
> text) and creates a more robust and easier to read format.

Suppling a patch certainly makes perfect sense when the maintainer can just 
apply the patch upstream but, AFAIK, that is not the case here.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: Metaprogramming features

[Chung-chieh Shan]

Richard Jones <rich@annexia.org> wrote in article <20081004190416.GA6466@annexia.org> in gmane.comp.lang.caml.inria:
> It's crying out for someone to invent a tool which actually
> understands the syntax of the code in the files (not just as plain
> text) and creates a more robust and easier to read format.

Here's some related work:
http://www.emn.fr/x-info/coccinelle/#sp

-- 
Edit this signature at http://www.digitas.harvard.edu/cgi-bin/ken/sig
2008-10-01 International Day of Older Persons http://www.who.int/ageing/
2008-11-20 Universal Children's Day http://www.unicef.org/
2008-11-25 Day for the Elimination of Violence Against Women http://unifem.org/
1948-12-10 Universal Declaration of Human Rights http://everyhumanhasrights.org