[Caml-list] novice puzzled by speed tests

[Caml-list] novice puzzled by speed tests

[dolfi]

Hi everybody,

I'm new to the list, and deeply impressed by the stability and speed of
OCaml, as well as its functional nature. My best congratulations to the
authors!

Toying around with 3.07, I found that ocamlopt.opt -unsafe (on Mandrake
9.1, Pentium 4, 2.4 GHz) actually produces slower code than ocamlopt.opt.
The program in question is a prime generator:

let count = 400000

let ptab =
  let ptab = Array.create count 0 in
  ( ptab.(0)<- 2; ptab.(1) <- 3; ptab.(2) <- 5; ptab );;

let rec loop nr toggle psize =
(   let max = truncate (sqrt (float nr)) in
    let rec floop i =
      if ptab.(i)>max then
        ( ptab.(psize) <- nr;
          loop (nr+toggle) (6-toggle) (succ psize) )
      else if nr mod ptab.(i) = 0 then
        loop (nr+toggle) (6-toggle) psize
      else
        floop (succ i)
    in if psize<count then floop 2 else ()
) in loop 5 2 3;

Printf.printf "prime %d: %d\n" count ptab.(pred count);;

On my box, the corresponding C program (gcc -O3) is slightly slower than
the ocamlopt.opt compiled O'Caml program, but about 25-30% faster than the
-unsafe one:

#include <stdio.h>
#include <math.h>

#define how_many 400000

int main()
{   unsigned int nr = 5, toggle = 2, max, primes_size = 3, i;
    unsigned int primes[how_many];

    primes[0] = 2; primes[1] = 3; primes[2] = 5;
  loop:
    nr += toggle; toggle = 6 - toggle;
    max = sqrt(nr);
    for (i = 2; primes[i] <= max; ++i)
        if (!(nr % primes[i])) goto loop;
    primes[primes_size++] = nr;
    if (primes_size < how_many) goto loop;

    printf("%i\n", primes[how_many - 1]);
    return 0;
}

Of course it's good that range checking increases the speed of programs,
but, being a long-time C user, I'm a little bit puzzled by miracles like
this. I suspected that the sense of the -unsafe flag was inverted, but it
isn't: the -unsafe program dies with SEGV when I deliberately introduce a
range overflow, the safe one gets an exception.

Till soon, Dolfi




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] novice puzzled by speed tests

[Xavier Leroy]

> Toying around with 3.07, I found that ocamlopt.opt -unsafe (on Mandrake
> 9.1, Pentium 4, 2.4 GHz) actually produces slower code than ocamlopt.opt.
> On my box, the corresponding C program (gcc -O3) is slightly slower than
> the ocamlopt.opt compiled O'Caml program, but about 25-30% faster than the
> -unsafe one:
> Of course it's good that range checking increases the speed of programs,
> but, being a long-time C user, I'm a little bit puzzled by miracles like
> this. I suspected that the sense of the -unsafe flag was inverted, but it
> isn't: the -unsafe program dies with SEGV when I deliberately introduce a
> range overflow, the safe one gets an exception.

Welcome to the wonderful world of modern processors.  It's not
uncommon to observe "absurd" speed differences of up to 20%.  By
"absurd" I mean for instance adding or removing dead code (never
executed) and observing a slowdown, or executing more code and
observing a speedup.

As far as I can guess, this is due to two processor features:

- Lots of instruction-level parallelism is available.  Thus, if your
main code doesn't use all of the computational units, adding extra code
(such as array bound checks) that can execute in parallel doesn't
reduce execution speed.

- Performance is very sensitive to code placement.  Things like code
cache conflicts, (mis-) alignment of branch targets, and oddities in the
instruction decoding logic can cause insertion *or deletion* of a few
instructions to have significant impact on execution speed.

These are just wild guesses.  The bottom line is that processors have
become so complex that explaining observed performances (let alone
predicting performances) has become nearly impossible, at least for
small performance variations (say, less than a factor of 1.5).  

(This makes compiler writers very unhappy, because they used to make a
living by cranking out 5% speed improvements, which are now lost in
the overall noise :-)

If you have access to a good performance analysis tool, such as
Intel's VTune, you could run it on your three executables and see if
some rational explanation comes out of VTune's figures.  But I
wouldn't bet on it.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] novice puzzled by speed tests

[Brian Hurt]

On Sun, 4 Jan 2004, Xavier Leroy wrote:

> > Toying around with 3.07, I found that ocamlopt.opt -unsafe (on Mandrake
> > 9.1, Pentium 4, 2.4 GHz) actually produces slower code than ocamlopt.opt.
> > On my box, the corresponding C program (gcc -O3) is slightly slower than
> > the ocamlopt.opt compiled O'Caml program, but about 25-30% faster than the
> > -unsafe one:
> > Of course it's good that range checking increases the speed of programs,
> > but, being a long-time C user, I'm a little bit puzzled by miracles like
> > this. I suspected that the sense of the -unsafe flag was inverted, but it
> > isn't: the -unsafe program dies with SEGV when I deliberately introduce a
> > range overflow, the safe one gets an exception.
> 
> Welcome to the wonderful world of modern processors.  It's not
> uncommon to observe "absurd" speed differences of up to 20%.  By
> "absurd" I mean for instance adding or removing dead code (never
> executed) and observing a slowdown, or executing more code and
> observing a speedup.
> 
> As far as I can guess, this is due to two processor features:
> 
> - Lots of instruction-level parallelism is available.  Thus, if your
> main code doesn't use all of the computational units, adding extra code
> (such as array bound checks) that can execute in parallel doesn't
> reduce execution speed.

Modern processors also do fairly aggressive branch prediction.  Bounds 
checking doesn't *normally* fail, so the processor very quickly starts 
predicting the branch with a very high degree of accuracy.  A correctly 
predicted branch is not exactly no cost, but very near to it.  A 
mispredicted branch is expensive- dozens of clock cycles.  

Hmm.  Actually, as I understand it, all array accesses call the same code
in Ocaml- that code then does the bounds checking.  This means that every
ocaml program has only one branch instruction which is the bounds checking
branch.  Thinking about it, in modern processors this may be better than
hoisting the bounds checking- even in situations where the most bounds
checking can be eliminated in hoisting.  This is because processors only
keep/"cache" information for branch prediction for so many branches-
generally about 4K branches IIRC.  So, program A doesn't hoist it's branch
checking, and branch checking uses the exact same branch.  This means that
only one entry in the branch history cache is used by bounds checking.  
But it executes that branch a lot.  Program B does hoist it's branch
checking- now there are a thousand different branch instructions in the
program to do branch checking.  Now 25% of the branch history cache is for 
bounds checking branches- all of which should be trivially predictable, 
but are pushing other branches out of the branch history.  Boom, the code 
runs slower.

This isn't what this programmer is hitting.  But it demonstrates that 
performance optimization is trickier than it might seem.

> 
> - Performance is very sensitive to code placement.  Things like code
> cache conflicts, (mis-) alignment of branch targets, and oddities in the
> instruction decoding logic can cause insertion *or deletion* of a few
> instructions to have significant impact on execution speed.

Cache conflicts especially.  Modern processors have 100-350 or more clock 
cycle latencies to go to main memory.  Very expensive- to the point where 
I'd argue that the run time of a program is mainly determined by the 
number of cache misses the program has, and not the number of instructions 
executed.  If I had to guess, he's hitting some weird cache behavior.  
Generally, instruction decode weirdness, branch target alignment, etc. 
only cost dozens of clock cycles and don't make that big of a change on 
the performance of code.  It's cache effects that can make surprisingly 
big swings.

You can have poor cache behavior even with working sets much smaller than
cache, due to the way cache works.  Say you have a 2-way set associative
cache of size C.  This means if you're heavily accessing locations X and
X+C, you're okay- but if you're heavily accessing locations X, X+C, and
X+2C, you're screwed.  Your cache can only hold two of those in memory at
once- accessing the third one pushes one of the original two out of cache.  
You make a "minor" change, and suddenly you're accessing locations X, X+C,
and X+2C+L (where L is the cache line size- generally 16-64 bytes), and
boom- all three address lines fit into cache now.  And suddenly your code
is 100 times faster.  This can happen with both data and code.  I've seen 
this happen in real code.

Here's an interesting paper for two reasons- first, it shows just how 
expensive minor changes can get due to cache effects, and second of all it 
has a solution (skew associative caches) that I'd like to see implemented, 
but it has a NIH problem:
http://citeseer.nj.nec.com/bodin94cache.html

Thinking about it, I'd love to see Ocaml do, as it's final linking stage, 
just a quick depth-first traversal sorting of the known-call graph of the 
functions.  I'd be this would do good things to Ocaml's performance.

> 
> These are just wild guesses.  The bottom line is that processors have
> become so complex that explaining observed performances (let alone
> predicting performances) has become nearly impossible, at least for
> small performance variations (say, less than a factor of 1.5).  

Easily.  I've seen performance swings of 100x due to simple changes.

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] camlp4

[Ker Lutyn]

Happy New Year!

I just tried to get camlp4 working from cvs:

- configure looks for exact equality between the ocaml version and the version
in camlp4/pcaml.ml

- camlp4/argl.ml does not handle Arg.Tuple and Arg.Bool

Since I wanted camlp4 for a quick one-off and I am under deadline, I can't look
into it more deeply. Let me just add my voice to those others who think it is
too bad that camlp4 has fallen off the map.

Just FYI, here's a cool camlp4 link:

http://www.venge.net/graydon/talks/mkc/html/index.html


__________________________________
Do you Yahoo!?
Find out what made the Top Yahoo! Searches of 2003
http://search.yahoo.com/top2003

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

camlp4

[Andy Ray]

Hi,

My project would really benefit from some simple camlp4 syntax
extensions, however, I am put off by the lack of a reference manual
for it.

At the moment I am tempted to go for camlp5 instead - not least
because I was able to work through it's manual and get some examples
working a while back.

The reality is I would prefer to use camlp4 as it appears to the
official ocaml supported way, but can't see how to get into it as a
beginner due to the lack of documentation (and that appears to have
been the case for quite some time now).  What should one do?

Cheers,
Andy

Re: [Caml-list] camlp4

[blue storm]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

- There is some documentation there :
http://brion.inria.fr/gallium/index.php/Camlp4
- If what you want is really simple, you can probably hack it from one of
the camlp4 extensions around
- If what you want is really simple, you could describe it; I know Camlp4
and would be glad to contribute something easy

[-- Attachment #2: Type: text/html, Size: 381 bytes --]

Re: [Caml-list] camlp4

[Tiphaine Turpin]

Andy Ray a écrit :
> What should one do?
>   
Complaining on the list about lack of camlp4 documentation is a good
start :-). To help you decide (and increase the pressure on the Ocaml
team), I would like to mention some of the difficulties that I
encountered when trying to understand camlp4 as a user:
- no overview of the organisation of the code (and no API either)
- the source folder tree is deep, and thus harder to grep
- the heavy use of functors obfuscates the dependencies (This is not a
remark against the design choice, but on readability in the absence of
documentation). Once you have found the implementation or interface of a
function that you need, you still have to find the path to the
appropriate module in PreCast which has it.
- a lot of .cmo and .cma, but what are their repective purposes ? This
is addressed in the wiki for the syntax extension part, but linking a
standalone program which uses camlp4 (even PreCast) to parse source
files is not so easy.

That being said, whith the wiki (whose structure is terrible) and a few
other google-suggested sources, you will get most of the informations
needed for the simplest use (syntax extension). Still, for a few grammar
entries, antiquotation names, and all basic manipulation functions
(locations, idents, etc.), you are on your own.

Two more links:
http://nicolaspouillard.fr/camlp4-changes.html
http://www.ocaml-tutorial.org/camlp4_3.10

Tiphaine

Re: [Caml-list] camlp4

[Guillaume Yziquel]

Tiphaine Turpin a écrit :
> Andy Ray a écrit :
>> What should one do?
>>   
> Complaining on the list about lack of camlp4 documentation is a good
> start :-). To help you decide (and increase the pressure on the Ocaml
> team), I would like to mention some of the difficulties that I
> encountered when trying to understand camlp4 as a user:

I fully agree. I've been working on understanding Camlp4 recently to 
handle one of Mauricio Fernandez' library on relational algebras.

You can, eventually, if you take time, get to understan how everything 
works out. But that involves a lot of trial and error, and I still 
believe I do not have the full picture in mind.

I'd just keep two things in mind concerning camlp4:

-1- It's not hugely different from Camlp5. It is different, but not 
hugely, at least from the newcommer's point of view. So my advice is 
simply to work on an existing camlp4 extension, and to try groking with 
existing camlp5 documentation, while doing trials and errors the whole 
way long. It is painful, but it's doable.

-2- A good hands-on introduction, to be read in conjunction with 
up-to-date camlp4 code:

	http://martin.jambon.free.fr/extend-ocaml-syntax.html

Now, if someone, or a group of people, has the courage to update 
Martin's page to the current camlp4, that would be hugely helpful: I'm 
pretty sure there's quite a lot of people willing to use camlp4 that are 
simply laid back by the lack of documentation.

Concerning the functorial design of the source code of camlp4, I fully 
agree with Tiphaine. It is probably a good design choice, and when you 
get to study it closely, it is quite remarkably done. However, a *huge* 
pain to read and to navigate through.

All the best,

-- 
      Guillaume Yziquel
http://yziquel.homelinux.org/

Re: [Caml-list] camlp4

[Guillaume Yziquel]

Guillaume Yziquel a écrit :
> Tiphaine Turpin a écrit :
>> Andy Ray a écrit :
>>> What should one do?
>>>   
>> Complaining on the list about lack of camlp4 documentation is a good
>> start :-). To help you decide (and increase the pressure on the Ocaml
>> team), I would like to mention some of the difficulties that I
>> encountered when trying to understand camlp4 as a user:
> 
> -2- A good hands-on introduction, to be read in conjunction with 
> up-to-date camlp4 code:
> 
>     http://martin.jambon.free.fr/extend-ocaml-syntax.html

http://ambassadortothecomputers.blogspot.com/search/label/camlp4

is also a good link, covering the essentials to get started with camlp4, 
I think.


-- 
      Guillaume Yziquel
http://yziquel.homelinux.org/

Re: [Caml-list] camlp4

[Jake Donham]

On Tue, Feb 9, 2010 at 7:30 AM, Guillaume Yziquel
<guillaume.yziquel@citycable.ch> wrote:
> http://ambassadortothecomputers.blogspot.com/search/label/camlp4
>
> is also a good link, covering the essentials to get started with camlp4, I
> think.

Thanks for the plug; the series is sadly incomplete but I hope to get
back to it very soon.

Jake

Re: [Caml-list] camlp4

[Martin DeMello]

On Sat, Feb 6, 2010 at 5:44 PM, Tiphaine Turpin
<Tiphaine.Turpin@irisa.fr> wrote:
> - the source folder tree is deep, and thus harder to grep

tangentially, i've found that a great way to deal with that is to
import the whole thing into a local git repository and then use git
grep. works like a charm, and is *fast*.

martin

Re: [Caml-list] camlp4

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 845 bytes --]

> - the source folder tree is deep, and thus harder to grep

I use find.

$ find ./ -exec grep "foo" {} \;



On Sun, Feb 7, 2010 at 12:19 PM, Martin DeMello <martindemello@gmail.com>wrote:

> On Sat, Feb 6, 2010 at 5:44 PM, Tiphaine Turpin
> <Tiphaine.Turpin@irisa.fr> wrote:
> > - the source folder tree is deep, and thus harder to grep
>
> tangentially, i've found that a great way to deal with that is to
> import the whole thing into a local git repository and then use git
> grep. works like a charm, and is *fast*.
>
> martin
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 1682 bytes --]

Re: [Caml-list] camlp4

[Yoann Padioleau]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]


On Feb 7, 2010, at 5:14 PM, Ashish Agarwal wrote:

> > - the source folder tree is deep, and thus harder to grep
> 
> I use find.
> 
> $ find ./ -exec grep "foo" {} \;


You can also use a shell like zsh or eshell and do

grep foo **/*.ml*


When you use eshell  the grep result is even clickable.


> 
> 
> 
> On Sun, Feb 7, 2010 at 12:19 PM, Martin DeMello <martindemello@gmail.com> wrote:
> On Sat, Feb 6, 2010 at 5:44 PM, Tiphaine Turpin
> <Tiphaine.Turpin@irisa.fr> wrote:
> > - the source folder tree is deep, and thus harder to grep
> 
> tangentially, i've found that a great way to deal with that is to
> import the whole thing into a local git repository and then use git
> grep. works like a charm, and is *fast*.
> 
> martin
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


[-- Attachment #2: Type: text/html, Size: 2765 bytes --]

Re: [Caml-list] camlp4

[Erik de Castro Lopo]

Ashish Agarwal wrote:

> > - the source folder tree is deep, and thus harder to grep
> 
> I use find.
> 
> $ find ./ -exec grep "foo" {} \;

If you're using GNU grep:

   grep -r foo .

Erik
-- 
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/

Re: [Caml-list] camlp4

[Ed Keith]

--- On Fri, 2/5/10, Andy Ray <evilkidder@googlemail.com> wrote:

> From: Andy Ray <evilkidder@googlemail.com>
> Subject: [Caml-list] camlp4
> To: caml-list@inria.fr
> Date: Friday, February 5, 2010, 8:16 PM
> Hi,
> 
> My project would really benefit from some simple camlp4
> syntax
> extensions, however, I am put off by the lack of a
> reference manual
> for it.
> 
> At the moment I am tempted to go for camlp5 instead - not
> least
> because I was able to work through it's manual and get some
> examples
> working a while back.
> 
> The reality is I would prefer to use camlp4 as it appears
> to the
> official ocaml supported way, but can't see how to get into
> it as a
> beginner due to the lack of documentation (and that appears
> to have
> been the case for quite some time now).  What should
> one do?
> 


I wrestled with this myself and finally decided to stick with camlp5 until there is documentation available for camlp4.

It appears to me that they are similar enough that skills with camlp5 will be transferable to camlp4 when documents becomes available.

    -EdK

Ed Keith
e_d_k@yahoo.com

Blog: edkeith.blogspot.com

Re: [Caml-list] camlp4

[Joseph Young]

On Sat, 6 Feb 2010, Andy Ray wrote:

> Hi,
>
> My project would really benefit from some simple camlp4 syntax
> extensions, however, I am put off by the lack of a reference manual
> for it.
>
> At the moment I am tempted to go for camlp5 instead - not least
> because I was able to work through it's manual and get some examples
> working a while back.
>
> The reality is I would prefer to use camlp4 as it appears to the
> official ocaml supported way, but can't see how to get into it as a
> beginner due to the lack of documentation (and that appears to have
> been the case for quite some time now).  What should one do?
>
> Cheers,
> Andy
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

 	Though slightly tangental, I use both camlp4 and camlp5 for the 
quotation system.  As already mentioned, the lack of documentation is a 
problem, but it is not insurmountable.  At this point, I prefer camlp4 to 
camlp5 for the following reasons.  First, I like how locations are handled 
in camlp4 as opposed to camlp5.  In camlp4, a function registered as a 
quotation accepts a string, a location, and optional location arguments. 
In camlp5, the function accepts only a string and the location information 
is passed in indirectly.  I find this direct handling of locations easier 
to manipulate.  Second, camlp4 allows me to register a quotation for a 
specific kind of ocaml ast (expression, pattern, whatever.)  In camlp5, 
every time I register a quotation, I must supply code for both expressions 
and patterns.  Third, although initially extremely confusing, I now prefer 
the functor organization that camlp4 uses.  The functors make it clear as 
to how the syntax is extended.  For example, the the original OCaml syntax 
is represented as an extension of the revised syntax.  In order to create 
the appropriate original syntax module in camlp4, we use the code:

module CamlSyntax=Camlp4OCamlParser.Make(
 	Camlp4OCamlRevisedParser.Make(
 		Camlp4.PreCast.Syntax));;

Here, we can see that PreCast adds the necessary starting material, the 
revised parser adds the revised syntax, and the original syntax is an 
extension of the revised.  Because of this organization, I find the 
relationship of syntax extensions more clear in camlp4.  In order to make 
this organization more understandable, make sure you use ocamlbrowser and 
not the module M=Stuff trick on the top level.  The nesting of modules in 
camlp4 is huge.

 	In any case, both work extremely well and much of what you should 
use comes down to preference.

camlp4

[Christian Sternagel]

When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
is the resulting code the naive translation, like in,

 [(x, y) | x <- xs, even xs, y <- ys]

=>

 List.flatten (
  List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
 )

or is there an optimization in order to avoid appends and minimize the
number of cons?

cheers

christian

Re: [Caml-list] camlp4

[Nicolas Pouillard]

Excerpts from christian.sternagel's message of Fri Jan 18 18:08:52 +0100 2008:
> When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
> is the resulting code the naive translation, like in,
> 
>  [(x, y) | x <- xs, even xs, y <- ys]
> 
> =>
> 
>  List.flatten (
>   List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
>  )
> 
> or is there an optimization in order to avoid appends and minimize the
> number of cons?

There is only a few very local optimizations.

However  you  can  answer  to your question by asking camlp4 to translate your
code and pretty-print the result.

$ camlp4o -parser Camlp4ListComprehension -str '[(x, y) | x <- xs; even xs; y <- ys]'
List.concat
  (List.map (fun x -> List.map (fun y -> (x, y)) ys)
     (List.filter (fun x -> even xs) xs))

As you can see List.concat is still there.

Note  also  that  the  syntax is to use `;' to separate qualifiers (generators
and filters) instead of `,' as your example.

Best regards,

-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] camlp4

[Olivier Andrieu]

On Jan 18, 2008 6:08 PM, Christian Sternagel
<Christian.Sternagel@uibk.ac.at> wrote:
> When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
> is the resulting code the naive translation, like in,
>
>  [(x, y) | x <- xs, even xs, y <- ys]
>
> =>
>
>  List.flatten (
>   List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
>  )
>
> or is there an optimization in order to avoid appends and minimize the
> number of cons?


FYI, my (old) syntax extension¹ for camlp4 <= 3.09 expands list
comprehensions to a combination of folds:

  [+ (x, y) | x <- xs | when even x | y <- ys]

=>

  List.fold_right
    (fun x __acc__ ->
       if even x then
         List.fold_right (fun y __acc__ -> (x, y) :: __acc__) ys __acc__
       else __acc__)
    xs []

Less cons operations, but it's not tail recursive.

[1] http://oandrieu.nerim.net/ocaml/#pa_compr
-- 
  Olivier

Re: [Caml-list] camlp4

[Nicolas Pouillard]

Excerpts from oandrieu's message of Fri Jan 18 20:30:59 +0100 2008:
> On Jan 18, 2008 6:08 PM, Christian Sternagel
> <Christian.Sternagel@uibk.ac.at> wrote:
> > When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
> > is the resulting code the naive translation, like in,
> >
> >  [(x, y) | x <- xs, even xs, y <- ys]
> >
> > =>
> >
> >  List.flatten (
> >   List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
> >  )
> >
> > or is there an optimization in order to avoid appends and minimize the
> > number of cons?
> 
> 
> FYI, my (old) syntax extension¹ for camlp4 <= 3.09 expands list
> comprehensions to a combination of folds:
> 
>   [+ (x, y) | x <- xs | when even x | y <- ys]
> 
> =>
> 
>   List.fold_right
>     (fun x __acc__ ->
>        if even x then
>          List.fold_right (fun y __acc__ -> (x, y) :: __acc__) ys __acc__
>        else __acc__)
>     xs []
> 
> Less cons operations, but it's not tail recursive.

Hum...  That's  nice and I could accept patches in that direction. However can
you  do  it  without  introducing  variables,  that's  always dangerous to add
variables because we should check that the user is not using the same.

> [1] http://oandrieu.nerim.net/ocaml/#pa_compr

-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] camlp4

[Christian Sternagel]

Quoting Nicolas Pouillard <nicolas.pouillard@gmail.com>:

> Excerpts from oandrieu's message of Fri Jan 18 20:30:59 +0100 2008:
> > On Jan 18, 2008 6:08 PM, Christian Sternagel
> > <Christian.Sternagel@uibk.ac.at> wrote:
> > > When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
> > > is the resulting code the naive translation, like in,
> > >
> > >  [(x, y) | x <- xs, even xs, y <- ys]
> > >
> > > =>
> > >
> > >  List.flatten (
> > >   List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
> > >  )
> > >
> > > or is there an optimization in order to avoid appends and minimize the
> > > number of cons?
> >
> >
> > FYI, my (old) syntax extension¹ for camlp4 <= 3.09 expands list
> > comprehensions to a combination of folds:
> >
> >   [+ (x, y) | x <- xs | when even x | y <- ys]
> >
> > =>
> >
> >   List.fold_right
> >     (fun x __acc__ ->
> >        if even x then
> >          List.fold_right (fun y __acc__ -> (x, y) :: __acc__) ys __acc__
> >        else __acc__)
> >     xs []
> >
> > Less cons operations, but it's not tail recursive.
>
> Hum...  That's  nice and I could accept patches in that direction. However
> can
> you  do  it  without  introducing  variables,  that's  always dangerous to
> add
> variables because we should check that the user is not using the same.
>
> > [1] http://oandrieu.nerim.net/ocaml/#pa_compr
>
> --
> Nicolas Pouillard aka Ertai
>
>
How about the transformation from Chapter 7 of [1] (by Philip Wadler)?
It should be similar to the `pseudo code':

type expr = ...;;
type patt = ...;;
type qualifier = Gen of patt * expr | Filt of expr;;
type compr = (expr * qualifier list);;
let rec expr = function
 | ...
 | (e, qs) -> transform [] (e, qs)
 | ...
and transform l = function
 | (e, []) -> expr e :: expr l
 | (e, Filt f :: qs) -> if expr f then transform l (e, qs) else expr l
 | (e, Gen (p, l1) :: qs) ->
  let rec h = function
   | [] -> expr l
   | u :: us -> (function p -> transform (h us) (e, qs) | _ -> h us) u
  in h (expr l1)
;;

(* where h, u, us are fresh variables not occurring in e, l1, l, or qs *)

Sorry I'm not yet familiar with camlp4 grammar extensions, but of course
above code would make use of them otherwise.

It is stated in [1] that the resulting code is optimal in that it
performs the minimum number of cons operations.

I'm not sure whether this isn't equivalent to oandrieu's code.

And I did ignore the hint that fresh variables make things
complicated :).

cheers

christian

[1] Simon P. Jones, 1987. The implementation of functional programming
languages. Available online at:
http://research.microsoft.com/~simonpj/papers/slpj-book-1987/index.htm

Re: [Caml-list] camlp4

[Nicolas Pouillard]

Excerpts from christian.sternagel's message of Sat Jan 19 16:09:01 +0100 2008:
> Quoting Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> 
> > Excerpts from oandrieu's message of Fri Jan 18 20:30:59 +0100 2008:
> > > On Jan 18, 2008 6:08 PM, Christian Sternagel
> > > <Christian.Sternagel@uibk.ac.at> wrote:
> > > > When using `camlp4o -parser Camlp4ListComprehension' as preprocessor,
> > > > is the resulting code the naive translation, like in,
> > > >
> > > >  [(x, y) | x <- xs, even xs, y <- ys]
> > > >
> > > > =>
> > > >
> > > >  List.flatten (
> > > >   List.map (fun x -> List.map (fun y -> (x, y)) ys) (List.filter even xs)
> > > >  )
> > > >
> > > > or is there an optimization in order to avoid appends and minimize the
> > > > number of cons?
> > >
> > >
> > > FYI, my (old) syntax extension¹ for camlp4 <= 3.09 expands list
> > > comprehensions to a combination of folds:
> > >
> > >   [+ (x, y) | x <- xs | when even x | y <- ys]
> > >
> > > =>
> > >
> > >   List.fold_right
> > >     (fun x __acc__ ->
> > >        if even x then
> > >          List.fold_right (fun y __acc__ -> (x, y) :: __acc__) ys __acc__
> > >        else __acc__)
> > >     xs []
> > >
> > > Less cons operations, but it's not tail recursive.
> >
> > Hum...  That's  nice and I could accept patches in that direction. However
> > can
> > you  do  it  without  introducing  variables,  that's  always dangerous to
> > add
> > variables because we should check that the user is not using the same.
> >
> > > [1] http://oandrieu.nerim.net/ocaml/#pa_compr
> >
> > --
> > Nicolas Pouillard aka Ertai
> >
> >
> How about the transformation from Chapter 7 of [1] (by Philip Wadler)?
> It should be similar to the `pseudo code':
> 
> type expr = ...;;
> type patt = ...;;
> type qualifier = Gen of patt * expr | Filt of expr;;
> type compr = (expr * qualifier list);;
> let rec expr = function
>  | ...
>  | (e, qs) -> transform [] (e, qs)
>  | ...
> and transform l = function
>  | (e, []) -> expr e :: expr l
>  | (e, Filt f :: qs) -> if expr f then transform l (e, qs) else expr l
>  | (e, Gen (p, l1) :: qs) ->
>   let rec h = function
>    | [] -> expr l
>    | u :: us -> (function p -> transform (h us) (e, qs) | _ -> h us) u
>   in h (expr l1)
> ;;
> 
> (* where h, u, us are fresh variables not occurring in e, l1, l, or qs *)
> 
> Sorry I'm not yet familiar with camlp4 grammar extensions, but of course
> above code would make use of them otherwise.

Yes this approach can be integrated with a camlp4 extension.

> It is stated in [1] that the resulting code is optimal in that it
> performs the minimum number of cons operations.

Nice.

> And I did ignore the hint that fresh variables make things
> complicated :).

Yes it can...

Best regards,

-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] camlp4

[Christian Sternagel]

> > How about the transformation from Chapter 7 of [1] (by Philip Wadler)?
> > It should be similar to the `pseudo code':
> > 
> > type expr = ...;;
> > type patt = ...;;
> > type qualifier = Gen of patt * expr | Filt of expr;;
> > type compr = (expr * qualifier list);;
> > let rec expr = function
> >  | ...
> >  | (e, qs) -> transform [] (e, qs)
> >  | ...
> > and transform l = function
> >  | (e, []) -> expr e :: expr l
> >  | (e, Filt f :: qs) -> if expr f then transform l (e, qs) else expr l
> >  | (e, Gen (p, l1) :: qs) ->
> >   let rec h = function
> >    | [] -> expr l
> >    | u :: us -> (function p -> transform (h us) (e, qs) | _ -> h us) u
> >   in h (expr l1)
> > ;;
> > 
> > (* where h, u, us are fresh variables not occurring in e, l1, l, or qs *)
> > 
> > Sorry I'm not yet familiar with camlp4 grammar extensions, but of course
> > above code would make use of them otherwise.
> 
> Yes this approach can be integrated with a camlp4 extension.
> 
> > It is stated in [1] that the resulting code is optimal in that it
> > performs the minimum number of cons operations.
> 
> Nice.
> 
> > And I did ignore the hint that fresh variables make things
> > complicated :).
> 
> Yes it can...
> 
> Best regards,
> 
> -- 
> Nicolas Pouillard aka Ertai
> 
I deduce that there is no standard way of introducing
`fresh' (w.r.t. the abstract syntax tree) variables
within a camlp4 syntax extension? Wouldn't that be nice? =)

Has anybody every implemented a solution to that problem?

cheers

christian

Re: [Caml-list] camlp4

[Nicolas Pouillard]

[-- Attachment #1: Type: multipart/signed, Size: 2257 bytes --]

Re: [Caml-list] camlp4

[Loup Vaillant]

2008/1/22, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> Excerpts from christian.sternagel's message of Tue Jan 22 14:33:55 +0100
> > I deduce that there is no standard way of introducing
> > `fresh' (w.r.t. the abstract syntax tree) variables
> > within a camlp4 syntax extension? Wouldn't that be nice? =)
>
> That  would be nice, but doing it cleanly would require a large amount of work
> and user visible changes.

Hello,

I am personally interested in how we "do it cleanly" (not specifically
for Ocaml), but I didn't found much documentation nor papers on the
subject. Do anyone have some pointers about that?

Thanks,
Loup

Re: [Caml-list] camlp4

[Nicolas Pouillard]

[-- Attachment #1: Type: multipart/signed, Size: 1402 bytes --]

Re: [Caml-list] camlp4

[Christian Sternagel]

I tried to implement the optimizations for list comprehensions
as stated in the book by simon peyton jones (1987). The problem
of using fresh variables I circumvented in an admittedly `ugly' way,
however (which is not fool proof since there could be programmers
that use variable names like __h__0, __h__1, ...).

The Starting point was the file Camlp4ListComprehension.ml from
camlp4.

Any comments are wellcome.

cheers

christian

Here is the code
-------------------------------------------------------------------------
open Camlp4;;

module Id = struct
 let name =    "ListComprehension";;
 let version = "$Id: ListComprehension";;
end

module Make (Syntax : Sig.Camlp4Syntax) = struct

open Sig;;
include Syntax;;

let rec safe_nth n = function
 | [] -> None
 | [(x,_)] -> if n = 1 then Some x else None
 | _ :: l -> safe_nth (n - 1) l
;;

let stream_peek_nth n s = safe_nth n (Stream.npeek n s);;

let lc_generator = Gram.Entry.of_parser "lc_generator" (fun s ->
 let rec skip_patt n = match stream_peek_nth n s with
  | Some (KEYWORD "<-") -> n
  | Some (KEYWORD ("[" | "[<")) -> skip_patt (ignore_upto "]" (n + 1) + 1)
  | Some (KEYWORD "(") -> skip_patt (ignore_upto ")" (n + 1) + 1)
  | Some (KEYWORD "{") -> skip_patt (ignore_upto "}" (n + 1) + 1)
  | Some (KEYWORD ("as" | "::" | ";" | "," | "_"))
  | Some (LIDENT _ | UIDENT _) -> skip_patt (n + 1)
  | Some _ | None -> raise Stream.Failure
 and ignore_upto end_kwd n = match stream_peek_nth n s with
  | Some (KEYWORD prm) when prm = end_kwd -> n 
  | Some (KEYWORD ("[" | "[<")) -> ignore_upto end_kwd (ignore_upto "]" (n + 1) + 1)
  | Some (KEYWORD "(") -> ignore_upto end_kwd (ignore_upto ")" (n + 1) + 1)
  | Some (KEYWORD "{") -> ignore_upto end_kwd (ignore_upto "}" (n + 1) + 1)
  | Some _ -> ignore_upto end_kwd (n + 1)
  | None -> raise Stream.Failure
 in skip_patt 1
);;

let var = ref 0;;
let fresh () = incr var; !var;;

let rec compr _loc e acc = function
 | [] -> <:expr< $e$::$acc$ >>
 | `filter b :: qs -> <:expr< if $b$ then $compr _loc e acc qs$ else $acc$ >>
 | `gen (p, l1) :: qs ->
 let h  = Format.sprintf "__h__%i" (fresh ()) in
 let u  = Format.sprintf "__u__%i" (fresh ()) in
 let us = Format.sprintf "__us__%i" (fresh ()) in
 <:expr<
  let rec $lid:h$ = function
   | [] -> $acc$
   | $lid:u$ :: $lid:us$ -> $if Ast.is_irrefut_patt p then
    <:expr< (fun $p$ -> $compr _loc e <:expr< ($lid:h$ $lid:us$) >> qs$) $lid:u$ >>
   else
    <:expr<
     (function $p$ ->
      $compr _loc e <:expr< ($lid:h$ $lid:us$) >> qs$ | _ -> $lid:h$ $lid:us$) $lid:u$
    >>
   $
  in $lid:h$ $l1$
 >>
 | _ -> <:expr< [] >>
;;

let list_comprehension = Gram.Entry.mk "list_comprehension";;

DELETE_RULE Gram expr: "["; sem_expr_for_list; "]" END;;

EXTEND Gram
 GLOBAL: expr list_comprehension;
 expr: LEVEL "simple" [
  [ "["; e = list_comprehension; "]" -> e]
 ];
 list_comprehension: [
  [ e = expr LEVEL "top"; ";"; mk = sem_expr_for_list ->
   <:expr< $e$::$mk <:expr< [] >>$ >>
  | e = expr LEVEL "top"; ";" -> <:expr< [$e$] >>
  | e = expr LEVEL "top"; "|"; l = LIST1 quantifier SEP ";" ->
   compr _loc e <:expr< [] >> l
  | e = expr LEVEL "top" -> <:expr< [$e$] >> ]
 ];
 quantifier: [
  [ lc_generator; p = patt; "<-"; e = expr LEVEL "top" -> `gen (p, e)
  | e = expr LEVEL "top" -> `filter e ]
 ];
END;;

end

let module M = Register.OCamlSyntaxExtension Id Make in ();;
-------------------------------------------------------------------------
On Tue, Jan 22, 2008 at 02:42:00PM +0100, Nicolas Pouillard wrote:
> Excerpts from christian.sternagel's message of Tue Jan 22 14:33:55 +0100 2008:
> > > > How about the transformation from Chapter 7 of [1] (by Philip Wadler)?
> > > > It should be similar to the `pseudo code':
> > > > 
> > > > type expr = ...;;
> > > > type patt = ...;;
> > > > type qualifier = Gen of patt * expr | Filt of expr;;
> > > > type compr = (expr * qualifier list);;
> > > > let rec expr = function
> > > >  | ...
> > > >  | (e, qs) -> transform [] (e, qs)
> > > >  | ...
> > > > and transform l = function
> > > >  | (e, []) -> expr e :: expr l
> > > >  | (e, Filt f :: qs) -> if expr f then transform l (e, qs) else expr l
> > > >  | (e, Gen (p, l1) :: qs) ->
> > > >   let rec h = function
> > > >    | [] -> expr l
> > > >    | u :: us -> (function p -> transform (h us) (e, qs) | _ -> h us) u
> > > >   in h (expr l1)
> > > > ;;
> > > > 
> > > > (* where h, u, us are fresh variables not occurring in e, l1, l, or qs *)
> > > > 
> > > > Sorry I'm not yet familiar with camlp4 grammar extensions, but of course
> > > > above code would make use of them otherwise.
> > > 
> > > Yes this approach can be integrated with a camlp4 extension.
> > > 
> > > > It is stated in [1] that the resulting code is optimal in that it
> > > > performs the minimum number of cons operations.
> > > 
> > > Nice.
> > > 
> > > > And I did ignore the hint that fresh variables make things
> > > > complicated :).
> > > 
> > > Yes it can...
> > > 
> > > Best regards,
> > > 
> > > -- 
> > > Nicolas Pouillard aka Ertai
> > > 
> > I deduce that there is no standard way of introducing
> > `fresh' (w.r.t. the abstract syntax tree) variables
> > within a camlp4 syntax extension? Wouldn't that be nice? =)
> > 
> 
> That  would be nice, but doing it cleanly would require a large amount of work
> and user visible changes.
> 
> -- 
> Nicolas Pouillard aka Ertai

Re: [Caml-list] camlp4

[Nicolas Pouillard]

[-- Attachment #1: Type: multipart/signed, Size: 6394 bytes --]

Re: [Caml-list] camlp4

[Christian Sternagel]

On Tue, Jan 22, 2008 at 07:20:55PM +0100, Nicolas Pouillard wrote:
> Excerpts from christian.sternagel's message of Tue Jan 22 17:43:20 +0100 2008:
> > I tried to implement the optimizations for list comprehensions
> > as stated in the book by simon peyton jones (1987). The problem
> > of using fresh variables I circumvented in an admittedly `ugly' way,
> > however (which is not fool proof since there could be programmers
> > that use variable names like __h__0, __h__1, ...).
> > 
> > The Starting point was the file Camlp4ListComprehension.ml from
> > camlp4.
> > 
> > Any comments are wellcome.
> 
> Nice  start,  however  can you make it more closer to the original code (as in
Yes. Here it is. The `fresh variable' problem, however,
is not really solved. Maybe, using some dedicated namespace
is indeed the easiest solution (hence I did that in my code).

cheers

christisn

@Ertai: sorry for the duplicate... the first time I forgot to
reply to the list.
--------------------------------------------------------------------------
open Camlp4;                                             (* -*- camlp4r -*- *)
(****************************************************************************)
(*                                                                          *)
(*                              Objective Caml                              *)
(*                                                                          *)
(*                            INRIA Rocquencourt                            *)
(*                                                                          *)
(*  Copyright  2007  Institut  National  de  Recherche en Informatique et   *)
(*  en Automatique.  All rights reserved.  This file is distributed under   *)
(*  the terms of the GNU Library General Public License, with the special   *)
(*  exception on linking described in LICENSE at the top of the Objective   *)
(*  Caml source tree.                                                       *)
(*                                                                          *)
(****************************************************************************)

(* Authors:
 * - Nao Hirokawa: initial version
 * - Nicolas Pouillard: revised syntax version
 * - Christian Sternagel: optimization of Chapter 7 of
 *   [Simon Peyton Jones, 1987. The Implementation of
 *    Functional Programming Languages]
 *)


module Id = struct
  value name = "Camlp4ListComprenhsion";
  value version = "$Id: Camlp4ListComprehension.ml,v 1.1.2.1 2007/05/27 16:23:35 pouillar Exp $";
end;

module Make (Syntax : Sig.Camlp4Syntax) = struct
  open Sig;
  include Syntax;

  value rec loop n =
    fun
    [ [] -> None
    | [(x, _)] -> if n = 1 then Some x else None
    | [_ :: l] -> loop (n - 1) l ];

  value stream_peek_nth n strm = loop n (Stream.npeek n strm);

  (* usual trick *)
  value test_patt_lessminus =
    Gram.Entry.of_parser "test_patt_lessminus"
      (fun strm ->
        let rec skip_patt n =
          match stream_peek_nth n strm with
          [ Some (KEYWORD "<-") -> n
          | Some (KEYWORD ("[" | "[<")) ->
              skip_patt (ignore_upto "]" (n + 1) + 1)
          | Some (KEYWORD "(") -> 
              skip_patt (ignore_upto ")" (n + 1) + 1)
          | Some (KEYWORD "{") -> 
              skip_patt (ignore_upto "}" (n + 1) + 1)
          | Some (KEYWORD ("as" | "::" | "," | "_"))
          | Some (LIDENT _ | UIDENT _) -> skip_patt (n + 1)
          | Some _ | None -> raise Stream.Failure ]
        and ignore_upto end_kwd n =
          match stream_peek_nth n strm with
          [ Some (KEYWORD prm) when prm = end_kwd -> n 
          | Some (KEYWORD ("[" | "[<")) ->
              ignore_upto end_kwd (ignore_upto "]" (n + 1) + 1)
          | Some (KEYWORD "(") ->
              ignore_upto end_kwd (ignore_upto ")" (n + 1) + 1)
          | Some (KEYWORD "{") -> 
              ignore_upto end_kwd (ignore_upto "}" (n + 1) + 1)
          | Some _ -> ignore_upto end_kwd (n + 1)
          | None -> raise Stream.Failure ]
        in
        skip_patt 1);

  value index = ref 0;
  value var base =
   Format.sprintf
    "__camlp4_list_comprehension__%s__%i"
    base
    (incr index; !index);

  value compr _loc e qs =
   let rec transform _loc e acc =
    fun
    [ [] -> <:expr< [ $e$ :: $acc$ ] >>
    | [`cond b :: qs] ->
     <:expr< if $b$ then $transform _loc e acc qs$ else $acc$ >>
    | [`gen (p, l) :: qs] ->
     let h = var "h" and u = var "u" and us = var "us" in
     <:expr<
      let rec $lid:h$ = fun
      [ [] -> $acc$
      | [ $lid:u$ :: $lid:us$ ] ->
       $if Ast.is_irrefut_patt p then
        <:expr< (fun
         [ $p$ -> $transform _loc e <:expr< ($lid:h$ $lid:us$) >> qs$ ]
        ) $lid:u$
        >>
       else
        <:expr< (fun
         [ $p$ -> $transform _loc e <:expr< ($lid:h$ $lid:us$) >> qs$
         | _ -> $lid:h$ $lid:us$ ]
        ) $lid:us$
        >>
        $
       ]
     in $lid:h$ $l$ 
     >>
    | _ -> <:expr< [] >> ]
   in
   transform _loc e <:expr< [] >> qs;

  DELETE_RULE Gram expr: "["; sem_expr_for_list; "]" END;

  value is_revised =
    try do {
      DELETE_RULE Gram expr: "["; sem_expr_for_list; "::"; expr; "]" END;
      True
    } with [ Not_found -> False ];

  value comprehension_or_sem_expr_for_list =
    Gram.Entry.mk "comprehension_or_sem_expr_for_list";

  EXTEND Gram
    GLOBAL: expr comprehension_or_sem_expr_for_list;

    expr: LEVEL "simple"
      [ [ "["; e = comprehension_or_sem_expr_for_list; "]" -> e ] ]
    ;

    comprehension_or_sem_expr_for_list:
      [ [ e = expr LEVEL "top"; ";"; mk = sem_expr_for_list ->
            <:expr< [ $e$ :: $mk <:expr< [] >>$ ] >>
        | e = expr LEVEL "top"; ";" -> <:expr< [$e$] >>
        | e = expr LEVEL "top"; "|"; l = LIST1 item SEP ";" -> compr _loc e l
        | e = expr LEVEL "top" -> <:expr< [$e$] >> ] ]
    ;

    item: 
      [ [ test_patt_lessminus; 
          p = patt; "<-" ; e = expr LEVEL "top" -> `gen (p, e)
        | e = expr LEVEL "top" -> `cond e ] ]
    ;

  END;

  if is_revised then
    EXTEND Gram
      GLOBAL: expr comprehension_or_sem_expr_for_list;

      comprehension_or_sem_expr_for_list:
      [ [ e = expr LEVEL "top"; ";"; mk = sem_expr_for_list; "::"; last = expr ->
            <:expr< [ $e$ :: $mk last$ ] >>
        | e = expr LEVEL "top"; "::"; last = expr ->
            <:expr< [ $e$ :: $last$ ] >> ] ]
      ;
    END
  else ();

end;

let module M = Register.OCamlSyntaxExtension Id Make in ();
--------------------------------------------------------------------------
> 
> -- 
> Nicolas Pouillard aka Ertai

[Caml-list] -unsafe and camlp4

["Dmitry Bely"]

Can two things live together? When I run ocamlopt -unsafe -pp "camlp4 ..." ..., compiler prints warning and -unsafe seems to be ignored. Am I doing something wrong?

-- Dmitry Bely

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] -unsafe and camlp4

[Xavier Leroy]

> Can two things live together? When I run ocamlopt -unsafe -pp
> "camlp4 ..." ..., compiler prints warning and -unsafe seems to be
> ignored. Am I doing something wrong?

"-unsafe" is a parsing-time option: it governs how the .[] and .()
notations are parsed into applications of functions from the Array and
String modules.  Hence, if you use camlp4 for parsing, you should pass
the -unsafe option to camlp4, not to ocamlc/ocamlopt.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] camlp4

[Dmitry Bely]

Xavier Leroy <xavier.leroy@inria.fr> writes:

>> Can two things live together? When I run ocamlopt -unsafe -pp
>> "camlp4 ..." ..., compiler prints warning and -unsafe seems to be
>> ignored. Am I doing something wrong?
>
> "-unsafe" is a parsing-time option: it governs how the .[] and .()
> notations are parsed into applications of functions from the Array and
> String modules.  Hence, if you use camlp4 for parsing, you should pass
> the -unsafe option to camlp4, not to ocamlc/ocamlopt.

Ah, I see. thank you very much.

There is another problem with camlp4. When it reports syntax error, it
doesn't count the lines, say:

File "test.ml", line 1, characters 12398-12401:
<...>

(line is always equal to 1, at least under Windows)

I compile Ocaml sources within XEmacs/Win32/tuareg mode, and Emacs consider
EOL in the buffer as ONE character, while camlp4 counts it as TWO chars. So
the error place is pointed incorrectly.

How do you think, where it could be fixed? (camlp4, tuareg,...)?

- Dmitry Bely


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] camlp4

[Damien Doligez]

On Tuesday, July 8, 2003, at 05:38 PM, Dmitry Bely wrote:

> There is another problem with camlp4. When it reports syntax error, it
> doesn't count the lines, say:
>
> File "test.ml", line 1, characters 12398-12401:
> <...>
>
> (line is always equal to 1, at least under Windows)
>
> I compile Ocaml sources within XEmacs/Win32/tuareg mode, and Emacs 
> consider
> EOL in the buffer as ONE character, while camlp4 counts it as TWO 
> chars. So
> the error place is pointed incorrectly.
>
> How do you think, where it could be fixed? (camlp4, tuareg,...)?

It needs to be fixed in camlp4. It will need a non-trivial amount
of work, so it will be done only after release 3.07.

In the meantime, you could try to convince emacs that CR LF is really
two characters, not one.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] camlp4

[Pierre CHATEL]

Hello, i'm kinda new to the list, so i hope my newbie questions will 
not disturb everyone :))

So, i was wondering if there were something like Scheme's hygienic 
macros with the Camlp4 preprocessor, or even if it does make sense with 
it.
More generally, i'm looking for some good point-to-point comparison 
between OCaml and Scheme on the macro side...if u know some good 
ressource on that, please point it to me !

Thanks,

Pierre CHATEL

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Camlp4

[Daniel de Rauglaudre]

Hi everybody,

I am in regret to announce that I have stopped developping Camlp4 and
other OCaml applications. I am sure that the Cristal team is going to
find some good programmers able to maintain the Camlp4 part of OCaml
and I wish good luck to the language.

-- 
Daniel de RAUGLAUDRE
http://cristal.inria.fr/~ddr/
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Camlp4

[Issac Trotts]

On Fri, Feb 07, 2003 at 12:11:21PM +0100, Daniel de Rauglaudre wrote:
> Hi everybody,
> 
> I am in regret to announce that I have stopped developping Camlp4 and
> other OCaml applications. I am sure that the Cristal team is going to
> find some good programmers able to maintain the Camlp4 part of OCaml
> and I wish good luck to the language.

Probably my newbie words don't count much, but I value your contribution 
and I wish you would continue to work on Camlp4.  Hopefully sometime you
will come back to it.

Issac Trotts

> -- 
> Daniel de RAUGLAUDRE
> http://cristal.inria.fr/~ddr/
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> 

-- 
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Camlp4

[Geoff Wozniak]

Daniel de Rauglaudre writes:
 > 
 > I am in regret to announce that I have stopped developping Camlp4 and
 > other OCaml applications. I am sure that the Cristal team is going to
 > find some good programmers able to maintain the Camlp4 part of OCaml
 > and I wish good luck to the language.
 > 

Sorry to see you stop development.  I appreciate the work you have done.
Good luck on your future endeavours.

-- 
Geoff(rey) Wozniak, MSc. Candidate
University of Western Ontario
Computer Science Department
London, Ontario, Canada
http://wozniak.ca/

He who knows best knows how little he knows.
                             -- Benjamin Franklin

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Camlp4

[Ohad Rodeh]

List,
   I'm writing to check whether Camlp4 remains or not in the Caml
distribution.

I have a 60000 line system written in Caml, and I was seriously considering
using
Camlp4 for conditional compilation of debugging versions of some of my
modules.

      Ohad.

PS
    While I don't develop Caml, I personally think that:
o Caml needs a preprocessor, at least for conditional compilation
o The preprocessor needs to be in the primary distribution (not as an
add-on)
o Since there is a lot of code written in the old syntax, changing the
syntax is out
   of the question.
o The above point means that a preprocessor is needed to experiment with
new
    syntax without breaking the old syntax.

-----------------------------------------------------------------------------------

Ohad Rodeh
tel: +972-9-9527641
IBM Haifa, storage research


                                                                                                                         
                      Christopher Quinn                                                                                  
                      <cq@htec.demon.co.uk>         To:                                                                  
                      Sent by:                      cc:       caml-list@inria.fr                                         
                      owner-caml-list@pauill        Subject:  [Caml-list] Re: Camlp4/OCaml                               
                      ac.inria.fr                                                                                        
                                                                                                                         
                                                                                                                         
                      17/05/2002 02:22                                                                                   
                      Please respond to                                                                                  
                      Christopher Quinn                                                                                  
                                                                                                                         
                                                                                                                         



How can camlp4 be unimportant or a waste of time!

Here are some things I find useful about it:

o conditional compilation

o ability to add command line options to switch between one's own 'safe'
modules and their 'unsafe' versions which can only be done in the case
of Arrays and Strings with the standard compiler.

o one can always modify one's own copy of the parser,
but that's a bother when something in the original changes, and besides,
camlp4's extension mechanism allows a compact and manageable format.

o no matter how one redefines the syntax, just running the source
through camlp4 can reconstruct it in the vanilla form. So no one ever
needs to suffer another's idea of perfection!

o camlp4 seems to me to be one way to avoid the endless creation of
mini-languages for specific tasks that end up trying to do stuff beyond
their limited design. Better to start with the expressivity of the likes
of caml, maybe cutting down, and augment with p4 extensions. I'm
planning to embed some sql style DDL for my project.

o I can write code without_the_god_damn_awful_underscore character
stressing my hands and use-an-alternative-even
(though-I-am-no-fan-of-lisp)!


I felt these reasons were compelling enough to justify  switching to
camlp4 as my default parser.

The above doubtless has no bearing on the technical arguments for not
having camlp4 in the distribution.
I just want to say I think camlp4 is important and indispensible.

- chris

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives:
http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ:
http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners