Re: [Caml-list] global record

Re: [Caml-list] global record

[Eric Breck]

[-- Attachment #1: Type: text/plain, Size: 3028 bytes --]


> Date: Wed, 19 Jul 2006 15:07:29 +0100
> From: Richard Jones <rich@annexia.org>
> Subject: Re: [Caml-list] global record
> To: Andreas Biegert <andreas.biegert@googlemail.com>
> Cc: caml-list@yquem.inria.fr

>This is a bit ugly, but we use it in our Adwords API toolkit:
>
>-------------------------------------------------- stdargs.mli
>val username : string
>val password : string
>val client : string option
>val token : string
>val update : bool
>val verbose : bool
>val args : string list
>
>-------------------------------------------------- stdargs.ml
>let username = ref ""
>let password = ref ""
>let client = ref ""
>let token = ref ""
>let update = ref false
>let verbose = ref false
>let args = ref []
>
>(* followed by some code which tries to read default values
> * from $HOME/.adwordsapidata -- code omitted
> *)
>
>(* parse the command line parameters *)
>let argspec = [
>  "--username", Arg.Set_string username, "Adwords account username.";
>  "--password", Arg.Set_string password, "Adwords account password.";
>  "--client", Arg.Set_string client, "Adwords account client (optional).";
>  "--token", Arg.Set_string token, "Adwords account token.";
>  "--update", Arg.Set update, "Perform updates.";
>  "--verbose", Arg.Set verbose, "Be verbose.";
>]
>
>let anon_fn str = args := str :: !args
>let usage =
>  Sys.executable_name ^ " [--options]"
>
>let () = Arg.parse argspec anon_fn usage
>
>let username = !username
>let password = !password
>let client = if !client = "" then None else Some !client
>let token = !token
>let update = !update
>let verbose = !verbose
>let args = List.rev !args
>
>---
>
>Then the code just stuff like:
>
>  open Stdargs
>
>  if verbose then printf "this is verbose mode\n"

<gauche_self_promotion>

I was also writing code that looked like that, and I got tired of the
duplication, and also the non-locality - each new option requires a
modification in several places.  So the pa_arg syntax extension I just
announced makes this a lot easier.  Except for the reading of values from the
dotfile, all the above code is generated from this declaration (not tested):

open Parseopt

type option myopts = {
username = "" : help = "Adwords account username"; string;
password = "" : help = "Adwords account password";  string;
client: help = "Adwords account client (optional)"; string;
token = "": help = "Adwords account token"; string;
update = false: help = "Perform updates"; action = set; bool;
verbose = false: help = "Be verbose"; action = set; bool;
}
let myopts = {myopts with keyspecs = [Long2]} in (** specify you want field
foo => --foo *)

let options, args = parse_argv myopts in

...

if options.verbose then print "this is verbose mode!\n"

...

You also get a detailed usage and help message and a function to convert the
returned options to a string, plus further customizability.

http://www.cs.cornell.edu/~ebreck/pa_arg

-E r i c

</gauche_self_promotion>

[-- Attachment #2: Type: text/html, Size: 4376 bytes --]

On language extensions (was Re: [Caml-list] global record)

[Daniel Bünzli]

Hello,

I would like to issue a warning about everybody rolling its own  
syntax extension to suit its taste.

Contrary to lisp's macros, each syntax extension is syntactically a  
new language. Hence not only do we need to learn a new semantics but  
also a new syntax. If I have to read someone else's code I want to  
read caml code and not caml code augmented with the author's  
syntactic obsessions.

In code readability, there is a trade-off between succinctness and  
syntactic regularity. Camlp4 extensions increase the former but  
decrease the latter. For me camlp4's usage should be limited to real  
domain specific languages (e.g. like coq does) and research, it  
should not be used to increase ocaml's succinctness.

This may be seen as a matter of personal taste: it is true that I  
lean towards syntactic regularity, i.e. less syntactic constructs.  
However there is another argument against using extensions: code  
maintenance. You have no guarantee that (1) an extension will not be  
broken by the next version of the language and (2) that the author  
will continue to maintain it.

And by the way, are we guaranteed that two arbitrary camlp4  
extensions will compose or may the order of application matter ?

Cheers,

Daniel

P.S. Even for domain specific languages many things can be done in  
pure ocaml by embedding the dsl in ocaml using meta-programming  
techniques.

Re: On language extensions (was Re: [Caml-list] global record)

[Richard Jones]

On Wed, Jul 19, 2006 at 09:41:46PM +0200, Daniel Bünzli wrote:
> And by the way, are we guaranteed that two arbitrary camlp4  
> extensions will compose or may the order of application matter ?

I'm probably wrong, but I think the only way to compose two camlp4
extensions is to combine the source code to both together and compile
a new extension.  Is there some other way?

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: On language extensions (was Re: [Caml-list] global record)

[Stefano Zacchiroli]

On Wed, Jul 19, 2006 at 08:57:44PM +0100, Richard Jones wrote:
> I'm probably wrong, but I think the only way to compose two camlp4
> extensions is to combine the source code to both together and compile
> a new extension.  Is there some other way?

Camlp4 extensions are regular .cmo/.cma files which, upon loading,
dynamically extends a grammar. You can "compose" multiple extensions by
passing all of them to your preferred camlp4 pre-processor.

Merely loading multiple extensions of course does not grant you obtain a
grammar able to parse anything at all ...

Cheers.

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

Re: On language extensions (was Re: [Caml-list] global record)

[Nicolas Pouillard]

On 7/19/06, Daniel Bünzli <daniel.buenzli@epfl.ch> wrote:
> Hello,
>
> I would like to issue a warning about everybody rolling its own
> syntax extension to suit its taste.
>
> Contrary to lisp's macros, each syntax extension is syntactically a
> new language. Hence not only do we need to learn a new semantics but
> also a new syntax. If I have to read someone else's code I want to
> read caml code and not caml code augmented with the author's
> syntactic obsessions.
>
> In code readability, there is a trade-off between succinctness and
> syntactic regularity. Camlp4 extensions increase the former but
> decrease the latter. For me camlp4's usage should be limited to real
> domain specific languages (e.g. like coq does) and research, it
> should not be used to increase ocaml's succinctness.
>

[...]

Camlp4 is "just" a pre-processor, you always can unsugar your code and
take back a plain ocaml code. This can be done to get rid of a no more
maintained extension, or to understand some code that use such an
extension.

$ camlp4o pa_foo.cmo pa_bar.cmo pr_o.cmo input.ml -o output.ml

You can now read output.ml as a plain ocaml file.

Thus, IMHO, people can add syntax extensions since we have an easy way
to get rid of them.

-- 
Nicolas Pouillard

Re: On language extensions (was Re: [Caml-list] global record)

[Martin Jambon]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 299 bytes --]

On Wed, 19 Jul 2006, Daniel Bünzli wrote:

> Hello,
>
> I would like to issue a warning about everybody rolling its own syntax 
> extension to suit its taste.

Well, it's good to suit your own taste when you write scripts for
yourself :-)


Martin

--
Martin Jambon, PhD
http://martin.jambon.free.fr

Re: On language extensions (was Re: [Caml-list] global record)

[Eric Breck]

> From: Daniel_Bünzli <daniel.buenzli@e...>
> Subject: On language extensions (was Re: [Caml-list] global record)
>
> Contrary to lisp's macros, each syntax extension is syntactically a
> new language. Hence not only do we need to learn a new semantics but
> also a new syntax. If I have to read someone else's code I want to
> read caml code and not caml code augmented with the author's
> syntactic obsessions.
>
> In code readability, there is a trade-off between succinctness and
> syntactic regularity. Camlp4 extensions increase the former but
> decrease the latter. For me camlp4's usage should be limited to real
> domain specific languages (e.g. like coq does) and research, it
> should not be used to increase ocaml's succinctness.

Succinctness is not precisely what I was after so much as locality -  
in this case, having each option declared in a single location - as  
opposed to the type definition in one place, the default value in  
another, the Arg.spec in another, code to print the chosen value of  
each option in another, and, optionally, the de-ref-ication in  
another.  Such locality is a basic principle of software engineering,  
and in this case I don't really know how to achieve it with only a  
library (and not a syntax extension).

I'm not advocating extending the language willy-nilly, but I feel  
that any new library already requires some amount of learning to use  
its particular API, and if a small, local extension to the language  
vastly shrinks (and localizes) code required to interface to the  
library, it seems reasonable to me (IMHO).

To put it another way, I do not agree with the adage that "syntactic  
sugar causes cancer of the semicolon." :)

As far as readability, I tried to choose syntax that would make clear  
to a casual reader what was going on, but clearly I'm blinded by  
having stared at this for awhile now.  As Nicolas Pouillard points  
out, you can always unsugar the code.

Anyway, I've been using variants of this library for half a year or  
so, and I've found it very convenient, so I thought I would offer it  
to others.

> This may be seen as a matter of personal taste: it is true that I
> lean towards syntactic regularity, i.e. less syntactic constructs.
> However there is another argument against using extensions: code
> maintenance. You have no guarantee that (1) an extension will not be
> broken by the next version of the language and (2) that the author
> will continue to maintain it.

I'm not sure why either 1 or 2 is more true for a syntax extension  
than for an arbitrary library, assuming the syntax extension doesn't  
try to make large-scale changes to the grammar.

> P.S. Even for domain specific languages many things can be done in
> pure ocaml by embedding the dsl in ocaml using meta-programming
> techniques.

I'm not sure what you mean here.  I understand meta-programming to  
mean (roughly) generating code programatically.  A solution along  
those lines to my problem (locality of command-line arguments) would  
be something like this: specify the options in some file, write some  
piece of code to process this and produce an ocaml file, which is  
then compiled and linked with the main program.  I actually  
originally did something along these lines, but then decided that  
since this approach also requires defining and processing a new  
syntax (of the option file), I would use camlp4 and integrate that  
syntax into my main program.  That is, camlp4 *is* meta-programming,  
it's just that it (a) integrates the DSL into the main program, and  
(b) hides the intermediate file.

But perhaps I'm misunderstanding your point - if you could clarify  
what you mean, I'd like to know what alternate approaches are.

Finally, I have the sense (perhaps I'm completely wrong) that the  
post I'm responding to was motivated in part by a thought that  
command-line option parsing is somehow not worthy of a domain- 
specific language / syntax extension.  I suppose it is something of a  
"syntactic obsession" with me to find a clear and concise way of  
specifying such options, since I write mostly small, command-line  
programs to which I want to be able to quickly add variant  
functionality.  Clearly, the benefit to any particular program is  
small - when you write enough of them, though, the lifetime savings  
in tearing-my-hair-out is, I think, a win.

best,

-E r i c

(PS: sorry if this doesn't get threaded appropriately, I read the  
digest of the list).

Re: On language extensions (was Re: [Caml-list] global record)

[skaller]

On Wed, 2006-07-19 at 21:12 -0400, Eric Breck wrote:
> > From: Daniel_Bünzli <daniel.buenzli@e...>

>  If I have to read someone else's code I want to
> > read caml code and not caml code augmented with the author's
> > syntactic obsessions.

> Succinctness is not precisely what I was after so much as locality -  

>  Such locality is a basic principle of software engineering,  
> and in this case I don't really know how to achieve it with only a  
> library (and not a syntax extension).

In my opinion, the difficulty here lies in the Ocaml 
programming model.

Precisely as you say above regarding locality .. the syntax
extensions should be localised. In particular, they should
be embedded in the file that uses them -- along with
documentation -- and not specified on the command line.

For example

syntax MyExtensions;;

which could itself be implemented in ocamlp4 perhaps,
and then *that* extension converted to a builtin part
of the language.

Felix implements syntax extensions with preprocessor
statements (which can be provided in a #included file).

Of course the syntax extension should contain documentation!

Having said that, I do agree excessive extensions make
code hard to read .. and probably defeat other tools
such as ocamldoc (which even if applied to expanded 
output may report on the expanded source when one would
prefer reporting on the unexpanded input).

This form of code generation is weak compared with
systems like MetaOcaml.

Similarly, Ocaml should have 

  import List;;
  .. List.fold_left ..

so that you are required to import a separately compiled
module before using it. As a sop to existing code, 

  open List;;

could imply import. grep for either 'open' or 'import'
is then enough to establish dependencies.

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: On language extensions (was Re: [Caml-list] global record)

[Stefano Zacchiroli]

On Thu, Jul 20, 2006 at 03:16:18PM +1000, skaller wrote:
> On Wed, 2006-07-19 at 21:12 -0400, Eric Breck wrote:
> >  Such locality is a basic principle of software engineering,  
> > and in this case I don't really know how to achieve it with only a  
> > library (and not a syntax extension).
> 
> In my opinion, the difficulty here lies in the Ocaml 
> programming model.
> 
> Precisely as you say above regarding locality .. the syntax
> extensions should be localised. In particular, they should
> be embedded in the file that uses them -- along with
> documentation -- and not specified on the command line.
> 
> For example
> 
> syntax MyExtensions;;

Fully agreed. Me myself try to enforce locality of syntax extensions to
avoid possible clashes. Unfortunately handling that in large project is
really painful, since you end up in having to differentiate "clusters"
of files which need to be built with different set of camlp4 flags, and
you need to do that in Makefile.

In my mind this is asking for trouble, and the usual solution I pursue
is to avoid using syntax extensions ... a pity.

Cheers.

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

Re: On language extensions (was Re: [Caml-list] global record)

[Jean-Marie Gaillourdet]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

On 20.07.2006, at 08:29, Stefano Zacchiroli wrote:
> Fully agreed. Me myself try to enforce locality of syntax  
> extensions to
> avoid possible clashes. Unfortunately handling that in large  
> project is
> really painful, since you end up in having to differentiate "clusters"
> of files which need to be built with different set of camlp4 flags,  
> and
> you need to do that in Makefile.

The OCamlMakefile in godi supports syntax extension declarations in  
the first line of ocaml source. Those declaration are wrapped in  
OCaml comments. Then make takes care that every single source file is  
compiled with the appropriate syntax extensions.

- --
   Jean-Marie

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEv0WONIUNP/I5YOgRAlGIAJ9Fr+vKiTbXCA0AMwvWOOG+wv7j+ACfQOEO
O00L8arLwjNmvXRxp76r4Ic=
=EkZi
-----END PGP SIGNATURE-----

Re: On language extensions (was Re: [Caml-list] global record)

[Bünzli Daniel]

Le 20 juil. 06 à 03:12, Eric Breck a écrit :

> Succinctness is not precisely what I was after so much as locality  
> - in this case, having each option declared in a single location -  
> as opposed to the type definition in one place, the default value  
> in another, the Arg.spec in another, code to print the chosen value  
> of each option in another, and, optionally, the de-ref-ication in  
> another.  Such locality is a basic principle of software  
> engineering, and in this case I don't really know how to achieve it  
> with only a library (and not a syntax extension).

The typechecker (indirectly) gives you that locality, it tells you  
exactly where changes are needed and frankly all this is in the same  
module, maybe even on the same page. If I have to maintain your code  
I'll understand quicker hand made code than your syntax extension  
(especially since I already know how the Arg module works).

> I'm not advocating extending the language willy-nilly, but I feel  
> that any new library already requires some amount of learning to  
> use its particular API, and if a small, local extension to the  
> language vastly shrinks (and localizes) code required to interface  
> to the library, it seems reasonable to me (IMHO).

Api is not new syntax. As a said before when you learn an api you  
only need to understand new semantics.

> As far as readability, I tried to choose syntax that would make  
> clear to a casual reader what was going on, but clearly I'm blinded  
> by having stared at this for awhile now.  As Nicolas Pouillard  
> points out, you can always unsugar the code.

For small examples this can be a solution. But I usually prefer not  
to read machine generated code. Most programmer (I hope) reread their  
code and try to make it readable and elegant, a machine won't.

>> You have no guarantee that (1) an extension will not be
>> broken by the next version of the language and (2) that the author
>> will continue to maintain it.
>
> I'm not sure why either 1 or 2 is more true for a syntax extension  
> than for an arbitrary library, assuming the syntax extension  
> doesn't try to make large-scale changes to the grammar.

I agree on this point for 2 but not on 1. Incompatible language  
changes are rare (inexistant ?) with ocaml.

>> P.S. Even for domain specific languages many things can be done in
>> pure ocaml by embedding the dsl in ocaml using meta-programming
>> techniques.
>
> I'm not sure what you mean here.  I understand meta-programming to  
> mean (roughly) generating code programatically.

I was not specifically talking about your problem here, though I  
could maybe be applied. It was more on using caml to write code that  
should be written in another language (e.g. regexps, sql, etc.),  
instead of writing a horrible language extension that will allow you  
to mix both languages or use caml strings, see [1] for more information.

I'm sorry to say that but I regard (maybe out of ignorance) camlp4 as  
very low level and brittle tool (not to say hack). An obvious proof  
of this to me is the composition problem which is according to  
previous messages is not solved by camlp4.

Best,

Daniel

[1]
@article{967844,
  author = {Conal Elliott and Sigbj\&\#248;rn Finne and Oege De Moor},
  title = {Compiling embedded languages},
  journal = {J. Funct. Program.},
  volume = {13},
  number = {3},
  year = {2003},
  issn = {0956-7968},
  pages = {455--481},
  doi = {http://dx.doi.org/10.1017/S0956796802004574},
  publisher = {Cambridge University Press},
  address = {New York, NY, USA},
  }