Re: [Caml-list] Camlp4/p5 type reflection [was: OCaml maintenance status / community fork (again)]

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Many people are still frustrated with the camlp4/p5 situation. IMHO, we
should give up on camlp4 inside the distribution, and only implement a
few of its features in the regular parser:

- Antiquotation syntax (i.e. << >> expressions) because this makes it
  very easy to incorporate foreign syntax elements. Of course, we
  would also need the ability to recursively invoke the ocaml parser
  for parts of the antiquoted expression (this would also open the
  door for transformations of the AST).

- A simple macro language (ifdef at least), for the cases antiquotations
  would be too complex.

That means we would give up on arbitrary syntax extensions in the core
distribution. There could be still camlp4 or p5 as a separate add-on,
but it will be a second-class citizen, and it would be clear that it is
a bit behind with the newest syntactical features. Also, it could be
developed outside the core team.

Gerd


Am Sonntag, den 11.12.2011, 11:29 +0100 schrieb Gabriel Scherer:
> > And Xavier's mail suggests that camlp4 is a maintenance burden for the OCaml team.
> > Why is it such a bad idea to drop camlp4 out of the distribution, and
> > just let camlp5 live?
> 
> First of all, I don't have a strong opinion here: I just voiced
> doubt. My reasoning for going so goes along two lines of argument:
> 
> 1. I'm not exactly sure how Camlp4 being in or out of the distribution
>    will change the maintainance burden. The main maintainance
>    difficulty with camlp{4,5} is that it needs to evolve its own
>    parser in parallel with the 'official' one (that's by design), with
>    changes to the language syntax. You need to change Camlp{4,5} when
>    Ocaml 3.N+1 introduces a new syntax, or it won't be usable on OCaml
>    3.N+1 code.
> 
>    There are already users relying on Camlp{4,5}, and those user
>    generally wish to use the new, exciting features of the next
>    version. If they can't, they will complex, regardless of whether
>    their preprocessor is in or out the distribution. That means that
>    when the OCaml team is about to release a new version with
>    syntactic changes, they have to worry about the preprocessor
>    anyway, or make users unhappy.
> 
>    So there is a "preprocessor burden" on the OCaml team, independently
>    of where the code is maintained and located. If the change means
>    "make it easier to distribute camlp4 fixes without bumping OCaml's
>    version number", why not. If it means "now we won't care about
>    Camlp4 state before releasing a new versions", this may mean
>    a degradation in the life of Camlp4 users. I doubt that's the idea.
> 
>    Being in or out the distribution also wouldn't change much, I think,
>    the possibility of external contributions. In my experience Nicolas
>    Pouillard and now Xavier Clerc have a good track record of
>    integrating external contributions (I have sent one or two bugfixes
>    on the tracker), and I'm confident they would be able to work with
>    Jérémie Dimino if he wished to contribute to camlp4's evolution more
>    frequently.
> 
> 2. I suppose -- purely personal guess -- the intention being the
>    consortium's suggestion is to try to move away slowly from
>    Camlp{4,5}, towards alternatives such as Alain Frisch's
>    "annotations" proposal. As I said previously, I would personally
>    welcome such a move, but I don't see said alternatives released
>    yet. I haven't had the opportunity to play with alternate tools,
>    have an idea of how a transition would work out, see if the
>    documentation is reasonably complete, etc.
> 
>    It would make more sense, in my opinion, to downplay camlp{4,5}
>    *once* we have played with alternatives and are confident that they
>    are mature enough to make a transition.
> 
>    Please also remember that the consortium members represent
>    relatively large, well-educated, experienced players in the OCaml
>    community. It probably wouldn't bother them much if, say,
>    ocamlbuild, ocamldoc, or ocamldbg where taken out of the official
>    distribution. They have important tooling in place and would adapt
>    relatively easily. The end user or OCaml beginner may not adapt to
>    such changes that easily. Now this is a matter where distributions,
>    such as Debian, can be of great help, by providing "complete"
>    packages regardless of what is or isn't in the official
>    distribution. However, I have handled just enough users reports
>    wondering why they didn't have "camlp4o" available, or
>    graphics.cma, or whatever, to know that this can also be a barrier
>    to use and adoption.
> 
> 
> Again, no strong opinion. I will welcome any change that strenghten
> the OCaml language. If you think distributing camlp4 out of the
> distribution would ease the live of OCaml developpers and maintainers,
> at no cost to the users, nor complicating the distribution side, then
> all is good. I just feel that it may be a bit too soon.
> 
> 
> On Sun, Dec 11, 2011 at 10:04 AM, Stéphane Glondu <steph@glondu.net> wrote:
> > Le 11/12/2011 00:34, Gabriel Scherer a écrit :
> >> The original Camlp4 tool was mostly developped by Daniel de
> >> Rauglaudre.
> >> [...]
> >> (I'm thinking of
> >> eg. Martin Jambon, which had extensive Camlp4 extensions, and the Coq
> >> team which has user-defined notations using Camlp4 and, huh, I really
> >> don't want to know the details); basically they didn't upgrade to
> >> 3.10 -- instead of porting the extensions, as was originally hoped.
> >> [...]
> >> Daniel, which apparently did not agree with some of the changes made,
> >> relatively suddenly restarted developpment of "his" branch of Camlp4,
> >> taken from the old sources, before refactoring. This was done as
> >> a separate project, outside the OCaml distribution (apparently Daniel
> >> and the OCaml team prefer not to work together).
> >> [...]
> >> Camlp4 is a piece of devil beauty. It does incredibly clever things,
> >> and is incredibly complex inside: Daniel is clearly a remarkable
> >> hacker, but his code is not easy to understand. I know that
> >> maintaining the whole thing is very hard; and that is the reason why
> >> Camlp4 tends to have problems to bump from one version to another,
> >> when non-neglectible syntaxic changes are made to the language.
> >> [...]
> >> (And I'm not sure it's a good idea to move Camlp4 out of the
> >> distribution as long as we don't have a viable alternative proposal
> >> and some users have started moving to it. Camlp4 will still need to be
> >> supported by someone anyway, and it needs to evolve in lockstep with
> >> OCaml language changes.)
> >
> > Coq has been adapted to work with both camlp4 and camlp5. In my
> > experience, Daniel has been much more responsive on camlp5 matters than
> > the OCaml team on camlp4 matters. And Xavier's mail suggests that camlp4
> > is a maintenance burden for the OCaml team.
> >
> > Why is it such a bad idea to drop camlp4 out of the distribution, and
> > just let camlp5 live? My feeling is that people using camlp5 care about
> > it, but people using camlp4 do so just because it's in the official
> > distribution and wouldn't care switching to camlp5 (or stop using this
> > kind of syntax extension) if camlp4 was officially deprecated in favor
> > of camlp5. At worst, someone will start maintaining camlp4 independently.
> >
> > Daniel already maintains and develops camlp5. I guess Jérémie would also
> > volunteer to join him (but he already offered to maintain camlp4
> > independently). In Debian, there are currently 49 packages depending on
> > camlp4, and 7 depending on camlp5. That also increases incentive /
> > possible help for maintaining it.
> >
> >
> > Cheers,
> >
> > --
> > Stéphane
> >
> 
>