From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 1B07B7FCCB for ; Mon, 27 Apr 2015 12:16:53 +0200 (CEST) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.223.171; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.223.171 as permitted sender) identity=mailfrom; client-ip=209.85.223.171; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ie0-f171.google.com) identity=helo; client-ip=209.85.223.171; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-ie0-f171.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0AvAgBECz5Vm6vfVdFcg19cBYMVxH6FRD4CgScHOxEBAQEBAQEBEQEBAQEBBgsLCSEuhCABAQEDARIRHQEUBx0BAwELBgMCCwMKKgICIgERAQUBHAYTCRIHhVmCGwEDCQgNlC+QXD4xizmBa4J2iFwKGScNVYRsAQEBAQEBAQMBAQEBAQEBARQBBQ6LKoJrgXQiBAeCaIFFBYZCjxSEZ4FRlBcSI4EMCYIIghE8MQEEgj8BAQE X-IPAS-Result: A0AvAgBECz5Vm6vfVdFcg19cBYMVxH6FRD4CgScHOxEBAQEBAQEBEQEBAQEBBgsLCSEuhCABAQEDARIRHQEUBx0BAwELBgMCCwMKKgICIgERAQUBHAYTCRIHhVmCGwEDCQgNlC+QXD4xizmBa4J2iFwKGScNVYRsAQEBAQEBAQMBAQEBAQEBARQBBQ6LKoJrgXQiBAeCaIFFBYZCjxSEZ4FRlBcSI4EMCYIIghE8MQEEgj8BAQE X-IronPort-AV: E=Sophos;i="5.11,656,1422918000"; d="scan'208";a="113650666" Received: from mail-ie0-f171.google.com ([209.85.223.171]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 27 Apr 2015 12:16:51 +0200 Received: by iebrs15 with SMTP id rs15so122568454ieb.3 for ; Mon, 27 Apr 2015 03:16:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=bnBmRUYnDLHpq43YsRfhOc4fCn06nn+3SXg0xvNOrao=; b=vT0gVlaEMT+aHXErWLzdnQFtWFYC5lPv2zB2QC1iYlCV6i28UV3V77E2d7xarlc+ly SSbXsWkgUbFJIbGA69ZdMNu4WHr5Pi/ltQyAMtx7KkrBS1Th7mN6KvvfHWXasOjqSXPG t8/504yllcqEJmHYAxEhdy2RR+cTEjnT+VX3pqgEO0rcl+uEzMf1NOcygXWZ6VuYJsy5 Uzf50uhKSYT7yfhXuSNTVmUgqwPNbPZYa/xm2FdinAtKhkTg/rOqnPP3lUESqg3RqRPq pIlz6slPnOzzadOgEkd/mpVf/uFR4SJ4TvYol5eoiyILtG7VexKdyMH2Lz8CeQVvXpkP QhCw== X-Received: by 10.50.43.202 with SMTP id y10mr9376253igl.35.1430129810183; Mon, 27 Apr 2015 03:16:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.53.10 with HTTP; Mon, 27 Apr 2015 03:16:10 -0700 (PDT) In-Reply-To: <1430128317.3427.70.camel@zotac> References: <54F5B3F7.3030705@cea.fr> <1425394551.4056.1.camel@thinkpad.lan.sumadev.de> <54F6D731.3090004@cea.fr> <1428953391.22412.40.camel@e130.lan.sumadev.de> <552CD705.9000508@cea.fr> <552CE242.9050307@glondu.net> <552D0BFE.7000904@cea.fr> <1430128317.3427.70.camel@zotac> From: Gabriel Scherer Date: Mon, 27 Apr 2015 12:16:10 +0200 Message-ID: To: Gerd Stolpmann Cc: =?UTF-8?Q?Fran=C3=A7ois_Bobot?= , =?UTF-8?Q?St=C3=A9phane_Glondu?= , OCaml Mailing List Content-Type: multipart/alternative; boundary=089e011604ba51c8590514b20bcc Subject: Re: [Caml-list] Dependencies between plugins --089e011604ba51c8590514b20bcc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Another (more general, less ad-hoc) way to have this semantics would be to introduce a "default value" for predicates that do not explictly appear in rules. Under the set of default values D, then a rule depending on predicates P would be true if the boolean assignment (D,P=3Dtrue) holds, wi= th the rightmost boolean assignment shadowing all others. I mean that the rule foo(x1,x2,x3) =3D ... would be applied whenever x1=3Dtrue, x2=3Dtrue, x3=3Dtrue with an empty def= ault environment, but with the default (y1=3Dtrue, x1=3Dfalse, y2=3Dfalse) it wo= uld only be applied whenever (y1=3Dtrue, x1=3Dtrue, y2=3Dfalse, x2=3Dtrue, x3= =3Dtrue). Then setting "plugin=3Dfalse" in the default environment (equivalently, -plugin) would give the expected semantics, I believe. In particular (thanks to Fran=C3=A7ois for his clear re-explanation on April 9) then the = rule archive(native) would not be matched by the query archive(native,plugin). I can see a use for another default variable (ocaml=3Dtrue) allowing people that could want to use ocamlfind for non-ocaml stuff to disable it explicitly, and then reuse the rule names for their own stuff. (In ocamlbuild it is useful to be able to talk about compilation problems that are not about OCaml). On Mon, Apr 27, 2015 at 11:51 AM, Gerd Stolpmann wrote: > Fran=C3=A7ois, > > I was thinking again about this issue. Introducing a third category > "shared" in addition to byte and native seems to be a bit odd. Actually, > what we really want to have is a second dimension plugin/executable in > addition to the already existing byte/native dimension, so that we can > have: > > - byte + executable (cmo) > - native + executable (cmx) > - byte + plugin (cmo, too) > - native + plugin (cmxs) > > That way we can have a separate cmo for byte+plugin, which may be useful > here and there. Also, byte and native are again symmetric. > > A typical META file would now specify > > archive(byte,executable) =3D "..." > archive(native,executable) =3D "..." > > if it doesn't support plugins, and specify > > archive(byte,executable) =3D "..." > archive(native,executable) =3D "..." > archive(byte,plugin) =3D "..." > archive(native,plugin) =3D "..." > > if it does so. (NB. "executable" because these are the objects for > creating executables.) > > The only remaining question is how to handle existing META files that > don't make this distinction. We don't have a version number in META > files, so we have to watch out for another criterion. I am thinking > about understanding > > archive(native) =3D "..." > > as > > archive(native,executable) =3D "..." > > if there is no other reference to the executable predicate. This would > be a special fixup after parsing META. After a transition phase (say, > two years from now on) we would consider archive(native) as an error. > The upcoming META lint will report this issue. > > This way, the existing META files can still be used for some time, > including those specifying plugins. There is no hurry in changing this > detail. However, you are absolutely right that the current use of > "plugin" breaks the way the predicates are defined, and in the long term > this is worth fixing. > > Gerd > > > > Am Dienstag, den 14.04.2015, 14:45 +0200 schrieb Fran=C3=A7ois Bobot: > > On 14/04/2015 11:47, St=C3=A9phane Glondu wrote: > > > Le 14/04/2015 10:59, Fran=C3=A7ois Bobot a =C3=A9crit : > > >>>> Are there any movement in this direction, or this patches will die? > > >>> > > >>> Don't think so. Slowness on my side. > > >> > > >> On my side, I haven't yet written the documentation. My main > impediment > > >> is to choose which predicates to use for the cmxs in the META file: > > >> 1) to keep archive(plugin,native) because it is the defacto standard > > >> 2) to move to something that is semantically right: > > >> archive(native_plugin) or archive(shared). > > > > > > Sorry, I didn't follow the whole discussion but... this looks like > > > hardcoding a special treatment of plugins for the native case, > > > forgetting the bytecode case. Would you introduce byte_plugin (or a > > > bytecode counterpart to "shared" which looks bad to me) as well? > > > > The fact is that native and bytecode are not symmetric: > > | byte | native > > static link | cmo | cmx > > dynamic link | cmo | cmxs > > > > So for bytecode we can still use `archive(byte)`. If someone wants its > library to be loaded > > differently in static linking and dynamic linking e can use > `archive(byte,plugin)`. > > > > You are right that I should give a full proposal (I'm going to go with > `shared` instead of > > `native_plugin` because it is short and correspond to the ocamlopt > option): > > > > 1. In META file: > > 1.1 use `archive(byte)` and `archive(native)` for the files to > statically link > > 1.2 use `archive(byte,plugin)` for the files to dynamically link in > bytecode if they are > > different from the static one > > 1.3 use `archive(shared)` for the files that are dynamically linked > in native code > > > > 2. During dynamic loading: > > 2.1. in bytecode: look for variable `archive` with predicates > `byte`,`plugin` and the other > > predicates used during compilation (`mt`, `mt_posix`, `mt_vm`, `gprof`, > ...) > > 2.2 in native: look for variable `archive` with predicates `shared`, > `plugin` and the other > > predicates used during static linking except `native` > > > > > > My goal is just that when you ask in native code "Does this library > define files for dynamic > > linking" the answer is not "yes, it defines these cmx". There are other > solutions (like asking that > > file to statically link are define with `archive(native,-plugin)`) but > they seem to be less > > conservative. > > > > > Even > > > code using Dynlink should be as generic (w.r.t. native/bytecode) as > > > possible... > > > > > > The examples of tools that use dependencies between plugins gathered at > the start of the discussion > > are already not generic (w.r.t. native/bytecode) : > > > > The following code makes a differences between bytecode and native code: > > > https://github.com/ocsigen/ocsigenserver/blob/master/src/baselib/ocsigen_= loader.ml#L165 > > > https://github.com/zoggy/stog/blob/e83c363c83465a7bfd1595816b3d9bc8331af5= 60/stog_dyn.ml#L119-L146 > > > > This one works only for native code, it seems: > > > https://github.com/hammerlab/ketrew/blob/master/src/lib/pure/ketrew_plugi= n.ml#L52 > > > > The proposed modification is to replace (for example in ocsigen): > > > > ```ocaml > > (if Ocsigen_config.is_native then "native" else "byte") > > ``` > > > > by > > > > ```ocaml > > (if Ocsigen_config.is_native then "shared" else "byte") > > ``` > > > > -- > > Fran=C3=A7ois > > > > > > -- > ------------------------------------------------------------ > Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de > My OCaml site: http://www.camlcity.org > Contact details: http://www.camlcity.org/contact.html > Company homepage: http://www.gerd-stolpmann.de > ------------------------------------------------------------ > --089e011604ba51c8590514b20bcc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Another (more general, less ad-hoc) wa= y to have this semantics would be to introduce a "default value" = for predicates that do not explictly appear in rules. Under the set of defa= ult values D, then a rule depending on predicates P would be true if the bo= olean assignment (D,P=3Dtrue) holds, with the rightmost boolean assignment = shadowing all others. I mean that the rule

=C2=A0 foo(x1,x2,x3= ) =3D ...

would be applied whenever x1=3Dtrue, x2=3Dtrue, x3= =3Dtrue with an empty default environment, but with the default (y1=3Dtrue,= x1=3Dfalse, y2=3Dfalse) it would only be applied whenever (y1=3Dtrue, x1= =3Dtrue, y2=3Dfalse, x2=3Dtrue, x3=3Dtrue).

Then setting "= ;plugin=3Dfalse" in the default environment (equivalently, -plugin) wo= uld give the expected semantics, I believe. In particular (thanks to Fran= =C3=A7ois for his clear re-explanation on April 9) then the rule archive(na= tive) would not be matched by the query archive(native,plugin).

I can see a use for another default variable (ocaml=3Dtrue) allowing peop= le that could want to use ocamlfind for non-ocaml stuff to disable it expli= citly, and then reuse the rule names for their own stuff. (In ocamlbuild it= is useful to be able to talk about compilation problems that are not about= OCaml).

On Mon, Apr 27, 2015 at 11:51 AM, Gerd Stolpmann <info@gerd-stolpman= n.de> wrote:
Fran=C3=A7ois,=

I was thinking again about this issue. Introducing a third category
"shared" in addition to byte and native seems to be a bit odd. Ac= tually,
what we really want to have is a second dimension plugin/executable in
addition to the already existing byte/native dimension, so that we can
have:

=C2=A0- byte + executable (cmo)
=C2=A0- native + executable (cmx)
=C2=A0- byte + plugin (cmo, too)
=C2=A0- native + plugin (cmxs)

That way we can have a separate cmo for byte+plugin, which may be useful
here and there. Also, byte and native are again symmetric.

A typical META file would now specify

archive(byte,executable) =3D "..."
archive(native,executable) =3D "..."

if it doesn't support plugins, and specify

archive(byte,executable) =3D "..."
archive(native,executable) =3D "..."
archive(byte,plugin) =3D "..."
archive(native,plugin) =3D "..."

if it does so. (NB. "executable" because these are the objects fo= r
creating executables.)

The only remaining question is how to handle existing META files that
don't make this distinction. We don't have a version number in META=
files, so we have to watch out for another criterion. I am thinking
about understanding

archive(native) =3D "..."

as

archive(native,executable) =3D "..."

if there is no other reference to the executable predicate. This would
be a special fixup after parsing META. After a transition phase (say,
two years from now on) we would consider archive(native) as an error.
The upcoming META lint will report this issue.

This way, the existing META files can still be used for some time,
including those specifying plugins. There is no hurry in changing this
detail. However, you are absolutely right that the current use of
"plugin" breaks the way the predicates are defined, and in the lo= ng term
this is worth fixing.

Gerd



Am Dienstag, den 14.04.2015, 14:45 +0200 schrieb Fran=C3=A7ois Bobot:
> On 14/04/2015 11:47, St=C3=A9phane Glondu wrote:
> > Le 14/04/2015 10:59, Fran=C3=A7ois Bobot a =C3=A9crit :
> >>>> Are there any movement in this direction, or this pat= ches will die?
> >>>
> >>> Don't think so. Slowness on my side.
> >>
> >> On my side, I haven't yet written the documentation. My m= ain impediment
> >> is to choose which predicates to use for the cmxs in the META= file:
> >> 1) to keep archive(plugin,native) because it is the defacto s= tandard
> >> 2) to move to something that is semantically right:
> >> archive(native_plugin) or archive(shared).
> >
> > Sorry, I didn't follow the whole discussion but... this looks= like
> > hardcoding a special treatment of plugins for the native case,
> > forgetting the bytecode case. Would you introduce byte_plugin (or= a
> > bytecode counterpart to "shared" which looks bad to me)= as well?
>
> The fact is that native and bytecode are not symmetric:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| byte=C2=A0 | n= ative
> static link=C2=A0 | cmo=C2=A0 =C2=A0| cmx
> dynamic link | cmo=C2=A0 =C2=A0| cmxs
>
> So for bytecode we can still use `archive(byte)`. If someone wants its= library to be loaded
> differently in static linking and dynamic linking e can use `archive(b= yte,plugin)`.
>
> You are right that I should give a full proposal (I'm going to go = with `shared` instead of
> `native_plugin` because it is short and correspond to the ocamlopt opt= ion):
>
> 1. In META file:
>=C2=A0 =C2=A0 1.1 use `archive(byte)` and `archive(native)` for the fil= es to statically link
>=C2=A0 =C2=A0 1.2 use `archive(byte,plugin)` for the files to dynamical= ly link in bytecode if they are
> different from the static one
>=C2=A0 =C2=A0 1.3 use `archive(shared)` for the files that are dynamica= lly linked in native code
>
> 2. During dynamic loading:
>=C2=A0 =C2=A0 2.1. in bytecode: look for variable `archive` with predic= ates `byte`,`plugin` and the other
> predicates used during compilation (`mt`, `mt_posix`, `mt_vm`, `gprof`= , ...)
>=C2=A0 =C2=A0 2.2=C2=A0 in native: look for variable `archive` with pre= dicates `shared`, `plugin` and the other
> predicates used during static linking except `native`
>
>
> My goal is just that when you ask in native code "Does this libra= ry define files for dynamic
> linking" the answer is not "yes, it defines these cmx".= There are other solutions (like asking that
> file to statically link are define with `archive(native,-plugin)`) but= they seem to be less
> conservative.
>
>=C2=A0 > Even
>=C2=A0 > code using Dynlink should be as generic (w.r.t. native/byte= code) as
>=C2=A0 > possible...
>
>
> The examples of tools that use dependencies between plugins gathered a= t the start of the discussion
> are already not generic (w.r.t. native/bytecode) :
>
> The following code makes a differences between bytecode and native cod= e:
> https://github.com/ocsigen/= ocsigenserver/blob/master/src/baselib/ocsigen_loader.ml#L165
> https://github.co= m/zoggy/stog/blob/e83c363c83465a7bfd1595816b3d9bc8331af560/stog_dyn.ml#L119= -L146
>
> This one works only for native code, it seems:
> https://github.com/hammerlab/ketr= ew/blob/master/src/lib/pure/ketrew_plugin.ml#L52
>
> The proposed modification is to replace (for example in ocsigen):
>
> ```ocaml
> (if Ocsigen_config.is_native then "native" else "byte&q= uot;)
> ```
>
> by
>
> ```ocaml
> (if Ocsigen_config.is_native then "shared" else "byte&q= uot;)
> ```
>
> --
> Fran=C3=A7ois
>
>

--
-----------------------= -------------------------------------
Gerd Stolpmann, Darmstadt, Germany=C2=A0 =C2=A0 gerd@gerd-stolpmann.de
My OCaml site:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 http://www.camlcity.org
Contact details:=C2=A0 =C2=A0 =C2=A0 =C2=A0 http://www.camlcity.org/contact.html
Company homepage:=C2=A0 =C2=A0 =C2=A0 =C2=A0http://www.gerd-stolpmann.de
------------------------------------------------------------

--089e011604ba51c8590514b20bcc--