Re: [Caml-list] Bug in Unix library on Mac?

Re: [Caml-list] Bug in Unix library on Mac?

[spiral voice]

Hi,

in bug 2423 I read that this bug was fixed in Ocaml CVS.
But how can I get that code?

The method described at http://camlcvs.inria.fr/cvsserver-eng.html
gives me 3.08+2, not 3.09 CVS

- spiralvoice

Re: [Caml-list] Bug in Unix library on Mac?

[Damien Doligez]

On Jan 13, 2005, at 08:24, spiral voice wrote:

> in bug 2423 I read that this bug was fixed in Ocaml CVS.

I assume you mean PR#3423, since 2423 doesn't exist...

> But how can I get that code?
>
> The method described at http://camlcvs.inria.fr/cvsserver-eng.html
> gives me 3.08+2, not 3.09 CVS

It's just the opposite.  The method described gives you the main trunk
(i.e. the 3.09 branch), and the fix is in the 3.08 branch.  It's easy to
get: follow the same method, but instead of typing "cvs checkout csl"
you type "cvs checkout -r release308 csl".

Or you can wait a few weeks for 3.08.3.

-- Damien

binary compatibility of 3.08.3

[Stefano Zacchiroli]

On Thu, Jan 13, 2005 at 05:53:07PM +0100, Damien Doligez wrote:
> Or you can wait a few weeks for 3.08.3.

BTW, do 3.08.3 break binary compatibility with 3.08.2?
(as happened when upgrading from 3.08.1 to 3.08.2)

TIA,
Cheers.

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

Re: [Caml-list] binary compatibility of 3.08.3

[Jacques Garrigue]

From: Stefano Zacchiroli <zack@debian.org>

> On Thu, Jan 13, 2005 at 05:53:07PM +0100, Damien Doligez wrote:
> > Or you can wait a few weeks for 3.08.3.
> 
> BTW, do 3.08.3 break binary compatibility with 3.08.2?
> (as happened when upgrading from 3.08.1 to 3.08.2)

Any bug fix in the compiler has a very high probability of breaking
binary compatibility. This is due to the fact interface digests depend
on the indices of internal identifiers, so that purely internal
changes may break compatibility, eventhough the cmi format is left
unchanged.
It is reasonnable to assume that 3.08.3 will not be binary compatible.

By the way, due to a bug fix in the variance inference, it will also
not be 100% source compatible, but I hope this has no impact on
existing programs (as only few programs use variance). (I sincerely
hope there were no unsound programs around...)

Jacques Garrigue

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Thu, Jan 13, 2005 at 03:02:39PM -0800, Jacques Garrigue wrote:
> From: Stefano Zacchiroli <zack@debian.org>
> 
> > On Thu, Jan 13, 2005 at 05:53:07PM +0100, Damien Doligez wrote:
> > > Or you can wait a few weeks for 3.08.3.
> > 
> > BTW, do 3.08.3 break binary compatibility with 3.08.2?
> > (as happened when upgrading from 3.08.1 to 3.08.2)
> 
> Any bug fix in the compiler has a very high probability of breaking
> binary compatibility. This is due to the fact interface digests depend
> on the indices of internal identifiers, so that purely internal
> changes may break compatibility, eventhough the cmi format is left
> unchanged.
> It is reasonnable to assume that 3.08.3 will not be binary compatible.

Notice that this is really not nice for a bugfix release, since this means we
have to rebuild all of the ocaml related packages on all arches, which may
take us month and such. Maybe we would be better off just backporting the
non-breaking fixes ? Maybe in future this situation could be somewhat improved ? 

> By the way, due to a bug fix in the variance inference, it will also
> not be 100% source compatible, but I hope this has no impact on
> existing programs (as only few programs use variance). (I sincerely
> hope there were no unsound programs around...)

Oh, well. ...

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Yaron Minsky]

It's worth mentioning that the pain of such upgrades is considerably
reduced by the use of a package manager like GODI.  It's hardly
perfect, but it makes such things much easier.

y


On Fri, 14 Jan 2005 14:36:32 +0100, Sven Luther <sven.luther@wanadoo.fr> wrote:
> On Thu, Jan 13, 2005 at 03:02:39PM -0800, Jacques Garrigue wrote:
> > From: Stefano Zacchiroli <zack@debian.org>
> >
> > > On Thu, Jan 13, 2005 at 05:53:07PM +0100, Damien Doligez wrote:
> > > > Or you can wait a few weeks for 3.08.3.
> > >
> > > BTW, do 3.08.3 break binary compatibility with 3.08.2?
> > > (as happened when upgrading from 3.08.1 to 3.08.2)
> >
> > Any bug fix in the compiler has a very high probability of breaking
> > binary compatibility. This is due to the fact interface digests depend
> > on the indices of internal identifiers, so that purely internal
> > changes may break compatibility, eventhough the cmi format is left
> > unchanged.
> > It is reasonnable to assume that 3.08.3 will not be binary compatible.
> 
> Notice that this is really not nice for a bugfix release, since this means we
> have to rebuild all of the ocaml related packages on all arches, which may
> take us month and such. Maybe we would be better off just backporting the
> non-breaking fixes ? Maybe in future this situation could be somewhat improved ?
> 
> > By the way, due to a bug fix in the variance inference, it will also
> > not be 100% source compatible, but I hope this has no impact on
> > existing programs (as only few programs use variance). (I sincerely
> > hope there were no unsound programs around...)
> 
> Oh, well. ...
> 
> Friendly,
> 
> Sven Luther
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] binary compatibility of 3.08.3

[Marcin 'Qrczak' Kowalczyk]

Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> writes:

> Any bug fix in the compiler has a very high probability of breaking
> binary compatibility. This is due to the fact interface digests depend
> on the indices of internal identifiers, so that purely internal
> changes may break compatibility, eventhough the cmi format is left
> unchanged.

Wouldn't it be better if exported names did not have unique numbers
in them?

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

Re: [Caml-list] binary compatibility of 3.08.3

[Xavier Leroy]

Sven Luther writes:

> Notice that this is really not nice for a bugfix release, since this means we
> have to rebuild all of the ocaml related packages on all arches, which may
> take us month and such.

I find this figure surprinsing.  Compiling the whole of GODI (which
contains roughly the same number of packages as Debian OCaml) doesn't
take months, more like one hour.  I know Debian has many architectures
and many of them are underpowered, but maybe that simply suggests that
Debian's dedication to supporting dead architectures is questionable.

> Maybe we would be better off just backporting the
> non-breaking fixes ?

Identifying those fixes would take you and the other Debian
maintainers an awful lot of time, and in the end you'd get something
that doesn't match any "official" release from us, which is something
that I'd rather avoid (it messes up bug reporting, for one thing).

> Maybe in future this situation could be somewhat improved ? 

On the Debian side, perhaps :-)  On the Caml side, I don't feel like
going out of my way to increase binary compatibility, which has never
been one of our design goals.

Cheers,

- Xavier Leroy

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Fri, Jan 14, 2005 at 10:01:00AM -0500, Yaron Minsky wrote:
> It's worth mentioning that the pain of such upgrades is considerably
> reduced by the use of a package manager like GODI.  It's hardly
> perfect, but it makes such things much easier.

We already have a package manager, thank you all the same. The user won't see
a thing, but it is a hard thing for us debian package maintainers to rebuild
all packages that are dependent on ocaml on all 11 officially supported debian
architectures, and a bunch of unofficial ones, especially given the fact that
there are many dependencies, and there is at least one day lag in the
dependency chain building, and that some of our architectures are real slow to
build (m68k, mips/mipsel and s390 are the usual blockers, often arm and hppa
too). And so near to the sarge release, it is a real question if we should
forego the 3.08.3 changes or launch such a wide rebuild, which may miss the
freeze/release date anyway, or worse even, may make some important part of the
packages not being in the release at all. Luckily no ocaml package is part of
base/standard, and not affected by the base freeze.

Stuff like GODI are just hacky workaround for systems that lack real
system-wide package management and good ocaml integration like debian does :)

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Berke Durak]

On Sun, Jan 16, 2005 at 02:25:29PM +0100, Sven Luther wrote:
> On Fri, Jan 14, 2005 at 10:01:00AM -0500, Yaron Minsky wrote:
> > It's worth mentioning that the pain of such upgrades is considerably
> > reduced by the use of a package manager like GODI.  It's hardly
> > perfect, but it makes such things much easier.
> 
> We already have a package manager, thank you all the same. The user won't see
> a thing, but it is a hard thing for us debian package maintainers to rebuild
> all packages that are dependent on ocaml on all 11 officially supported debian
> architectures, and a bunch of unofficial ones, especially given the fact that
> there are many dependencies, and there is at least one day lag in the
> dependency chain building, and that some of our architectures are real slow to
> build (m68k, mips/mipsel and s390 are the usual blockers, often arm and hppa
> too). And so near to the sarge release, it is a real question if we should
> forego the 3.08.3 changes or launch such a wide rebuild, which may miss the
> freeze/release date anyway, or worse even, may make some important part of the
> packages not being in the release at all. Luckily no ocaml package is part of
> base/standard, and not affected by the base freeze.

By the way, is m68k/s390/etc. stuff built on m68k/etc. ?  Why not
cross-compile on fast CPUs ?
-- 
Berke Durak

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Sat, Jan 15, 2005 at 01:07:19PM +0100, Xavier Leroy wrote:
> Sven Luther writes:
> 
> > Notice that this is really not nice for a bugfix release, since this means we
> > have to rebuild all of the ocaml related packages on all arches, which may
> > take us month and such.
> 
> I find this figure surprinsing.  Compiling the whole of GODI (which
> contains roughly the same number of packages as Debian OCaml) doesn't
> take months, more like one hour.  I know Debian has many architectures
> and many of them are underpowered, but maybe that simply suggests that
> Debian's dedication to supporting dead architectures is questionable.

Well, this is understandable, since GODI builds in a isolated way, on one
architecture, without any dependency on the result of the autobuilder. There
is at least one day lag between each package build, and then there is the
problems of some packages not really maintained by the debian/ocaml core team,
which we may miss, or other unrelated problem in some random library or
toolchain component which may bring a delay to a much needed build.

And no, m68k is by far not the major problem here, ia64 and arm had much more
troubles with the 3.07 to 3.08 transition, but i doubt you are suggesting
debian should drop support for those ? 

Also, we are hoping to release soon, and the debian release manager will
assuredly not wait for ocaml stuff to rebuild if it is the only thing missing.
The release delay have hurt enough as it is.

> > Maybe we would be better off just backporting the
> > non-breaking fixes ?
> 
> Identifying those fixes would take you and the other Debian
> maintainers an awful lot of time, and in the end you'd get something
> that doesn't match any "official" release from us, which is something
> that I'd rather avoid (it messes up bug reporting, for one thing).

Just forgetting about the fix and include it in the next version of sarge is
the only sane possibility then.

> > Maybe in future this situation could be somewhat improved ? 
> 
> On the Debian side, perhaps :-)  On the Caml side, I don't feel like

Bah. There are improvement possible, sure, in the autobuilder setup, and with
the api-version management used for the ocaml packages, but it has served us
well upto now, until it horribly broke with unannounced and unexpected binary
brokeness in 3.08.2.

> going out of my way to increase binary compatibility, which has never
> been one of our design goals.

Well, if binary compatibility could be maintained at least on the bug fix
branch, it would be good already. 

But if what Jacques says is true, and the binary compatibility is only broken
by the way the diggests are calculated, maybe there may be a solution there ? 

Or is the binary-compatibility breakage really needed ? 

Anyway, it would be nice if the breakage would at least be announced in the
release documentation. Especially if it doesn't touch all the modules and
libraries, as was the case for 3.08.2.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Sun, Jan 16, 2005 at 02:33:09PM +0100, Berke Durak wrote:
> On Sun, Jan 16, 2005 at 02:25:29PM +0100, Sven Luther wrote:
> > On Fri, Jan 14, 2005 at 10:01:00AM -0500, Yaron Minsky wrote:
> > > It's worth mentioning that the pain of such upgrades is considerably
> > > reduced by the use of a package manager like GODI.  It's hardly
> > > perfect, but it makes such things much easier.
> > 
> > We already have a package manager, thank you all the same. The user won't see
> > a thing, but it is a hard thing for us debian package maintainers to rebuild
> > all packages that are dependent on ocaml on all 11 officially supported debian
> > architectures, and a bunch of unofficial ones, especially given the fact that
> > there are many dependencies, and there is at least one day lag in the
> > dependency chain building, and that some of our architectures are real slow to
> > build (m68k, mips/mipsel and s390 are the usual blockers, often arm and hppa
> > too). And so near to the sarge release, it is a real question if we should
> > forego the 3.08.3 changes or launch such a wide rebuild, which may miss the
> > freeze/release date anyway, or worse even, may make some important part of the
> > packages not being in the release at all. Luckily no ocaml package is part of
> > base/standard, and not affected by the base freeze.
> 
> By the way, is m68k/s390/etc. stuff built on m68k/etc. ?  Why not
> cross-compile on fast CPUs ?

As said, we have enough m68k autobuilder to do the work, so why go for more
complicated and error-prone setups. As said, during the last transition, it
was not really m68k which was the problem.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Jacques Garrigue]

From: Sven Luther <sven.luther@wanadoo.fr>

> Well, if binary compatibility could be maintained at least on the bug fix
> branch, it would be good already. 
> 
> But if what Jacques says is true, and the binary compatibility is only broken
> by the way the diggests are calculated, maybe there may be a
> solution there ?
> 
> Or is the binary-compatibility breakage really needed ? 

Actually, what I gave is only one possible reason for breakage.
Another that comes to mind is that .cmx's contain code to be inlined.
If any file was modified in the standard library (to fix a bug there),
then there is a good chance that this code changed, and as a result
the digest will make it binary incompatible.

Binary compatibility as you get it in C is just a hack: you drop some
consistency checks, and hope that the user is clever enough to not use
incompatible libraries. Ocaml chooses the safe way. This could be made
a bit more resilient, particularly for bytecode, but you would still
have breakages in the bug fix branch.

> Anyway, it would be nice if the breakage would at least be announced in the
> release documentation. Especially if it doesn't touch all the modules and
> libraries, as was the case for 3.08.2.

You can safely assume that every new release breaks binary compatibility.
(I.e. that some digests in the libraries have changed.)

Jacques Garrigue

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Sun, Jan 16, 2005 at 08:26:37AM -0800, Jacques Garrigue wrote:
> From: Sven Luther <sven.luther@wanadoo.fr>
> 
> > Well, if binary compatibility could be maintained at least on the bug fix
> > branch, it would be good already. 
> > 
> > But if what Jacques says is true, and the binary compatibility is only broken
> > by the way the diggests are calculated, maybe there may be a
> > solution there ?
> > 
> > Or is the binary-compatibility breakage really needed ? 
> 
> Actually, what I gave is only one possible reason for breakage.
> Another that comes to mind is that .cmx's contain code to be inlined.
> If any file was modified in the standard library (to fix a bug there),
> then there is a good chance that this code changed, and as a result
> the digest will make it binary incompatible.

Oh well.

> Binary compatibility as you get it in C is just a hack: you drop some
> consistency checks, and hope that the user is clever enough to not use
> incompatible libraries. Ocaml chooses the safe way. This could be made
> a bit more resilient, particularly for bytecode, but you would still
> have breakages in the bug fix branch.

What would be nice in this light, would be an exact list of the digests, and
thus the modules, that changed, so we could only rebuild the packages that are
actually affected. That said, such behavior is a major drawback for using
ocaml in real projects, i believe.

> > Anyway, it would be nice if the breakage would at least be announced in the
> > release documentation. Especially if it doesn't touch all the modules and
> > libraries, as was the case for 3.08.2.
> 
> You can safely assume that every new release breaks binary compatibility.
> (I.e. that some digests in the libraries have changed.)

Yep, understood that now, 3.08.2 came as a big surprise though, as we somehow
expected no binary compatibility change for a bug fix release.

But now we know about it, and i will enable the full-rebuild process for the
3.08.3 release, hoping that it will be in time for the debian/sarge release.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Damien Doligez]

On Jan 16, 2005, at 14:37, Sven Luther wrote:

> Also, we are hoping to release soon, and the debian release manager 
> will
> assuredly not wait for ocaml stuff to rebuild if it is the only thing 
> missing.
> The release delay have hurt enough as it is.
[...]
> Just forgetting about the fix and include it in the next version of 
> sarge is
> the only sane possibility then.

I don't see it as a big problem if you are lagging by one bugfix 
version.
The differences are small anyway, and it's not like 3.08.2 is horribly
broken.

> Anyway, it would be nice if the breakage would at least be announced 
> in the
> release documentation. Especially if it doesn't touch all the modules 
> and
> libraries, as was the case for 3.08.2.

I would have announced it if I had known about it.  From now on, I will 
put
a reminder about binary compatibility in the release notes for each 
bugfix
release.


> That said, such behavior is a major drawback for using
> ocaml in real projects, i believe.

I'd say it's a drawback of blind upgrading, which nobody does in a
real project anyway.

-- Damien

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Sun, Jan 16, 2005 at 10:08:01PM +0100, Damien Doligez wrote:
> On Jan 16, 2005, at 14:37, Sven Luther wrote:
> 
> >Also, we are hoping to release soon, and the debian release manager 
> >will
> >assuredly not wait for ocaml stuff to rebuild if it is the only thing 
> >missing.
> >The release delay have hurt enough as it is.
> [...]
> >Just forgetting about the fix and include it in the next version of 
> >sarge is
> >the only sane possibility then.
> 
> I don't see it as a big problem if you are lagging by one bugfix 
> version.
> The differences are small anyway, and it's not like 3.08.2 is horribly
> broken.

Yep, we are going to try to make it, it should be possible to hasten the
process some if we stage the uploads just right, and implement some smart
(build) dependency tree travelling to make sure that we forget no package. 
The matter is complicated because ocaml uses a virtual ocaml-3.08 package as
build-dependency, to be able to make sure we don't have a package builded
against a binary-incompatible version. This will become ocaml-3.08.3, but the
debian tools have trouble with dependencies on virtual packages, especially
the autobuilders. Oh well, we will handle.

> >Anyway, it would be nice if the breakage would at least be announced 
> >in the
> >release documentation. Especially if it doesn't touch all the modules 
> >and
> >libraries, as was the case for 3.08.2.
> 
> I would have announced it if I had known about it.  From now on, I will 

Ah, so we were not the only one surprised :)

> put
> a reminder about binary compatibility in the release notes for each 
> bugfix
> release.

Cool.

> >That said, such behavior is a major drawback for using
> >ocaml in real projects, i believe.
> 
> I'd say it's a drawback of blind upgrading, which nobody does in a
> real project anyway.

Well, it was a bug fix release. I looked over the changelog, and everything
seemed reasonable in it. I did not look over the actual source code, and
nothing hinted at binary compatibility problems in the changelog, or maybe i
just missed it. If each individual changelog entry which breaks binary
compatibility listed the affected modules, that would be a great solution to
this problem.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[William Lovas]

On Sun, Jan 16, 2005 at 02:25:29PM +0100, Sven Luther wrote:
> Stuff like GODI are just hacky workaround for systems that lack real
> system-wide package management and good ocaml integration like debian does :)

Of course, GODI is a good stopgap measure until the whole world runs
Debian.

William, tongue firmly in cheek ;)

Re: [Caml-list] binary compatibility of 3.08.3

[Jacques Garrigue]

From: Sven Luther <sven.luther@wanadoo.fr>
> > Binary compatibility as you get it in C is just a hack: you drop some
> > consistency checks, and hope that the user is clever enough to not use
> > incompatible libraries. Ocaml chooses the safe way. This could be made
> > a bit more resilient, particularly for bytecode, but you would still
> > have breakages in the bug fix branch.
> 
> What would be nice in this light, would be an exact list of the digests, and
> thus the modules, that changed, so we could only rebuild the
> packages that are actually affected. That said, such behavior is a
> major drawback for using ocaml in real projects, i believe.

Not as simple as that. Actually, I tried comparing digests for the
current stable version and 3.08.2, and here are the differences I get.
You can of course do it yourself.
It is not as bad as I would have expected, which explains why you
could believe that binary compatibility is sometimes kept.
(Actually some work was done in the past to avoid having recompiling
everything after any small change in the compiler)

tet4-lib> find . -name \*.cmi -exec cmp "{}" /usr/local/lib/ocaml/"{}" \;
./toploop.cmi /usr/local/lib/ocaml/./toploop.cmi differ
./ocamldoc/odoc_sig.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_sig.cmi differ
./ocamldoc/odoc_opt.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_opt.cmi differ
./ocamldoc/odoc_dep.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_dep.cmi differ
./ocamldoc/odoc_ast.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_ast.cmi differ
./ocamldoc/odoc.cmi /usr/local/lib/ocaml/./ocamldoc/odoc.cmi differ
./camlp4/ast2pt.cmi /usr/local/lib/ocaml/./camlp4/ast2pt.cmi differ
./camlp4/pa_o.cmi /usr/local/lib/ocaml/./camlp4/pa_o.cmi differ
./camlp4/pa_o_fast.cmi /usr/local/lib/ocaml/./camlp4/pa_o_fast.cmi differ

tet4-lib> find . -name \*.cmx -exec cmp "{}" /usr/local/lib/ocaml/"{}" \;
./camlp4/pr_dump.cmx /usr/local/lib/ocaml/./camlp4/pr_dump.cmx differ
./camlp4/pa_r.cmx /usr/local/lib/ocaml/./camlp4/pa_r.cmx differ
./camlp4/pa_o.cmx /usr/local/lib/ocaml/./camlp4/pa_o.cmx differ
./camlp4/pa_o_fast.cmx /usr/local/lib/ocaml/./camlp4/pa_o_fast.cmx differ

So, any program that extends ocamldoc or accesses the toploop module
must be recompiled.
Concerning camlp4, I'm not expert enough to know whether a change in
the above interfaces has consequences, but you should assume this is
the case.
As of this writing, the .cmx's don't seem to have changed much.

Unfortunately, the above is only about changes in the libraries.
There were also bug fixes in the compiler, which result in slightly
different .cmi's for some programs using classes.
For instance, the GObj module in lablgtk. As a result, anything
depending on lablgtk would have to be recompiled, but you cannot
deduce it from the above listing.

> > You can safely assume that every new release breaks binary compatibility.
> > (I.e. that some digests in the libraries have changed.)
> 
> Yep, understood that now, 3.08.2 came as a big surprise though, as we somehow
> expected no binary compatibility change for a bug fix release.
> 
> But now we know about it, and i will enable the full-rebuild process for the
> 3.08.3 release, hoping that it will be in time for the debian/sarge release.

The most reasonable thing to do.
If some individuals want to take the risk of not recompiling what
seems to work, this is ok for them, but usually you don't want to do
this in a distribution.

If you're ready to do the checks by hand, there is yet another option.
The binary incompatibilities are not platform dependent. So if you
have a problem with the speed of some platforms, you can just
recompile everything on a single platform, and mark everything which
installs modified files as needing a recompilation on all
platforms. Hopefully, this should be sufficient (at least for binary
compatibility).

        Jacques Garrigue

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Thu, Jan 20, 2005 at 02:53:23PM +0900, Jacques Garrigue wrote:
> From: Sven Luther <sven.luther@wanadoo.fr>
> > > Binary compatibility as you get it in C is just a hack: you drop some
> > > consistency checks, and hope that the user is clever enough to not use
> > > incompatible libraries. Ocaml chooses the safe way. This could be made
> > > a bit more resilient, particularly for bytecode, but you would still
> > > have breakages in the bug fix branch.
> > 
> > What would be nice in this light, would be an exact list of the digests, and
> > thus the modules, that changed, so we could only rebuild the
> > packages that are actually affected. That said, such behavior is a
> > major drawback for using ocaml in real projects, i believe.
> 
> Not as simple as that. Actually, I tried comparing digests for the
> current stable version and 3.08.2, and here are the differences I get.
> You can of course do it yourself.

Nice. Stefano tried something such to some degree to analyse the 3.08.2
breakage.

> It is not as bad as I would have expected, which explains why you
> could believe that binary compatibility is sometimes kept.

Exactly, and the fact that maybe i had to big expectations for the bug-fix
branches, but well.

> (Actually some work was done in the past to avoid having recompiling
> everything after any small change in the compiler)

Cool.

> tet4-lib> find . -name \*.cmi -exec cmp "{}" /usr/local/lib/ocaml/"{}" \;
> ./toploop.cmi /usr/local/lib/ocaml/./toploop.cmi differ

If i remember right, the toploop is not part of the runtime, and only used by
ocamlmktop, right ? No library should depend on it, so there should be no
major problem.

> ./ocamldoc/odoc_sig.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_sig.cmi differ
> ./ocamldoc/odoc_opt.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_opt.cmi differ
> ./ocamldoc/odoc_dep.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_dep.cmi differ
> ./ocamldoc/odoc_ast.cmi /usr/local/lib/ocaml/./ocamldoc/odoc_ast.cmi differ
> ./ocamldoc/odoc.cmi /usr/local/lib/ocaml/./ocamldoc/odoc.cmi differ
> ./camlp4/ast2pt.cmi /usr/local/lib/ocaml/./camlp4/ast2pt.cmi differ
> ./camlp4/pa_o.cmi /usr/local/lib/ocaml/./camlp4/pa_o.cmi differ
> ./camlp4/pa_o_fast.cmi /usr/local/lib/ocaml/./camlp4/pa_o_fast.cmi differ
> 
> tet4-lib> find . -name \*.cmx -exec cmp "{}" /usr/local/lib/ocaml/"{}" \;
> ./camlp4/pr_dump.cmx /usr/local/lib/ocaml/./camlp4/pr_dump.cmx differ
> ./camlp4/pa_r.cmx /usr/local/lib/ocaml/./camlp4/pa_r.cmx differ
> ./camlp4/pa_o.cmx /usr/local/lib/ocaml/./camlp4/pa_o.cmx differ
> ./camlp4/pa_o_fast.cmx /usr/local/lib/ocaml/./camlp4/pa_o_fast.cmx differ

So ocamldoc and camlp4, the usual culprits. What broke most in 3.08.2 where
the thread module.

> So, any program that extends ocamldoc or accesses the toploop module
> must be recompiled.
> Concerning camlp4, I'm not expert enough to know whether a change in
> the above interfaces has consequences, but you should assume this is
> the case.
> As of this writing, the .cmx's don't seem to have changed much.

Ok.

> Unfortunately, the above is only about changes in the libraries.
> There were also bug fixes in the compiler, which result in slightly
> different .cmi's for some programs using classes.

Ah, but would those not be detected in the above ? Could you elaborate more
about those changes ? 

> For instance, the GObj module in lablgtk. As a result, anything
> depending on lablgtk would have to be recompiled, but you cannot
> deduce it from the above listing.

Yep, but once we know lablgtk needs a rebuild, we can travel the dependency
tree from there, no problem.

Actually one could imagine a little tool checking all the existing source
codes for presence of the affected modules. Reusing some code of ocamldep
maybe ? 

> > > You can safely assume that every new release breaks binary compatibility.
> > > (I.e. that some digests in the libraries have changed.)
> > 
> > Yep, understood that now, 3.08.2 came as a big surprise though, as we somehow
> > expected no binary compatibility change for a bug fix release.
> > 
> > But now we know about it, and i will enable the full-rebuild process for the
> > 3.08.3 release, hoping that it will be in time for the debian/sarge release.
> 
> The most reasonable thing to do.

Yes, but a huge amount of work in the current state, so if we could avoid it ...

> If some individuals want to take the risk of not recompiling what
> seems to work, this is ok for them, but usually you don't want to do
> this in a distribution.

Bah.

> If you're ready to do the checks by hand, there is yet another option.
> The binary incompatibilities are not platform dependent. So if you
> have a problem with the speed of some platforms, you can just
> recompile everything on a single platform, and mark everything which
> installs modified files as needing a recompilation on all
> platforms. Hopefully, this should be sufficient (at least for binary
> compatibility).

Ah, ok, nice to know.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Christophe TROESTLER]

On Thu, 13 Jan 2005, Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> wrote:
> 
> It is reasonnable to assume that 3.08.3 will not be binary compatible.

BTW, what about then changing the library locations
(e.g. /usr/lib/ocaml/3.08) to include the "minor number"
(e.g. /usr/lib/ocaml/3.08.3)? [Applies also to ld.conf, /usr/local,
the default findlib location,...]  That would make easy to have
several versions of OCaml installed on the same machine (especially
for /usr/local) and will make it clear one has to recompile existing
libraries.

My 2¢,
ChriS

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Thu, Jan 27, 2005 at 04:40:50PM +0100, Christophe TROESTLER wrote:
> On Thu, 13 Jan 2005, Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> wrote:
> > 
> > It is reasonnable to assume that 3.08.3 will not be binary compatible.
> 
> BTW, what about then changing the library locations
> (e.g. /usr/lib/ocaml/3.08) to include the "minor number"
> (e.g. /usr/lib/ocaml/3.08.3)? [Applies also to ld.conf, /usr/local,
> the default findlib location,...]  That would make easy to have
> several versions of OCaml installed on the same machine (especially
> for /usr/local) and will make it clear one has to recompile existing
> libraries.

Sure, and that is what we will have to do. This is a lengthy process though,
and we had hoped to avoid this. This means not only changing the LIBDIR name,
but also changing the virtual versioned package from ocaml-3.08 to
ocaml-3.08.3, which our autobuilders still have trouble with.

We should really have done this for 3.08.2, but 3.08.1 was ok.

Also notice that on debian systems, you should not install into plain
/usr/local, but into /usr/local/lib/ocaml/3.08.3 too. the default ld.conf
knows about that, but sadly ocaml doesn't know how to look for various LIBDIR
path by itself.

Friendly,

Sven Luther

Re: [Caml-list] binary compatibility of 3.08.3

[Christophe TROESTLER]

On Sun, 30 Jan 2005, Sven Luther <sven.luther@wanadoo.fr> wrote:
> 
> Also notice that on debian systems, you should not install into
> plain /usr/local, but into /usr/local/lib/ocaml/3.08.3

That's what I do -- with the advantage it is readily build from
`ocamlc -version` (well in fact I install in
/usr/local/stow/<package>/lib/ocaml/3.08.3 but that's another story).

Thanks to you and other debian maintainers for making our life easier!

Regards,
ChriS

Re: [Caml-list] binary compatibility of 3.08.3

[Stefan Monnier]

> Sure, and that is what we will have to do. This is a lengthy process though,
> and we had hoped to avoid this. This means not only changing the LIBDIR name,
> but also changing the virtual versioned package from ocaml-3.08 to
> ocaml-3.08.3, which our autobuilders still have trouble with.

Somehow I can't understand why someone would go through so much trouble
when using the name "3.09" instead of "3.08.3" would solve the problem much
more cheaply.


        Stefan

Re: [Caml-list] binary compatibility of 3.08.3

[Sven Luther]

On Sun, Jan 30, 2005 at 10:28:32AM -0500, Stefan Monnier wrote:
> > Sure, and that is what we will have to do. This is a lengthy process though,
> > and we had hoped to avoid this. This means not only changing the LIBDIR name,
> > but also changing the virtual versioned package from ocaml-3.08 to
> > ocaml-3.08.3, which our autobuilders still have trouble with.
> 
> Somehow I can't understand why someone would go through so much trouble
> when using the name "3.09" instead of "3.08.3" would solve the problem much
> more cheaply.

well, we have now a devel branch which will maturate in a 3.09 in some time
(this summer usually :). This is a good thinf since previously we were forced
to get the fixes individually out of CVS.

That said, using 3.08.3 or 3.09 is really something fully transparent to the
user anyway.

Friendly,

Sven Luther

Re: [Caml-list] Bug in Unix library on Mac?

[Damien Doligez]

On Jan 10, 2005, at 19:40, spiral voice wrote:

> some users are having problems with MLDonkey, an Ocaml P2P application.
>
> "I think I've verified that the problem is in the OCAML unix library, 
> not in mldonkey."
>
> The problem is described here:
> http://savannah.nongnu.org/bugs/?func=detailitem&item_id=11384

This is already in the Ocaml bug tracking system, and it will be fixed 
soon.

-- Damien