From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 7438F7FEEB for ; Sat, 18 Jun 2016 17:15:07 +0200 (CEST) IronPort-PHdr: 9a23:aC/SahLgoLD+C207xNmcpTZWNBhigK39O0sv0rFitYgVKvvxwZ3uMQTl6Ol3ixeRBMOAu6MC2rCd7P6ocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC3oLmjavtpdX6WEZhunmUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO9MxGlldhq5lhf44dqsrtY4q3wD89pozcNLUL37cqIkVvQYSW1+ayFm0vb2rgHORhej4X4VU2Ne0kYZQluN0BavcZ77qCr3sqJG0ymXJ8DsBeQ7UD647qpvDgTjiCodOiQR/2Tei8g2h6Ve9kGPvRt6lsTxaZuJNfxJROXqW94HReZc6ctLHWQVGoSnc6MKBvAHMPsepI748Qhd5SCiDBWhUbu8ggRDgWX7iOhniuk= Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=post@marneu.com; spf=Pass smtp.mailfrom=post@marneu.com; spf=None smtp.helo=postmaster@zulu1878.startdedicated.de Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of post@marneu.com) identity=pra; client-ip=62.75.236.16; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="post@marneu.com"; x-sender="post@marneu.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of post@marneu.com designates 62.75.236.16 as permitted sender) identity=mailfrom; client-ip=62.75.236.16; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="post@marneu.com"; x-sender="post@marneu.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@zulu1878.startdedicated.de) identity=helo; client-ip=62.75.236.16; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="post@marneu.com"; x-sender="postmaster@zulu1878.startdedicated.de"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0B2BADNZGVX/xDsSz5DGoQUSwExgn6mSoUajAKBeiJghRUCgVcUAQEBAQEBAQFkJ4IxghoBAQEDAQwXSAMLBQsJAhgNHQICISQSBhMSiAQDDwwBCS2SH50djDADCoNeAQEBAQEFAQEBAQEBARIOhS+Cb4JWgkOEfiuCLwEEhksMkWs0AYMtgliGKoF6gjeHCoVhhyJoh2sCHjaCO4E3bAGKRwEBAQ X-IPAS-Result: A0B2BADNZGVX/xDsSz5DGoQUSwExgn6mSoUajAKBeiJghRUCgVcUAQEBAQEBAQFkJ4IxghoBAQEDAQwXSAMLBQsJAhgNHQICISQSBhMSiAQDDwwBCS2SH50djDADCoNeAQEBAQEFAQEBAQEBARIOhS+Cb4JWgkOEfiuCLwEEhksMkWs0AYMtgliGKoF6gjeHCoVhhyJoh2sCHjaCO4E3bAGKRwEBAQ X-IronPort-AV: E=Sophos;i="5.26,487,1459807200"; d="asc'?scan'208";a="181717436" Received: from wanjo.de (HELO zulu1878.startdedicated.de) ([62.75.236.16]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Jun 2016 17:15:06 +0200 Received: from localhost (fuenf.wanjo.de.local [127.0.0.1]) by zulu1878.startdedicated.de (Postfix) with ESMTP id 3CAD24D82E45; Sat, 18 Jun 2016 17:15:05 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at zulu1878.startdedicated.de Received: from zulu1878.startdedicated.de ([127.0.0.1]) by localhost (fuenf.wanjo.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7VGRr98uGl0Q; Sat, 18 Jun 2016 17:15:04 +0200 (CEST) Received: from martins-macbook-pro.fritz.box (x4e301e41.dyn.telefonica.de [78.48.30.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by zulu1878.startdedicated.de (Postfix) with ESMTPSA id D462C4D825FD; Sat, 18 Jun 2016 17:15:03 +0200 (CEST) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_43036AC2-811F-44B7-BC7D-B8561F3DE24E"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.6b2 From: =?utf-8?Q?=22Martin_R=2E_Neuh=C3=A4u=C3=9Fer=22?= In-Reply-To: Date: Sat, 18 Jun 2016 17:15:03 +0200 Cc: OCaml List Message-Id: References: <7182303B-EBFF-4CFC-A7DF-B5C7A059509F@marneu.com> To: Gabriel Scherer X-Mailer: Apple Mail (2.3124) Subject: Re: [Caml-list] Presumed bug in OCaml's garbage collector or in its Weak module... --Apple-Mail=_43036AC2-811F-44B7-BC7D-B8561F3DE24E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > Am 17.06.2016 um 21:23 schrieb Gabriel Scherer : >=20 >=20 > 3. The finalizer is never called for blocks that are still alive; stated = otherwise, a block that has been finalized can never been presented as a va= lue to a C-stub anymore. >=20 > I think that this does not hold in general: the finalizer should only be = called when there is no reference to the value around, *but* the finalizer = action can create a new reference (for example you may insert the value int= o a (live) global cache), so the value may be live after the finalizer has = been called -- and be sent to a C stub, etc. It is definitely true that finalizers that are registered with Gc.finalise = may generate new values (or make the value live again). But I do not see how to create a new value within a finalizer that is attac= hed to a custom block on the C-layer. As the manual states, one must neither trigger a garbage collection in such= a finalizer, nor call-back into OCaml code. >=20 > This remark may not apply to your actual reproduction case (your finalize= r shouldn't make the value reachable from the OCaml heap). >=20 > (There were changes between the 4.03 development cycle that would explain= the difference in behavior between beta1 and beta2, see in particular http= ://caml.inria.fr/mantis/view.php?id=3D7210 and https://github.com/ocaml/oca= ml/pull/22 ; but I'll leave memory-runtime experts comment on that.) >=20 > On Fri, Jun 17, 2016 at 2:09 PM, "Martin R. Neuh=C3=A4u=C3=9Fer" wrote: > Dear all, >=20 > after an intense week of debugging some large C-bindings, I presume to ha= ve found a bug in OCaml=E2=80=99s garbage collection code or in its Weak mo= dule. > To summarize, it seems as if OCaml=E2=80=99s garbage collector and its We= ak module sometimes release a custom block (by calling its finalizer) too e= arly, i.e. when it is still reachable. >=20 > I hesitate a bit before opening an =E2=80=9Eofficial=E2=80=9C issue on Ma= ntis as I might very well overlook some detail. Therefore I=E2=80=99d like = to double-check some of the assumptions that I have made when writing my C-= stubs. Any corrections are highly welcome: > 1. Custom blocks may be moved around in memory by the GC, but they are ne= ver duplicated. > 2. The finalizer for each custom block that is allocated by caml_alloc_cu= stom is called at most once. > 3. The finalizer is never called for blocks that are still alive; stated = otherwise, a block that has been finalized can never been presented as a va= lue to a C-stub anymore. >=20 > There is a small example program available on github: https://github.com/= martin-neuhaeusser/ocaml_bug where the above assumptions seem to be violate= d. >=20 > The idea is as follows: > Each custom block that the example allocates contains a pointer to struct= ure that is malloc=E2=80=99ed outside the OCaml heap; each custom block uni= quely refers to such a structure. > Upon creation of a custom block, a special flag-value is stored in its co= rresponding (newly created) structure to indicate that the block is alive. = Once a custom block=E2=80=99s finalizer is called, the flag is changed to i= ndicate finalization of the block. I assume that after the finalizer return= s, the custom block is unreachable (and perhaps its memory reclaimed by the= GC); however, for debugging, I intentionally keep the custom block=E2=80= =99s structure allocated on the unmanaged heap. Whenever a custom block is = presented to the C-stubs, they check that the corresponding flag-value with= in its corresponding non-managed heap structure indicates liveness of the b= lock. As it turns out, this invariant is violated, if the Weak module and t= he GC are stressed. >=20 > Of course, it might very well be that there is a bug in my C-stubs that i= s responsible for that behavior. I tested with different versions of OCaml,= with different outcomes: The example program terminates correctly with OCa= ml 3.12.1, 4.00.0, 4.00.1, 4.01.0, and 4.03.0+beta1 whereas it crashes due = to the above error when compiled with OCaml 4.02.0, 4.02.1, 4.02.2, 4.02.3,= 4.03.0+beta2, and the released 4.03.0. The tests were run on 64bit Debian = Linux and 64bit MacOS X. >=20 > I'd very much appreciate any help, hints to bugs in my code, or requests = for clarification. >=20 > Thanks and best regards, > Martin >=20 > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs >=20 --Apple-Mail=_43036AC2-811F-44B7-BC7D-B8561F3DE24E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iEYEARECAAYFAldlZXcACgkQbEbxnBb9spiVGwCfVKfjA/OeEDgcDKTaeDhY56Y9 kisAn0E6guJksov2QGWR1DDO8iJwiw9s =c4RW -----END PGP SIGNATURE----- --Apple-Mail=_43036AC2-811F-44B7-BC7D-B8561F3DE24E--