From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id D2A857FDC8 for ; Tue, 12 Jan 2016 11:33:33 +0100 (CET) IronPort-PHdr: 9a23:ElhzPxQF6cVNvtbVKkTKAdHA0Npsv+yvbD5Q0YIujvd0So/mwa64bR2N2/xhgRfzUJnB7Loc0qyN4/6mCTdLucjJmUtBWaIPfidNsd8RkQ0kDZzNImzAB9muURYHGt9fXkRu5XCxPBsdMs//Y1rPvi/6tmZKSV3BPAZ4bt74BpTVx5zukbvipduDOE4Q2nKUWvBbElaflU3prM4YgI9veO4a6yDihT92QdlQ3n5iPlmJnhzxtY+a9Z9n9DlM6bp6r5YTGY2zRakzTKRZATI6KCh1oZSz7ViQBTeIszE6X3gWmxdVGBPI9lWye57rrir8/KIp3SCAIczwC7Y5RDSr4rpwUxLyoDwGOjs09nqRgct12vF1uhWk8jNy2YKcW52SMOJ7d6XbNYcbQ2RGdslcTSAEGZ+7a5MKBuwHe+pV+dqu72ASpAezUFH/TNjkzSVF0zqvhfU3 Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=info@gerd-stolpmann.de; spf=None smtp.mailfrom=info@gerd-stolpmann.de; spf=None smtp.helo=postmaster@mout.kundenserver.de Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=pra; client-ip=212.227.126.133; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=mailfrom; client-ip=212.227.126.133; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mout.kundenserver.de) identity=helo; client-ip=212.227.126.133; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="postmaster@mout.kundenserver.de"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BcAQAk1pRWlIV+49RehAxtiFmsA4kgGAqFbQKBIzwQAQEBAQEBAQEQAQEBAQcLCwkfMIItgggBAQRVJBALOwtXBhMJiCkBCb9XAQEBAQEBBAEBAQEBFQmFWoR3gQSEMTGCWAxBgTYFh2SPL4EVAoQsiBaCKIZ8BIVUjlM5glEkgUFxAYRmgUoBAQE X-IPAS-Result: A0BcAQAk1pRWlIV+49RehAxtiFmsA4kgGAqFbQKBIzwQAQEBAQEBAQEQAQEBAQcLCwkfMIItgggBAQRVJBALOwtXBhMJiCkBCb9XAQEBAQEBBAEBAQEBFQmFWoR3gQSEMTGCWAxBgTYFh2SPL4EVAoQsiBaCKIZ8BIVUjlM5glEkgUFxAYRmgUoBAQE X-IronPort-AV: E=Sophos;i="5.20,556,1444687200"; d="asc'?scan'208";a="160130625" Received: from mout.kundenserver.de ([212.227.126.133]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Jan 2016 11:33:13 +0100 Received: from office1.lan.sumadev.de ([188.107.82.221]) by mrelayeu.kundenserver.de (mreue004) with ESMTPSA (Nemesis) id 0LipzN-1Zkz5b3FE5-00cyjX; Tue, 12 Jan 2016 11:33:06 +0100 Received: from [192.168.65.10] (unknown [192.168.65.10]) by office1.lan.sumadev.de (Postfix) with ESMTPSA id 083ECDC05D; Tue, 12 Jan 2016 11:33:05 +0100 (CET) Message-ID: <1452594780.13126.63.camel@e130.lan.sumadev.de> From: Gerd Stolpmann To: "Neuhaeusser, Martin" Cc: "caml-list@inria.fr" Date: Tue, 12 Jan 2016 11:33:00 +0100 In-Reply-To: <965631B03C670145ABB9F693E51622530D0D763B@DENBGAT9EK5MSX.ww902.siemens.net> References: <965631B03C670145ABB9F693E51622530D0D763B@DENBGAT9EK5MSX.ww902.siemens.net> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-ALNce3dAg8H4a4ZQ6II0" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 X-Provags-ID: V03:K0:EpW1tKfC6i8MQ/oDHa3pScvAxy4+XxvGmCnlcqrIPuXmFbcbJad erMfCXaSu4wzZAb9/qkNessm1aCEMzh0v9uTs9all91QNRnrIo4eeEq16/FWoCMjsVeeIk6 jsXNN8WY17zpiXvn8K0vv/RazZG3Xur+3jJqk+fbw+zbOgQKoP8cuSTAW/JSXL2pwG9xvZP tqVNNZompww5zOmDNxYKw== X-UI-Out-Filterresults: notjunk:1;V01:K0:QuBuCvNnv+U=:7QsqszzA6glb/jPbWauYrR ZFueUkCVxDVkg1CZFft2E5/MsGoUEV8vSAU2fusBN//AQO/DreogHtE3FTpO0uJnrmKJjPMdG P8FECkWY31JvbktKyb2pJJYrgO2SYa71sPNv2S8+eALfkskF1+WRrqzxv73C63+N12z4dbHnh K4mUjheCddsPydNAHgix3HRqpjDWWxZyLY0Vn78J+0n2rX8D/jx4aFyB4UOhuGm3LMiLQfedn X6L4WfhwtiaRaTguz3LJ7PnC3L0LAFWEF8x+hu5NOI8/uC1iYAJrDK7GiKbNzwvS5nGFXa2AW RfLEAHZrfAHqAri/NsNODcd08Jlld2I/P4Nff8b4sfeAljJ0jWoCpg35Eq3zLDBh/ulZTtn1W 4E+ABZjfbw19h1mN31TCsPKornIQe7vp+aLgTO9g6lO9kE8fxKYbybJc4vZPTO3/mLRl5WZfC kW09ZrokjY/Kz26aBkPXVs8c9CDZhq+mxcFdmAuLG0wI+MdSx7vT7avBnazJh8eCZ3DFG8xky NhaxL1TC+tFtIz1yixZckdQR9xPtJsU7vGCVpmswcNcknCTpQL4CaBXUmqfoAE6bPpHulm+Lc 3HYQjPet/AjY8uS2h/JAYAapbG6hYxf1yWg7PALJRCNiaA9tbwsjMbzr7GcyacseBVzVmuwcI Q0949zes2KL/qAxmcbdExIz47tldfLCBG5bqe7tc2SiOV7UTesOROkesiMYipXmlf4Jg= Subject: Re: [Caml-list] The OCaml garbage collector, finalisers, and the right way of disposing native pointers in C bindings --=-ALNce3dAg8H4a4ZQ6II0 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: quoted-printable Dealing with naked pointers from OCaml is notoriously difficult. As you found out, you have no good ways of controlling GC cycles, and to limit bad effects of that. Also, there is a dangling pointer problem - essentially the naked pointer can be mistaken as heap pointer between the time the memory has been freed and the naked pointer is set to null. Note that this aspect is practically impossible to do right from OCaml code, and even in a C function it is easy to get wrong, resulting in random crashes that occur infrequently. For these reasons naked pointers are strongly discouraged. The way to go is to wrap pointers into custom blocks (http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html#sec458), and to do all pointer management in C. Gerd Am Dienstag, den 12.01.2016, 09:12 +0000 schrieb Neuhaeusser, Martin: > Dear all, >=20 > during our work on some SMT solver bindings, a couple of question came up= regarding the behavior of the OCaml garbage collector (see also https://gi= thub.com/Z3Prover/z3/issues/411). Assume we have defined a record type cont= aining a native pointer to some object from an external C-DLL: >=20 > type ocaml_record =3D { > native_ptr np; > [... some more fields ...] > } >=20 > When creating an ocaml_record, we register a finalizer that makes the C l= ibrary dispose the data belonging to the native pointer once the GC collect= s the OCaml record value: >=20 > let f ocaml_record =3D NativeLib.dispose ocaml_record.np >=20 > let create [...] =3D > let new_ocaml_record =3D { ... } in > Gc.finalise f new_ocaml_record; > new_ocaml_record >=20 > When calling one of the C-stubs, we pass the native pointer from the OCam= l record value: >=20 > let get_native_pointer ocaml_record =3D ocaml_record.np >=20 > NativeLib.native_function (get_native_pointer ocaml_record) >=20 > However, this has the problem that if ocaml_record has become otherwise u= nreachable, the GC might collect ocaml_record directly after evaluating (ge= t_native_pointer ocaml_record), triggering the finalizer which disposes the= data pointed at by ocaml_record.native_ptr. The successive call to NativeL= ib.native_function (i.e. the C-stub) results in a segfault, as it tries to = access data that has previously been freed by the finalizer. >=20 > I assume this is a common problem in writing interfaces to C libraries. I= f so, is there a preferred way how to tackle it? > Two approaches that came to my mind are >=20 > 1. One could design the C-stubs such that they accept values of type ocam= l_record and extract the native pointer within the C stub. In the C-stubs, = the GC must not collect an item that has been "pinned" by the CAMLparamX ma= cros, right? > 2. One could invent some function that takes an ocaml_record, but does no= thing with it and whose calls do not get optimized away by the compiler... = Evaluating such a function after the call to NativeLib.native_function woul= d prevent the GC from collecting ocaml_record. However, this feels like a v= ery ugly hack.=20 >=20 > Are there any better ideas? Any help and suggestions are highly appreciat= ed. >=20 > Best, > Martin >=20 --=20 ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de My OCaml site: http://www.camlcity.org Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------ --=-ALNce3dAg8H4a4ZQ6II0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAABAgAGBQJWlNZdAAoJEAaM4b9ZLB5Tb54H/3O2xKwvXe3liCCK2hTH96AX 8vbdYN+40hw/TjNyNGfAfVwSyfuxyND/f00kLysBjKn7ssmUm6K4MjIKWCEowic/ AezadRsZWr2cLWqpl5AGwpBiA+94d9VeywoXmuzKqlvOhTw6T4nJucrsOni8bsGc 34/I7q+Paz0gbRMNqOJ0VhXN7muQ+uVYlQSN/yj4FTC5CsnGBrr67LIbpQd6Ke+Q X16skOOGx1vQZOLFyriL2Toflg8YAKQww7V9T+Rxfh0VNCKQ52yhNXM28YNBZIYC uOtz0+rr6+DJPbrsPcy0AmDdwYdDwR62n1uQmfG6WOSbrMWbcNZ42fdBrLJn9Bs= =+fQ4 -----END PGP SIGNATURE----- --=-ALNce3dAg8H4a4ZQ6II0--