From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 036517FB13 for ; Thu, 4 Dec 2014 11:04:36 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of kennethadammiller@gmail.com) identity=pra; client-ip=209.85.218.45; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of kennethadammiller@gmail.com designates 209.85.218.45 as permitted sender) identity=mailfrom; client-ip=209.85.218.45; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-oi0-f45.google.com) identity=helo; client-ip=209.85.218.45; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="postmaster@mail-oi0-f45.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArwBAOcwgFTRVdotlGdsb2JhbABag1hYBIMBw0+GFgKBEAcWAQEBAQERAQEBAQcLCwkSMIQCAQEBAwESER0BGxENAwELBgMCCw0nAwICIgERAQUBCxEGEyKIBgEDCQkNonGQTT0xizCBbIMHizQKGScNWYUFAQEBAQYBAQEBARcBAQQOjSGDMwuCcYFRBYpTiVOEBYIvgSIQjnyBdhIlgRCCCYI6ITCCRQEBAQ X-IPAS-Result: ArwBAOcwgFTRVdotlGdsb2JhbABag1hYBIMBw0+GFgKBEAcWAQEBAQERAQEBAQcLCwkSMIQCAQEBAwESER0BGxENAwELBgMCCw0nAwICIgERAQUBCxEGEyKIBgEDCQkNonGQTT0xizCBbIMHizQKGScNWYUFAQEBAQYBAQEBARcBAQQOjSGDMwuCcYFRBYpTiVOEBYIvgSIQjnyBdhIlgRCCCYI6ITCCRQEBAQ X-IronPort-AV: E=Sophos;i="5.07,514,1413237600"; d="scan'208";a="91618806" Received: from mail-oi0-f45.google.com ([209.85.218.45]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 04 Dec 2014 11:04:35 +0100 Received: by mail-oi0-f45.google.com with SMTP id a141so12131607oig.18 for ; Thu, 04 Dec 2014 02:04:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=w5dHnXUqfdb/UoG/6d9rZOeggRVdU0zZsuouHYW9y8A=; b=nGBECH/fLSToWWwFntcQMexDZrdKwC9+jUJloAD3oVx2vzRgtKi8fEejm5UkFKwqk8 7H3/yu+T3ivkRSLjXT1z3QnWdvIcr8J7QJfkgBAc3JzjwQTc+8qlhrM7Q9Z0vruPrEPq jHA3tr9q9D6z+W+W3S6lLTAkQebh/wb2bJHXudJnlcitbkcILvWU7hl3NeAAfnq6iJmB 60Bp0QdaETLNVAVCu6S8FUlyVt2p2QfIuDrBvKljKYwSTQXxVgKyhILvlkVbZsdvc7sf rnOC9pWzsK0+f4XJmRVFqvD4M8j72xEX9u5l8ytWwgqaspa6T6GjaOyppAfLnN4RTaM8 8spA== MIME-Version: 1.0 X-Received: by 10.182.199.106 with SMTP id jj10mr6106538obc.27.1417687474210; Thu, 04 Dec 2014 02:04:34 -0800 (PST) Received: by 10.182.197.1 with HTTP; Thu, 4 Dec 2014 02:04:34 -0800 (PST) In-Reply-To: <5480309B.3020402@fugmann.net> References: <54801373.3010506@fugmann.net> <5480309B.3020402@fugmann.net> Date: Thu, 4 Dec 2014 05:04:34 -0500 Message-ID: From: Kenneth Adam Miller To: caml users Content-Type: multipart/alternative; boundary=e89a8ff1ca2a4daa0f0509611666 Subject: Re: [Caml-list] Potential OCaml-ZMQ memory management problems --e89a8ff1ca2a4daa0f0509611666 Content-Type: text/plain; charset=UTF-8 Ok I'll give that a shot when I get a chance to edit it. Thanks, gc.major... I'm new to ocaml and I'm still reading through RWO, on chapter 9 learning about how modules compose nicely :) On Thu, Dec 4, 2014 at 4:59 AM, Anders Fugmann wrote: > The garbage collector settings seems quite sane to me. You could try to > force a major collection every call to iltrans. > > Gc.major () > > It could be that the segfault is really an out of memory situation. Maybe > you have a leak in your code somewhere (Not releasing references to large > objects). > > /Anders > > > On 12/04/2014 09:02 AM, Kenneth Adam Miller wrote: > >> I'm using zmq and ocaml-zmq as installed by opam; I'm not at my work >> computer or I would provide those details. >> >> Yes, I will try to work on creating a reproducible script >> >> Could it possibly be a call to Tunegc.set_gc () from >> https://github.com/argp/bap/blob/master/ocaml/tunegc.ml ? >> >> I notice that this error didn't appear in single calls to my utility; >> basically, I'm using iltrans from bap and I'm using it in a long living >> process, unlike the current implementation which expects a few >> transformations and then process death. I did notice that when I run it, >> if I'm watching system monitor that it begins consuming vast amounts of >> memory (quickly grows from a few megabytes to 800+) before it hits >> segfault. Would there be any way to restore the aggressiveness of the GC >> between calls to iltrans? >> >> On Thu, Dec 4, 2014 at 2:55 AM, Anders Fugmann > > wrote: >> >> Hi Kenneth, >> >> The Ocaml-zmq code copies data from received messages into memory >> under >> the control of the ocaml garbage collector, and immediatly frees the >> ZMQ buffers. >> >> You do not need to explicitly free the data received from call to >> recv. >> >> Can you produce a small sample code that exposes the problem? We are >> using the ocaml-zmq binding extensively, and have not seen any >> problems. >> >> If I were to take a wild guess, it either a mismatch between library >> versions or the garbage-collector collecting the socket or zmq >> context. >> >> What version of the ocaml-zmq bindings and libzmq (c impl) are you >> using? >> >> /Anders >> >> >> >> On 12/04/2014 07:09 AM, Kenneth Adam Miller wrote: >> >> I'm using ocaml-zmq (https://github.com/issuu/__ocaml-zmq >> ) and I think I'm >> encountering a memory management issue. I could be wrong >> however, but >> basically the issue (I think) is I have a rather large set of >> messages >> to send via zmq, and I'm getting a segfault. >> >> Does anybody know if I need to free the strings received from >> zmq recv >> functions in ocaml? If so how do I do that from ocaml? >> >> There's no code because this is just a general novice ocaml >> questions. >> >> >> >> > --e89a8ff1ca2a4daa0f0509611666 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Ok I'll give that a shot when I get a chance to edit i= t. Thanks, gc.major... I'm new to ocaml and I'm still reading throu= gh RWO, on chapter 9 learning about how modules compose nicely :)

On Thu, Dec 4, 2014 a= t 4:59 AM, Anders Fugmann <anders@fugmann.net> wrote:
The garbage collector settings seems quite san= e to me. You could try to force a major collection every call to iltrans.
Gc.major ()

It could be that the segfault is really an out of memory situation. Maybe y= ou have a leak in your code somewhere (Not releasing references to large ob= jects).

/Anders


On 12/04/2014 09:02 AM, Kenneth Adam Miller wrote:
I'm using zmq and ocaml-zmq as installed by opam; I'm not at my wor= k
computer or I would provide those details.

Yes, I will try to work on creating a reproducible script

Could it possibly be a call to Tunegc.set_gc () from
https://github.com/argp/bap/blob/master/ocaml/tunegc.ml<= /a> ?

I notice that this error didn't appear in single calls to my utility; basically, I'm using iltrans from bap and I'm using it in a long li= ving
process, unlike the current implementation which expects a few
transformations and then process death. I did notice that when I run it,
if I'm watching system monitor that it begins consuming vast amounts of=
memory (quickly grows from a few megabytes to 800+) before it hits
segfault. Would there be any way to restore the aggressiveness of the GC
between calls to iltrans?

On Thu, Dec 4, 2014 at 2:55 AM, Anders Fugmann <anders@fugmann.net
 <mailto:anders@f= ugmann.net>> wrote:

=C2=A0 =C2=A0 Hi Kenneth,

=C2=A0 =C2=A0 The Ocaml-zmq code copies data from received messages into me= mory under
=C2=A0 =C2=A0 the control of the ocaml garbage collector, and immediatly fr= ees the
=C2=A0 =C2=A0 ZMQ buffers.

=C2=A0 =C2=A0 You do not need to explicitly free the data received from cal= l to recv.

=C2=A0 =C2=A0 Can you produce a small sample code that exposes the problem?= We are
=C2=A0 =C2=A0 using the ocaml-zmq binding extensively, and have not seen an= y problems.

=C2=A0 =C2=A0 If I were to take a wild guess, it either a mismatch between = library
=C2=A0 =C2=A0 versions or the garbage-collector collecting the socket or zm= q context.

=C2=A0 =C2=A0 What version of the ocaml-zmq bindings and libzmq (c impl) ar= e you
=C2=A0 =C2=A0 using?

=C2=A0 =C2=A0 /Anders



=C2=A0 =C2=A0 On 12/04/2014 07:09 AM, Kenneth Adam Miller wrote:

 =C2=A0 =C2=A0 =C2=A0 =C2=A0 I'm using ocaml-zmq (https://github.com/issuu/__= ocaml-zmq
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <https://github.com/issuu/ocaml-zmq>) an= d I think I'm
=C2=A0 =C2=A0 =C2=A0 =C2=A0 encountering a memory management issue. I could= be wrong
=C2=A0 =C2=A0 =C2=A0 =C2=A0 however, but
=C2=A0 =C2=A0 =C2=A0 =C2=A0 basically the issue (I think) is I have a rathe= r large set of
=C2=A0 =C2=A0 =C2=A0 =C2=A0 messages
=C2=A0 =C2=A0 =C2=A0 =C2=A0 to send via zmq, and I'm getting a segfault= .

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Does anybody know if I need to free the strings= received from
=C2=A0 =C2=A0 =C2=A0 =C2=A0 zmq recv
=C2=A0 =C2=A0 =C2=A0 =C2=A0 functions in ocaml? If so how do I do that from= ocaml?

=C2=A0 =C2=A0 =C2=A0 =C2=A0 There's no code because this is just a gene= ral novice ocaml
=C2=A0 =C2=A0 =C2=A0 =C2=A0 questions.





--e89a8ff1ca2a4daa0f0509611666--