From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 43EF37FB13 for ; Thu, 4 Dec 2014 17:45:14 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of kennethadammiller@gmail.com) identity=pra; client-ip=209.85.214.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of kennethadammiller@gmail.com designates 209.85.214.174 as permitted sender) identity=mailfrom; client-ip=209.85.214.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ob0-f174.google.com) identity=helo; client-ip=209.85.214.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="postmaster@mail-ob0-f174.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4BAB+PgFTRVdaum2dsb2JhbABZg1hYBIMBw0+GFgKBFgcWAQEBAQERAQEBAQEGCwsJFC6EAgEBAQMBEhEdARsRDQMBCwYDAgsNJwMCAiEBAREBBQELEQYTIogGAQMJCQ2jOpBNPTGLMIFsgweLOAoZJw1ZhQUBAQEHAQEBAQEXAQEEDo0hgQGCMguCcYFRBYpTiVOCIoFjPYFygSMQjEiCNIF2EiWBEIIJgjohMIJFAQEB X-IPAS-Result: Av4BAB+PgFTRVdaum2dsb2JhbABZg1hYBIMBw0+GFgKBFgcWAQEBAQERAQEBAQEGCwsJFC6EAgEBAQMBEhEdARsRDQMBCwYDAgsNJwMCAiEBAREBBQELEQYTIogGAQMJCQ2jOpBNPTGLMIFsgweLOAoZJw1ZhQUBAQEHAQEBAQEXAQEEDo0hgQGCMguCcYFRBYpTiVOCIoFjPYFygSMQjEiCNIF2EiWBEIIJgjohMIJFAQEB X-IronPort-AV: E=Sophos;i="5.07,516,1413237600"; d="scan'208";a="91685301" Received: from mail-ob0-f174.google.com ([209.85.214.174]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 04 Dec 2014 17:45:12 +0100 Received: by mail-ob0-f174.google.com with SMTP id nt9so6729120obb.5 for ; Thu, 04 Dec 2014 08:45:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=JKZ25j72KjSc7wm7CfWU/ltNi1RDIzBgfp6pZgztxIs=; b=EYTEkZVAvME2JX9OlrtfeuWQ7QXXNosjkCanCeqC+qSP+gg0Z6/vveujcoz9FJzTgO bLZlnlkjmUk3r6IVP61JoJ93sPlB+uXGuVszSwSZqjekxKGJ5+dinuO5OW9snqzRZ0xd kIklTFaO2yeLpEpF1aXiHnwaxHC6icwJFteSnaBZX14nVWNulCuwVE7UvGX7UKVEYsgg cwSgHsTuXEbBiovIWtVrqjXUOcfawbB86i1ZhOFxohRziOZSpTDtozDyNET0PyxNNezF +9sKAyEpfyfB4WaGWtr7tJvR5p3MQrnYNCZ288Tg/6dFCWYRtBb9S/JnrL3hp9l6JN3V k1RQ== MIME-Version: 1.0 X-Received: by 10.60.94.197 with SMTP id de5mr7138054oeb.64.1417711511806; Thu, 04 Dec 2014 08:45:11 -0800 (PST) Received: by 10.182.197.1 with HTTP; Thu, 4 Dec 2014 08:45:11 -0800 (PST) In-Reply-To: References: <54801373.3010506@fugmann.net> <5480309B.3020402@fugmann.net> Date: Thu, 4 Dec 2014 11:45:11 -0500 Message-ID: From: Kenneth Adam Miller To: caml users Content-Type: multipart/alternative; boundary=089e011825ee0e28ab050966afea Subject: Re: [Caml-list] Potential OCaml-ZMQ memory management problems --089e011825ee0e28ab050966afea Content-Type: text/plain; charset=UTF-8 Actually, I literally just found the location of the error... I'm trying to send some data from ocaml that is causing ZMQ to segfault... On Thu, Dec 4, 2014 at 11:39 AM, Kenneth Adam Miller < kennethadammiller@gmail.com> wrote: > ok, I tried it with Gc.major() and Gc.minor(); and that didn't eliminate > the segfault. > > There's an issue with debugging too; when I compile my unit tests for > debugging or byte code, it reports parameters not being found (as in the > string returned from ZMQ is empty), even though a simple recompile with > native code makes that specific mysterious error go away. In any case, > because it does not receive the parameters while compiled for debug or byte > code, it cannot proceed to the point where the segfault occurs. Also, just > removing some of the preceeding tests in order that they not be in the way > when compiled for byte or debug code causes the segfault to not occur... > > How do I debug this in binary mode? > > On Thu, Dec 4, 2014 at 5:04 AM, Kenneth Adam Miller < > kennethadammiller@gmail.com> wrote: > >> Ok I'll give that a shot when I get a chance to edit it. Thanks, >> gc.major... I'm new to ocaml and I'm still reading through RWO, on chapter >> 9 learning about how modules compose nicely :) >> >> On Thu, Dec 4, 2014 at 4:59 AM, Anders Fugmann >> wrote: >> >>> The garbage collector settings seems quite sane to me. You could try to >>> force a major collection every call to iltrans. >>> >>> Gc.major () >>> >>> It could be that the segfault is really an out of memory situation. >>> Maybe you have a leak in your code somewhere (Not releasing references to >>> large objects). >>> >>> /Anders >>> >>> >>> On 12/04/2014 09:02 AM, Kenneth Adam Miller wrote: >>> >>>> I'm using zmq and ocaml-zmq as installed by opam; I'm not at my work >>>> computer or I would provide those details. >>>> >>>> Yes, I will try to work on creating a reproducible script >>>> >>>> Could it possibly be a call to Tunegc.set_gc () from >>>> https://github.com/argp/bap/blob/master/ocaml/tunegc.ml ? >>>> >>>> I notice that this error didn't appear in single calls to my utility; >>>> basically, I'm using iltrans from bap and I'm using it in a long living >>>> process, unlike the current implementation which expects a few >>>> transformations and then process death. I did notice that when I run it, >>>> if I'm watching system monitor that it begins consuming vast amounts of >>>> memory (quickly grows from a few megabytes to 800+) before it hits >>>> segfault. Would there be any way to restore the aggressiveness of the GC >>>> between calls to iltrans? >>>> >>>> On Thu, Dec 4, 2014 at 2:55 AM, Anders Fugmann >>> > wrote: >>>> >>>> Hi Kenneth, >>>> >>>> The Ocaml-zmq code copies data from received messages into memory >>>> under >>>> the control of the ocaml garbage collector, and immediatly frees the >>>> ZMQ buffers. >>>> >>>> You do not need to explicitly free the data received from call to >>>> recv. >>>> >>>> Can you produce a small sample code that exposes the problem? We are >>>> using the ocaml-zmq binding extensively, and have not seen any >>>> problems. >>>> >>>> If I were to take a wild guess, it either a mismatch between library >>>> versions or the garbage-collector collecting the socket or zmq >>>> context. >>>> >>>> What version of the ocaml-zmq bindings and libzmq (c impl) are you >>>> using? >>>> >>>> /Anders >>>> >>>> >>>> >>>> On 12/04/2014 07:09 AM, Kenneth Adam Miller wrote: >>>> >>>> I'm using ocaml-zmq (https://github.com/issuu/__ocaml-zmq >>>> ) and I think I'm >>>> encountering a memory management issue. I could be wrong >>>> however, but >>>> basically the issue (I think) is I have a rather large set of >>>> messages >>>> to send via zmq, and I'm getting a segfault. >>>> >>>> Does anybody know if I need to free the strings received from >>>> zmq recv >>>> functions in ocaml? If so how do I do that from ocaml? >>>> >>>> There's no code because this is just a general novice ocaml >>>> questions. >>>> >>>> >>>> >>>> >>> >> > --089e011825ee0e28ab050966afea Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Actually, I literally just found the location of the error= ... I'm trying to send some data from ocaml that is causing ZMQ to segf= ault...

On T= hu, Dec 4, 2014 at 11:39 AM, Kenneth Adam Miller <kennethadammil= ler@gmail.com> wrote:
ok, I tried it with Gc.major() and Gc.minor(); and that didn= 9;t eliminate the segfault.

There's an issue with de= bugging too; when I compile my unit tests for debugging or byte code, it re= ports parameters not being found (as in the string returned from ZMQ is emp= ty), even though a simple recompile with native code makes that specific my= sterious error go away. In any case, because it does not receive the parame= ters while compiled for debug or byte code, it cannot proceed to the point = where the segfault occurs. Also, just removing some of the preceeding tests= in order that they not be in the way when compiled for byte or debug code = causes the segfault to not occur...

How do I debug t= his in binary mode?=C2=A0

On Thu, D= ec 4, 2014 at 5:04 AM, Kenneth Adam Miller <kennethadammiller@gm= ail.com> wrote:
Ok I'll give that a shot when I get a chance to edit it. Thanks, = gc.major... I'm new to ocaml and I'm still reading through RWO, on = chapter 9 learning about how modules compose nicely :)

On Thu, Dec 4, 2014 at= 4:59 AM, Anders Fugmann <anders@fugmann.net> wrote:
The garbage collector settings seems quite sane= to me. You could try to force a major collection every call to iltrans.

Gc.major ()

It could be that the segfault is really an out of memory situation. Maybe y= ou have a leak in your code somewhere (Not releasing references to large ob= jects).

/Anders


On 12/04/2014 09:02 AM, Kenneth Adam Miller wrote:
I'm using zmq and ocaml-zmq as installed by opam; I'm not at my wor= k
computer or I would provide those details.

Yes, I will try to work on creating a reproducible script

Could it possibly be a call to Tunegc.set_gc () from
https://github.com/argp/bap/blob/master/ocaml/tunegc.ml<= /a> ?

I notice that this error didn't appear in single calls to my utility; basically, I'm using iltrans from bap and I'm using it in a long li= ving
process, unlike the current implementation which expects a few
transformations and then process death. I did notice that when I run it,
if I'm watching system monitor that it begins consuming vast amounts of=
memory (quickly grows from a few megabytes to 800+) before it hits
segfault. Would there be any way to restore the aggressiveness of the GC
between calls to iltrans?

On Thu, Dec 4, 2014 at 2:55 AM, Anders Fugmann <anders@fugmann.net
 <mailto:anders@f= ugmann.net>> wrote:

=C2=A0 =C2=A0 Hi Kenneth,

=C2=A0 =C2=A0 The Ocaml-zmq code copies data from received messages into me= mory under
=C2=A0 =C2=A0 the control of the ocaml garbage collector, and immediatly fr= ees the
=C2=A0 =C2=A0 ZMQ buffers.

=C2=A0 =C2=A0 You do not need to explicitly free the data received from cal= l to recv.

=C2=A0 =C2=A0 Can you produce a small sample code that exposes the problem?= We are
=C2=A0 =C2=A0 using the ocaml-zmq binding extensively, and have not seen an= y problems.

=C2=A0 =C2=A0 If I were to take a wild guess, it either a mismatch between = library
=C2=A0 =C2=A0 versions or the garbage-collector collecting the socket or zm= q context.

=C2=A0 =C2=A0 What version of the ocaml-zmq bindings and libzmq (c impl) ar= e you
=C2=A0 =C2=A0 using?

=C2=A0 =C2=A0 /Anders



=C2=A0 =C2=A0 On 12/04/2014 07:09 AM, Kenneth Adam Miller wrote:

 =C2=A0 =C2=A0 =C2=A0 =C2=A0 I'm using ocaml-zmq (https://github.com/issuu/__= ocaml-zmq
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <https://github.com/issuu/ocaml-zmq>) an= d I think I'm
=C2=A0 =C2=A0 =C2=A0 =C2=A0 encountering a memory management issue. I could= be wrong
=C2=A0 =C2=A0 =C2=A0 =C2=A0 however, but
=C2=A0 =C2=A0 =C2=A0 =C2=A0 basically the issue (I think) is I have a rathe= r large set of
=C2=A0 =C2=A0 =C2=A0 =C2=A0 messages
=C2=A0 =C2=A0 =C2=A0 =C2=A0 to send via zmq, and I'm getting a segfault= .

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Does anybody know if I need to free the strings= received from
=C2=A0 =C2=A0 =C2=A0 =C2=A0 zmq recv
=C2=A0 =C2=A0 =C2=A0 =C2=A0 functions in ocaml? If so how do I do that from= ocaml?

=C2=A0 =C2=A0 =C2=A0 =C2=A0 There's no code because this is just a gene= ral novice ocaml
=C2=A0 =C2=A0 =C2=A0 =C2=A0 questions.







--089e011825ee0e28ab050966afea--