From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 5F5CD7F946 for ; Fri, 20 Jun 2014 17:36:08 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.216.45; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.216.45 as permitted sender) identity=mailfrom; client-ip=209.85.216.45; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-qa0-f45.google.com) identity=helo; client-ip=209.85.216.45; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-qa0-f45.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AoIBAHdUpFPRVdgtm2dsb2JhbABZg19agm2qFZZTAYECCBYPAQEBAQEGCwsJFCiEAwEBAQQSER0BGxILAQMMBgULDQICCR0CAiEBAREBBQEKEgYTEhCICwEDEQ2gK2qLJ4Fygw+SHwoZJwMKZYVMEQEFDIEeiymCIweCd4FMAQOEYwWTY4F5gUOMFYQRGCmEdDsv X-IPAS-Result: AoIBAHdUpFPRVdgtm2dsb2JhbABZg19agm2qFZZTAYECCBYPAQEBAQEGCwsJFCiEAwEBAQQSER0BGxILAQMMBgULDQICCR0CAiEBAREBBQEKEgYTEhCICwEDEQ2gK2qLJ4Fygw+SHwoZJwMKZYVMEQEFDIEeiymCIweCd4FMAQOEYwWTY4F5gUOMFYQRGCmEdDsv X-IronPort-AV: E=Sophos;i="5.01,514,1400018400"; d="scan'208";a="81180657" Received: from mail-qa0-f45.google.com ([209.85.216.45]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 20 Jun 2014 17:36:07 +0200 Received: by mail-qa0-f45.google.com with SMTP id v10so3324370qac.32 for ; Fri, 20 Jun 2014 08:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Y0yjcrkM2eGt8vMnbnm/v39qxkd8HIe8ospzyhvqJDQ=; b=0uAMEcSN0JEUTPixvCmeZzu5tiVxxAAmy+dqUiyoaexyUbOAT298PVMe2LOLivur6m 4O1hQpBGFSsRSYG2A6RVyxs1lcOOy54aT1Zv9o+cZLwIISiS+w5cciKVQ5Rh96+NgMRg bacKCTgYCtN0Ev4DWaNl49KOHwMPsSkbHp1BCqJ96P+aMaqM4Q70Lllh4gdafe1FjADi NRRsidaDPyg1U7/NODlVDpSIOW8ssFfxYFSldkdaA+rgj4n2GmbI2xviabaLt3zEDz/q cAXCVQ02XjbVnOgkuaKnQy+3pL0gH3hLhoEhJhn3g4GYzanA0RHGw2wThMJHVnFeQ9vq 8enA== X-Received: by 10.224.11.137 with SMTP id t9mr6226797qat.4.1403278566758; Fri, 20 Jun 2014 08:36:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.28.75 with HTTP; Fri, 20 Jun 2014 08:35:25 -0700 (PDT) In-Reply-To: References: From: Gabriel Scherer Date: Fri, 20 Jun 2014 17:35:25 +0200 Message-ID: To: Jeremy Yallop Cc: jean-vincent.loddo@lipn.univ-paris13.fr, Caml List Content-Type: text/plain; charset=UTF-8 Subject: Re: [Caml-list] Memory leaks generated by Scanf.fscanf? It looks like ephemerons would be a perfect fit to fix this issue, but unfortunately they're not yet available. It should be possible instead, at each call of the memo function, to iterate on the table and remove any item for file-descriptor which has been closed (I don't think checking whether a file-descriptor is closed is provided by an OCaml-land function right now, but it'd be easy to add to the runtime). That would make Scanning.from_channel slower (linear in the number of opened channels, though we could easily amortize by checking for all N new channels), but remove the leak, I think. On Fri, Jun 20, 2014 at 3:01 PM, Jeremy Yallop wrote: > On 20 June 2014 13:29, wrote: >> working on Marionnet (https://launchpad.net/marionnet), I noticed a serious >> memory leak making the system unusable after a few tens of minutes. After >> investigation, the problem seems to be related to Scanf.fscanf. > > It looks like your leak is caused by the 'memo' table in the Scanf > module that associates a lookahead buffer with each input channel: > > https://github.com/ocaml/ocaml/blob/trunk/stdlib/scanf.ml#L393 > > as explained by a comment in the Scanf code: > > https://github.com/ocaml/ocaml/blob/trunk/stdlib/scanf.ml#L268-L320 > > Entries are added to the table for each input channel used for > scanning, but there's no mechanism for removing entries. This would > be worth raising on Mantis. > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs