From: Gabriel Scherer <gabriel.scherer@gmail.com>
To: Jeremy Yallop <yallop@gmail.com>
Cc: jean-vincent.loddo@lipn.univ-paris13.fr, Caml List <caml-list@inria.fr>
Subject: Re: [Caml-list] Memory leaks generated by Scanf.fscanf?
Date: Fri, 20 Jun 2014 17:35:25 +0200 [thread overview]
Message-ID: <CAPFanBGrfvs_Y9z2Ue6U-RQDVZ-h0KYViBrFNjCrVhSTdNQ5Jw@mail.gmail.com> (raw)
In-Reply-To: <CAAxsn=GkMuQz_hNqRLxsRbRpK0GxOf_3ZFFq_02bkM_Y95h3Cg@mail.gmail.com>
It looks like ephemerons would be a perfect fit to fix this issue, but
unfortunately they're not yet available.
It should be possible instead, at each call of the memo function, to
iterate on the table and remove any item for file-descriptor which has
been closed (I don't think checking whether a file-descriptor is
closed is provided by an OCaml-land function right now, but it'd be
easy to add to the runtime). That would make Scanning.from_channel
slower (linear in the number of opened channels, though we could
easily amortize by checking for all N new channels), but remove the
leak, I think.
On Fri, Jun 20, 2014 at 3:01 PM, Jeremy Yallop <yallop@gmail.com> wrote:
> On 20 June 2014 13:29, <jean-vincent.loddo@lipn.univ-paris13.fr> wrote:
>> working on Marionnet (https://launchpad.net/marionnet), I noticed a serious
>> memory leak making the system unusable after a few tens of minutes. After
>> investigation, the problem seems to be related to Scanf.fscanf.
>
> It looks like your leak is caused by the 'memo' table in the Scanf
> module that associates a lookahead buffer with each input channel:
>
> https://github.com/ocaml/ocaml/blob/trunk/stdlib/scanf.ml#L393
>
> as explained by a comment in the Scanf code:
>
> https://github.com/ocaml/ocaml/blob/trunk/stdlib/scanf.ml#L268-L320
>
> Entries are added to the table for each input channel used for
> scanning, but there's no mechanism for removing entries. This would
> be worth raising on Mantis.
>
> --
> Caml-list mailing list. Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
next prev parent reply other threads:[~2014-06-20 15:36 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-06-20 12:29 jean-vincent.loddo
2014-06-20 13:01 ` Jeremy Yallop
2014-06-20 15:35 ` Gabriel Scherer [this message]
2014-06-22 17:11 ` Benoît Vaugon
2014-06-23 9:06 ` François Bobot
2014-06-27 14:32 ` Jeremy Yallop
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPFanBGrfvs_Y9z2Ue6U-RQDVZ-h0KYViBrFNjCrVhSTdNQ5Jw@mail.gmail.com \
--to=gabriel.scherer@gmail.com \
--cc=caml-list@inria.fr \
--cc=jean-vincent.loddo@lipn.univ-paris13.fr \
--cc=yallop@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).