From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id q3KFRN3f001084 for ; Fri, 20 Apr 2012 17:27:23 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtoBACl/kU/RVdW2kGdsb2JhbABDhWerSQgiAQEBAQkJDQcUBCOCCQEBAQMBEgIMIAEBGh4ECwsEBzshAQERAQUBHBkIGodeAQMGBZwXCopkZQZTgnMBhTcKGScNV4h2Bol1g02CC4EaiGGFModqizyDIT2EDw X-IronPort-AV: E=Sophos;i="4.75,453,1330902000"; d="scan'208";a="140962648" Received: from mail-yx0-f182.google.com ([209.85.213.182]) by mail4-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 20 Apr 2012 17:27:12 +0200 Received: by yenl9 with SMTP id l9so7634036yen.27 for ; Fri, 20 Apr 2012 08:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=de6BduPanxcH1dVCQJ7LUVSnG+Ms4LWNKA/lu5PIRK4=; b=oHcBlnepmq4uJppBStF7IOXzDA0ZMO1mQ0K9zhGPlkJceBCdd44gPYvJQLMIusT2Pe i+hycR5GcZ6QNZkDOlaTrPzv8cJLYHjeZWkovRkZxGE8Aej0a42eyDR9+1PhwV8+KwsT xUUT//jUuA2dS6kbU53Yjxi8WfnWxB8PhC8J5nZolClk5X3YdutBvY26Mj/rqve9s7X7 wXp+VwMSj+uxZmvnHmrlqXsAOVucCmLYFwqzOq3aNs/End5BFAmt0HJeCgO70u+n9gu2 uCXIezZHGylrUxKvfHsG0N+X5v2xpm7ySjD9BEEeX5x874COz34MQgyQV80mV74ZXkY8 PbKQ== MIME-Version: 1.0 Received: by 10.236.79.195 with SMTP id i43mr6337446yhe.53.1334935631758; Fri, 20 Apr 2012 08:27:11 -0700 (PDT) Received: by 10.236.145.36 with HTTP; Fri, 20 Apr 2012 08:27:11 -0700 (PDT) Received: by 10.236.145.36 with HTTP; Fri, 20 Apr 2012 08:27:11 -0700 (PDT) In-Reply-To: References: Date: Fri, 20 Apr 2012 19:27:11 +0400 Message-ID: From: SerP To: caml-list@inria.fr Content-Type: multipart/alternative; boundary=bcaec5396dfe2103d204be1dec9c Subject: [Caml-list] Re: Memory fragmentation --bcaec5396dfe2103d204be1dec9c Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable Ocaml 3.12.0., Lwt 2.3.1 On Apr 20, 2012 7:12 PM, "SerP" wrote: > Hi! > We have developped a daemon on ocaml using the lwt lib(libev). It > processes about 800 requests per second, but it increases to 2 Gb in memo= ry > for a hour of work. We have studied the problem for a long time, using > mtrace, mallinfo and other tools, and we tried to change GC params. We > found out that setting up GC.minor_heap_size=3D10Mb =C1 > major_heap_increment=3D2=EDb will make the daemon grow slower. mallinf=CF= shows > that the memory (1,5G) is allocated using mmap (blkhd field in mallinfo > struct) and arena size - about 20M In this case, first calls of GC.compact > compress the daemon to 200 Mb (live -110Mb) , and mallinfo show decreasing > of total size of memory allocated by mmap. The daemon keeps growing, but = it > allocates the memory to arena (using brk), and calls of GC.compact leads = to > decrease of heap_bytes to 200M, but arena size (1.5 Gb) does not decrease, > just doing less uordblks and more fordblks (fileds I.e., after first calls > of GC.compact, the daemon starts using brk much more than mmap, and cant > memory given back to the system. > > If Gc.allocation_policy is first fit, memory usage is stable, and it's > grow very little, but worked very slowly. > > Any suggestions are very welcome. Thanks > --bcaec5396dfe2103d204be1dec9c Content-Type: text/html; charset=KOI8-R Content-Transfer-Encoding: quoted-printable

Ocaml 3.12.0., Lwt 2.3.1

On Apr 20, 2012 7:12 PM, "SerP" <serp256@gmail.com&= gt; wrote:

Hi!
We have developped a daemon on ocaml using the lwt lib(libev). It processes= about 800 requests per second, but it increases to 2 Gb in memory for a ho= ur of work. We have studied the problem for a long time, using mtrace, mall= info and other tools, and we tried to change GC params. We found out that s= etting up GC.minor_heap_size=3D10Mb =C1 major_heap_increment=3D2=EDb will m= ake the daemon grow slower. mallinf=CF shows that the memory (1,5G) is allo= cated using mmap (blkhd field in mallinfo struct) and arena size - about 20= M In this case, first calls of GC.compact compress the daemon to 200 Mb (li= ve -110Mb) , and mallinfo show decreasing of total size of memory allocated= by mmap. The daemon keeps growing, but it allocates the memory to arena (u= sing brk), and calls of GC.compact leads to decrease of heap_bytes to 200M,= but arena size (1.5 Gb) does not decrease, just doing less uordblks and mo= re fordblks (fileds I.e., after first calls of GC.compact, the daemon start= s using brk much more than mmap, and cant memory given back to the system.<= /p>

If Gc.allocation_policy is first fit, memory usage is stable, and it'= ;s grow very little, but worked very slowly.

Any suggestions are very welcome. Thanks

--bcaec5396dfe2103d204be1dec9c--