From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id q3KFCCBR000544 for ; Fri, 20 Apr 2012 17:12:14 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtkBAGJ7kU/RVaC2kGdsb2JhbABDhWerSQgiAQEBAQkJDQcUBCOCCgEBBBICDCABGx4DDAYJAQY7IgERAQUBHDuHXgEDC5wdCotJBlOCc4U4ChknDVeIdgEFC403gguBGgSIXYUyh2qOXT2EDw X-IronPort-AV: E=Sophos;i="4.75,453,1330902000"; d="scan'208";a="140960648" Received: from mail-gy0-f182.google.com ([209.85.160.182]) by mail4-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 20 Apr 2012 17:12:13 +0200 Received: by ghrr20 with SMTP id r20so7599879ghr.27 for ; Fri, 20 Apr 2012 08:12:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Jzp3i9vaooTJyYs7s16WRRmSF7ApsAcmQIubNOr/u84=; b=m5uB0iYr37mra5jSNAvI9RZjfvRUsqdJXpTXgpcalrFCYpAyLD+hd1Cz2Bfnsbts0g Fj/UdhKN2c5kA0URq9MAD4xwKtYneuf6tugMMLuV45I5oQLGl6/YRbbzA8M3h2ZJ30kW EdjWkmO2B4fUBZ31GVk+9OeTUrvNelOjKffSn2NzmuvrEwOLF5Y46ZDC38HOGprbJv1w zlEblRyi/e/ttoEKOhnoEk8QWWC2KBgAYSG74BskmJKBUN9JjL+FV/LZ+s+AsZHpIMSz O72DvkSu9bl4Q+YGXhkimUoq+tO9jP69x6c6vIh916tkIA1fw+zLpiXEBb9+uACDyCsS 7lcg== MIME-Version: 1.0 Received: by 10.236.82.134 with SMTP id o6mr6364982yhe.51.1334934732425; Fri, 20 Apr 2012 08:12:12 -0700 (PDT) Received: by 10.236.145.36 with HTTP; Fri, 20 Apr 2012 08:12:12 -0700 (PDT) Received: by 10.236.145.36 with HTTP; Fri, 20 Apr 2012 08:12:12 -0700 (PDT) In-Reply-To: References: Date: Fri, 20 Apr 2012 19:12:12 +0400 Message-ID: From: SerP To: caml-list@inria.fr Content-Type: multipart/alternative; boundary=20cf300e5463864aa904be1db643 Subject: [Caml-list] Memory fragmentation --20cf300e5463864aa904be1db643 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable Hi! We have developped a daemon on ocaml using the lwt lib(libev). It processes about 800 requests per second, but it increases to 2 Gb in memory for a hour of work. We have studied the problem for a long time, using mtrace, mallinfo and other tools, and we tried to change GC params. We found out that setting up GC.minor_heap_size=3D10Mb =C1 major_heap_increment=3D2=EDb = will make the daemon grow slower. mallinf=CF shows that the memory (1,5G) is allocated using mmap (blkhd field in mallinfo struct) and arena size - about 20M In this case, first calls of GC.compact compress the daemon to 200 Mb (live -110Mb) , and mallinfo show decreasing of total size of memory allocated by mmap. The daemon keeps growing, but it allocates the memory to arena (using brk), and calls of GC.compact leads to decrease of heap_bytes to 200M, but arena size (1.5 Gb) does not decrease, just doing less uordblks and more fordblks (fileds I.e., after first calls of GC.compact, the daemon starts using brk much more than mmap, and cant memory given back to the system. If Gc.allocation_policy is first fit, memory usage is stable, and it's grow very little, but worked very slowly. Any suggestions are very welcome. Thanks --20cf300e5463864aa904be1db643 Content-Type: text/html; charset=KOI8-R Content-Transfer-Encoding: quoted-printable

Hi!
We have developped a daemon on ocaml using the lwt lib(libev). It processes= about 800 requests per second, but it increases to 2 Gb in memory for a ho= ur of work. We have studied the problem for a long time, using mtrace, mall= info and other tools, and we tried to change GC params. We found out that s= etting up GC.minor_heap_size=3D10Mb =C1 major_heap_increment=3D2=EDb will m= ake the daemon grow slower. mallinf=CF shows that the memory (1,5G) is allo= cated using mmap (blkhd field in mallinfo struct) and arena size - about 20= M In this case, first calls of GC.compact compress the daemon to 200 Mb (li= ve -110Mb) , and mallinfo show decreasing of total size of memory allocated= by mmap. The daemon keeps growing, but it allocates the memory to arena (u= sing brk), and calls of GC.compact leads to decrease of heap_bytes to 200M,= but arena size (1.5 Gb) does not decrease, just doing less uordblks and mo= re fordblks (fileds I.e., after first calls of GC.compact, the daemon start= s using brk much more than mmap, and cant memory given back to the system.<= /p>

If Gc.allocation_policy is first fit, memory usage is stable, and it'= ;s grow very little, but worked very slowly.

Any suggestions are very welcome. Thanks

--20cf300e5463864aa904be1db643--