Tracking memory usage: GC output not same order as unix top command

Tracking memory usage: GC output not same order as unix top command

[Hugo Ferreira]

Hello,

I have a problem trying to identify where my algorithm consumes
so much memory. I have created a set of functions (see end of message)
that uses the GC module to obtain the memory used in the heap and print
that out to determine where memory is being consumed.

I use the "Gc.quick_stat ()" and measure memory consumption via
the "stat.Gc.heap_words * 8" value (64 bit machine). The output
shows memory usage below the 100M mark, however the unix command
"top" shows usage in the order of Gigabytes (at least 4.8). This
memory consumption grows gradually.

I would like to know:
1) Can I use "Gc.quick_stat" and "stat.Gc.heap_words" to measure
    total memory consumption?
2) Is my use of the GC module as shown below ok?
   For example:
   let h1 = Gc_mem.get_quick_stat () in
   let _ = do_something....
   let h2 = Gc_mem.get_quick_stat () in
   let dh = Gc_mem.delta_heap_used h1 h2 in
   let h1 = Gc_mem.string_of_stat_heap_used h1 in
   let h2 = Gc_mem.string_of_stat_heap_used h2 in
   let dh = Gc_mem.string_of_heap_used dh in
   Printf.printf "GC: get_maximal' (0) heap_before(%s) ; heap_after(%s) 
; heap_delta(%s)\n%!" h1 h2 dh


TIA,
Hugo F.


type multiplier =
   | U of int
   | K of float
   | M of float
   | G of float

let convert_units heap_words =
   let heap_words_k = (float_of_int heap_words) /. 1024.0 in
   if heap_words_k <= 1.0
   then U(heap_words)
   else let heap_words_m = heap_words_k /. 1024.0 in
        if heap_words_m <= 1.0
        then K(heap_words_k)
        else let heap_words_g = heap_words_m /. 1024.0 in
             if heap_words_g <= 1.0
             then M(heap_words_m)
             else G(heap_words_g)


let get_quick_stat unit = Gc.quick_stat ()

let delta_heap_used stat_1 stat_2 =
   let heap_words_1 = stat_1.Gc.heap_words * 8 in (* 64 bit machine *)
   let heap_words_2 = stat_2.Gc.heap_words * 8 in (* 64 bit machine *)
   let heap_words = abs ( heap_words_2 - heap_words_1 ) in
   heap_words

let string_of_heap_used heap_words =
   let value = convert_units heap_words in
   match value with
   | U(i) -> Printf.sprintf "%d" i
   | K(i) -> Printf.sprintf "%.3fK" i
   | M(i) -> Printf.sprintf "%.3fM" i
   | G(i) -> Printf.sprintf "%.3fG" i

let string_of_stat_heap_used stat =
   let heap_used = stat.Gc.heap_words * 8 in (* 64 bit machine *)
   string_of_heap_used heap_used

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Richard Jones]

On Mon, Aug 30, 2010 at 10:43:42AM +0100, Hugo Ferreira wrote:
> The output
> shows memory usage below the 100M mark, however the unix command
> "top" shows usage in the order of Gigabytes (at least 4.8). This
> memory consumption grows gradually.

The output of top isn't a reliable way to measure memory usage.

Really you should be looking at /proc/<pid>/maps.  That will show you
if, for example, the heap is becoming fragmented or if there's some
memory leak.  There is a nice utility for examining maps files and
system memory usage, but the name of it escapes me at the moment.

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Richard Jones]

On Mon, Aug 30, 2010 at 11:02:52AM +0100, Richard Jones wrote:
> On Mon, Aug 30, 2010 at 10:43:42AM +0100, Hugo Ferreira wrote:
> > The output
> > shows memory usage below the 100M mark, however the unix command
> > "top" shows usage in the order of Gigabytes (at least 4.8). This
> > memory consumption grows gradually.
> 
> The output of top isn't a reliable way to measure memory usage.
> 
> Really you should be looking at /proc/<pid>/maps.  That will show you
> if, for example, the heap is becoming fragmented or if there's some
> memory leak.  There is a nice utility for examining maps files and
> system memory usage, but the name of it escapes me at the moment.

I might have been thinking of this one:

http://lwn.net/Articles/329458/

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Hugo Ferreira]

Hi,

Richard Jones wrote:
> On Mon, Aug 30, 2010 at 11:02:52AM +0100, Richard Jones wrote:
>> On Mon, Aug 30, 2010 at 10:43:42AM +0100, Hugo Ferreira wrote:
>>> The output
>>> shows memory usage below the 100M mark, however the unix command
>>> "top" shows usage in the order of Gigabytes (at least 4.8). This
>>> memory consumption grows gradually.
>> The output of top isn't a reliable way to measure memory usage.
>>
>> Really you should be looking at /proc/<pid>/maps.  That will show you
>> if, for example, the heap is becoming fragmented or if there's some
>> memory leak. 

No looking for exact or detailed data. Just trying to identify which
function is the culprit. An order of magnitude in difference makes me
think I am doing something wrong.

>> There is a nice utility for examining maps files and
>> system memory usage, but the name of it escapes me at the moment.
> 
> I might have been thinking of this one:
> 
> http://lwn.net/Articles/329458/
> 

I looked at this tool. Going to ask the admin if he can install
this because I cannot interpret the output from the "/proc/<pid>/maps".

Thanks for the pointer,
Hugo F.

> Rich.
> 

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Richard Jones]

On Mon, Aug 30, 2010 at 11:33:48AM +0100, Hugo Ferreira wrote:
> I looked at this tool. Going to ask the admin if he can install
> this because I cannot interpret the output from the "/proc/<pid>/maps".

Here's a Perl script that I wrote quite a long time ago.  I don't know
if it still works, but worth looking at.

Rich.

----------------------------------------------------------------------
#!/usr/bin/perl -wT
# Parse /proc/*/maps file into a readable summary.
# $Id: maps.pl,v 1.1 2006/11/01 10:35:56 rich Exp $

no warnings qw(portable);

foreach my $filename (@ARGV) {
  my %devices;

  open MAPS, "<$filename" or die "$filename: $!";

  while (<MAPS>) {
    if (m/^([[:xdigit:]]+)-([[:xdigit:]]+) ([-rwxps]+) ([[:xdigit:]]+) ([[:xdigit:]]{2}:[[:xdigit:]]{2}) (\d+)\s*(.*)?/) {
      my $start = hex $1;
      my $end = hex $2;
      my $perms = $3;
      my $offset = hex $4;
      my $device = $5;
      my $inode = $6;
      my $filename = $7;

      my $size = $end - $start;

      # Create a record.
      my %rec =
	(
	 start => $start,
	 end => $end,
	 perms => $perms,
	 offset => $offset,
	 device => $device,
	 inode => $inode,
	 filename => $filename,
	 size => $size
	);

      # Key for storing this.
      my $key;
      if ($device ne "00:00" && $inode != 0) {
	$key = "$filename ($device $inode)"
      } elsif ($filename ne "") {
	$key = $filename
      } else {
	$key = "anonymous mapping"
      }

      # Store it.
      $devices{$key} = [] if !exists $devices{$key};
      push @{$devices{$key}}, \%rec
    } else {
      warn "ignored: $_\n"
    }
  }

  close MAPS;

  # Get the list of devices.
  my @devices = keys %devices;

  # For each device, print a summary.
  foreach (@devices) {
    print "$_:\n";
    my @recs = @{$devices{$_}};
    my $sum = 0;
    $sum += $_->{size} foreach @recs;
    printf ("  %d bytes %.1f MB\n", $sum, $sum/1024/1024);
    print "  segments:\n";
    foreach (@recs) {
      printf ("    %x-%x (%d bytes %.1f MB) %s %d\n",
	      $_->{start}, $_->{end}, $_->{size}, $_->{size}/1024/1024,
	      $_->{perms}, $_->{offset});
    }
  }
}
----------------------------------------------------------------------

-- 
Richard Jones
Red Hat

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Pietro Abate]

On Mon, Aug 30, 2010 at 10:43:42AM +0100, Hugo Ferreira wrote:
> I have a problem trying to identify where my algorithm consumes
> so much memory. I have created a set of functions (see end of message)
> that uses the GC module to obtain the memory used in the heap and print
> that out to determine where memory is being consumed.

have you tried ocamlviz ?
http://ocamlviz.forge.ocamlcore.org/


-- 
----
http://en.wikipedia.org/wiki/Posting_style

Re: [Caml-list] Tracking memory usage: GC output not same order as unix top command

[Hugo Ferreira]

Pietro Abate wrote:
> On Mon, Aug 30, 2010 at 10:43:42AM +0100, Hugo Ferreira wrote:
>> I have a problem trying to identify where my algorithm consumes
>> so much memory. I have created a set of functions (see end of message)
>> that uses the GC module to obtain the memory used in the heap and print
>> that out to determine where memory is being consumed.
> 
> have you tried ocamlviz ?
> http://ocamlviz.forge.ocamlcore.org/
> 
> 

No. Already been suggested though.

Thanks.
Hugo F.