Re: [Caml-list] Overflow with Spacetime

[Kenneth Adam Miller <kennethadammiller@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

Ok, that's understandable. No complaints on my end, spacetime is awesome.

On Thu, Aug 10, 2017 at 9:12 AM, Leo White <lwhite@janestreet.com> wrote:

> > Possibly I ran into a space leak while trying to find a space leak.
>
> 7GB for a 900MB profile sounds about right unfortunately. The format of
> spacetime profiles makes it difficult for the viewer to do much better than
> that. I hope to get around to changing the format to fix this, but probably
> won't until some time next year.
>
> I would recommend trying to get a smaller profile. Taking snapshots less
> often will help, as will running the program for a shorter time (even with
> the same number of snapshots). Failing that try getting access to a machine
> with more memory.
>
> On 9 August 2017 at 08:37, Kenneth Adam Miller <
> kennethadammiller@gmail.com> wrote:
>
>> Hello all,
>>
>>
>> I wrote a particular application, and after profiling I can see that the
>> garbage collector is using 95% of the time in the application and that it
>> uses up to 6 gigs of memory on a 1 megabyte input file for a run that
>> totals out to be about 2 minutes. A 27 kilobyte input file takes seconds to
>> run, so I know that there is some explosive leak or otherwise terrible
>> issue with the space complexity here. I doubled down, and got compilation
>> to run to completion using _tags to make sure that there were symbols and
>> profiling turned on (thanks Ivan, Drup).
>>
>> Additionally, I compiled my application with spacetime and ran it to
>> obtain a `spacetime-<pid>` binary, which I then attempted to get a report
>> on with prof_spacetime. prof_spacetime went up to about 7.3 gigs in memory
>> usage before it repeatedly would die with `Killed` or `Stack overflow` as
>> the error. I'm trying to figure out what else there is I can do to make
>> sure that the process runs to completion. I double checked to make sure
>> that I was running the prof_spacetime command built with just a 4.04.0
>> compiler to ensure that it wouldn't profile itself while trying to "Process
>> series". My spacetime dump is about 900 Mb, so I understand that it would
>> use some memory processing it. Possibly I ran into a space leak while
>> trying to find a space leak.
>>
>> In the meantime, I will try and find some intermediate size input to get
>> something I can work on - thought I should report this.
>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 3308 bytes --]