[Caml-list] Timing module initializations (working but crashy)

[Caml-list] Timing module initializations (working but crashy)

[Adrien]

Hi,

A few months ago I think I saw some module initializations in lablgtk2
take a lot of time. By module initializations, I mean the evaluation of
toplevel expressions from source files at application startup.

I found out about ld's --wrap option which can wrap a symbol. I then
wrap all the camlSomeModule__entry symbols into a C function. When I
start the program, I get an output like:

  Initialiazing Pervasives.
  Done initializing Pervasives in 0.236261 milliseconds.
  Initialiazing Array.
  Done initializing Array in 21.480000 microseconds.
  [...]
  Initialiazing Random.
  Done initializing �H in 11.417000 microseconds.
  [...]
  Initialiazing GPack.
  Done initializing g in 2.782904 milliseconds.
  Initialiazing GButton.

  Program received signal SIGSEGV, Segmentation fault.
  0x000000000057618b in caml_oldify_local_roots ()

The bogus module name after module initialization is the first sign that
something has gone wrong (the string has not been changed: it's the char*
_pointer_ to the module name which has been changed).

My program is an ocaml wrapper to GCC (as a linker) which generates
wrapper functions for the modules named in the environment variable
"MODULES" (space-separated). The code is (currently) at:
  http://notk.org/~adrien/link.ml
You can build with: "ocamlopt unix.cmxa str.cmxa link.ml -o gcc".
Now, make sure that the directory with this "gcc" executable is in your
PATH and is the first element (I never said this was very clean :-) ).

In the output C code, commenting out the two lines which change indent,
makes the segfault go away but the memory corruption still happens.

How could I fix this code? And what are the constraints which I should
have taken into account?

I've managed to make the program survive long enough to finish starting
by setting the minor heap to 10M words with OCAMLRUNPARAM but I'd
really prefer something clean.


PS: In order to put all possible module names in the MODULES environment
variable, I use the following:
  pmake \
  && MODULES="$(nm caravel.native | grep 'T caml.*__entry$' \
    | cut -f3 -d' ' | sed -e 's/caml\(.*\)__entry/\1/g' | xargs)" \
  && rm src/_build/browser/caravel.native \
  && MODULES="${MODULES}" pmake \
  && gdb ./caravel.native
This builds the project, finds all the camlSomeModule__entry symbols
which are referenced in the output (native code) binary, removes the
final binary and builds the project again, thereby forcing a relink,
with MODULES set.

PS2: most initializations are very fast but I've already found some like
GWindow which takes 22ms here, 1000 times more than most other modules.
And less than 5% of the modules seem to be responsible for almost all
the initialization time.


Thanks,
Adrien Nader

Re: [Caml-list] Timing module initializations (working but crashy)

[Adrien]

Hi,

Kaustuv Chaudhuri mentionned a few things about this issue on IRC.

First, one thing I had forgotten in my previous email: I've tried making
the functions usual ocaml-callable C functions (with
CAML{param,local,return}*) but that didn't change anything. The
segfaults still occurs in the exact same place.

I've also removed the intermediate function. It generates ten times more
C code but removing the indirection should reduce the number of possible
issues. However the issue still happens at the same place.

Finally, by printing the addresses of the entry functions, we got the
following output:
  [...]
  (* this is GtkMain disabling auto-compaction *)
  New max overhead: 1000000%
  [...]

  Initialiazing OgtkFileProps through 0x50d170.
  Done initialiazing OgtkFileProps through 0x50d3bc.
  23.016000 microseconds spent initializing OgtkFileProps.

  Initialiazing GContainer through 0x50e8b0.
  Done initialiazing GContainer through 0x1.
  483.846000 microseconds spent initializing GContainer.

  Initialiazing GPack through 0x511fa0.
  Done initialiazing GPack through 0x1.
  1931.545000 microseconds spent initializing GPack.

  Initialiazing GButton through 0x5165c0.
  <
  Program received signal SIGSEGV, Segmentation fault.
  0x0000000000581176 in caml_oldify_local_roots ()

I don't really understand why (and, to be honest, how) the function
pointer would change but maybe there's something I don't fully know with
function pointers and printf's %p format.

I've put a new file online with the aforementionned changes:
  http://notk.org/~adrien/link2.ml

Any input is particularly appreciated. :-)

By the way, a few reasons I've started this: startup speed, potential
very slow initializations for which there was no instrumentation,
detecting useless code that impacts runtime, and the fact that I want to
make an application "not get start slower" but I had no hard numbers on
how how fast it currently starts.

-- 
Adrien Nader