Strange hugeness of .o, .cmo, and .cmi files

Strange hugeness of .o, .cmo, and .cmi files

[John Prevost]

I've been working on a curses library written entirely in O'Caml, and
recently copied my sources over a slow ppp link to my machine at home.
Quite by chance, I also got a copy of the binaries, and noticed the
sizes of the .cmo and .o files for one module in particular.

This module is not terribly complex (at least from my understanding),
but it is rather large in another sense.  I was wondering if anybody
could explain what's happening internally and how I could be more
efficient.

type entry = { term_names : string;
               string_table : string;
               booleans : bool array;
               numbers : int array;
               strings : int array }

type terminfo = {
    auto_left_margin : bool;
    auto_right_margin : bool;
[...461 lines similar, with some : int option and some : string options...]
}

[...small amount of code for reading terminfo files into entries...]

let process_entry e =
  let get_bool = get_bool e in
  let get_number = get_number e in
  let get_string = get_string e in

  { auto_left_margin               = get_bool 0;
    auto_right_margin              = get_bool 1;
[... another 461 lines ...]
  }

[...a few more little routines...]

Obviously, there are a lot of fields in the record type, and I assume that this is where the blowup is happening, but I'm not at all sure where.

Another module, with apparently comparable amounts of code, maybe a
bit more complex is tparm.ml.  Here's a comparison:

-rw-r--r--   1 prevost  users         721 Mar 31 01:07 tparm.cmi
-rw-r--r--   1 prevost  users        6400 Mar 31 01:00 tparm.cmo
-rw-r--r--   1 prevost  users         398 Mar 31 01:07 tparm.cmx
-rw-r--r--   1 prevost  users       15784 Mar 31 01:07 tparm.o

-rw-r--r--   1 prevost  users       59432 Mar 31 01:17 terminfo.cmi
-rw-r--r--   1 prevost  users       11396 Mar 31 01:00 terminfo.cmo
-rw-r--r--   1 prevost  users        1009 Mar 31 01:07 terminfo.cmx
-rw-r--r--   1 prevost  users      259076 Mar 31 01:07 terminfo.o

(compiling with -compact or without produces the same object size)

A bigger .cmi file was expected from the huge record, and I can deal
with, since it isn't used in every executable linked against it.

I'd expected a bigger object, but not 20x as big!

So, is there anything I can do to get around this, preferably without
needing to use -compact?  Would a different way of accessing fields be
better?  What if I went back to what I had before I decided a record
would be a good idea, which was using get_string entry some_field for
access?  (for example:

let auto_left_margin e = get_bool e 1

or

let auto_left_margin = get_bool 1

with a different formulation of get_bool.)


Since everything else uses the terminfo module, and since it gets
linked into the programs whole-hog, I'd very much like to cut down on
the space use.

Thanks,
John

Re: Strange hugeness of .o, .cmo, and .cmi files

[Xavier Leroy]

> [Surprisingly large .cmo / .o files]

It's hard to say what's happening without seeing the source or the
generated assembly code.  If that kind of things doesn't scare you,
just compile with ocamlopt -S and look at the resulting .s file.
However, I'll venture a guess:

> Obviously, there are a lot of fields in the record type, and I
> assume that this is where the blowup is happening, but I'm not at
> all sure where.

OCaml 2.01 had a known code size problem with the
{ record with lbl = newval } construct, when "record" belongs to a
type with many labels.  This was fixed in OCaml 2.02, however.

Regards,

- Xavier Leroy

Re: Strange hugeness of .o, .cmo, and .cmi files

[John Prevost]

Xavier Leroy <Xavier.Leroy@inria.fr> writes:

> It's hard to say what's happening without seeing the source or the
> generated assembly code.  If that kind of things doesn't scare you,
> just compile with ocamlopt -S and look at the resulting .s file.
> However, I'll venture a guess:

The source is almost exactly as you see, for the most part.  I can
package it along to you if you'd like.

The start of the notably huge part of the .s file is at the end of
this message.

> OCaml 2.01 had a known code size problem with the
> { record with lbl = newval } construct, when "record" belongs to a
> type with many labels.  This was fixed in OCaml 2.02, however.

I've been using 2.02 nearly since it came out, so it's not a 2.01 vs
2.02 problem.  All I'm doing is:

let get_bool e n = try e.booleans.(n) with _ -> false

let get_number e n =
  try
    let contents = e.numbers.(n) in
    if contents = -1 then None else Some contents
  with _ -> None

let get_string e n =
  try
    let start = e.strings.(n) in
    if start = -1 then
      None
    else
      let finish = String.index_from e.string_table start '\000' in
      Some (String.sub e.string_table start (finish - start))
  with _ -> None

let process_entry e =
  let get_bool = get_bool e in
  let get_number = get_number e in
  let get_string = get_string e in

  { auto_left_margin               = get_bool 0;
    auto_right_margin              = get_bool 1;
    no_esc_ctlc                    = get_bool 2;
[ ... for 463 total lines ... ]
    enter_right_hl_mode            = get_string 389;
    enter_top_hl_mode              = get_string 390;
    enter_vertical_hl_mode         = get_string 391 }

I'd guess from the below that it's for some reason making a long entry
in the frametable for each one of the lines above.  I don't know
enough about how ocamlopt generates code to know what that means.

John.


Partial .s file:

Terminfo_frametable:
	.long	502
	.long	.L648
	.word	4
	.word	0
	.align	4
	.long	.L646
	.word	16
	.word	1
	.word	8
	.align	4
	.long	.L645
	.word	12
	.word	1
	.word	0
	.align	4
	.long	.L644
	.word	12
	.word	1
	.word	0
	.align	4
	.long	.L643
	.word	12
	.word	0
	.align	4
	.long	.L642
	.word	12
	.word	0
	.align	4
	.long	.L640
	.word	12
	.word	0
	.align	4
	.long	.L639
	.word	12
	.word	1
	.word	4
	.align	4
	.long	.L638
	.word	12
	.word	1
	.word	0
	.align	4
	.long	.L636
	.word	1852
	.word	462
	.word	0
	.word	4
	.word	12
	.word	16
	.word	20
	.word	24
	.word	28
	.word	32
	.word	36
	.word	40
[ ... counting up by 4 to ... ]
	.word	1844
	.word	3
	.align	4
	.long	.L633
	.word	1852
	.word	461
	.word	0
	.word	4
	.word	12
[... again, counting down the .word 461 until it is .word 1, with one
     less entry after each time... ]
[... a few more smaller bits... ...]