No unused code linking?

No unused code linking?

[Dmitry Bely]

I am thinking of reducing the size of ocamlopt-generated executables that
easily grows to 1Mb and above. The problem as I see it is that any module is
linked as a single whole (as all its functions are placed into the single
object file), no matter what is actually used. So almost all
stdlib/otherlibs become quickly linked into even not very complex project.

In C world there is a solution that lets the linker to do its job smartly
for monolitic object files: place each function into the separate section
(gcc's "-ffunction-sections" or msvc's "-Gy" options) and then ask linker
to eliminate unused sections during the link stage. Can anything similar be
done with Ocamlopt compiler?

Of course, placing each function into the separate section is not a problem -
modifications of the code generator would be minimal. The real problem are
the frame tables - the single table references all functions in the
module. Maybe they can be split and linked separately? What Ocaml
developers think of that?

- Dmitry Bely

Re: [Caml-list] No unused code linking?

[malc]

On Wed, 9 Nov 2005, Dmitry Bely wrote:

> I am thinking of reducing the size of ocamlopt-generated executables that
> easily grows to 1Mb and above. The problem as I see it is that any module is
> linked as a single whole (as all its functions are placed into the single
> object file), no matter what is actually used. So almost all
> stdlib/otherlibs become quickly linked into even not very complex project.
>
> In C world there is a solution that lets the linker to do its job smartly
> for monolitic object files: place each function into the separate section
> (gcc's "-ffunction-sections" or msvc's "-Gy" options) and then ask linker
> to eliminate unused sections during the link stage. Can anything similar be
> done with Ocamlopt compiler?
>
> Of course, placing each function into the separate section is not a problem -
> modifications of the code generator would be minimal. The real problem are
> the frame tables - the single table references all functions in the
> module. Maybe they can be split and linked separately? What Ocaml
> developers think of that?
>

I have played with this idea myself, the problem is not only frametables
but also the module header (which more or less contains pointers to all
externally visible(don't take my word on it, been a while, might as well
be ALL) symbols.


All in all this would require quite an effort in restructuring almost
everything. I wouldn't count on it.

-- 
mailto:malc@pulsesoft.com

Re: [Caml-list] No unused code linking?

[Dmitry Bely]

malc <malc@pulsesoft.com> writes:

>> I am thinking of reducing the size of ocamlopt-generated executables that
>> easily grows to 1Mb and above. The problem as I see it is that any module is
>> linked as a single whole (as all its functions are placed into the single
>> object file), no matter what is actually used. So almost all
>> stdlib/otherlibs become quickly linked into even not very complex project.
>>
>> In C world there is a solution that lets the linker to do its job smartly
>> for monolitic object files: place each function into the separate section
>> (gcc's "-ffunction-sections" or msvc's "-Gy" options) and then ask linker
>> to eliminate unused sections during the link stage. Can anything similar be
>> done with Ocamlopt compiler?
>>
>> Of course, placing each function into the separate section is not a problem -
>> modifications of the code generator would be minimal. The real problem are
>> the frame tables - the single table references all functions in the
>> module. Maybe they can be split and linked separately? What Ocaml
>> developers think of that?
>
> I have played with this idea myself, the problem is not only frametables
> but also the module header (which more or less contains pointers to all
> externally visible(don't take my word on it, been a while, might as well
> be ALL) symbols.

IMHO, each header item can be placed into the separate DATA section, so
it's not the real problem (the frame table is).

> All in all this would require quite an effort in restructuring almost
> everything. I wouldn't count on it.

Probably you are right, but I still hope for a miracle :-) 

- Dmitry Bely

Re: [Caml-list] No unused code linking?

[malc]

On Wed, 9 Nov 2005, Dmitry Bely wrote:

> malc <malc@pulsesoft.com> writes:
>
>>> I am thinking of reducing the size of ocamlopt-generated executables that
>>> easily grows to 1Mb and above. The problem as I see it is that any module is
>>> linked as a single whole (as all its functions are placed into the single
>>> object file), no matter what is actually used. So almost all
>>> stdlib/otherlibs become quickly linked into even not very complex project.
>>>
>>> In C world there is a solution that lets the linker to do its job smartly
>>> for monolitic object files: place each function into the separate section
>>> (gcc's "-ffunction-sections" or msvc's "-Gy" options) and then ask linker
>>> to eliminate unused sections during the link stage. Can anything similar be
>>> done with Ocamlopt compiler?
>>>
>>> Of course, placing each function into the separate section is not a problem -
>>> modifications of the code generator would be minimal. The real problem are
>>> the frame tables - the single table references all functions in the
>>> module. Maybe they can be split and linked separately? What Ocaml
>>> developers think of that?
>>
>> I have played with this idea myself, the problem is not only frametables
>> but also the module header (which more or less contains pointers to all
>> externally visible(don't take my word on it, been a while, might as well
>> be ALL) symbols.
>
> IMHO, each header item can be placed into the separate DATA section, so
> it's not the real problem (the frame table is).

No it can not. The moduler header must be contiguous and not rearranged by
linker. In absence of .cmx (and name -> mangled name table) OCaml uses the
header to call functions by position. Thus, as it is now, no part of
header can be eliminated, which in reality means that almost everything,
from linkers perspective, is reachable and can not be removed by 
--gc-functions.

>
>> All in all this would require quite an effort in restructuring almost
>> everything. I wouldn't count on it.
>
> Probably you are right, but I still hope for a miracle :-)
>

Right rigth, that's exactly what's required

-- 
mailto:malc@pulsesoft.com

Re: [Caml-list] No unused code linking?

[Florian Weimer]

>> IMHO, each header item can be placed into the separate DATA section, so
>> it's not the real problem (the frame table is).
>
> No it can not. The moduler header must be contiguous and not rearranged by
> linker. In absence of .cmx (and name -> mangled name table) OCaml uses the
> header to call functions by position. Thus, as it is now, no part of
> header can be eliminated, which in reality means that almost everything,
> from linkers perspective, is reachable and can not be removed by 
> --gc-functions.

The module header oculd be generated by the linker, omitting
references to unused functions (filling the place with a null pointer
or something like that).

Re: [Caml-list] No unused code linking?

[Dmitry Bely]

malc <malc@pulsesoft.com> writes:

>> IMHO, each header item can be placed into the separate DATA section, so
>> it's not the real problem (the frame table is).
>
> No it can not. The moduler header must be contiguous and not rearranged by
> linker. In absence of .cmx (and name -> mangled name table) OCaml uses the
> header to call functions by position. Thus, as it is now, no part of
> header can be eliminated, which in reality means that almost everything,
> from linkers perspective, is reachable and can not be removed by
> --gc-functions.

>>> All in all this would require quite an effort in restructuring almost
>>> everything. I wouldn't count on it.
>>
>> Probably you are right, but I still hope for a miracle :-)
>>
> Right rigth, that's exactly what's required

As the Ocaml team people remain silent I am afraid your analysis is
correct. OK, we cannot easily do the job during the code generation but
maybe we can rearrange the source files? Say, if we have

[foo.mli]
val f1: ...
val f2: ...

[foo.ml]
let f1 = ...
let f2 = ...

could we transform them (with some camlp4-based source-level tool) to

[foo_f1.mli]
val f1: ...

[foo_f2.mli]
val f2: ...

[foo.mli]
open Foo_f1
open Foo_f2

[foo_f1.ml]
let f1 = ...

[foo_f2.ml]
let f2 = ...

Isn't it equivalent to the initial version but can be linked separately?

- Dmitry Bely