Options order for ocamlc/opt/opt.opt

Options order for ocamlc/opt/opt.opt

[Romain Beauxis]

	Hi all !

While trying to limit linking to non-needed libraries, I tried to use 
gcc's -Wl,--as-needed option.

Documentation states:
"This option affects ELF DT_NEEDED tags for dynamic libraries mentioned on the 
command line after the --as-needed option. Normally, the linker will add a 
DT_NEEDED tag for each dynamic library mentioned on the command line, 
regardless of whether the library is actually needed. --as-needed causes 
DT_NEEDED tags to only be emitted for libraries that satisfy some symbol 
reference from regular objects which is undefined at the point that the 
library was linked. --no-as-needed restores the default behaviour."

(-Wl means pass the option from gcc to the linker..)

So, as said, this option should be place *before* -l options..

However, when trying to use -cclib with this value, I get:
   '-lthreadsnat' '-lunix' '-lpthread' '-lunix' '-Wl,--as-needed'
which doesn't help...

Also, in order to work, one has also to take care or argument type order in 
gcc call:
'Basically, what the linker does is look for the symbols missing in a given 
file (either an object file, a static archive or a library) only in the files 
coming after it. When using the normal linking, without --as-needed, this is 
not a problem, although there might be some internal drawbacks on the linking 
stage, the files are linked together without considering ordering. But with 
the flag, the libraries that aren't used for resolving symbols are discarded 
and thus not linked.

The fix in this case is to simply fix the linking order so that the libraries 
given to the linker are all after the object files and the static archives."

This is not either the case since I have:
  '-lthreadsnat' '-lunix' '-lpthread' '-lunix' '-Wl,--as-needed' 'float_pcm_c.o' 'oss_io_c.o' '/usr/lib/ocaml/3.10.0/libasmrun.a' -lm  -ldl

This also explains why forcing it on top of the list of arguments via -ccopt 
doesn't help either...

There's a good documentation where the above statement is taken there:
  http://www.gentoo.org/proj/en/qa/asneeded.xml

What do you think ?


Romain

Re: Options order for ocamlc/opt/opt.opt

[Romain Beauxis]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

Le Friday 23 November 2007 02:43:34, vous avez écrit :
> The fix in this case is to simply fix the linking order so that the
> libraries given to the linker are all after the object files and the static
> archives."
(...)
> There's a good documentation where the above statement is taken there:
>   http://www.gentoo.org/proj/en/qa/asneeded.xml
>
> What do you think ?

Well, I'm quite confused with this...

After spending some time finding out how to change those arguments order, 
I ended up with a patch that orders them just perfectly, 
so that the following is passed:

  gcc <other options> <-I options> -Wl,--as-needed <base_file.o> <project_files.o> <static_libraries.a> <dynamic list of -lfoo> -o object

This is exactly what's refered as a good practice in the above link, but still I got my linking wrong...

Ha, and of course this option works on a simple hello_world with the same gcc...


Another nice possibility would be to be able to filter out -l values, since they are automagically 
added, we don't have control over them...


Romain



[-- Attachment #2: link_args_order.patch --]
[-- Type: text/x-diff, Size: 1673 bytes --]

--- ocaml-3.10.0.orig/asmcomp/asmlink.ml
+++ ocaml-3.10.0/asmcomp/asmlink.ml
@@ -239,20 +239,20 @@
   | "cc" ->
       let cmd =
         if not !Clflags.output_c_object then
-          Printf.sprintf "%s %s -o %s %s %s %s %s %s %s %s %s"
+          Printf.sprintf "%s %s %s %s %s %s %s %s %s %s -o %s"
             !Clflags.c_linker
             (if !Clflags.gprofile then Config.cc_profile else "")
-            (Filename.quote output_name)
             (Clflags.std_include_flag "-I")
-            (String.concat " " (List.rev !Clflags.ccopts))
-            (Filename.quote startup_file)
-            (Ccomp.quote_files (List.rev file_list))
             (Ccomp.quote_files
               (List.map (fun dir -> if dir = "" then "" else "-L" ^ dir)
                         !load_path))
-            (Ccomp.quote_files (List.rev !Clflags.ccobjs))
+           (String.concat " " !Clflags.ccopts)
+           (Filename.quote startup_file)
+            (Ccomp.quote_files (List.rev file_list))
             (Filename.quote runtime_lib)
+           (Ccomp.quote_files (List.rev !Clflags.ccobjs))
             c_lib
+            (Filename.quote output_name)
         else
           Printf.sprintf "%s -o %s %s %s"
             Config.native_partial_linker
@@ -320,7 +320,7 @@
   List.iter
     (fun (info, file_name, crc) -> check_consistency file_name info crc)
     units_tolink;
-  Clflags.ccobjs := !Clflags.ccobjs @ !lib_ccobjs;
+  Clflags.ccobjs := !lib_ccobjs @ !Clflags.ccobjs;
   Clflags.ccopts := !lib_ccopts @ !Clflags.ccopts; (* put user's opts first *)
   let startup = Filename.temp_file "camlstartup" ext_asm in
   make_startup_file ppf startup units_tolink;

Re: [Caml-list] Re: Options order for ocamlc/opt/opt.opt

[Julien Moutinho]

On Fri, Nov 23, 2007 at 05:40:27PM +0100, Romain Beauxis wrote:
> Le Friday 23 November 2007 02:43:34, vous avez écrit :
> > The fix in this case is to simply fix the linking order so that the
> > libraries given to the linker are all after the object files and the static
> > archives."
> (...)
> > There's a good documentation where the above statement is taken there:
> >   http://www.gentoo.org/proj/en/qa/asneeded.xml
> >
> > What do you think ?
> 
> Well, I'm quite confused with this...
> 
> After spending some time finding out how to change those arguments order, 
> I ended up with a patch that orders them just perfectly, 
> so that the following is passed:
> 
>   gcc <other options> <-I options> -Wl,--as-needed <base_file.o> <project_files.o> <static_libraries.a> <dynamic list of -lfoo> -o object
> 
> This is exactly what's refered as a good practice in the above link, but still I got my linking wrong...
> 
> Ha, and of course this option works on a simple hello_world with the same gcc...
Options passed to -cclib are in !Clflags.ccobjs to which -l<foo> flags are appended,
hence your patch:
          Printf.sprintf "%s %s %s %s %s %s %s %s %s %s -o %s"
            !Clflags.c_linker
            (if !Clflags.gprofile then Config.cc_profile else "")
            (Clflags.std_include_flag "-I")
            (Ccomp.quote_files
              (List.map (fun dir -> if dir = "" then "" else "-L" ^ dir)
                        !load_path))
            (String.concat " " !Clflags.ccopts)
            (Ccomp.quote_files (List.rev !Clflags.ccobjs))
            (Filename.quote startup_file)
            (Ccomp.quote_files (List.rev file_list))
            (Ccomp.quote_optfile runtime_lib)
            c_lib
            (Filename.quote output_name)

gives (it can easily be seen by wrapping /usr/bin/gcc with a shell
script /usr/local/bin/gcc echoing its arguments):

  gcc -I/usr/local/lib/ocaml -L/usr/local/lib/ocaml /tmp/camlstartupf0fb45.o \
    /usr/local/lib/ocaml/std_exit.o test.o /usr/local/lib/ocaml/stdlib.a \
    /usr/local/lib/ocaml/libasmrun.a -Wl,--as-needed -lm -ldl -o test

Which is not what you want, isn't it.

But now, what exactly is wrong for you in:
  ocamlopt -o test -ccopt '-Wl,--as-needed' -cclib '-lunix' test.ml

which gives (with a vanilla release310 branch):

  gcc -o test -I/usr/local/lib/ocaml -Wl,--as-needed \
    /tmp/camlstartupd18cfa.o /usr/local/lib/ocaml/std_exit.o test.o \
    /usr/local/lib/ocaml/stdlib.a -L/usr/local/lib/ocaml -lunix \
    /usr/local/lib/ocaml/libasmrun.a -lm -ldl

HTH,
  Julien.

Re: [Caml-list] Re: Options order for ocamlc/opt/opt.opt

[Julien Moutinho]

On Sat, Nov 24, 2007 at 04:10:46AM +0100, Julien Moutinho wrote:
> Options passed to -cclib are in !Clflags.ccobjs to which -l<foo> flags are appended,
> hence your patch:
>           Printf.sprintf "%s %s %s %s %s %s %s %s %s %s -o %s"
>             !Clflags.c_linker
>             (if !Clflags.gprofile then Config.cc_profile else "")
>             (Clflags.std_include_flag "-I")
>             (Ccomp.quote_files
>               (List.map (fun dir -> if dir = "" then "" else "-L" ^ dir)
>                         !load_path))
>             (String.concat " " !Clflags.ccopts)
>             (Ccomp.quote_files (List.rev !Clflags.ccobjs))
>             (Filename.quote startup_file)
>             (Ccomp.quote_files (List.rev file_list))
>             (Ccomp.quote_optfile runtime_lib)
>             c_lib
>             (Filename.quote output_name)
Sorry, copy/paste error your patch is:
          Printf.sprintf "%s %s %s %s %s %s %s %s %s %s -o %s"
            !Clflags.c_linker
            (if !Clflags.gprofile then Config.cc_profile else "")
            (Clflags.std_include_flag "-I")
            (Ccomp.quote_files
              (List.map (fun dir -> if dir = "" then "" else "-L" ^ dir)
                        !load_path))
            (String.concat " " !Clflags.ccopts)
            (Filename.quote startup_file)
            (Ccomp.quote_files (List.rev file_list))
            (Ccomp.quote_optfile runtime_lib)
            (Ccomp.quote_files (List.rev !Clflags.ccobjs)) (* here is !Clflags.ccobjs *)
            c_lib
            (Filename.quote output_name)

It changes nothing to what I said previously.

Re: [Caml-list] Re: Options order for ocamlc/opt/opt.opt

[Romain Beauxis]

Le Saturday 24 November 2007 04:15:44 Julien Moutinho, vous avez écrit :
> It changes nothing to what I said previously.

Hey, it seems I was wrong all the way...

In fact ldd output seems to include indirect needed libraries.

The real way to check linked libraries is:
objdump -x _your-binary_ | grep NEEDED

And, yes it works with current ocaml :-)



Romain