[Caml-list] shared libraries [pcre-ocaml]

[Caml-list] shared libraries [pcre-ocaml]

[Richard Nyberg]

Hello I've some problems with ocaml and shared libraries. I'm running
a recent FreeBSD-4-STABLE and have ocaml 3.04 installed from ports.

pcre-ocaml is built with shared libraries and is installed in
/usr/local/lib/ocaml/site-lib/pcre

ocaml's ld.conf contains the following lines:

----------ld.conf
/usr/local/lib/ocaml
/usr/local/lib/ocaml/labltk
/usr/local/lib/ocaml/site-lib/pcre
----------

If I use ocamlopt to compile everything is fine but when I use ocamlc
the following error occurs.

----------compiling
# ocamlc -o myprog -I +site-lib/netstring -I +site-lib/pcre \
         unix.cma pcre.cma netstring.cma myprog.ml

Error on dynamically loaded library: Shared object "libpcre.so.0" not found
----------

I ran truss to see the syscalls made by ocamlc (relevant bit below) and
to me it seems as if it finds dllpcre.so but not libpcre.so.0 on which
dllpcre.so depends. However if I run ldconfig to include pcre-ocaml's
installation directory everything works just fine.

So, should I have to use ldconfig or is there something wrong?

---------truss
stat("/usr/local/lib/ocaml/site-lib/netstring/dllpcre.so",0xbfbff524) ERR#2 'No 
such file or directory'
stat("/usr/local/lib/ocaml/site-lib/pcre/dllpcre.so",0xbfbff524) = 0 (0x0)
sigprocmask(0x1,0x28072c80,0xbfbff4e4)           = 0 (0x0)
open("/usr/local/lib/ocaml/site-lib/pcre/dllpcre.so",0x0,056) = 6 (0x6)
fstat(6,0xbfbff494)                              = 0 (0x0)
read(0x6,0xbfbfe464,0x1000)                      = 4096 (0x1000)
mmap(0x0,12288,0x5,0x2,6,0x0)                    = 672776192 (0x2819c000)
mmap(0x2819e000,4096,0x3,0x12,6,0x1000)          = 672784384 (0x2819e000)
close(6)                                         = 0 (0x0)
access("/usr/local/lib/ocaml/contrib/libpcre.so.0",0) ERR#2 'No such file or dir
ectory'
access("/usr/lib/libpcre.so.0",0)                ERR#2 'No such file or director
y'
access("/usr/lib/compat/libpcre.so.0",0)         ERR#2 'No such file or director
y'
access("/usr/X11R6/lib/libpcre.so.0",0)          ERR#2 'No such file or director
y'
access("/usr/local/lib/libpcre.so.0",0)          ERR#2 'No such file or director
y'
access("/usr/lib/elf/libpcre.so.0",0)            ERR#2 'No such file or director
y'
access("/usr/lib/libpcre.so.0",0)                ERR#2 'No such file or director
y'
-------------

	-Richard
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Markus Mottl]

On Mon, 22 Apr 2002, Richard Nyberg wrote:
> Error on dynamically loaded library: Shared object "libpcre.so.0" not found

> So, should I have to use ldconfig or is there something wrong?

It seems to me that you hadn't set the environment variable
"OCAML_LIB_INSTALL" to your installation directory before "make"ing the
library, i.e. it's default is "+contrib" now:

[snip]
> access("/usr/local/lib/ocaml/contrib/libpcre.so.0",0) ERR#2 'No such file or directory'
[snip]

The path leading to this directory is added to the generated OCaml-library
so that it can find the shared C-library. Usually things should also work
when you set your LD_LIBRARY_PATH appropriately in your shell or even add
this path to your global /etc/ld.so.conf. However, the preferred solution
is surely to let the library now where to find things it depends on.

Regards,
Markus Mottl

-- 
Markus Mottl                                             markus@oefai.at
Austrian Research Institute
for Artificial Intelligence                  http://www.oefai.at/~markus
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Richard Nyberg]

> It seems to me that you hadn't set the environment variable
> "OCAML_LIB_INSTALL" to your installation directory before "make"ing the
> library, i.e. it's default is "+contrib" now:

No I didn't, and I cannot understand how I managed to miss it!
I will rebuild it immediately. Thanks!!!

> [snip]
> > access("/usr/local/lib/ocaml/contrib/libpcre.so.0",0) ERR#2 'No such file or directory'
> [snip]
> 
> The path leading to this directory is added to the generated OCaml-library
> so that it can find the shared C-library. Usually things should also work
> when you set your LD_LIBRARY_PATH appropriately in your shell or even add
> this path to your global /etc/ld.so.conf. However, the preferred solution
> is surely to let the library now where to find things it depends on.

Definitely :)

	-Richard
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[John Max Skaller]

Richard Nyberg wrote:

>Hello I've some problems with ocaml and shared libraries. I'm running
>a recent FreeBSD-4-STABLE and have ocaml 3.04 installed from ports.
>
>pcre-ocaml is built with shared libraries and is installed in
>/usr/local/lib/ocaml/site-lib/pcre
>
>ocaml's ld.conf contains the following lines:
>
>----------ld.conf
>/usr/local/lib/ocaml
>/usr/local/lib/ocaml/labltk
>/usr/local/lib/ocaml/site-lib/pcre
>----------
>
>If I use ocamlopt to compile everything is fine but when I use ocamlc
>the following error occurs.
>
>----------compiling
># ocamlc -o myprog -I +site-lib/netstring -I +site-lib/pcre \
>         unix.cma pcre.cma netstring.cma myprog.ml
>
>Error on dynamically loaded library: Shared object "libpcre.so.0" not found
>----------
>
>I ran truss to see the syscalls made by ocamlc (relevant bit below) and
>to me it seems as if it finds dllpcre.so but not libpcre.so.0 on which
>dllpcre.so depends. However if I run ldconfig to include pcre-ocaml's
>installation directory everything works just fine.
>
>So, should I have to use ldconfig or is there something wrong?
>

Argg... crappy unix hackery. The dynamic loader typically searches for
relative filenames in the ldconfig'd search path, augmented by LD_PATH
environment variable. Absolute filenames just get opened directly.

You can see below that ocaml is manually stat'ing dllpcre.so in places
it knows about before doing a dlopen on an absolute filename.
Unfortunately, it depends on another shared lib, and ld is
searching for that by itself.

It doesn't find it, since /usr/local/ocaml/site-lib/pcre isn't on the 
ldconfig path.
This suggests the binding to dllpcre.so.0 installed in dllpcre.so is a 
relative
filename. So does the ld.conf file above .. and if you run ldconfig on that,
of course now the load works.

Running ldconfig on the ld.conf file will work.
Another solution is simply to install dllpcre.so.0 in /usr/local/lib.
Another is to put a symbolic link there. None of these solutions is
very good ..

[Check out Python's mechanism for a much saner system ..]

>
>---------truss
>stat("/usr/local/lib/ocaml/site-lib/netstring/dllpcre.so",0xbfbff524) ERR#2 'No 
>such file or directory'
>stat("/usr/local/lib/ocaml/site-lib/pcre/dllpcre.so",0xbfbff524) = 0 (0x0)
>sigprocmask(0x1,0x28072c80,0xbfbff4e4)           = 0 (0x0)
>open("/usr/local/lib/ocaml/site-lib/pcre/dllpcre.so",0x0,056) = 6 (0x6)
>fstat(6,0xbfbff494)                              = 0 (0x0)
>read(0x6,0xbfbfe464,0x1000)                      = 4096 (0x1000)
>mmap(0x0,12288,0x5,0x2,6,0x0)                    = 672776192 (0x2819c000)
>mmap(0x2819e000,4096,0x3,0x12,6,0x1000)          = 672784384 (0x2819e000)
>close(6)                                         = 0 (0x0)
>access("/usr/local/lib/ocaml/contrib/libpcre.so.0",0) ERR#2 'No such file or dir
>ectory'
>access("/usr/lib/libpcre.so.0",0)                ERR#2 'No such file or director
>y'
>access("/usr/lib/compat/libpcre.so.0",0)         ERR#2 'No such file or director
>y'
>access("/usr/X11R6/lib/libpcre.so.0",0)          ERR#2 'No such file or director
>y'
>access("/usr/local/lib/libpcre.so.0",0)          ERR#2 'No such file or director
>y'
>access("/usr/lib/elf/libpcre.so.0",0)            ERR#2 'No such file or director
>y'
>access("/usr/lib/libpcre.so.0",0)                ERR#2 'No such file or director
>y'
>-------------
>
>	-Richard
>-------------------
>

-- 
John Max Skaller, mailto:skaller@ozemail.com.au
snail:10/1 Toxteth Rd, Glebe, NSW 2037, Australia.
voice:61-2-9660-0850




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Jacques Garrigue]

From: John Max Skaller <skaller@ozemail.com.au>
> Richard Nyberg wrote:
> 
> >Hello I've some problems with ocaml and shared libraries. I'm running
> >a recent FreeBSD-4-STABLE and have ocaml 3.04 installed from ports.
> >
> >pcre-ocaml is built with shared libraries and is installed in
> >/usr/local/lib/ocaml/site-lib/pcre
> >
> >If I use ocamlopt to compile everything is fine but when I use ocamlc
> >the following error occurs.
> >
> >----------compiling
> ># ocamlc -o myprog -I +site-lib/netstring -I +site-lib/pcre \
> >         unix.cma pcre.cma netstring.cma myprog.ml
> >
> >Error on dynamically loaded library: Shared object "libpcre.so.0" not found
> >----------
> >
> >I ran truss to see the syscalls made by ocamlc (relevant bit below) and
> >to me it seems as if it finds dllpcre.so but not libpcre.so.0 on which
> >dllpcre.so depends. However if I run ldconfig to include pcre-ocaml's
> >installation directory everything works just fine.
> >
> >So, should I have to use ldconfig or is there something wrong?
> 
> Argg... crappy unix hackery. The dynamic loader typically searches for
> relative filenames in the ldconfig'd search path, augmented by LD_PATH
> environment variable. Absolute filenames just get opened directly.
> 
> You can see below that ocaml is manually stat'ing dllpcre.so in places
> it knows about before doing a dlopen on an absolute filename.
> Unfortunately, it depends on another shared lib, and ld is
> searching for that by itself.
> 
> It doesn't find it, since /usr/local/ocaml/site-lib/pcre isn't on the 
> ldconfig path.
> This suggests the binding to dllpcre.so.0 installed in dllpcre.so is a 
> relative
> filename. So does the ld.conf file above .. and if you run ldconfig on that,
> of course now the load works.
> 
> Running ldconfig on the ld.conf file will work.
> Another solution is simply to install dllpcre.so.0 in /usr/local/lib.
> Another is to put a symbolic link there. None of these solutions is
> very good ..

Almost correct answer: the problem is of course with libpcre.so.0, not
dllpcre.so. So if you put libpcre.so.0 in any place where the system
loader can find it (like /usr/local/lib, but also any directory in
LD_LIBRARY_PATH), things will work smooth.

More basically, I would say that pcre-ocaml is not packaged correctly:
for this kind of use, either the code of libpcre.so should be included
in dllcpre.so (not too hard), or libcpre.so.0 should be installed to a
public directory. Unfortunately the system loader does not provide any
interface to change the load path cleanly.

Best regards,

Jacques Garrigue
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Richard Nyberg]

> > Argg... crappy unix hackery. The dynamic loader typically searches for
> > relative filenames in the ldconfig'd search path, augmented by LD_PATH
> > environment variable. Absolute filenames just get opened directly.
> > 
> > You can see below that ocaml is manually stat'ing dllpcre.so in places
> > it knows about before doing a dlopen on an absolute filename.
> > Unfortunately, it depends on another shared lib, and ld is
> > searching for that by itself.
> > 
> > It doesn't find it, since /usr/local/ocaml/site-lib/pcre isn't on the 
> > ldconfig path.
> > This suggests the binding to dllpcre.so.0 installed in dllpcre.so is a 
> > relative
> > filename. So does the ld.conf file above .. and if you run ldconfig on that,
> > of course now the load works.
> > 
> > Running ldconfig on the ld.conf file will work.
> > Another solution is simply to install dllpcre.so.0 in /usr/local/lib.
> > Another is to put a symbolic link there. None of these solutions is
> > very good ..
> 
> Almost correct answer: the problem is of course with libpcre.so.0, not
> dllpcre.so. So if you put libpcre.so.0 in any place where the system
> loader can find it (like /usr/local/lib, but also any directory in
> LD_LIBRARY_PATH), things will work smooth.
> 
> More basically, I would say that pcre-ocaml is not packaged correctly:
> for this kind of use, either the code of libpcre.so should be included
> in dllcpre.so (not too hard), or libcpre.so.0 should be installed to a
> public directory. Unfortunately the system loader does not provide any
> interface to change the load path cleanly.

Well, you are both right of course, but the real problem here was me.
There is no problem with pcre-ocaml if you tell it where to look for
its libraries. I had just missed setting OCAML_LIB_INSTALL correctly
when I installed pcre-ocaml.

So, it's my bad and I promise to _read_ the documentation instead of
just _looking_ at it next time ;)

	-Richard
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Markus Mottl]

On Tue, 23 Apr 2002, Jacques Garrigue wrote:
> Almost correct answer: the problem is of course with libpcre.so.0, not
> dllpcre.so. So if you put libpcre.so.0 in any place where the system
> loader can find it (like /usr/local/lib, but also any directory in
> LD_LIBRARY_PATH), things will work smooth.

The only thing that needs to be changed is setting the installation path
(OCAML_LIB_INSTALL) correctly before "make"ing the library (as described
in the INSTALL-file; please do me all a favour and read it _before_
installation ;)

The linker stores the path to this directory in the generated
OCaml-library so that it can find any associated shared C-libraries.

The only known problem so far with the installation of Pcre-OCaml is
portability on MacOS X: shared libraries are not (yet) supported there,
because "libtool", which is used by the shipped C-library, obviously
misbehaves on this platform. Just compile the library statically then
as also described in the INSTALL-file.

> More basically, I would say that pcre-ocaml is not packaged correctly:
> for this kind of use, either the code of libpcre.so should be included
> in dllcpre.so (not too hard), or libcpre.so.0 should be installed to
> a public directory. Unfortunately the system loader does not provide
> any interface to change the load path cleanly.

My suggestion on the bug tracker was to automatically update the path to
shared C-libraries at runtime so that it also contains the directory in
which the associated OCaml-library resides.  This would make things much
easier for me, but as Xavier noted, this could cause endless troubles
with portability, because some systems may react differently to this hack.

Regards,
Markus Mottl

-- 
Markus Mottl                                             markus@oefai.at
Austrian Research Institute
for Artificial Intelligence                  http://www.oefai.at/~markus
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[John Max Skaller]

Jacques Garrigue wrote:

>>Argg... crappy unix hackery.
>>
>
>Almost correct answer
>

Yeah. The correct answer is to fix the stupid dynamic
loader hackery used in unix systems. Obviously,
there should be a way to load a library from
the directory in which the caller is installed:
that is, the filename

    here/dllpcre.so.0

should be allowed, where 'here' means
" the place that dllpcre.so" was found.
That way would always work provided
both libraries are kept in the same directory.
It is the moral equivalent of what most unix
C compilers do with

#include "relative.h"

where the file is searched for in the directory
in which the including file is located.

Another solution would be to bind the inode
number directly into the caller so that only
physically removing the library could cause
a load failure (and in this case, that would seem
desirable .. you could even consider shared libraries
as being like directories .. so that removal
would fail while there were references ..)

So .. when are the Ocaml team going to design
an operating system based on theory not
historical hackery? <j/2>

-- 
John Max Skaller, mailto:skaller@ozemail.com.au
snail:10/1 Toxteth Rd, Glebe, NSW 2037, Australia.
voice:61-2-9660-0850




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] shared libraries [pcre-ocaml]

[Xavier Leroy]

> Yeah. The correct answer is to fix the stupid dynamic
> loader hackery used in unix systems. Obviously,
> there should be a way to load a library from
> the directory in which the caller is installed:

Agreed.  The Win32 dynamic loader gets this right.  Another workaround
would be to have an API to manipulate the search path of the dynamic
loader at run-time, but setenv("LD_LIBRARY_PATH", ...) doesn't work
because usually the value of LD_LIBRARY_PATH is cached at program
start-up time.

- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners