* PIC
@ 2005-12-25 2:07 skaller
2005-12-27 10:03 ` [Caml-list] PIC Xavier Leroy
0 siblings, 1 reply; 9+ messages in thread
From: skaller @ 2005-12-25 2:07 UTC (permalink / raw)
To: caml-list
I notice Ocaml 3.09 has -fPIC option. Thanks! This is a step
towards dynamic loading in native code.
I wonder, is it actually supported -- or even possible
to load native code (on suitable platforms) at startup
from C? From an Ocaml mainline?
What's left to be able to 'dlopen()' a function library?
>From C? From Ocaml? Will that ever be possible?
[Another question, whether one can wrap native code
so it can be called from bytecode.. this would allow
bytecode programs to generate and compile bytecode
extensions .. without the bulk of the code running
slowly]
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [Caml-list] PIC 2005-12-25 2:07 PIC skaller @ 2005-12-27 10:03 ` Xavier Leroy 2005-12-27 12:23 ` skaller ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Xavier Leroy @ 2005-12-27 10:03 UTC (permalink / raw) To: skaller; +Cc: caml-list > I notice Ocaml 3.09 has -fPIC option. Thanks! This is a step > towards dynamic loading in native code. > > I wonder, is it actually supported -- or even possible > to load native code (on suitable platforms) at startup > from C? From an Ocaml mainline? > > What's left to be able to 'dlopen()' a function library? > From C? From Ocaml? Will that ever be possible? Encapsulation of compiled OCaml code as a shared library that can be called from C is possible, on x86 at least. The procedure is similar to that described in section 18.8 of the reference manual, namely, use "ocamlc -output-obj" or "ocamlopt -output-obj" to package the Caml code as a .o file, then use "gcc -shared" (or equivalent commands to build shared objects) to group together this .o file, the C stub code that calls back into OCaml, and the OCaml run-time system. This is a very good approach to use OCaml code from other languages (e.g. Java) or as plug-ins (e.g. in Apache). The reason it works particularly well for x86 is that dynamic loaders for x86 (both under Unix/Linux and Windows) can relocate arbitrary code, not necessarily PIC code. This is unfortunately not the case for all target architectures. In particular, dynamic loaders for x86_64 (AMD64) are much less forgiving. The -fPIC option to ocamlopt you mention was added to the AMD64 code generator in an attempt to generate "more position-independent" code. However, it does not quite work yet. A difficulty is the paucity of documentation on what, exactly, the Linux AMD64 dynamic loader can relocate. Dynamic loading of OCaml code from OCaml code raises many other issues. It is currently supported for bytecode only, and will not be available for native code in the forseeable future. I have already discussed this on this list earlier. > [Another question, whether one can wrap native code > so it can be called from bytecode.. this would allow > bytecode programs to generate and compile bytecode > extensions .. without the bulk of the code running > slowly] Again, I believe this has been discussed at length before. The short answer is: no. A longer answer is the asmdynlink library by Fabrice Le Fessant, which (used to) enable this kind of things with some speed penalty on the interpretation of the bytecode. - Xavier Leroy ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2005-12-27 10:03 ` [Caml-list] PIC Xavier Leroy @ 2005-12-27 12:23 ` skaller 2005-12-28 9:14 ` Sven Luther 2005-12-31 21:31 ` David Fox 2 siblings, 0 replies; 9+ messages in thread From: skaller @ 2005-12-27 12:23 UTC (permalink / raw) To: Xavier Leroy; +Cc: caml-list On Tue, 2005-12-27 at 11:03 +0100, Xavier Leroy wrote: > > I notice Ocaml 3.09 has -fPIC option. Thanks! This is a step > > towards dynamic loading in native code. Thanks for your summary! > Encapsulation of compiled OCaml code as a shared library that can be > called from C is possible, on x86 at least. [] > This is a very good approach to use OCaml code from other languages > (e.g. Java) or as plug-ins (e.g. in Apache). Indeed! If I understand correctly, this requires redistribution of "LGPL with Linking Exception" parts of Ocaml only? > The reason it works particularly well for x86 is that dynamic loaders > for x86 (both under Unix/Linux and Windows) can relocate arbitrary > code, not necessarily PIC code. This is unfortunately not the case > for all target architectures. > In particular, dynamic loaders for > x86_64 (AMD64) are much less forgiving. I suspect you're being polite .. 'broken' seems to the correct term ;( The same techniques should work for the x86_64 as the x86 .. just making sure 64 bit values are supported (by the loader and ELF formats) should be enough. > The -fPIC option to ocamlopt > you mention was added to the AMD64 code generator in an attempt to > generate "more position-independent" code. However, it does not quite > work yet. A difficulty is the paucity of documentation on what, > exactly, the Linux AMD64 dynamic loader can relocate. I see. Ok, thanks for that info. Having looked into this myself .. I'm not surprised .. Sigh. Is the ia64 any better? > Dynamic loading of OCaml code from OCaml code raises many other > issues. Type safety, version compatibility, etc being a few. But one can work around this problem, perhaps messily and riskily, if the codes can be loaded and connected together with C glue. > It is currently supported for bytecode only, and will not be > available for native code in the forseeable future. I have already > discussed this on this list earlier. Yes, thanks. I hope I covered the combinations reasonably well, so your responses are fairly clear summary of the situation, which is what I was looking for: thanks! -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2005-12-27 10:03 ` [Caml-list] PIC Xavier Leroy 2005-12-27 12:23 ` skaller @ 2005-12-28 9:14 ` Sven Luther 2005-12-31 21:31 ` David Fox 2 siblings, 0 replies; 9+ messages in thread From: Sven Luther @ 2005-12-28 9:14 UTC (permalink / raw) To: Xavier Leroy; +Cc: skaller, caml-list On Tue, Dec 27, 2005 at 11:03:24AM +0100, Xavier Leroy wrote: > The reason it works particularly well for x86 is that dynamic loaders > for x86 (both under Unix/Linux and Windows) can relocate arbitrary > code, not necessarily PIC code. This is unfortunately not the case > for all target architectures. In particular, dynamic loaders for > x86_64 (AMD64) are much less forgiving. The -fPIC option to ocamlopt > you mention was added to the AMD64 code generator in an attempt to > generate "more position-independent" code. However, it does not quite > work yet. A difficulty is the paucity of documentation on what, > exactly, the Linux AMD64 dynamic loader can relocate. What is the situation for other architectures, in particular, i would be interested in the powerpc case. Friendly, Sven Luther ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2005-12-27 10:03 ` [Caml-list] PIC Xavier Leroy 2005-12-27 12:23 ` skaller 2005-12-28 9:14 ` Sven Luther @ 2005-12-31 21:31 ` David Fox 2006-01-01 10:49 ` Ker Lutyn 2 siblings, 1 reply; 9+ messages in thread From: David Fox @ 2005-12-31 21:31 UTC (permalink / raw) To: Xavier Leroy; +Cc: skaller, caml-list Xavier Leroy wrote: >Dynamic loading of OCaml code from OCaml code raises many other >issues. It is currently supported for bytecode only, and will not be >available for native code in the forseeable future. I have already >discussed this on this list earlier. > > I can seem to locate this discussion - do you remember any keywords I might search for? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2005-12-31 21:31 ` David Fox @ 2006-01-01 10:49 ` Ker Lutyn 2006-01-03 23:27 ` Nathaniel Gray 0 siblings, 1 reply; 9+ messages in thread From: Ker Lutyn @ 2006-01-01 10:49 UTC (permalink / raw) To: caml-list <http://groups.google.com/groups?q=group:fa.caml+author:xavier+loading+native> --- David Fox <david.fox@linspire.com> wrote: > Xavier Leroy wrote: > > >Dynamic loading of OCaml code from OCaml code raises many other > >issues. It is currently supported for bytecode only, and will not be > >available for native code in the forseeable future. I have already > >discussed this on this list earlier. > > > > > I can seem to locate this discussion - do you remember any keywords I > might search for? > > _______________________________________________ > Caml-list mailing list. Subscription management: > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list > Archives: http://caml.inria.fr > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > __________________________________________ Yahoo! DSL – Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2006-01-01 10:49 ` Ker Lutyn @ 2006-01-03 23:27 ` Nathaniel Gray 2006-01-04 2:38 ` skaller 0 siblings, 1 reply; 9+ messages in thread From: Nathaniel Gray @ 2006-01-03 23:27 UTC (permalink / raw) To: Ker Lutyn; +Cc: caml-list Maybe these links will be more useful: http://groups.google.com/group/fa.caml/browse_thread/thread/2827ec9d553e7760/183ac101fef47132?lnk=st&q=group%3Afa.caml+author%3Axavier+loading+native&rnum=10#183ac101fef47132 http://groups.google.com/group/fa.caml/browse_thread/thread/efc434c41cc88ded/663f229e008648e0?lnk=st&q=group%3Afa.caml+author%3Axavier+binary+compatibility&rnum=4#663f229e008648e0 In the first (newer) post linked above Xavier says: """ Feature 3- (dynamic code loading) is already available in bytecode through the Dynlink API. I understand there's a demand for having it in native-code as well, and that might be possible without too much fuss, at least on selected operating systems. """ This seems to conflict with Xavier's statements earlier in this thread. I hope dynamic loading is still being considered, as I can't see using OCaml for medium-to-large scale applications programming without it. Who wants to write an app without plugins these days? Cheers, -n8 On 1/1/06, Ker Lutyn <ker527mail@yahoo.com> wrote: > <http://groups.google.com/groups?q=group:fa.caml+author:xavier+loading+native> > > --- David Fox <david.fox@linspire.com> wrote: > > > Xavier Leroy wrote: > > > > >Dynamic loading of OCaml code from OCaml code raises many other > > >issues. It is currently supported for bytecode only, and will not be > > >available for native code in the forseeable future. I have already > > >discussed this on this list earlier. > > > > > > > > I can seem to locate this discussion - do you remember any keywords I > > might search for? > > -- >>>-- Nathaniel Gray -- Caltech Computer Science ------> >>>-- Mojave Project -- http://mojave.cs.caltech.edu --> ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2006-01-03 23:27 ` Nathaniel Gray @ 2006-01-04 2:38 ` skaller 2006-01-04 4:54 ` Nathaniel Gray 0 siblings, 1 reply; 9+ messages in thread From: skaller @ 2006-01-04 2:38 UTC (permalink / raw) To: Nathaniel Gray; +Cc: Ker Lutyn, caml-list On Tue, 2006-01-03 at 15:27 -0800, Nathaniel Gray wrote: > Who wants to write an app without plugins these days? Who is willing to write applications that are unsound? Once Ocaml generates -fPIC it can be linked with a C wrapper to make a shared library, and your mainline (or another wrapped library) can embed a function that loads such libraries with dlopen() and retrieves a record of functions, then casts it to the expected type .. voila, you have native code plugins. Note even bytecode cannot *unload* plugins .. which is needed to execute user scripts in web pages with long running server side scripting support. -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Caml-list] PIC 2006-01-04 2:38 ` skaller @ 2006-01-04 4:54 ` Nathaniel Gray 0 siblings, 0 replies; 9+ messages in thread From: Nathaniel Gray @ 2006-01-04 4:54 UTC (permalink / raw) To: skaller; +Cc: Ker Lutyn, caml-list On 1/3/06, skaller <skaller@users.sourceforge.net> wrote: > On Tue, 2006-01-03 at 15:27 -0800, Nathaniel Gray wrote: > > > Who wants to write an app without plugins these days? > > Who is willing to write applications that are unsound? > > Once Ocaml generates -fPIC it can be linked with a C wrapper > to make a shared library, and your mainline (or another > wrapped library) can embed a function that loads such > libraries with dlopen() and retrieves a record of functions, > then casts it to the expected type .. > voila, you have native code plugins. This could solve part of the problem, namely alleviating the need for recompiling the app for each plugin the user adds. However, given the OCaml team's indifference towards binary compatibility, trying to keep everybody on the same version of the compiler would be too restrictive to be practical. -n8 -- >>>-- Nathaniel Gray -- Caltech Computer Science ------> >>>-- Mojave Project -- http://mojave.cs.caltech.edu --> ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2006-01-04 4:54 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-12-25 2:07 PIC skaller 2005-12-27 10:03 ` [Caml-list] PIC Xavier Leroy 2005-12-27 12:23 ` skaller 2005-12-28 9:14 ` Sven Luther 2005-12-31 21:31 ` David Fox 2006-01-01 10:49 ` Ker Lutyn 2006-01-03 23:27 ` Nathaniel Gray 2006-01-04 2:38 ` skaller 2006-01-04 4:54 ` Nathaniel Gray
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).