* ocaml, llvm and generating code at runtime @ 2010-01-01 16:45 Joel Reymont 2010-01-01 17:39 ` [Caml-list] " Basile STARYNKEVITCH 2010-01-01 19:31 ` Jon Harrop 0 siblings, 2 replies; 8+ messages in thread From: Joel Reymont @ 2010-01-01 16:45 UTC (permalink / raw) To: caml-list Does anybody have example code that shows how to generate OCaml bindings at runtime with LLVM? My goal is to compile an AST into code that uses OCaml functions within the same binary that's doing the compiling. I don't think it can be done with OCaml since it requires a standalone assembler, linker, etc. Correct me if I'm wrong, though. Mine is a web-based compiler with potentially many concurrent sessions. Running gas, ld, etc. seems a much heavier and less scalable approach that generating code at runtime. Thanks and Happy New Year, Joel --- http://wagerlabs.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 16:45 ocaml, llvm and generating code at runtime Joel Reymont @ 2010-01-01 17:39 ` Basile STARYNKEVITCH 2010-01-01 19:08 ` Jon Harrop 2010-01-01 19:31 ` Jon Harrop 1 sibling, 1 reply; 8+ messages in thread From: Basile STARYNKEVITCH @ 2010-01-01 17:39 UTC (permalink / raw) To: Joel Reymont; +Cc: caml-list Joel Reymont wrote: > Does anybody have example code that shows how to generate OCaml bindings at runtime with LLVM? > > My goal is to compile an AST into code that uses OCaml functions within the same binary that's doing the compiling. > > I don't think it can be done with OCaml since it requires a standalone assembler, linker, etc. You could generate some (Ocaml) source code, compile it (using ocamlopt -shared), and then dynamically load the just generated shared module using Dynlink.loadfile. IIRC, the "experimental" ocamlnat binary did that. > Correct me if I'm wrong, though. Mine is a web-based compiler with potentially many concurrent sessions. Running gas, ld, etc. seems a much heavier and less scalable approach that generating code at runtime. Indeed, forking an ocamlopt & then loading the shared module is more heavy, and you have a slight latency issue: if the generated code is not big (i.e. less than a thousand lines of Ocaml code), it could take a few seconds (or tenths of seconds). LLVM is rumored to be a bit faster, but is also rumored to be slow as a pure JIT (just in time) code generated (w.r.t. to other non Ocaml implementations - eg SBCL or CLISP common lisp). Polyml http://polyml.org/ is also supposed to be a JIT-ed SML implementation (it is SML not Ocaml). A few years ago, metaocaml existed, but seems dead today. However, inside a web server, you also have another issue: garbage collection (or simply disposal) of generated code. And this happens even with LLVM. You probably should dispose explicitly & manually of session data and generated code. Ten years ago, SML/NJ was supposed to also garbage-collect code. Ocaml don't do that, and today's Ocaml Dynlink module don't have any unloading code (it would be unsafe). A possible work-around could be to restart your web compiling server once in a while (e.g. twice a day). Happy new year to everyone! Regards. -- Basile STARYNKEVITCH http://starynkevitch.net/Basile/ email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359 8, rue de la Faiencerie, 92340 Bourg La Reine, France *** opinions {are only mines, sont seulement les miennes} *** ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 17:39 ` [Caml-list] " Basile STARYNKEVITCH @ 2010-01-01 19:08 ` Jon Harrop 2010-01-01 20:23 ` Basile STARYNKEVITCH 0 siblings, 1 reply; 8+ messages in thread From: Jon Harrop @ 2010-01-01 19:08 UTC (permalink / raw) To: caml-list; +Cc: Basile STARYNKEVITCH On Friday 01 January 2010 17:39:15 Basile STARYNKEVITCH wrote: > LLVM is rumored to be a bit faster, but is also rumored to be slow as a > pure JIT (just in time) code generated (w.r.t. to other non Ocaml > implementations - eg SBCL or CLISP common lisp). Are you saying that LLVM's JIT is slow to generate code or that the code it generates runs slow? > Polyml http://polyml.org/ > is also supposed to be a JIT-ed SML implementation (it is SML not Ocaml). A > few years ago, metaocaml existed, but seems dead today. Walid said that MetaOCaml was going to relive back in August 2008: http://ocamlnews.blogspot.com/2008/07/metaprogramming-with-metaocaml.html?showComment=1217617620000#c4695232398067055037 > Happy new year to everyone! Happy New Year! -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 19:08 ` Jon Harrop @ 2010-01-01 20:23 ` Basile STARYNKEVITCH 2010-01-01 22:29 ` Jon Harrop 0 siblings, 1 reply; 8+ messages in thread From: Basile STARYNKEVITCH @ 2010-01-01 20:23 UTC (permalink / raw) To: Jon Harrop; +Cc: caml-list Jon Harrop wrote: > On Friday 01 January 2010 17:39:15 Basile STARYNKEVITCH wrote: >> LLVM is rumored to be a bit faster, but is also rumored to be slow as a >> pure JIT (just in time) code generated (w.r.t. to other non Ocaml >> implementations - eg SBCL or CLISP common lisp). > > Are you saying that LLVM's JIT is slow to generate code or that the code it > generates runs slow? I heard that LLVM code generation time is significantly higher (i.e. slower) than other JIT technologies. So machine code generation time is apparently significant which might be an issue inside a web server) but performance of the generated code is supposedly good (inside a web server this is important only if the generated code runs a lot, in particular more than in a single session). I don't have enough personal experience to validate that claim. However, both MONO & PARROT sites are saying something similar: http://www.mono-project.com/Mono_LLVM http://trac.parrot.org/parrot/wiki/JITRewrite http://cliffhacks.blogspot.com/2007/03/experimenting-with-llvm.html But again, I may be wrong. Only real benchmarks on real applications can tell. I believe that libjit & GNU lightning should probably both generate machine code quicker than LLVM does, but the performance of the generated code (by libjit or by lightning) is worse than when using LLVM. And some benchmarks on http://www.phoronix.com/scan.php?page=article&item=apple_llvm_gcc&num=1 suggest that LLVM generated machine code is less efficient than GCC generated machine code. Again, take all this with a grain of salt... Regards. -- Basile STARYNKEVITCH http://starynkevitch.net/Basile/ email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359 8, rue de la Faiencerie, 92340 Bourg La Reine, France *** opinions {are only mines, sont seulement les miennes} *** ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 20:23 ` Basile STARYNKEVITCH @ 2010-01-01 22:29 ` Jon Harrop 2010-01-01 22:33 ` Yoann Padioleau 0 siblings, 1 reply; 8+ messages in thread From: Jon Harrop @ 2010-01-01 22:29 UTC (permalink / raw) To: Basile STARYNKEVITCH; +Cc: caml-list On Friday 01 January 2010 20:23:35 Basile STARYNKEVITCH wrote: > I heard that LLVM code generation time is significantly higher (i.e. > slower) than other JIT technologies. So machine code generation time is > apparently significant which might be an issue inside a web server) but > performance of the generated code is supposedly good (inside a web server > this is important only if the generated code runs a lot, in particular more > than in a single session). > > I don't have enough personal experience to validate that claim. > > However, both MONO & PARROT sites are saying something similar: > > http://www.mono-project.com/Mono_LLVM > > http://trac.parrot.org/parrot/wiki/JITRewrite > > http://cliffhacks.blogspot.com/2007/03/experimenting-with-llvm.html > > But again, I may be wrong. Only real benchmarks on real applications can > tell. > > I believe that libjit & GNU lightning should probably both generate machine > code quicker than LLVM does, but the performance of the generated code (by > libjit or by lightning) is worse than when using LLVM. > > And some benchmarks on > http://www.phoronix.com/scan.php?page=article&item=apple_llvm_gcc&num=1 > suggest that LLVM generated machine code is less efficient than GCC > generated machine code. > > Again, take all this with a grain of salt... That's a fair assessment but LLVM is only about 2x slower than the fastest compilers and it generates code that runs 2-10x faster. For example, compiling the 155kLOC of LLVM IR generated by HLVM's test suite takes 9.65s. I think it was a big mistake for the Go developers at Google and the Mono developers at Novell to not build upon LLVM. My main concern about other JITs is maturity: AFAIK LLVM has orders of magnitude more users that all of the others combined. -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 22:29 ` Jon Harrop @ 2010-01-01 22:33 ` Yoann Padioleau 2010-01-02 0:06 ` Jon Harrop 0 siblings, 1 reply; 8+ messages in thread From: Yoann Padioleau @ 2010-01-01 22:33 UTC (permalink / raw) To: Jon Harrop; +Cc: Basile STARYNKEVITCH, caml-list [-- Attachment #1: Type: text/plain, Size: 773 bytes --] On Jan 1, 2010, at 2:29 PM, Jon Harrop wrote: > > I think it was a big mistake for the Go developers at Google and the > Mono > developers at Novell to not build upon LLVM. > > My main concern about other JITs is maturity: AFAIK LLVM has orders of > magnitude more users that all of the others combined. And ? C has many orders of magnitude more users than ocaml ... > > -- > Dr Jon Harrop, Flying Frog Consultancy Ltd. > http://www.ffconsultancy.com/?e > > _______________________________________________ > Caml-list mailing list. Subscription management: > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list > Archives: http://caml.inria.fr > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > [-- Attachment #2: Type: text/html, Size: 1180 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 22:33 ` Yoann Padioleau @ 2010-01-02 0:06 ` Jon Harrop 0 siblings, 0 replies; 8+ messages in thread From: Jon Harrop @ 2010-01-02 0:06 UTC (permalink / raw) To: Yoann Padioleau; +Cc: caml-list On Friday 01 January 2010 22:33:42 Yoann Padioleau wrote: > On Jan 1, 2010, at 2:29 PM, Jon Harrop wrote: > > I think it was a big mistake for the Go developers at Google and the > > Mono developers at Novell to not build upon LLVM. > > > > My main concern about other JITs is maturity: AFAIK LLVM has orders of > > magnitude more users that all of the others combined. > > And ? That maturity gives me confidence in LLVM's reliability and on-going development. > C has many orders of magnitude more users than ocaml ... Indeed. -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] ocaml, llvm and generating code at runtime 2010-01-01 16:45 ocaml, llvm and generating code at runtime Joel Reymont 2010-01-01 17:39 ` [Caml-list] " Basile STARYNKEVITCH @ 2010-01-01 19:31 ` Jon Harrop 1 sibling, 0 replies; 8+ messages in thread From: Jon Harrop @ 2010-01-01 19:31 UTC (permalink / raw) To: caml-list On Friday 01 January 2010 16:45:34 Joel Reymont wrote: > Does anybody have example code that shows how to generate OCaml bindings at > runtime with LLVM? The OCaml Journal article "Calling C code from OCaml" (10th July 2008) described an OCaml program that invoked a C function both via a stub written in C and via JIT compiled interface code using LLVM (no more stinking C stubs!). I would like to write more on JIT-compiled interface code in the future and perhaps write a library to make it easy. > My goal is to compile an AST into code that uses OCaml functions within the > same binary that's doing the compiling. > > I don't think it can be done with OCaml since it requires a standalone > assembler, linker, etc. Correct me if I'm wrong, though. Mine is a > web-based compiler with potentially many concurrent sessions. Running gas, > ld, etc. seems a much heavier and less scalable approach that generating > code at runtime. Sounds like you really want HLVM. :-) > Thanks and Happy New Year, Joel Happy New Year! -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-01-01 22:52 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-01-01 16:45 ocaml, llvm and generating code at runtime Joel Reymont 2010-01-01 17:39 ` [Caml-list] " Basile STARYNKEVITCH 2010-01-01 19:08 ` Jon Harrop 2010-01-01 20:23 ` Basile STARYNKEVITCH 2010-01-01 22:29 ` Jon Harrop 2010-01-01 22:33 ` Yoann Padioleau 2010-01-02 0:06 ` Jon Harrop 2010-01-01 19:31 ` Jon Harrop
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).