Tried `opam switch ocaml-4.03.0+trunk+flambda` on the Transit format encoder/decoder i have. I wanted to see how much faster flambda would make the code, since I've heard 20% and 30% thrown around. It is not very optimized code, and in particular, the encoder path is rather elegant, but awfully slow. Well, not anymore: 4.02: Name Time/Run mWd/Run mjWd/Run Prom/Run Percentage ------------ ---------- ------------ ---------- ---------- ------------ decode 2.12ms 352.86kw 34.86kw 34.86kw 27.88% encode 5.07ms 647.93kw 263.69kw 250.40kw 66.70% round_trip 7.61ms 1_000.79kw 298.54kw 285.26kw 100.00% 4.03.0+trunk+flambda: │ Name │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │ │ decode │ 2.04ms │ 319.83kw │ 35.94kw │ 35.94kw │ 43.97% │ │ encode │ 2.65ms │ 422.67kw │ 130.88kw │ 117.59kw │ 56.95% │ │ round_trip │ 4.65ms │ 742.50kw │ 164.85kw │ 151.56kw │ 100.00% │ Pretty impressive result. Note the heavyweight lifting is due to the yajl JSON parser and this poses a lower bound. But I think the speedup in the encode-path is rather good. Note that the benchmark is probably flawed and some time passed between these two runs, so there might be a confounder hidden in other fixes, either to yajl, or to other parts of the compiler toolchain. However, I think the result itself stands since in practice, my encoding time was just cut in half. On Thu, Mar 10, 2016 at 4:49 PM, Gabriel Scherer wrote: > One point that is tangentially related to your message is that the > flambda people observed that it's easy to miss cross-module > optimizations because .cmx files are missing -- the compiler is silent > about this. Leo White added a new warning (58) when a module does not > find the .cmx of one of its dependencies, which interacts with -opaque > (initially introduced in 4.02.0 when compiling implementation files) > in the following way. In 4.03, you can compile an *interface* with > -opaque, announcing the intent not to provide an .cmx file (or to > choose among several implementations at link-time) for its > implementation(s). Warning 58 will not warn about a missing .cmx if > the dependency's interface was compiled opaque. > https://github.com/ocaml/ocaml/pull/319 > > I think the long-term plan is to encourage people to enable the > warning, and explicitly use -opaque on .cmi when it is their intent > not to distribute .cmx files. That said, those things may be refined > once we get more experience of flambda in the wild. > > On Thu, Mar 10, 2016 at 10:32 AM, Markus Mottl > wrote: > > Ok, that explains things. Is it realistic to assume that the size of > > .cmx files can be substantially reduced? It seems there is a natural > > tradeoff between "optimize well" and "compile fast". I suspect it may > > be inevitable to add more compilation files. We actually already have > > that situation with native code libraries: the .cmxa file is enough to > > compile a project, but if the .cmx files of contained modules are > > visible in the path, too, then, and only then, the compiler can and > > will do cross-module inlining - which takes longer, of course. > > > > What about the following approach? - There is one "minimal" set of > > compilation files that always allows you to quickly obtain a running > > (albeit slow / large) executable. Additional compilation files then > > monotonically augment this information and can be produced and > > consumed optionally depending on compilation flags. The nice thing > > about this approach is that you don't necessarily have to recompile > > the whole project with different flags whenever you need a different > > compile time / performance tradeoff. E.g. if Flambda information is > > available for an unchanged file, you don't have to rebuild it when > > needed. If you just want to compile quickly, you don't have to read > > data you don't need. Separate compilation files would also integrate > > much better with build tools (timestamping, etc.). > > > > I guess we would already be looking at OCaml version 5 for such a change > :) > > > > On Thu, Mar 10, 2016 at 2:20 AM, Mark Shinwell > wrote: > >> By "enabled at configure time" I mean that you need to pass the > >> "-flambda" option to the configure script when building the compiler. > >> > >> The main reason Flambda isn't enabled by default is because we need to > >> do further work to improve compile-time performance. There are also > >> concerns about .cmx file size. Flambda produces larger .cmx files: it > >> stores the entire intermediate representation of the compilation unit > >> so that no subsequent cross-module inlining decision is compromised. > >> > >> There is a mode, -Oclassic, which uses Flambda but mimics the > >> behaviour of the existing compiler; unfortunately this isn't really > >> fast enough yet either and .cmx sizes aren't small enough. > >> > >> When we manage to address some of these issues further, hopefully for > >> 4.04, we will revisit whether Flambda should be enabled by default. > >> > >> One of the main reasons there is a configure option rather than a > >> runtime switch is to avoid having to re-engineer the compiler's build > >> system to permit multiple builds of the various libraries (the stdlib, > >> for example) with differing options that affect what appears in the > >> .cmx files (e.g. with and without Flambda). Even if code were used to > >> allow Flambda to read non-Flambda .cmx files, performance degradation > >> would result. > >> > >> Mark > >> > >> On 10 March 2016 at 01:43, Markus Mottl wrote: > >>> I agree with Yotam. Assuming that Flambda produces correct code and > >>> doesn't cause any serious performance issues either with the generated > >>> code or with excessive compile times, I'd prefer building it into the > >>> compiler by default. I'd be fine if I had to pass an extra flag at > >>> compile time to actually run Flambda optimizers, but it should at > >>> least be available. It doesn't have to be perfect to be useful. > >>> > >>> On Wed, Mar 9, 2016 at 8:32 PM, Yotam Barnoy > wrote: > >>>> While we await the manual, can you explain what you mean by 'enabled > at > >>>> configure time'? Will a -flambda -O-something argument passed to the > normal > >>>> 4.03 compiler enable flambda optimizations? Flambda is clearly the > star of > >>>> the 4.03 release, so not enabling it using command line options seems > >>>> counter-intuitive (if this is the case). > >>>> > >>>> -Yotam > >>>> > >>>> On Wed, Mar 9, 2016 at 7:59 PM, Markus Mottl > wrote: > >>>>> > >>>>> I've just tested Flambda, and it seems to already be doing a pretty > >>>>> decent job on some non-trivial examples (e.g. inlining combinations > of > >>>>> functors and first class functions). I hope there will be a stable > >>>>> 4.03 OPAM switch that enables it. I'm looking forward to being able > >>>>> to write more elegant, abstract code that's still efficient. > >>>>> > >>>>> Regards, > >>>>> Markus > >>>>> > >>>>> On Wed, Mar 9, 2016 at 2:14 AM, Mark Shinwell < > mshinwell@janestreet.com> > >>>>> wrote: > >>>>> > It will not be enabled by default in 4.03. For the majority of > >>>>> > programs, in the current state, it should improve performance > (mainly > >>>>> > by lowering allocation). It should never generate wrong code. > >>>>> > However we know of examples that don't improve as much as we would > >>>>> > like, which we will try to address for 4.04. > >>>>> > > >>>>> > There will be a draft version of the new Flambda manual chapter > >>>>> > available shortly (hopefully this week). Amongst other things this > >>>>> > documents what you found about the configure options and the flags' > >>>>> > operation. > >>>>> > > >>>>> > Mark > >>>>> > > >>>>> > On 9 March 2016 at 03:55, Markus Mottl > wrote: > >>>>> >> Hi Alain, > >>>>> >> > >>>>> >> I see, thanks. It was a little confusing, because the command > line > >>>>> >> options for tuning flambda were still available even without > Flambda > >>>>> >> being enabled. > >>>>> >> > >>>>> >> Will Flambda be enabled by default in OCaml 4.03 or is it still > >>>>> >> considered to be too experimental? It could turn out to become > one of > >>>>> >> the most impactful new features in terms of how I write code. > >>>>> >> > >>>>> >> Regards, > >>>>> >> Markus > >>>>> >> > >>>>> >> On Tue, Mar 8, 2016 at 5:53 PM, Alain Frisch < > alain.frisch@lexifi.com> > >>>>> >> wrote: > >>>>> >>> Hi Markus, > >>>>> >>> > >>>>> >>> flambda needs to be enabled explicitly at configure time with the > >>>>> >>> "-flambda" > >>>>> >>> flag. The new optimizer will then be used unconditionally, and > you > >>>>> >>> can > >>>>> >>> tweak it using command-line parameters passed to ocamlopt (see > >>>>> >>> "ocamlopt > >>>>> >>> -h"). > >>>>> >>> > >>>>> >>> > >>>>> >>> Alain > >>>>> >>> > >>>>> >>> > >>>>> >>> On 08/03/2016 23:10, Markus Mottl wrote: > >>>>> >>>> > >>>>> >>>> Hi, > >>>>> >>>> > >>>>> >>>> I'm trying out OCaml 4.03.0+beta1 right now and wanted to test > >>>>> >>>> Flambda > >>>>> >>>> optimizations. But looking at the generated assembly, it > doesn't > >>>>> >>>> seem > >>>>> >>>> to be doing much if anything on the simple test examples that I > >>>>> >>>> thought would benefit. > >>>>> >>>> > >>>>> >>>> To give an example of what I expected to see, lets consider this > >>>>> >>>> code: > >>>>> >>>> > >>>>> >>>> ----- > >>>>> >>>> let map_pair f (x, y) = f x, f y > >>>>> >>>> > >>>>> >>>> let succ x = x + 1 > >>>>> >>>> let map_pair_succ1 pair = map_pair succ pair > >>>>> >>>> let map_pair_succ2 (x, y) = succ x, succ y > >>>>> >>>> ----- > >>>>> >>>> > >>>>> >>>> I would have thought that the "succ" function would be inlined > in > >>>>> >>>> "map_pair_succ1" as the compiler would do for "map_pair_succ2". > >>>>> >>>> But the generated code looks like this: > >>>>> >>>> > >>>>> >>>> ----- > >>>>> >>>> L101: > >>>>> >>>> movq %rax, %rdi > >>>>> >>>> movq %rdi, 8(%rsp) > >>>>> >>>> movq %rbx, (%rsp) > >>>>> >>>> movq 8(%rbx), %rax > >>>>> >>>> movq (%rdi), %rsi > >>>>> >>>> movq %rdi, %rbx > >>>>> >>>> call *%rsi > >>>>> >>>> L102: > >>>>> >>>> movq %rax, 16(%rsp) > >>>>> >>>> movq (%rsp), %rax > >>>>> >>>> movq (%rax), %rax > >>>>> >>>> movq 8(%rsp), %rbx > >>>>> >>>> movq (%rbx), %rdi > >>>>> >>>> call *%rdi > >>>>> >>>> ----- > >>>>> >>>> > >>>>> >>>> Is Flambda supposed to work out of the box with the current > beta? > >>>>> >>>> What flags or annotations should I use for testing? Any > showcase > >>>>> >>>> examples I should try out that are expected to be improved? > >>>>> >>>> > >>>>> >>>> Regards, > >>>>> >>>> Markus > >>>>> >>>> > >>>>> >>> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> -- > >>>>> >> Markus Mottl http://www.ocaml.info > markus.mottl@gmail.com > >>>>> >> > >>>>> >> -- > >>>>> >> Caml-list mailing list. Subscription management and archives: > >>>>> >> https://sympa.inria.fr/sympa/arc/caml-list > >>>>> >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > >>>>> >> Bug reports: http://caml.inria.fr/bin/caml-bugs > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Markus Mottl http://www.ocaml.info > markus.mottl@gmail.com > >>>>> > >>>>> -- > >>>>> Caml-list mailing list. Subscription management and archives: > >>>>> https://sympa.inria.fr/sympa/arc/caml-list > >>>>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > >>>>> Bug reports: http://caml.inria.fr/bin/caml-bugs > >>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Markus Mottl http://www.ocaml.info > markus.mottl@gmail.com > > > > > > > > -- > > Markus Mottl http://www.ocaml.info markus.mottl@gmail.com > > > > -- > > Caml-list mailing list. Subscription management and archives: > > https://sympa.inria.fr/sympa/arc/caml-list > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > Bug reports: http://caml.inria.fr/bin/caml-bugs > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > -- J.