Re: ocamllex speed [was Re: [Caml-list] mboxlib reloaded ;-)]

[Bruno De Fraine <Bruno.De.Fraine@vub.ac.be>]

On 24 Sep 2007, at 21:54, Alain Frisch wrote:

> Bruno De Fraine wrote:
>> Neither version does anything useful, except print the current  
>> directory when it encounters the string "current_directory". I  
>> tested this on a 57M text file (that has only a few  
>> "current_directory" occurrences). The ocamllex-version takes about  
>> 3.5s, while the Str-version takes only 0.35s. What causes this  
>> difference? Perhaps there is a high overhead in calling the  
>> translate function for every input character in such big input  
>> files, but I don't know how this can be avoided.
>
> Did you try to compile the same programs with the native compiler?

Yes, the timings were for the ocamlopt compiler. Following skaller's  
suggestion, I also tried:

rule translate = parse
| [^ ' ' '\n' '\t']+ as word
    { if word = "current_directory" then print_endline (Sys.getcwd ());
      translate lexbuf }
| _ { translate lexbuf }
| eof { () }

This brings it down to 1.87s. Of course, only occurrences of  
"current_directory" that are white-space separated are picked up.

Regards,
Bruno