[Caml-list] Slow... very slow...

[Caml-list] Slow... very slow...

[Oliver Bandel]

Hello,

I'm reading in data from an external
program, and do some conversion on it.


I first have used btreeop and read out data.
Then I have used the ocaml-program and let it do
it's work.

For the btreeop's I have redirected the output to a file.

(I was not shure, if /dev/null is faster and may
 create prolems in the measurment, because the ocaml-program
 can't discard the data; so I decided to not discard it
 via /dev/null for the invocation of the btreeop's and
 direct it to a file, so that there may be similar
 conditions in the fact that the data isn't discarded
 by the system.)




On an old Intel-machine ("CPU: AMD Am5x86-WB stepping 04")
the times ("real times") for the btreeop-operations are:
0.905 s + 3.959 s + 3.715 s

The ocaml-Program (bytecode) uses 48.395 s
and the native-code program needs 41.612 s


On a PowerBook G4 the situation is:

Btreeop's:
0.543 s + 0.200 s + 0.538 s

Ocaml-bytecode:  4.78 s
Ocaml-nativecode: 4.097 s


But here during the third call of btreeop an error-message
will be created  ("btreeop: dbop_next failed").

(But this problem is reported from the call of the ocaml-program too.)



Why is the Ocaml-program so slow, compared to the pure
reading of the data? Normally reading from the disk
should need the most time, and working in memory
for simple tasks should be no problem.

Is my code crap, or what is the problem here?


And: Why is the ocaml program on the 486 sooooooo much slower
than the btreeop-calls, and the ocaml-program on the G4
not sooo much? (Maybe it's because OCaml on the G4 uses
the G4's altivec-optimizations? Or is OCaml not optimized
for the altivec and it will have other reasons?)

On both computers I'm running Ocaml 3.04 with Linux.

Intel: Debian with 2.2.20-idepci
G4:    Suse   with 2.4.2





The code:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

exception Problem


let gsyms_channel  = Unix.open_process_in "/usr/bin/btreeop ./GSYMS"
let gtags_channel  = Unix.open_process_in "/usr/bin/btreeop ./GTAGS"
let grtags_channel = Unix.open_process_in "/usr/bin/btreeop ./GRTAGS"
let gpath_channel  = Unix.open_process_in "/usr/bin/btreeop ./GPATH"



(* ------------------------------------------------------------------- *)

let split_gsyms_line line =
   let firstsplit = Str.bounded_split (Str.regexp "[ \t]+") line 3
   in
     match firstsplit with
       id::path::commaseplist::[] -> (id,path, Str.split (Str.regexp ",") commaseplist)
     | _ -> raise Problem


let split_gtags_line line =
   let firstsplit = Str.bounded_split (Str.regexp "[ \t]+") line 4
   in
     match firstsplit with
       id::linenum::path::line::[] -> (id, int_of_string linenum, path, line)
     | _ -> raise Problem


let split_grtags_line line =
   let firstsplit = Str.bounded_split (Str.regexp "[ \t]+") line 4
   in
     match firstsplit with
       id::linenum::path::line::[] -> (id, int_of_string linenum, path, line)
     | _ -> raise Problem

(* ----------------------------------------------------------------------- *)


let input_lines_with_converter  chan converter =
  let rec input_lines_helper res =
    let sl =
       try
         Some (input_line chan)
       with
         End_of_file -> None in
    match sl with
       None -> List.rev res
     | Some l -> input_lines_helper (converter l :: res) in
  input_lines_helper []




let gsyms_data  = input_lines_with_converter gsyms_channel split_gsyms_line
let gtags_data  = input_lines_with_converter gtags_channel split_gtags_line
let grtags_data = input_lines_with_converter grtags_channel split_grtags_line


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



Thanx in Advance,
             Oliver


P.S.: Should I use imperative code, or writing better functional
      one? Or falling back to C-programming? (oouch!)
      Well... programming in OCaml is fun... but 40 seconds on
      the old machine, when the btreeop's need about 8 seconds...
      ...well... thats 500 % more for only working on the
      data, having no hard-disk stress...


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Slow... very slow...

[Florian Hars]

Oliver Bandel wrote:
> Why is the Ocaml-program so slow,

The tool of choice if you don't understand where your program spends it's time 
isn't mutt, but gprof or ocamlprof.

The main problem may be here:

>    let firstsplit = Str.bounded_split (Str.regexp "[ \t]+") line 3

If Str.regexp is implemented in the same way as Pcre.regexp (where I first 
encountered this problem), the compiler regards the call to the external C 
library prividing the regexp code as an unsafe operation that may have side 
effects and so must be repeated on every invocation of Str.bounded_split.
Try moving it to the toplevel:

let tabre = Str.regexp "[ \t]+"

and then change all your functions to use it like:

     let firstsplit = Str.bounded_split tabre line 3

Yours, Florian.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners