Re: zcat vs CamlZip

[Bardur Arantsson <spam@scientician.net>]

Sam Steingold wrote:
> I read through a huge *.gz file.
> I have two versions of the code:
[--snip--]
> 
> let buf = Buffer.create 1024
> let gz_input_line gz_in char_counter line_counter =
>   Buffer.clear buf;
>   let finish () = incr line_counter; Buffer.contents buf in
>   let rec loop () =
>     let ch = Gzip.input_char gz_in in

This is your most likely culprit. Any kind of "do this for every 
character" is usually insanely expensive when you can do it in bulk.
(This is especially true when needing to do system calls, or if the 
called function cannot be inlined.)

-- 
Bardur Arantsson
<bardurREMOVE@THISscientician.net>

If you can't join 'em, beat 'em. Preferably with a big stick.