Re: [Caml-list] Re: zcat vs CamlZip

[John Carr <jfc@MIT.EDU>]

> This is your most likely culprit. Any kind of "do this for every 
> character" is usually insanely expensive when you can do it in bulk.

I wrote a program that read data from a text file, which
could optionally be compressed.  I defined my text file
format to have nearly-fixed length lines so I could call
Gzip.really_input.  My program doesn't spend much of its
time reading the text file so I didn't spend much time
making input fast.  I just did what I thought the obvious
optimization of reading a block of characters in the
normal case.

let input_line =
  begin function
      Uncompressed c ->
 	input_line c
    | Compressed c ->
	begin match Gzip.input_char c with
	  '#' -> while Gzip.input_char c <> '\n' do () done; "#"
	| 'S' -> 
	    let buf = String.make 11 'S' in
	    Gzip.really_input c buf 1 10;
	    if String.unsafe_get buf 10 = '\n' then
	      String.unsafe_set buf 10 ' '
	    else begin
	      if Gzip.input_char c <> '\n' then
		failwith "bad override file"
	    end;
	    buf
	| _ -> failwith "bad override file"
	end
  end

(Lines are variable-length comments beginning '#' or data
lines beginning with 'S' followed by 9 or 10 characters.)