Speed of ZSH

Speed of ZSH

[Andrej Borsenkow]

I recently needed to do a quick prototyping, and selected zsh just for the
easy programming. The task was: I get a file with records of length 2040
characters and need to extract several (~100) fixed width fields and write
them out separated by tabs. I had a list of fields in the form n1-n2, so I
just vi'ed them :-) to get $line[n1,n2].

The problems I got were:

1. I found, that it is near to impossible to ``read'' line without breaking
it on IFS. ``read -k'' did not work :-( After some play I found, that
unsetting IFS helps - not very nice. How useful would it be to add a flag
that says "read entire line in scalar variable without splitting"?

2. It is impossible to ``print'' arguments separated by user defined string
(only NL or NUL). I had to make an array out of fields, and then print it
with explicit join:

fields=( $line[n1,n2] ...)
print ${(j/\t/)fields}

(Aside: how useful would it be to add flag to ``print'' to define
separator?)

After doing it I found, that ZSH needs several seconds for a file with 28
records! That is really too much for such a simple task. I suspect, the
reason is constant reallocation of memory when first array is created
(remember, it has about 100 elements) and then when it is joined.

If anybody interested, I could recompile with profiling.

I don't use zsh-mem mostly because I don't understand, what it actually is
:-)

/andrej

Re: Speed of ZSH

[Bart Schaefer]

On May 21, 12:11pm, Andrej Borsenkow wrote:
} Subject: Speed of ZSH
}
} I recently needed to do a quick prototyping, and selected zsh just for the
} easy programming. The task was: I get a file with records of length 2040
} characters and need to extract several (~100) fixed width fields and write
} them out separated by tabs. I had a list of fields in the form n1-n2, so I
} just vi'ed them :-) to get $line[n1,n2].
} 
} The problems I got were:
} 
} 1. I found, that it is near to impossible to ``read'' line without breaking
} it on IFS. ``read -k'' did not work :-(

You have to combine it with -u to get the desired effect, and you need
to use -r to prevent backslash-interpretation.  I tried to explain about
this in my rewrite of the "read" info in 3.1.5, but maybe I didn't do a
good enough job.  (It's also a bit odd that a space must precede the
argument to -k and must not precede the argument to -u, at least as the
3.0.5 code stands.)

    read -ru0 -k 2040 block < file

} 2. It is impossible to ``print'' arguments separated by user defined string
} (only NL or NUL).

Unless there are embedded NULs in the input, just use "tr":

    print -rN -- "$block[n1,n2]" "$block[n3,n4]" ... | tr '\0' '\11'

But:

} I had to make an array out of fields, and then print it
} with explicit join:
} 
} fields=( $line[n1,n2] ...)
} print ${(j/\t/)fields}

If you're already spelling it all out like that, why not use a single
quoted string and put in the tabs yourself?

    print -r -- "$block[n1,n2]	$block[n3,n4]	..."

It won't be pretty, but it'll get the job done.

} After doing it I found, that ZSH needs several seconds for a file with 28
} records! That is really too much for such a simple task. I suspect, the
} reason is constant reallocation of memory when first array is created
} (remember, it has about 100 elements) and then when it is joined.

You're probably right.  There are good and bad ways to write programs in
any language.  Simply doing the read/print isn't that slow (though about
8 times slower than dd, look at the user seconds):

zagzig[31] time while read -ru0 -k 2040 block ; do print -r -- "$block"; done < /etc/termcap | dd bs=2040 of=/dev/null
213+198 records in
213+198 records out
while read -ru0 -k 2040 block; do; print -r -- "$block"; done < /etc/termcap 0.79s user 0.07s system 93% cpu 0.917 total
dd bs=2040 of=/dev/null  0.01s user 0.02s system 3% cpu 0.875 total

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com