Large LS_COLORS makes auto_cd very slow

Large LS_COLORS makes auto_cd very slow

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

I am using zsh 4.3.17.

This series of initialization (with the attached, admittedly rather
big DIRCOLORS.256), makes auto_cd very slow.

zsh -f
setopt AUTO_CD
eval $(dircolors -b ~/DIRCOLORS.256)
autoload -U compinit; compinit
zstyle ':completion:*' list-colors ${${(s.:.)LS_COLORS}/\*/\=\*}
zstyle ':completion:*' group-name ''

What I am trying to achieve with the substitution in the list-colors
style is to make the explicit coloring of files with file endings take
precedent over, say, executables.

With the above lines loaded, the following completion takes over a
full second on my Linux system:

/us<tab>

whereas completion after an explicit cd is nearly instantaneous:

cd /us<tab>

It seems rather odd to me that coloring would render this so slow. I
suppose the delay is caused by zsh trying to color some list of
possible completions, but why would it be so very slow?

[-- Attachment #2: DIRCOLORS.256 --]
[-- Type: application/octet-stream, Size: 14685 bytes --]

# Configuration file for dircolors, a utility to help you set the
# LS_COLORS environment variable used by GNU ls with the --color option.
# Copyright (C) 1996, 1999-2011 Free Software Foundation, Inc.
# Copying and distribution of this file, with or without modification,
# are permitted provided the copyright notice and this notice are preserved.
# The keywords COLOR, OPTIONS, and EIGHTBIT (honored by the
# slackware version of dircolors) are recognized but ignored.
# Below, there should be one TERM entry for each termtype that is colorizable

TERM Eterm
TERM ansi
TERM color-xterm
TERM con132x25
TERM con132x30
TERM con132x43
TERM con132x60
TERM con80x25
TERM con80x28
TERM con80x30
TERM con80x43
TERM con80x50
TERM con80x60
TERM cons25
TERM console
TERM cygwin
TERM dtterm
TERM eterm-color
TERM gnome
TERM gnome-256color
TERM jfbterm
TERM konsole
TERM kterm
TERM linux
TERM linux-c
TERM mach-color
TERM mlterm
TERM putty
TERM rxvt
TERM rxvt-256color
TERM rxvt-cygwin
TERM rxvt-cygwin-native
TERM rxvt-unicode
TERM rxvt-unicode-256color
TERM rxvt-unicode256
TERM screen
TERM screen-256color
TERM screen-256color-bce
TERM screen-bce
TERM screen-w
TERM screen.rxvt
TERM screen.linux
TERM terminator
TERM vt100
TERM xterm
TERM xterm-16color
TERM xterm-256color
TERM xterm-88color
TERM xterm-color
TERM xterm-debian

# Below are the color init strings for the basic file types. A color init
# string consists of one or more of the following numeric codes:
# Attribute codes:
# 00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
# Text color codes:
# 30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
# Background color codes:
# 40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white
#NORMAL 00 # no color code at all
#FILE 00 # regular file: use no color at all
RESET 0 # reset to "normal" color
DIR 01;36 # directory
LINK 01;34 # symbolic link. (If you set this to 'target' instead of a
 # numerical value, the color is as for the file pointed to.)
MULTIHARDLINK 00 # regular file with more than one link
FIFO 40;33 # pipe
SOCK 01;35 # socket
DOOR 01;35 # door
BLK 40;33;01 # block device driver
CHR 40;33;01 # character device driver
ORPHAN 40;31;01 # symlink to nonexistent file, or non-stat'able file
SETUID 37;41 # file that is setuid (u+s)
SETGID 30;43 # file that is setgid (g+s)
CAPABILITY 30;41 # file with capability
STICKY_OTHER_WRITABLE 01;36 # dir that is sticky and other-writable (+t,o+w)
OTHER_WRITABLE 01;36 # dir that is other-writable (o+w) and not sticky
STICKY 01;36 # dir with the sticky bit set (+t) and not other-writable
EXEC 00 # This is for files with execute permission:

.m3u8 38;5;28
.M3U8 38;5;28
.m3u 38;5;28
.M3U 38;5;28
.pls 38;5;28
.PLS 38;5;28
.cue 38;5;28
.CUE 38;5;28
.xspf 38;5;28
.XSPF 38;5;28
.cfg 38;5;34
.CFG 38;5;34
.conf 38;5;34
.CONF 38;5;34
.ini 38;5;34
.INI 38;5;34
.yml 38;5;34
.YML 38;5;34
.flv 38;5;40
.FLV 38;5;40
.swf 38;5;40
.SWF 38;5;40
.swfl 38;5;40
.SWFL 38;5;40
.avi 38;5;41
.AVI 38;5;41
.mpeg 38;5;42
.MPEG 38;5;42
.mpg 38;5;42
.MPG 38;5;42
.mpe 38;5;42
.MPE 38;5;42
.mp4 38;5;42
.MP4 38;5;42
.asf 38;5;43
.ASF 38;5;43
.asx 38;5;43
.ASX 38;5;43
.wm 38;5;43
.WM 38;5;43
.wmv 38;5;43
.WMV 38;5;43
.wmx 38;5;43
.WMX 38;5;43
.wvx 38;5;43
.WVX 38;5;43
.webm 38;5;44
.WEBM 38;5;44
.mpv 38;5;45
.MPV 38;5;45
.mkv 38;5;45
.MKV 38;5;45
.qt 38;5;46
.QT 38;5;46
.mov 38;5;46
.MOV 38;5;46
.ogv 38;5;47
.OGV 38;5;47
.3gp 38;5;48
.3GP 38;5;48
.3g2 38;5;48
.3G2 38;5;48
.ts 38;5;49
.TS 38;5;49
.vob 38;5;49
.VOB 38;5;49
.7z 38;5;58
.7Z 38;5;58
.zip 38;5;59
.ZIP 38;5;59
.gz 38;5;60
.GZ 38;5;60
.z 38;5;60
.Z 38;5;60
.tar 38;5;61
.TAR 38;5;61
.bz2 38;5;62
.BZ2 38;5;62
.tbz2 38;5;62
.tBZ2 38;5;62
.tar.bz2 38;5;62
.TAR.bz2 38;5;62
.tb2 38;5;62
.TB2 38;5;62
.rar 38;5;63
.RAR 38;5;63
.gz 38;5;64
.GZ 38;5;64
.tar.gz 38;5;64
.TAR.gz 38;5;64
.tgz 38;5;64
.TGZ 38;5;64
.ace 38;5;65
.ACE 38;5;65
.a 38;5;66
.A 38;5;66
.arj 38;5;67
.ARJ 38;5;67
.jar 38;5;68
.JAR 38;5;68
.rpm 38;5;68
.RPM 38;5;68
.cab 38;5;68
.CAB 38;5;68
.deb 38;5;68
.DEB 38;5;68
.udeb 38;5;68
.UDEB 38;5;68
.apk 38;5;68
.APK 38;5;68
.xpi 38;5;68
.XPI 38;5;68
.egg 38;5;68
.EGG 38;5;68
.pkg 38;5;68
.PKG 38;5;68
.jad 38;5;68
.JAD 38;5;68
.gem 38;5;68
.GEM 38;5;68
.msi 38;5;68
.MSI 38;5;68
.iso 38;5;70
.ISO 38;5;70
.dmg 38;5;70
.DMG 38;5;70
.vcd 38;5;70
.VCD 38;5;70
.nrg 38;5;70
.NRG 38;5;70
.hqx 38;5;71
.HQX 38;5;71
.sit 38;5;72
.SIT 38;5;72
.sitx 38;5;72
.SITX 38;5;72
.lha 38;5;73
.LHA 38;5;73
.lzh 38;5;73
.LZH 38;5;73
.lzx 38;5;73
.LZX 38;5;73
.zshrc 38;5;76
.ZSHRC 38;5;76
.zprofile 38;5;76
.ZPROFILE 38;5;76
.zshenv 38;5;76
.ZSHENV 38;5;76
.zlogout 38;5;76
.ZLOGOUT 38;5;76
.bashrc 38;5;77
.BASHRC 38;5;77
.bash_profile 38;5;77
.BASH_profile 38;5;77
.bash_logout 38;5;77
.BASH_logout 38;5;77
.profile 38;5;77
.PROFILE 38;5;77
.dircolors 38;5;78
.DIRCOLORS 38;5;78
.vimrc 38;5;82
.VIMRC 38;5;82
.gvimrc 38;5;82
.GVIMRC 38;5;82
.exrc 38;5;82
.EXRC 38;5;82
.sqlite 38;5;99
.SQLITE 38;5;99
.db 38;5;99
.DB 38;5;99
.mdf 38;5;99
.MDF 38;5;99
.ldf 38;5;99
.LDF 38;5;99
.mdb 38;5;99
.MDB 38;5;99
.odb 38;5;99
.ODB 38;5;99
.jpeg 38;5;101
.JPEG 38;5;101
.jpg 38;5;101
.JPG 38;5;101
.jpe 38;5;101
.JPE 38;5;101
.gif 38;5;102
.GIF 38;5;102
.png 38;5;103
.PNG 38;5;103
.pnm 38;5;104
.PNM 38;5;104
.pbm 38;5;104
.PBM 38;5;104
.pgm 38;5;104
.PGM 38;5;104
.ppm 38;5;104
.PPM 38;5;104
.svg 38;5;105
.SVG 38;5;105
.svgz 38;5;105
.SVGZ 38;5;105
.pcx 38;5;106
.PCX 38;5;106
.tiff 38;5;107
.TIFF 38;5;107
.tif 38;5;107
.TIF 38;5;107
.ico 38;5;108
.ICO 38;5;108
.icns 38;5;108
.ICNS 38;5;108
.bmp 38;5;109
.BMP 38;5;109
.psd 38;5;110
.PSD 38;5;110
.ai 38;5;110
.AI 38;5;110
.xbm 38;5;111
.XBM 38;5;111
.xpm 38;5;111
.XPM 38;5;111
.epub 38;5;127
.EPUB 38;5;127
.prc 38;5;127
.PRC 38;5;127
.mobi 38;5;127
.MOBI 38;5;127
.lit 38;5;127
.LIT 38;5;127
.djvu 38;5;127
.DJVU 38;5;127
.djv 38;5;127
.DJV 38;5;127
.cbr 38;5;132
.CBR 38;5;132
.cbz 38;5;132
.CBZ 38;5;132
.pdf 38;5;134
.PDF 38;5;134
.chm 38;5;134
.CHM 38;5;134
.dvi 38;5;137
.DVI 38;5;137
.xhtml 38;5;142
.XHTML 38;5;142
.xht 38;5;142
.XHT 38;5;142
.xml 38;5;142
.XML 38;5;142
.xsl 38;5;142
.XSL 38;5;142
.xsd 38;5;142
.XSD 38;5;142
.wsdl 38;5;142
.WSDL 38;5;142
.opml 38;5;142
.OPML 38;5;142
.mml 38;5;143
.MML 38;5;143
.mnl 38;5;143
.MNL 38;5;143
.rdf 38;5;144
.RDF 38;5;144
.xul 38;5;144
.XUL 38;5;144
.rss 38;5;145
.RSS 38;5;145
.atom 38;5;145
.ATOM 38;5;145
.x3dv 38;5;146
.X3DV 38;5;146
.x3d 38;5;146
.X3D 38;5;146
.kml 38;5;147
.KML 38;5;147
.kmz 38;5;147
.KMZ 38;5;147
.jnlp 38;5;148
.JNLP 38;5;148
.csv 38;5;149
.CSV 38;5;149
.tsv 38;5;149
.TSV 38;5;149
.psv 38;5;149
.PSV 38;5;149
.vcs 38;5;150
.VCS 38;5;150
.vcf 38;5;150
.VCF 38;5;150
.msg 38;5;150
.MSG 38;5;150
.eml 38;5;150
.EML 38;5;150
.ics 38;5;150
.ICS 38;5;150
.icz 38;5;150
.ICZ 38;5;150
.txt 38;5;151
.TXT 38;5;151
.log 38;5;151
.LOG 38;5;151
.nfo 38;5;151
.NFO 38;5;151
.sig 38;5;151
.SIG 38;5;151
.manifest 38;5;151
.MANIFEST 38;5;151
.nzb 38;5;155
.NZB 38;5;155
.torrent 38;5;155
.TORRENT 38;5;155
.aac 38;5;160
.AAC 38;5;160
.mpga 38;5;161
.MPGA 38;5;161
.mpega 38;5;161
.MPEGA 38;5;161
.mp2 38;5;161
.MP2 38;5;161
.mp3 38;5;161
.MP3 38;5;161
.m4a 38;5;161
.M4A 38;5;161
.m4b 38;5;161
.M4B 38;5;161
.m4r 38;5;161
.M4R 38;5;161
.wav 38;5;162
.WAV 38;5;162
.wma 38;5;162
.WMA 38;5;162
.oga 38;5;163
.OGA 38;5;163
.ogg 38;5;163
.OGG 38;5;163
.spx 38;5;163
.SPX 38;5;163
.flac 38;5;164
.FLAC 38;5;164
.ape 38;5;164
.APE 38;5;164
.apl 38;5;164
.APL 38;5;164
.mpc 38;5;164
.MPC 38;5;164
.aif 38;5;165
.AIF 38;5;165
.aiff 38;5;165
.AIFF 38;5;165
.aifc 38;5;165
.AIFC 38;5;165
.mid 38;5;166
.MID 38;5;166
.midi 38;5;166
.MIDI 38;5;166
.kar 38;5;166
.KAR 38;5;166
.ra 38;5;167
.RA 38;5;167
.rm 38;5;167
.RM 38;5;167
.ram 38;5;167
.RAM 38;5;167
.pfa 38;5;169
.PFA 38;5;169
.pfb 38;5;169
.PFB 38;5;169
.gsf 38;5;169
.GSF 38;5;169
.pcf 38;5;169
.PCF 38;5;169
.otf 38;5;169
.OTF 38;5;169
.ttf 38;5;169
.TTF 38;5;169
.fon 38;5;169
.FON 38;5;169
.dfont 38;5;169
.DFONT 38;5;169
.ps 38;5;170
.PS 38;5;170
.eps 38;5;170
.EPS 38;5;170
.epsi 38;5;170
.EPSI 38;5;170
.epsf 38;5;170
.EPSF 38;5;170
.eps2 38;5;170
.EPS2 38;5;170
.eps3 38;5;170
.EPS3 38;5;170
.gpg 38;5;172
.GPG 38;5;172
.pgp 38;5;172
.PGP 38;5;172
.asc 38;5;172
.ASC 38;5;172
.pem 38;5;172
.PEM 38;5;172
.crt 38;5;172
.CRT 38;5;172
.csr 38;5;172
.CSR 38;5;172
.sdc 38;5;173
.SDC 38;5;173
.sxc 38;5;173
.SXC 38;5;173
.stc 38;5;173
.STC 38;5;173
.ods 38;5;173
.ODS 38;5;173
.ots 38;5;173
.OTS 38;5;173
.xls 38;5;173
.XLS 38;5;173
.xlb 38;5;173
.XLB 38;5;173
.xlt 38;5;173
.XLT 38;5;173
.xlsx 38;5;173
.XLSX 38;5;173
.xlam 38;5;173
.XLAM 38;5;173
.xlsb 38;5;173
.XLSB 38;5;173
.xlsm 38;5;173
.XLSM 38;5;173
.xltm 38;5;173
.XLTM 38;5;173
.xlsx 38;5;173
.XLSX 38;5;173
.xltx 38;5;173
.XLTX 38;5;173
.xltx 38;5;173
.XLTX 38;5;173
.gnumeric 38;5;173
.GNUMERIC 38;5;173
.ksp 38;5;173
.KSP 38;5;173
.sdf 38;5;174
.SDF 38;5;174
.sxm 38;5;174
.SXM 38;5;174
.odf 38;5;174
.ODF 38;5;174
.sdw 38;5;176
.SDW 38;5;176
.sgl 38;5;176
.SGL 38;5;176
.sxw 38;5;176
.SXW 38;5;176
.sxg 38;5;176
.SXG 38;5;176
.stw 38;5;176
.STW 38;5;176
.odt 38;5;176
.ODT 38;5;176
.odm 38;5;176
.ODM 38;5;176
.ott 38;5;176
.OTT 38;5;176
.oth 38;5;176
.OTH 38;5;176
.docm 38;5;176
.DOCM 38;5;176
.dotm 38;5;176
.DOTM 38;5;176
.doc 38;5;176
.DOC 38;5;176
.dot 38;5;176
.DOT 38;5;176
.docx 38;5;176
.DOCX 38;5;176
.dotx 38;5;176
.DOTX 38;5;176
.rtx 38;5;176
.RTX 38;5;176
.rtf 38;5;176
.RTF 38;5;176
.abw 38;5;176
.ABW 38;5;176
.pages 38;5;176
.PAGES 38;5;176
.wpd 38;5;176
.WPD 38;5;176
.kwd 38;5;176
.KWD 38;5;176
.kwt 38;5;176
.KWT 38;5;176
.sds 38;5;177
.SDS 38;5;177
.odc 38;5;177
.ODC 38;5;177
.chrt 38;5;177
.CHRT 38;5;177
.sda 38;5;178
.SDA 38;5;178
.sxd 38;5;178
.SXD 38;5;178
.std 38;5;178
.STD 38;5;178
.odg 38;5;178
.ODG 38;5;178
.otg 38;5;178
.OTG 38;5;178
.odi 38;5;178
.ODI 38;5;178
.cdr 38;5;178
.CDR 38;5;178
.pat 38;5;178
.PAT 38;5;178
.cdt 38;5;178
.CDT 38;5;178
.cpt 38;5;178
.CPT 38;5;178
.vsd 38;5;178
.VSD 38;5;178
.kil 38;5;178
.KIL 38;5;178
.sdd 38;5;180
.SDD 38;5;180
.sxi 38;5;180
.SXI 38;5;180
.sti 38;5;180
.STI 38;5;180
.odp 38;5;180
.ODP 38;5;180
.otp 38;5;180
.OTP 38;5;180
.ppt 38;5;180
.PPT 38;5;180
.pps 38;5;180
.PPS 38;5;180
.ppam 38;5;180
.PPAM 38;5;180
.pptm 38;5;180
.PPTM 38;5;180
.sldm 38;5;180
.SLDM 38;5;180
.ppsm 38;5;180
.PPSM 38;5;180
.potm 38;5;180
.POTM 38;5;180
.pptx 38;5;180
.PPTX 38;5;180
.sldx 38;5;180
.SLDX 38;5;180
.ppsx 38;5;180
.PPSX 38;5;180
.potx 38;5;180
.POTX 38;5;180
.key 38;5;180
.KEY 38;5;180
.wps 38;5;180
.WPS 38;5;180
.kpr 38;5;180
.KPR 38;5;180
.kpt 38;5;180
.KPT 38;5;180
.rst 38;5;182
.RST 38;5;182
.textile 38;5;183
.TEXTILE 38;5;183
.markdown 38;5;184
.MARKDOWN 38;5;184
.md 38;5;184
.MD 38;5;184
.mkd 38;5;184
.MKD 38;5;184
.texinfo 38;5;185
.TEXINFO 38;5;185
.texi 38;5;185
.TEXI 38;5;185
.info 38;5;185
.INFO 38;5;185
.t 38;5;186
.T 38;5;186
.tr 38;5;186
.TR 38;5;186
.ronn 38;5;186
.RONN 38;5;186
.roff 38;5;186
.ROFF 38;5;186
.man 38;5;186
.MAN 38;5;186
.me 38;5;186
.ME 38;5;186
.ms 38;5;186
.MS 38;5;186
.rb 38;5;197
.RB 38;5;197
.gemspec 38;5;197
.GEMSPEC 38;5;197
.c++ 38;5;198
.C++ 38;5;198
.cpp 38;5;198
.CPP 38;5;198
.cxx 38;5;198
.CXX 38;5;198
.cc 38;5;198
.CC 38;5;198
.c 38;5;198
.C 38;5;198
.h++ 38;5;199
.H++ 38;5;199
.hpp 38;5;199
.HPP 38;5;199
.hxx 38;5;199
.HXX 38;5;199
.hh 38;5;199
.HH 38;5;199
.h 38;5;199
.H 38;5;199
.scpt 38;5;200
.SCPT 38;5;200
.applescript 38;5;200
.APPLESCRIPT 38;5;200
.csproj 38;5;201
.CSPROJ 38;5;201
.vbproj 38;5;201
.VBPROJ 38;5;201
.sln 38;5;201
.SLN 38;5;201
.cl 38;5;202
.CL 38;5;202
.el 38;5;202
.EL 38;5;202
.cs 38;5;203
.CS 38;5;203
.js 38;5;204
.JS 38;5;204
.json 38;5;204
.JSON 38;5;204
.coffee 38;5;204
.COFFEE 38;5;204
.es 38;5;204
.ES 38;5;204
.pl 38;5;205
.PL 38;5;205
.pm 38;5;205
.PM 38;5;205
.pod 38;5;205
.POD 38;5;205
.vb 38;5;206
.VB 38;5;206
.vbs 38;5;206
.VBS 38;5;206
.boo 38;5;207
.BOO 38;5;207
.diff 38;5;208
.DIFF 38;5;208
.patch 38;5;208
.PATCH 38;5;208
.html 38;5;209
.HTML 38;5;209
.htm 38;5;209
.HTM 38;5;209
.shtml 38;5;209
.SHTML 38;5;209
.pht 38;5;210
.PHT 38;5;210
.php 38;5;210
.PHP 38;5;210
.phps 38;5;210
.PHPS 38;5;210
.php3 38;5;210
.PHP3 38;5;210
.php3p 38;5;210
.PHP3P 38;5;210
.php4 38;5;210
.PHP4 38;5;210
.php5 38;5;210
.PHP5 38;5;210
.phtml 38;5;210
.PHTML 38;5;210
.asp 38;5;211
.ASP 38;5;211
.aspx 38;5;211
.ASPX 38;5;211
.ascx 38;5;211
.ASCX 38;5;211
.asmx 38;5;211
.ASMX 38;5;211
.ashx 38;5;211
.ASHX 38;5;211
.haml 38;5;212
.HAML 38;5;212
.erb 38;5;212
.ERB 38;5;212
.rhtml 38;5;212
.RHTML 38;5;212
.py 38;5;213
.PY 38;5;213
.tac 38;5;213
.TAC 38;5;213
.wsgi 38;5;213
.WSGI 38;5;213
.lua 38;5;213
.LUA 38;5;213
.hs 38;5;214
.HS 38;5;214
.lhs 38;5;214
.LHS 38;5;214
.jsp 38;5;215
.JSP 38;5;215
.sh 38;5;216
.SH 38;5;216
.bash 38;5;216
.BASH 38;5;216
.zsh 38;5;216
.ZSH 38;5;216
.csh 38;5;216
.CSH 38;5;216
.tcsh 38;5;216
.TCSH 38;5;216
.ksh 38;5;216
.KSH 38;5;216
.cmd 38;5;216
.CMD 38;5;216
.bat 38;5;216
.BAT 38;5;216
.awk 38;5;216
.AWK 38;5;216
.sed 38;5;216
.SED 38;5;216
.com 38;5;216
.COM 38;5;216
.command 38;5;216
.COMMAND 38;5;216
.java 38;5;217
.JAVA 38;5;217
.egg-.info 38;5;218
.EGG-.info 38;5;218
.pth 38;5;218
.PTH 38;5;218
.tex 38;5;219
.TEX 38;5;219
.ltx 38;5;219
.LTX 38;5;219
.sty 38;5;219
.STY 38;5;219
.cls 38;5;219
.CLS 38;5;219
.tfm 38;5;219
.TFM 38;5;219
.bib 38;5;219
.BIB 38;5;219
.latex 38;5;219
.LATEX 38;5;219
.lyx 38;5;220
.LYX 38;5;220
.tm 38;5;220
.TM 38;5;220
.vim 38;5;221
.VIM 38;5;221
.css 38;5;222
.CSS 38;5;222
.scss 38;5;222
.SCSS 38;5;222
.sass 38;5;222
.SASS 38;5;222
.sql 38;5;223
.SQL 38;5;223
.d 38;5;224
.D 38;5;224
.r 38;5;224
.R 38;5;224
.tcl 38;5;225
.TCL 38;5;225
.tk 38;5;225
.TK 38;5;225
.go 38;5;226
.GO 38;5;226
.erl 38;5;227
.ERL 38;5;227
.ml 38;5;227
.ML 38;5;227
.scala 38;5;227
.SCALA 38;5;227
.asm 38;5;228
.ASM 38;5;228
.p 38;5;229
.P 38;5;229
.pas 38;5;229
.PAS 38;5;229
.gitignore 38;5;238
.GITIGNORE 38;5;238
.hgignore 38;5;238
.HGIGNORE 38;5;238
.bzrignore 38;5;238
.BZRIGNORE 38;5;238
.hg 38;5;239
.HG 38;5;239
.bzr 38;5;239
.BZR 38;5;239
.svn 38;5;239
.SVN 38;5;239
.git 38;5;239
.GIT 38;5;239
.cvs 38;5;239
.CVS 38;5;239
.swp 38;5;240
.SWP 38;5;240
.swo 38;5;240
.SWO 38;5;240
.bak 38;5;240
.BAK 38;5;240
.old 38;5;240
.OLD 38;5;240
.sik 38;5;240
.SIK 38;5;240
.tmp 38;5;240
.TMP 38;5;240
.pkl 38;5;242
.PKL 38;5;242
.dat 38;5;242
.DAT 38;5;242
.ser 38;5;242
.SER 38;5;242
.md5 38;5;244
.MD5 38;5;244
.sfv 38;5;244
.SFV 38;5;244
.sha1 38;5;244
.SHA1 38;5;244
.st5 38;5;244
.ST5 38;5;244
.ffp 38;5;244
.FFP 38;5;244
.par 38;5;244
.PAR 38;5;244
.pyc 38;5;247
.PYC 38;5;247
.pyo 38;5;247
.PYO 38;5;247
.o 38;5;247
.O 38;5;247
.class 38;5;247
.CLASS 38;5;247
.zwc 38;5;247
.ZWC 38;5;247
.dll 38;5;247
.DLL 38;5;247
.exe 38;5;247
.EXE 38;5;247
.app 38;5;247
.APP 38;5;247

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 3,  9:08pm, Jesper Nygårds wrote:
}
} /us<tab>
} 
} whereas completion after an explicit cd is nearly instantaneous:
} 
} cd /us<tab>
} 
} It seems rather odd to me that coloring would render this so slow. I
} suppose the delay is caused by zsh trying to color some list of
} possible completions, but why would it be so very slow?

/us<tab> as the first word in the buffer is in command position, so is
completing any/all of commands, executables, builtins, functions,
aliases, suffix-aliases, reserved-words, jobs and parameters.

For each of these categories, _setup line 12 rebuilds the value of
the _comp_colors array to add new patterns such that any pattern that
started with '=' gets copied with a prefix matching the tag currently
being tested; e.g. '(commands)=...' or '(jobs)=...'

This is done even for tags that won't have any matches because the
colors array has to be ready for the internals to use when a match is
found, there's no way to "call back" to build it on demand.

The expensive bit is that _comp_colors is declared as a unique array,
so every time it gets rebuilt the resulting 1700+ entries are all
compared against one another to make sure there is no duplication.
Repeat that nine times and it takes a while.

With "cd" in front, it's restricted to directories only, so the array
is only rebuilt once.

Addenda for -workers:

Anybody want to have a stab at creating a vastly more efficient version
of Src/params.c:arrayuniq() ?  I came up with a faster way to remove the
duplicate elements, but the problem here is to efficiently determine
that there are no duplicates to remove.  Perhaps throw the elements into
a hash table and then make one pass looking up each element.

Re: Large LS_COLORS makes auto_cd very slow

[Jesper Nygårds]

On Wed, Apr 4, 2012 at 9:52 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> /us<tab> as the first word in the buffer is in command position, so is
> completing any/all of commands, executables, builtins, functions,
> aliases, suffix-aliases, reserved-words, jobs and parameters.

I was guessing it had something to do with it being in command
position. Thank you for the detailed explanation, Bart.

Re: Large LS_COLORS makes auto_cd very slow

[Václav Zeman]

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]

On 4 April 2012 09:52, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Apr 3,  9:08pm, Jesper Nygårds wrote:
> }
> } /us<tab>
> }
> } whereas completion after an explicit cd is nearly instantaneous:
> }
> } cd /us<tab>
> }
> } It seems rather odd to me that coloring would render this so slow. I
> } suppose the delay is caused by zsh trying to color some list of
> } possible completions, but why would it be so very slow?
>
> /us<tab> as the first word in the buffer is in command position, so is
> completing any/all of commands, executables, builtins, functions,
> aliases, suffix-aliases, reserved-words, jobs and parameters.
>
> For each of these categories, _setup line 12 rebuilds the value of
> the _comp_colors array to add new patterns such that any pattern that
> started with '=' gets copied with a prefix matching the tag currently
> being tested; e.g. '(commands)=...' or '(jobs)=...'
>
> This is done even for tags that won't have any matches because the
> colors array has to be ready for the internals to use when a match is
> found, there's no way to "call back" to build it on demand.
>
> The expensive bit is that _comp_colors is declared as a unique array,
> so every time it gets rebuilt the resulting 1700+ entries are all
> compared against one another to make sure there is no duplication.
> Repeat that nine times and it takes a while.
>
> With "cd" in front, it's restricted to directories only, so the array
> is only rebuilt once.
>
> Addenda for -workers:
>
> Anybody want to have a stab at creating a vastly more efficient version
> of Src/params.c:arrayuniq() ?  I came up with a faster way to remove the
> duplicate elements, but the problem here is to efficiently determine
> that there are no duplicates to remove.  Perhaps throw the elements into
> a hash table and then make one pass looking up each element.
I wonder if the attached patch does what you want here. It is fairly
untested and incomplete as I have not been able to find out how to
make sure that search.h gets included.

-- 
VZ

[-- Attachment #2: hash_arrayuniq.diff --]
[-- Type: application/octet-stream, Size: 1880 bytes --]

=== modified file 'Src/params.c'
--- Src/params.c	2012-03-13 09:47:01 +0000
+++ Src/params.c	2012-04-05 09:25:52 +0000
@@ -3456,7 +3456,7 @@
 
 /**/
 static void
-arrayuniq(char **x, int freeok)
+simple_arrayuniq(char **x, int freeok)
 {
     char **t, **p = x;
 
@@ -3471,6 +3471,62 @@
 }
 
 /**/
+static void
+arrayuniq(char **x, int freeok)
+{
+    char **it, **write_it;
+    size_t array_size;
+    int ret;
+    ENTRY item;
+    ENTRY * found_item;
+
+    if (*x == NULL)
+	return;
+
+    for (it = x; *it != NULL; ++it)
+	;
+
+    array_size = it - x;
+    if (array_size <= 10u) {
+	/* fallback to simpler routine */
+	simple_arrayuniq (x, freeok);
+	return;
+    }
+
+    ret = hcreate (array_size);
+    assert (ret);
+    if (! ret) {
+	/* fallback to routine without memory allocation needs */
+	simple_arrayuniq (x, freeok);
+	return;
+    }
+
+    item.data = NULL;
+    
+    for (it = x, write_it = x; *it;) {
+	item.key = *it;
+	found_item = hsearch (item, FIND);
+	if (! found_item) {
+	    found_item = hsearch (item, ENTER);
+	    assert (found_item);
+	    *write_it = *it;
+	    if (it != write_it)
+		*it = NULL;
+	    ++write_it;
+	}
+	else {
+	    if (freeok)
+		zsfree (*it);
+	    *it = NULL;
+	}
+	++it;
+    }
+    
+    
+    hdestroy ();
+}
+
+/**/
 void
 uniqarray(char **x)
 {

=== modified file 'configure.ac'
--- configure.ac	2012-03-05 10:06:28 +0000
+++ configure.ac	2012-04-05 09:21:42 +0000
@@ -610,7 +610,7 @@
 		 termios.h sys/param.h sys/filio.h string.h memory.h \
 		 limits.h fcntl.h libc.h sys/utsname.h sys/resource.h \
 		 locale.h errno.h stdio.h stdarg.h varargs.h stdlib.h \
-		 unistd.h sys/capability.h \
+		 unistd.h sys/capability.h search.h \
 		 utmp.h utmpx.h sys/types.h pwd.h grp.h poll.h sys/mman.h \
 		 netinet/in_systm.h pcre.h langinfo.h wchar.h stddef.h \
 		 sys/stropts.h iconv.h ncurses.h ncursesw/ncurses.h \

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 5, 11:30am, Vaclav Zeman wrote:
}
} > Anybody want to have a stab at creating a vastly more efficient version
} > of Src/params.c:arrayuniq() ?
} I wonder if the attached patch does what you want here. It is fairly
} untested and incomplete as I have not been able to find out how to
} make sure that search.h gets included.

Zsh has its own hash table implementation which ought to be used rather
than introducing any new external dependencies, but otherwise this is
the kind of thing I was suggesting.

Or perhaps someone else has an even more clever idea.  Anything better
than O(N^2) would help.

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 5,  8:51am, Bart Schaefer wrote:
}
} Or perhaps someone else has an even more clever idea.  Anything better
} than O(N^2) would help.

Incidentally the requirement that the output remain in the same order as
the input puts a dent in many of the algorithms you'll find with google
on this question.

Re: Large LS_COLORS makes auto_cd very slow

[Philippe Troin]

On Thu, 2012-04-05 at 09:33 -0700, Bart Schaefer wrote:
> On Apr 5,  8:51am, Bart Schaefer wrote:
> }
> } Or perhaps someone else has an even more clever idea.  Anything better
> } than O(N^2) would help.
> 
> Incidentally the requirement that the output remain in the same order as
> the input puts a dent in many of the algorithms you'll find with google
> on this question.

The patch in
<CAKw7uVh4X_VoJxqtjjC9Cvv2ZS2-xfr29kHyAqyG3V1=DyPQTg@mail.gmail.com>
uses a hash table.
The array uniquification while preserving order with hash is only O(n^2)
in the worst case.  If hsearch's hash implementation is reasonably good,
this algorithm should give you O(kn) where k is a small constant between
1 and 2.

Or so it seems to me. ;-)

Phil.

Re: Large LS_COLORS makes auto_cd very slow

[Václav Zeman]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]

On 5 April 2012 17:51, Bart Schaefer wrote:
> On Apr 5, 11:30am, Vaclav Zeman wrote:
> }
> } > Anybody want to have a stab at creating a vastly more efficient version
> } > of Src/params.c:arrayuniq() ?
> } I wonder if the attached patch does what you want here. It is fairly
> } untested and incomplete as I have not been able to find out how to
> } make sure that search.h gets included.
>
> Zsh has its own hash table implementation which ought to be used rather
> than introducing any new external dependencies, but otherwise this is
> the kind of thing I was suggesting.
>
> Or perhaps someone else has an even more clever idea.  Anything better
> than O(N^2) would help.
I have reimplemented the change using ZSH's own hash table. Please see
the attached patch.

-- 
VZ

[-- Attachment #2: hash_arrayuniq-2.diff.txt --]
[-- Type: text/plain, Size: 1877 bytes --]

=== modified file 'Src/params.c'
--- Src/params.c	2012-03-13 09:47:01 +0000
+++ Src/params.c	2012-04-06 09:42:20 +0000
@@ -29,6 +29,7 @@
 
 #include "zsh.mdh"
 #include "params.pro"
+#include "hashtable.pro"
 
 #include "version.h"
 #ifdef CUSTOM_PATCHLEVEL
@@ -3456,7 +3457,7 @@
 
 /**/
 static void
-arrayuniq(char **x, int freeok)
+simple_arrayuniq(char **x, int freeok)
 {
     char **t, **p = x;
 
@@ -3471,6 +3472,73 @@
 }
 
 /**/
+static void
+arrayuniq_freenode(HashNode hn)
+{
+    (void)hn;
+}
+
+/**/
+static void
+arrayuniq(char **x, int freeok)
+{
+    char **it, **write_it;
+    size_t array_size;
+    int ret;
+    HashNode found_item;
+    struct hashnode new_node;
+    HashTable ht;
+
+    if (*x == NULL)
+	return;
+
+    for (it = x; *it != NULL; ++it)
+	;
+
+    array_size = it - x;
+    if (array_size <= 10u) {
+	/* fallback to simpler routine */
+	simple_arrayuniq(x, freeok);
+	return;
+    }
+
+    ht = newhashtable((int)array_size * 2, "arrayuniq", NULL);
+    /* ??? error checking */
+    ht->hash        = hasher;
+    ht->emptytable  = emptyhashtable;
+    ht->filltable   = NULL;
+    ht->cmpnodes    = strcmp;
+    ht->addnode     = addhashnode;
+    ht->getnode     = gethashnode2;
+    ht->getnode2    = gethashnode2;
+    ht->removenode  = removehashnode;
+    ht->disablenode = disablehashnode;
+    ht->enablenode  = enablehashnode;
+    ht->freenode    = arrayuniq_freenode;
+    ht->printnode   = NULL;
+    
+    for (it = x, write_it = x; *it;) {
+	found_item = gethashnode(ht, *it);
+	if (! found_item) {
+	    found_item = addhashnode2(ht, *it, &new_node);
+	    /*assert(! found_item);*/
+	    *write_it = *it;
+	    if (it != write_it)
+		*it = NULL;
+	    ++write_it;
+	}
+	else {
+	    if (freeok)
+		zsfree(*it);
+	    *it = NULL;
+	}
+	++it;
+    }
+    
+    deletehashtable(ht);
+}
+
+/**/
 void
 uniqarray(char **x)
 {

Re: Large LS_COLORS makes auto_cd very slow

[Mark van Dijk]

On Fri, 6 Apr 2012 11:49:47 +0200
Václav Zeman <vhaisman@gmail.com> wrote:

> On 5 April 2012 17:51, Bart Schaefer wrote:
> > On Apr 5, 11:30am, Vaclav Zeman wrote:
> > }
> > } > Anybody want to have a stab at creating a vastly more efficient
> > version } > of Src/params.c:arrayuniq() ?
> > } I wonder if the attached patch does what you want here. It is
> > fairly } untested and incomplete as I have not been able to find
> > out how to } make sure that search.h gets included.
> >
> > Zsh has its own hash table implementation which ought to be used
> > rather than introducing any new external dependencies, but
> > otherwise this is the kind of thing I was suggesting.
> >
> > Or perhaps someone else has an even more clever idea.  Anything
> > better than O(N^2) would help.
> I have reimplemented the change using ZSH's own hash table. Please see
> the attached patch.
> 

Works for me.

-Mark

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 6, 11:49am, Vaclav Zeman wrote:
}
} I have reimplemented the change using ZSH's own hash table. Please see
} the attached patch.

Thank you.  There are a couple of minor things that could be tweaked,
but this is pretty good.

And indeed completing after "/us" is a lot faster than it was before.

Unfortunately completing after "/b" is still rather slow.  Now the
shell is spending most of its time in complist.c:getcoldef().  Ah, well.

-- 
Barton E. Schaefer

Re: Large LS_COLORS makes auto_cd very slow

[Valodim Skywalker]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

> For each of these categories, _setup line 12 rebuilds the value of
> the _comp_colors array to add new patterns such that any pattern that
> started with '=' gets copied with a prefix matching the tag currently
> being tested; e.g. '(commands)=...' or '(jobs)=...'
> 
> This is done even for tags that won't have any matches because the
> colors array has to be ready for the internals to use when a match is
> found, there's no way to "call back" to build it on demand.
> 
> The expensive bit is that _comp_colors is declared as a unique array,
> so every time it gets rebuilt the resulting 1700+ entries are all
> compared against one another to make sure there is no duplication.
> Repeat that nine times and it takes a while.

Are these steps all hard requirements? I don't have a good overview of
things, but some of these parts seem like they could be rethought to
work faster or the cpu/memory tradeoff shifted a bit towards more memory
use or something. Does the array really need to be unique? How much
fluctuation is in these contexts, this is a frequent operation so maybe
caching the entries, possibly only for known often called contexts,
could help?

 - V

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 6,  8:30pm, Valodim Skywalker wrote:
} 
} > For each of these categories, _setup line 12 rebuilds the value of
} > the _comp_colors array to add new patterns such that any pattern that
} > started with '=' gets copied with a prefix matching the tag currently
} > being tested; e.g. '(commands)=...' or '(jobs)=...'
} > 
} > This is done even for tags that won't have any matches because the
} > colors array has to be ready for the internals to use when a match is
} > found, there's no way to "call back" to build it on demand.
} 
} Are these steps all hard requirements? I don't have a good overview of
} things, but some of these parts seem like they could be rethought to
} work faster or the cpu/memory tradeoff shifted a bit towards more memory
} use or something. Does the array really need to be unique?

Using the hash seive for arrayuniq seems to resolve that part of the
problem.  The array doesn't have to be unique, but making it not so will
only make the other part of the problem [getcoldef()] worse.

} How much fluctuation is in these contexts, this is a frequent
} operation so maybe caching the entries, possibly only for known often
} called contexts, could help?

It's conceivable that getcoldef() could use a cache.  The parse of each
string passed to getcoldef() is not dependent on context, although the
strings themselves are, so getcoldef() could just stash away every one
it ever sees and return it the next time the same parse is needed.

All the heap allocations [zhalloc()] in getcoldef() would have to become
global allocations [zcalloc()], patcompile() would need to be called
with the PAT_ZDUP flag, and the module boot_() and cleanup_() routines
would then allocate and delete a hash table for the cache.

However, this would only help on second and subsequent completions in
a context that had been completed before.  E.g., the first completion
after "/b" in command position would still have a noticable delay; it
checks at least eight different contexts.

Re: Large LS_COLORS makes auto_cd very slow

[Václav Zeman]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On 04/06/2012 11:49 AM, Václav Zeman wrote:
> On 5 April 2012 17:51, Bart Schaefer wrote:
>> On Apr 5, 11:30am, Vaclav Zeman wrote:
>> }
>> } > Anybody want to have a stab at creating a vastly more efficient version
>> } > of Src/params.c:arrayuniq() ?
>> } I wonder if the attached patch does what you want here. It is fairly
>> } untested and incomplete as I have not been able to find out how to
>> } make sure that search.h gets included.
>>
>> Zsh has its own hash table implementation which ought to be used rather
>> than introducing any new external dependencies, but otherwise this is
>> the kind of thing I was suggesting.
>>
>> Or perhaps someone else has an even more clever idea.  Anything better
>> than O(N^2) would help.
> I have reimplemented the change using ZSH's own hash table. Please see
> the attached patch.
>
I have put the changes onto github: <git://github.com/wilx/zsh.git>

-- 
VZ



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Large LS_COLORS makes auto_cd very slow

[Bart Schaefer]

On Apr 9, 10:23am, Václav Zeman wrote:
}
} > I have reimplemented the change using ZSH's own hash table. Please see
} > the attached patch.
} >
} I have put the changes onto github: <git://github.com/wilx/zsh.git>

Unfortunately on some further testing it doesn't quite work properly.

You can't declare

    struct hashnode new_node;

and then call

    addhashnode2(ht, *it, &new_node);

because a pointer to new_node will be placed in the hash table and then
THE CONTENTS OF new_node will be overwritten on the next call to
addhashnode2().  

You have to pass allocated space.  It can be allocated with zhalloc() so
that it needn't be explicitly free()d.

The good news is that fixing this makes completion after "/b" much faster!
The getcoldef() issue that we discussed before turns out to have been
happening because the hash seive was failing to remove all the duplicate
elements, not because of any inherent problem with getcoldef().

I'll commit a repaired version later.

Now the question is at what array length it makes sense to switch from
the simple algorithm to the hash seive.  I suspect Václav's choice of
10 elements may be a little too small given allocation overhead, but I
will leave it that way for the first commit because it's easier to
write the Test/D04parameter.ztst entry for a small number of elements.

-- 
Barton E. Schaefer

Completing all possible candidates

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

For quite a while, I've used the following when I want completion from the
output of a previous command:

_my-prev-result() {
    local hstring
    # Run last command again, save output
    hstring=$(eval $(fc -l -n -1))
    # Split items on new-line into an array, quote each item
    compadd - ${(@f)hstring}
}

zle -C my-prev-comp menu-complete _my-prev-result
bindkey '\ee' my-prev-comp

zle -C my-rev-prev-comp reverse-menu-complete _my-prev-result
bindkey '\eE' my-rev-prev-comp

This allows me to for example use "locate foo", and then complete the next
command with the found files (the drawback is of course that the previous
command could take a while to run). I know about the  "keeper" functions in
the zsh distribution, but this is simpler.

Anyway, I recently realized that I sometimes could have use for putting ALL
the possibilities on the command line with a keystroke, à la the
"all-matches" example under _all_matches in the zsh manual.

In fact, I suspect the _all_matches completer should be involved somehow,
but I have failed to find out how.

Simply stated, I want a key that says: "run the previous command line, and
put all the resulting output on the command line". How to do this?

Re: Completing all possible candidates

[Bart Schaefer]

On Jan 11, 12:30pm, Jesper Nygårds wrote:
}
} Simply stated, I want a key that says: "run the previous command line, and
} put all the resulting output on the command line". How to do this?

Take a look at Functions/Zle/keeper in the distributed set of examples.
It doesn't capture the same results that you are looking for, but it
has functions for copying those results to the command line.

Re: Completing all possible candidates

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

Great thanks, Bart! That worked very well. If anyone else is interested,
here's my result:

_my-prev-result() {
    local hstring
    if [[ $WIDGET = *-all-* ]]; then
        compstate[insert]=all
    fi
    # Run last command again, save output in hstring
    hstring=$(eval $(fc -l -n -1))
    # Split items on new-line into an array, quote each item
    compadd - ${(@f)hstring}
}

zle -C my-prev-comp menu-complete _my-prev-result
bindkey '\ee' my-prev-comp

zle -C my-all-prev-comp complete-word _my-prev-result
bindkey '^xE' my-all-prev-comp




On Fri, Jan 11, 2013 at 3:32 PM, Bart Schaefer <schaefer@brasslantern.com>wrote:

> On Jan 11, 12:30pm, Jesper Nygårds wrote:
> }
> } Simply stated, I want a key that says: "run the previous command line,
> and
> } put all the resulting output on the command line". How to do this?
>
> Take a look at Functions/Zle/keeper in the distributed set of examples.
> It doesn't capture the same results that you are looking for, but it
> has functions for copying those results to the command line.
>

Re: Completing all possible candidates

[Bart Schaefer]

On Jan 15,  8:28am, Jesper Nygårds wrote:
}
} _my-prev-result() {
}     local hstring
}     if [[ $WIDGET = *-all-* ]]; then
}         compstate[insert]=all
}     fi
}     # Run last command again, save output in hstring
}     hstring=$(eval $(fc -l -n -1))

You probably ought to put double quotes around the inner $(...), because
without them in that context it's subject to word splitting and then the
eval will split the resulting words again.

}     # Split items on new-line into an array, quote each item
}     compadd - ${(@f)hstring}
} }

I'm glad the example worked well for you.

Re: Completing all possible candidates

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

On Thu, Jan 17, 2013 at 4:14 AM, Bart Schaefer <schaefer@brasslantern.com>wrote:

>
> You probably ought to put double quotes around the inner $(...), because
> without them in that context it's subject to word splitting and then the
> eval will split the resulting words again.
>
> Thank you. As always, the generosity you show with your time and knowledge
is remarkable, Bart. I've been on this list for 11 years now, and it amazes
me to think back of all the fantastic help you've given to everyone over
those years.