From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-workers-return-34471-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 23101 invoked by alias); 8 Feb 2015 16:20:13 -0000
Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Workers List <zsh-workers.zsh.org>
List-Post: <mailto:zsh-workers@zsh.org>
List-Help: <mailto:zsh-workers-help@zsh.org>
X-Seq: 34471
Received: (qmail 5097 invoked from network); 8 Feb 2015 16:19:58 -0000
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU autolearn=ham version=3.3.2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=thequod.de; h=
	content-transfer-encoding:content-type:content-type:subject
	:subject:mime-version:user-agent:from:from:date:date:message-id
	:received:received; s=postfix2; t=1423412392; bh=KwfsmFn5Rl8EmbF
	pVpLKbNrh3Uz59iF37l8GRn8iQcc=; b=QJ/iHUeGxkx1M8xwmHkm0oJ2whtCUUp
	6e1ZkiPJPAUoI1xsS7NPQ/0U8noTB+1NLXFsOpOG5BWcAyPwDq0duIt/dVFSfYks
	dB8wMPnphp1vs1JtHbuynLTxNPaXsSPrfzp+XEwFTJz3anIL3B+mr6wh/JVhXjTj
	xJdno01vbDzs=
Message-ID: <54D78CA8.7010802@thequod.de>
Date: Sun, 08 Feb 2015 17:19:52 +0100
From: Daniel Hahler <genml+zsh-workers@thequod.de>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Zsh Hackers' List <zsh-workers@zsh.org>
Subject: Performance of _store_cache and _retrieve_cache
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I've noticed that the completion systems cache mechanism
(_retrieve_cache and _store_cache) is slow with large lists (~50000).

_store_cache saves the array like this:

    _zsh_all_pkgs=( '02exercicio' '0x10c-asm'  ... )

and _retrieve_cache then sources it from a file.

The problem is that `source ./pip_allpkgs.slow` takes about 8 seconds,
and is slower than generating the list anew!


When converting the list to be line-separated, the following is much
faster (less than a second):

   _zsh_all_pkgs=(${(f)"$(<pip_allpkgs)"})

This also applies to using the "formatted"/"typed" source file as-is:
Even when using the slow list as is:

   _zsh_all_pkgs=(${$(<pip_allpkgs.slow)})


The initial list is generated using:

      _zsh_all_pkgs=( $(curl -s https://pypi.python.org/simple/ \
        | sed -n '/<a href/ s/.*>\([^<]\{1,\}\).*/\1/p' \
        | tr '\n' ' ') )


Regards,
Daniel.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iD8DBQFU14yofAK/hT/mPgARAkEFAKD2P5vOvKLwOQffWmq8OTTCTxFHTQCfQzzL
bqUGpSQvauen9qgus9FB5rQ=
=72Ab
-----END PGP SIGNATURE-----