From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/73198 Path: news.gmane.org!not-for-mail From: James Cloos Newsgroups: gmane.emacs.gnus.general Subject: fast list Date: Sun, 17 Oct 2010 01:15:41 -0400 Message-ID: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1287293041 23656 80.91.229.12 (17 Oct 2010 05:24:01 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sun, 17 Oct 2010 05:24:01 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M21570@lists.math.uh.edu Sun Oct 17 07:24:00 2010 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1P7Liv-0006wN-WB for ding-account@gmane.org; Sun, 17 Oct 2010 07:23:58 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1P7Lhx-00085v-Ma; Sun, 17 Oct 2010 00:22:57 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1P7Lhn-00085b-Ki for ding@lists.math.uh.edu; Sun, 17 Oct 2010 00:22:47 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.72) (envelope-from ) id 1P7Lhl-0006iV-L7 for ding@lists.math.uh.edu; Sun, 17 Oct 2010 00:22:46 -0500 Original-Received: from eagle.jhcloos.com ([207.210.242.212]) by quimby.gnus.org with esmtp (Exim 3.36 #1 (Debian)) id 1P7Lhk-00011V-00 for ; Sun, 17 Oct 2010 07:22:44 +0200 Original-Received: by eagle.jhcloos.com (Postfix, from userid 10) id 33E694015C; Sun, 17 Oct 2010 05:22:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jhcloos.com; s=eagle; t=1287292957; bh=TnWfxd2R2367ywSue+/I/yRtGpZaVeoOmUi5RCOg5nU=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; b=vSq3Q21UKQcqAIWaYr8ME8psy8VrFP3g65dhF0XYFjuus7atM1gAQuA9gumtw0Qi1 VObgp3VnyUDCwbXlkHxHme/xPtplPNI0SasdqIWGoFHGMPmkHaKbOWGc9TF8g3YEQn +N0HaSWWO2ArPlrnRNktUPw+06gJYsP7H3CDPwKs= Original-Received: from carbon.jhcloos.org (localhost [127.0.0.1]) by carbon.jhcloos.org (Postfix) with ESMTP id 8C5231E6E46 for ; Sun, 17 Oct 2010 05:15:41 +0000 (UTC) User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/24.0.50 (gnu/linux) Face: iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAAI1J REFUOE+lU9ESgCAIg64P1y+ngUdxhl5H8wFbbM0OmUiEhKkCYaZThXCo6KE5sCbA1DDX3genvO4d eBQgEMaM5qy6uWk4SfBYfdu9jvBN9nSVDOKRtwb+I3epboOsOX5pZbJNsBJFvmQQ05YMfieIBnYX FK2N6dOawd97r/e8RjkTLzmMsiVgrAoEugtviCM3v2WzjgAAAABJRU5ErkJggg== Copyright: Copyright 2009 James Cloos OpenPGP: ED7DAEA6; url=http://jhcloos.com/public_key/0xED7DAEA6.asc OpenPGP-Fingerprint: E9E9 F828 61A4 6EA9 0F2B 63E7 997A 9F17 ED7D AEA6 Original-Lines: 45 X-Hashcash: 1:30:101017:ding@gnus.org::U87G6+FGoPtAb7oS:000ZBR3L X-Spam-Score: -2.0 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:73198 Archived-At: I fixed my modification of dbmail-2.2 to need only a single sql SELECT to do an IMAP LIST/LSUB. Previously it grabbed the UIDVALIDITY values for the to-be-listed-groups in one SELECT, then iterated through that list SELECTing the info required for LIST. Now that latter data is returned as part of the initial SELECT. That improved LIST from ~1k seconds for ~20k groups to about 1/2 second. Nice. But gnus still needs to do an EXAMINE Or STATUS or SELECT on each returned group, and that is slow. Ir turns out that the IMAP working group already thought of that, though. RFC 5819 defines an extension to LIST which returns STATUS replies interleaved with the normal LIST replies. Using that should speed gnus startup considerably. Gnus needs to note the LIST-STATUS capability; when it exists gnus should send commands akin to: C: A01 LIST "" % RETURN (STATUS (MESSAGES UNSEEN)) specifying what status info it wants for each group. RETURN (STATUS (MESSAGES RECENT UIDNEXT UIDVALIDITY UNSEEN)) would be OK here. The reply includes w/in the untagged part of the reply status info after each line of the list info, specifying the status info for that mailbox. With that my startup time should get down to something which doesn't involve stepping away for a pot of tea. Or three. :-/ I expect to have the dbmail side of LIST-STATUS done tomorrow (it doesn't look too difficult; at least not after I just finished a dbmail hacking session) and should be able to test out any gnus support. -JimC -- James Cloos OpenPGP: 1024D/ED7DAEA6