From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/70312 Path: news.gmane.org!not-for-mail From: Lars Magne Ingebrigtsen Newsgroups: gmane.emacs.gnus.general Subject: Faster, lolcat. Faster! Date: Thu, 02 Sep 2010 04:22:44 +0200 Organization: Programmerer Ingebrigtsen Message-ID: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1283394275 32672 80.91.229.12 (2 Sep 2010 02:24:35 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 2 Sep 2010 02:24:35 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M18698@lists.math.uh.edu Thu Sep 02 04:24:29 2010 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OqzTY-0007Xw-SF for ding-account@gmane.org; Thu, 02 Sep 2010 04:24:29 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1OqzS9-0004gf-CN; Wed, 01 Sep 2010 21:23:01 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1OqzS7-0004gT-Nn for ding@lists.math.uh.edu; Wed, 01 Sep 2010 21:22:59 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtp (Exim 4.72) (envelope-from ) id 1OqzS3-0005sa-0i for ding@lists.math.uh.edu; Wed, 01 Sep 2010 21:22:59 -0500 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 3.36 #1 (Debian)) id 1OqzS2-0005fv-00 for ; Thu, 02 Sep 2010 04:22:54 +0200 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OqzRz-0006rb-Um for ding@gnus.org; Thu, 02 Sep 2010 04:22:51 +0200 Original-Received: from cm-84.215.34.171.getinternet.no ([84.215.34.171]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 02 Sep 2010 04:22:51 +0200 Original-Received: from larsi by cm-84.215.34.171.getinternet.no with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 02 Sep 2010 04:22:51 +0200 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 60 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cm-84.215.34.171.getinternet.no Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEU8HSIUEBgPChPLa04O CRENBw4RDRUKBAoJfg0VAAACZElEQVQ4jVWUQW/bMAyFlRlbrjbS+rwKis4OVOwHCNR2HVppudZq wHOCzdPf36NkZ5tgO4G/kE8iH6OGumY2FKL3PGxL1ec45hBTNMTzf4BZG0ohasvj/Bfg5zmkFEO0 PIxbTAVaEwEksqyZNzByRh6sc4lR5zuYa6KCdVOfktc6r4CZcxJwc85NRyLdQtTIfElpKaUDcCcB 8yrOWiQkAGvntb0DpErlZwPPtO5LdqXPAF0DE3k/I2SUCINdJbeBJnIHP1bgPEFl1cD5yvUfkHmY ccBhvFAo3/FOCTBEmevJhyyp7hERIjUV6nZMsdzu4qGpK/IUFpREXsqNGhsLorzPRwE4x4T7lGoq BqCvNeLduf2rcx9FQw6vsGsq6MXVTb9QlvRSgRWgw4LiFjdde/e8UAOzgg+CtKlc9/1++rYglYUp BoWA2j8stccDrmgR0oxzC1Ff8EQEasKzoiO1VOXaqVLO1LaLCFjnXJMdBCwoLmrFozLiqQo6B5Da KdAPqzd5VKQswWdsScqOz0uo8gDnCDNa+JfFVxnlRbLfKOFjtSKARPDwEATcGuAN4MvlUZz7UsHT vJv7ah+bs051ATz498H13QqsDhswFtXvup0ArY3GlFEFmp9cPx0qmLOHGRKug5sg3Q0AIMrnbJpG 3JFnFhN1k6RCTZpGShjnt5PrDqddBcak2AAi3gfVd5MDgPNoDUjarxPYSwTmtdTXZfmg7ZuqCyDr IBJR+jiS/lzfiwaG2dTxxyBGNENGfR2DzCQ6cnmb59HKROGArHVTp/SCpj2xTEd1+6qSKMJQ6Fuu fzR/AK+fEgnWoUXkAAAAAElFTkSuQmCC Mail-Copies-To: never X-Now-Playing: XXL's =?iso-8859-1?Q?=5F=BFSpicchiology=3F?= (1)_: "Last In The Society" User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/24.0.50 (gnu/linux) Cancel-Lock: sha1:PJQJLcNfQikKEJRFsd6G4Ap3JWs= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:70312 Archived-At: Now that Gnus is bug-free (because I haven't hacked much on Gnus the past years) and has HTML support (because I have this week), I think it's time to have a long good look at why `g' is so slow. I mean, it's just querying some mail servers and some nntp servers. I though take less than a second, total, if you're not contacting half the servers in the world. Just to see where we were, I did a `g' and straced Emacs. [larsi@quimbies ~/pgnus]$ grep stat /tmp/s2 | wc -l 5832 [larsi@quimbies ~/pgnus]$ grep open /tmp/s2 | wc -l 305 Right. It stat()-et 5832 files. 5832 files!!1! And opened and read and wrote 305 files. Geez. So I think there should be room for improvement. My plan is to do the following: 0) Remove almost all of my Gnus settings, so that I get the "default" user experience. 1) General optimisations. Instrument `file-exists-p' and friends and see what's accessing all these files, and try to clamp down on that, as well as doing general performance analysis and fixing up functions/areas that are particularly slow. 2) Feature reduction. If there are features that take a lot of time when you do a `g', but don't give compelling results, then they should default to "off". 3) Streaming. I think many of the things that talk to servers over the net can be more streaming than they are now. I had a peek at pop3, just to take an example, and I think I gleaned from it that it requests one message, and then waits for it to arrive, and then requests the next, etc. If you just request all the articles, and then wait for them all to arrive, it'll be oodles faster. I think there's probably some opportunities in that area, both for pop3, nntp and imap, as the obvious candidates (although I haven't actually looked at the code for any of these in depth). 4) Multi-threading. This is the most difficult bit, but could yield good results. If you're requesting active files from two different nntp servers, and getting mail, there's really no reason why those three activities shouldn't happen at the same time. Implementing asynchronous things in Emacs is a pain, because you have to use process filters and the like, but it seems to me (at 4:15, and I couldn't sleep) that it should be possible to gather up the actions that can be async, launch them all, wait for them all to finish, and then continue on with the data that we've now gotten. But this is a way, way bigger project than the first three points here, so I'll tackle it last... if ever... So there you go. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen