From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/78663 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: Asynchroneous image retrieval in HTML rendering Date: Mon, 02 May 2011 12:12:59 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <87fwoxvxes.fsf@lifelogs.com> References: <87bozwfw84.fsf@gmail.com> <87r58ll8jm.fsf@gmail.com> <87liys4x04.fsf@gmail.com> <87tydewilm.fsf@gmail.com> <87liyp35fy.fsf@gmail.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1304356541 13366 80.91.229.12 (2 May 2011 17:15:41 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 2 May 2011 17:15:41 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M26965@lists.math.uh.edu Mon May 02 19:15:37 2011 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1QGwie-0001mW-99 for ding-account@gmane.org; Mon, 02 May 2011 19:15:36 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1QGwiF-0001BR-Rn; Mon, 02 May 2011 12:15:11 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1QGwiE-0001BI-SV for ding@lists.math.uh.edu; Mon, 02 May 2011 12:15:10 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1QGwiA-0003Fw-Tf for ding@lists.math.uh.edu; Mon, 02 May 2011 12:15:10 -0500 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1QGwi9-0004eN-F7 for ding@gnus.org; Mon, 02 May 2011 19:15:05 +0200 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QGwi9-0001Tv-7a for ding@gnus.org; Mon, 02 May 2011 19:15:05 +0200 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 May 2011 19:15:05 +0200 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 May 2011 19:15:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 32 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux) Cancel-Lock: sha1:PuNVnJfUgnyZuzb9Q2ITxqFXoVE= X-Spam-Score: -0.7 (/) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:78663 Archived-At: On Mon, 02 May 2011 16:30:09 +0200 Lars Magne Ingebrigtsen wrote: LMI> Antoine Levitt writes: >> That doesn't change much for my test cases, so I guess the DNS lookup is >> indeed the bottleneck. LMI> Have you verified that the DNS lookup is really the problem? Because I LMI> have more theories. :-) (Or, as scientists call it, "guesses".) LMI> One thing about the async image retrieval is that it's massively LMI> parallel. If there's 500 images in the article, Gnus will call LMI> `url-retrieve' 500 times in rapid succession, which will then fire off LMI> 500 HTTP connections (in parallel, more or less). LMI> I suspect that Emacs doesn't really like having this number of async LMI> socket sentinels going on at the same time, which means that the main LMI> Emacs thread gets no CPU time, which gives us interactive pauses. LMI> (And it's really rude towards the server. It's basically a one-Emacs LMI> DoS attack.) LMI> So I'm going to rewrite this by adding a new library, called, er, LMI> url-queue.el, which has the same interface as `url-retrieve', but which LMI> manages the parallelism in a more sensible way, by not having more than LMI> (for instance) four connections going at the same time. It may be nice at this point to include libcurl in Emacs. I would start by testing an article with 500 local images vs. one with 500 remote images. That will eliminate the network variable. Ted