Re: Asynchroneous image retrieval in HTML rendering

[Lars Magne Ingebrigtsen <larsi@gnus.org>]

Antoine Levitt <antoine.levitt@gmail.com> writes:

> That doesn't change much for my test cases, so I guess the DNS lookup is
> indeed the bottleneck.

Have you verified that the DNS lookup is really the problem?  Because I
have more theories.  :-) (Or, as scientists call it, "guesses".)

One thing about the async image retrieval is that it's massively
parallel.  If there's 500 images in the article, Gnus will call
`url-retrieve' 500 times in rapid succession, which will then fire off
500 HTTP connections (in parallel, more or less).

I suspect that Emacs doesn't really like having this number of async
socket sentinels going on at the same time, which means that the main
Emacs thread gets no CPU time, which gives us interactive pauses.

(And it's really rude towards the server.  It's basically a one-Emacs
DoS attack.)

So I'm going to rewrite this by adding a new library, called, er,
url-queue.el, which has the same interface as `url-retrieve', but which
manages the parallelism in a more sensible way, by not having more than
(for instance) four connections going at the same time.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/