From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/75270 Path: news.gmane.org!not-for-mail From: Steinar Bang Newsgroups: gmane.emacs.gnus.general Subject: Re: url-retrieve parallelism Date: Sun, 19 Dec 2010 10:02:18 +0100 Organization: Probably a good idea Message-ID: <87ipyqw2xx.fsf@dod.no> References: <87r5dew4bo.fsf@dod.no> <87mxo2w40w.fsf@dod.no> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1292749374 8350 80.91.229.12 (19 Dec 2010 09:02:54 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sun, 19 Dec 2010 09:02:54 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M23624@lists.math.uh.edu Sun Dec 19 10:02:50 2010 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1PUFAI-00087d-7T for ding-account@gmane.org; Sun, 19 Dec 2010 10:02:50 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1PUFA6-0004EV-Gk; Sun, 19 Dec 2010 03:02:38 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1PUFA4-0004EJ-SW for ding@lists.math.uh.edu; Sun, 19 Dec 2010 03:02:36 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.72) (envelope-from ) id 1PUFA2-0008Lk-Rk for ding@lists.math.uh.edu; Sun, 19 Dec 2010 03:02:36 -0600 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1PUFA2-0007iw-4W for ding@gnus.org; Sun, 19 Dec 2010 10:02:34 +0100 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PUFA1-0007we-OE for ding@gnus.org; Sun, 19 Dec 2010 10:02:33 +0100 Original-Received: from cm-84.208.226.192.getinternet.no ([84.208.226.192]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Dec 2010 10:02:33 +0100 Original-Received: from sb by cm-84.208.226.192.getinternet.no with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Dec 2010 10:02:33 +0100 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 8 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cm-84.208.226.192.getinternet.no Mail-Copies-To: never User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (gnu/linux) Cancel-Lock: sha1:Wtm8bNmwMiLBZfzDWrGqgJd7qiM= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:75270 Archived-At: One thing you could do, if curl is installed on the system: Start curl in a separate process, giving it all the URLs you want to download, and then route the curl output into an emacs buffer and parse it for progress information and error messages http://curl.haxx.se/docs/manpage.html curl will do things the network friendly way, and pipeline if it can, AFAIK.