From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/53858 Path: main.gmane.org!not-for-mail From: Gaute B Strokkenes Newsgroups: gmane.emacs.gnus.general Subject: PATCH: Make IMAP mail sources more robust, was :Re: Working around IMAP server b0rk3n-ness Date: Fri, 29 Aug 2003 03:47:17 +0100 Organization: The Church of Emacs Sender: ding-owner@lists.math.uh.edu Message-ID: <87bru9xjre.fsf_-_@srcf.ucam.org> References: <87adc3cqth.fsf@mit.edu> <87lltedm6g.fsf@srcf.ucam.org> NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1062126011 8839 80.91.224.253 (29 Aug 2003 03:00:11 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Fri, 29 Aug 2003 03:00:11 +0000 (UTC) Original-X-From: ding-owner+M2398@lists.math.uh.edu Fri Aug 29 05:00:10 2003 Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19sZUf-0003mn-00 for ; Fri, 29 Aug 2003 05:00:09 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19sZTl-00010i-00; Thu, 28 Aug 2003 21:59:13 -0500 Original-Received: from sclp3.sclp.com ([64.157.176.121]) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19sZTg-00010d-00 for ding@lists.math.uh.edu; Thu, 28 Aug 2003 21:59:08 -0500 Original-Received: (qmail 53965 invoked by alias); 29 Aug 2003 02:59:08 -0000 Original-Received: (qmail 53960 invoked from network); 29 Aug 2003 02:59:08 -0000 Original-Received: from mta05-svc.ntlworld.com (62.253.162.45) by sclp3.sclp.com with SMTP; 29 Aug 2003 02:59:08 -0000 Original-Received: from belldandy ([81.100.93.186]) by mta05-svc.ntlworld.com (InterMail vM.4.01.03.37 201-229-121-137-20020806) with ESMTP id <20030829024842.VJNL1523.mta05-svc.ntlworld.com@belldandy> for ; Fri, 29 Aug 2003 03:48:42 +0100 Original-Received: from gs234 by belldandy with local (Exim 3.36 #1 (Debian)) id 19sZID-0004tb-00 for ; Fri, 29 Aug 2003 03:47:17 +0100 Mail-Copies-To: never Original-To: ding@gnus.org Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAG1BMVEWWWBTly7aGMwb+/vz9 +ffIilb///+yaCxNCwHGVC3gAAACU0lEQVR4nF2TQWvjMBBGVUGwj3WxaY/aAWNfi0zuwZPuHp0Q ob079OwYjHrsChb0s3dGctJ0ddTzm9F8lsR7WiNiWWL/flviul9Uh0NR4vY/gJUxJjOEtt/AubBy UsotRfH2DbxWE0hom3w5XJUEKis9hACt2hR4B86VDOtqTNHfgeEGvLiUN7Ddze26Dx6y1CUa+ygo NwFQm0N/BeMzCZAtNIoQrd9EhcH5mWpnztB8ZpJBlH23jaAbuE4EhZ2Bam0TeJ0DRMCKVP6CegWk q4z2TVFUx0946FOPcSe9ELE3rSHAj15rBuddS0C4BGyAutfpVAQkKdZOQiwMMIHxlUDrgeej5Ns7 I/ccSWNBLda1XwYBEG1Qswy1yXJf879nA4cjkNI0H+FzoUyuRodPA5eqZwFONG34i30ccOxeKFUy flNe1szqsgLdvXwKCj6zjTXODHV9M46gZPCLlYKCGTYO8S0aOAvJzRfD0w+1XcGIsxLUw/sYl5sM 9n0yjkoSUDOByrijwRRiAq13NhqLJYB3ACazrgN2bIwanyJQKXhHAJPRPQruEaCJP2Shy4DJ2As+ FQjJZH7AZIy6w0pQiiRYAh8nTEZHYDfRqSDeB5OXq4EEcAC4NocvoBl4+n1cyl54nyLRNA3iIwRP wNGFOK1g1Gz8jNfaCjdtqJLm2LXuqMc+50cDIJQpr6X0OHa4X58ULAw03QbRd7GUiG/K04mj0bNB xQhwJkEsJiZ1Bbgb6CnF79cBE+jwF91qniOCP3fghUFKxOShjIATw/wOfIQTg39LpFHtxhlKywAA AABJRU5ErkJggg== In-Reply-To: <87lltedm6g.fsf@srcf.ucam.org> (Gaute B. Strokkenes's message of "Thu, 28 Aug 2003 13:03:03 +0100") User-Agent: Gnus/5.1003 (Gnus v5.10.3) Emacs/21.3.50 (gnu/linux) Precedence: bulk Xref: main.gmane.org gmane.emacs.gnus.general:53858 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:53858 On 28 aug 2003, biggaute@uwc.net wrote: > You're not alone, if it makes you feel any better. I have a similar > problem every morning when I try to get ~500 new messages. Gnus > constructs a line of ~2000 bytes; the process then hangs > indefinitely. I haven't quite tracked down what's going wrong, but > the prime suspect is uw-imapd not being able to cope with long lines > (we all know how robust c-client and friends are in that respect) > with some sort of openssl / emacs sub-process brokenness as a more > distant second. Usually I just run fetchmail once every morning. I have now determined that the problem definitely lies within openssl. I can reproduce the problem by manually running openssl and logging in; telnet on port 143 does not exhibit the problem. openssl seems to misbehave in an interesting way if you send it too much data at once; this patch works around the problem by sending less data. However, the openssl bug is not the only reason to apply the patch. The UW imapd has an 8k long command buffer. While this was not the problem in this case, it would only take a couple of thousand messages to reach that limit. (For me, that's a couple of days' worth of mail.) > What's going on here is that remove is a list of message numbers > that should be expunged. The imap-range-to-message-set / > gnus-compress-sequence combo is supposed to create a compact > representation using the imap range syntax (i.e. "1:5" rather than > "1,2,3,4,5"). However, gnus constructs remove by repeatedly pushing > message numbers on to it--and since gnus fetches messages in > sequence, the list of message numbers will be in reverse order, > which defeats gnus-compress-sequence ("5,4,3,2,1"). The other part of the patch essentially partially undoes the changes between revisions 6.32 and 6.33 of mail-source.el, made by Lars on 17 March. As reported previously, the effect of the change is that sometimes the wrong buffer is passed from mail-source-fetch to imap-open. This would occur when there were a " *imap source*" buffer already present, such as that left over by interrupting mail-source-fetch-imap with C-g. (Because of the previous bug I have done this quite frequently.) 2003-08-29 Gaute Strokkenes * mail-source.el (mail-source-fetch-imap): Pass correct buffer to imap-open. Reverse remove before calling gnus-compress-sequence. Index: mail-source.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/mail-source.el,v retrieving revision 6.38 diff -u -r6.38 mail-source.el --- mail-source.el 14 Jul 2003 08:03:13 -0000 6.38 +++ mail-source.el 29 Aug 2003 02:49:41 -0000 @@ -966,14 +966,13 @@ (defun mail-source-fetch-imap (source callback) "Fetcher for imap sources." (mail-source-bind (imap source) - (let* ((from (format "%s:%s:%s" server user port)) - (found 0) - (buffer-name " *imap source*") - (buf (get-buffer-create (generate-new-buffer-name buffer-name))) - (mail-source-string (format "imap:%s:%s" server mailbox)) - (imap-shell-program (or (list program) imap-shell-program)) - remove) - (if (and (imap-open server port stream authentication buffer-name) + (let ((from (format "%s:%s:%s" server user port)) + (found 0) + (buf (generate-new-buffer " *imap source*")) + (mail-source-string (format "imap:%s:%s" server mailbox)) + (imap-shell-program (or (list program) imap-shell-program)) + remove) + (if (and (imap-open server port stream authentication buf) (imap-authenticate user (or (cdr (assoc from mail-source-password-cache)) password) buf) @@ -1006,6 +1005,7 @@ (nnheader-ms-strip-cr)) (incf found (mail-source-callback callback server)) (when (and remove fetchflag) + (setq remove (nreverse remove)) (imap-message-flags-add (imap-range-to-message-set (gnus-compress-sequence remove)) fetchflag nil buf)) -- Gaute Strokkenes http://www.srcf.ucam.org/~gs234/ Actually, what I'd like is a little toy spaceship!!