Problems with IMAP-over-ssl mail source.

Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

I am now quite happily using imap over ssl to get my mail.  However,
there are a few problems.

Firstly, if I interrupt such a mail source with C-g then it won't
work again--the next time I hit g I get an error:

[I have removed my passwords from this backtrace]

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  format(nil)
  apply(format nil)
  error(nil)
  mail-source-fetch-imap((imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) #[(file orig-file) "Â\bÃÄ\b	.?.\x0f.Å	Æ\"Ç%." [file orig-file nnmail-split-incoming nnml-save-mail nil nnmail-get-split-group (imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) nnml-active-number] 7])
  mail-source-fetch((imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) #[(file orig-file) "Â\bÃÄ\b	.?.\x0f.Å	Æ\"Ç%." [file orig-file nnmail-split-incoming nnml-save-mail nil nnmail-get-split-group (imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) nnml-active-number] 7])
  nnmail-get-new-mail(nnml nnml-save-nov "~/Mail/" nil)
  nnml-request-scan(nil "")
  gnus-request-scan(nil (nnml ""))
  gnus-read-active-file-1((nnml "") nil)
  gnus-read-active-file()
  gnus-group-get-new-news(nil)
  call-interactively(gnus-group-get-new-news)

Inspection shows that (imap-error-text buf) returns nil.

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
Oh, I get it!!  ``The BEACH goes on,'' huh, SONNY??

Re: Problems with IMAP-over-ssl mail source.

[Simon Josefsson]

Gaute B Strokkenes <gs234@cam.ac.uk> writes:

> I am now quite happily using imap over ssl to get my mail.  However,
> there are a few problems.
>
> Firstly, if I interrupt such a mail source with C-g then it won't
> work again--the next time I hit g I get an error:
>
> [I have removed my passwords from this backtrace]
>
> Debugger entered--Lisp error: (wrong-type-argument stringp nil)
>   format(nil)
>   apply(format nil)
>   error(nil)
>   mail-source-fetch-imap((imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) #[(file orig-file) "Â\bÃÄ\b	.?.\x0f.Å	Æ\"Ç%." [file orig-file nnmail-split-incoming nnml-save-mail nil nnmail-get-split-group (imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) nnml-active-number] 7])
>   mail-source-fetch((imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) #[(file orig-file) "Â\bÃÄ\b	.?.\x0f.Å	Æ\"Ç%." [file orig-file nnmail-split-incoming nnml-save-mail nil nnmail-get-split-group (imap :server "imap.srcf.ucam.org" :password "xxx" :stream ssl) nnml-active-number] 7])
>   nnmail-get-new-mail(nnml nnml-save-nov "~/Mail/" nil)
>   nnml-request-scan(nil "")
>   gnus-request-scan(nil (nnml ""))
>   gnus-read-active-file-1((nnml "") nil)
>   gnus-read-active-file()
>   gnus-group-get-new-news(nil)
>   call-interactively(gnus-group-get-new-news)
>
> Inspection shows that (imap-error-text buf) returns nil.

This should be fixed, although the problem generating the backtrace is
probably still around.  Could you set `imap-log' and send the content
of *imap-log* when this happens?

Re: Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

On 14 jul 2003, jas@extundo.com wrote:

> Gaute B Strokkenes <gs234@cam.ac.uk> writes:
>
>> I am now quite happily using imap over ssl to get my mail.
>> However, there are a few problems.
>>
>> Firstly, if I interrupt such a mail source with C-g then it won't
>> work again--the next time I hit g I get an error:
>>
>> [I have removed my passwords from this backtrace]

[snip]

>>
>> Inspection shows that (imap-error-text buf) returns nil.
>
> This should be fixed, although the problem generating the backtrace
> is probably still around.  Could you set `imap-log' and send the
> content of *imap-log* when this happens?

depth=0 /C=UK/ST=Cambridgeshire/L=Cambridge/O=University of Cambridge/OU=Student-Run Computing Facility/CN=imap.srcf.ucam.org/emailAddress=sysadmins@srcf.ucam.org
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=UK/ST=Cambridgeshire/L=Cambridge/O=University of Cambridge/OU=Student-Run Computing Facility/CN=imap.srcf.ucam.org/emailAddress=sysadmins@srcf.ucam.org
verify return:1
* OK [CAPABILITY IMAP4REV1 X-NETSCAPE LOGIN-REFERRALS STARTTLS AUTH=LOGIN] localhost IMAP4rev1 2001.315 at Mon, 14 Jul 2003 08:52:43 +0100 (BST)^M

(the CR at the end of the last line has been replaced with the two
characters ^M for your safety and convenience.)

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
I OWN six pink HIPPOS!!

Re: Problems with IMAP-over-ssl mail source.

[Simon Josefsson]

Gaute B Strokkenes <gs234@cam.ac.uk> writes:

>>> Inspection shows that (imap-error-text buf) returns nil.
>>
>> This should be fixed, although the problem generating the backtrace
>> is probably still around.  Could you set `imap-log' and send the
>> content of *imap-log* when this happens?
>
> depth=0 /C=UK/ST=Cambridgeshire/L=Cambridge/O=University of Cambridge/OU=Student-Run Computing Facility/CN=imap.srcf.ucam.org/emailAddress=sysadmins@srcf.ucam.org
> verify error:num=18:self signed certificate
> verify return:1
> depth=0 /C=UK/ST=Cambridgeshire/L=Cambridge/O=University of Cambridge/OU=Student-Run Computing Facility/CN=imap.srcf.ucam.org/emailAddress=sysadmins@srcf.ucam.org
> verify return:1
> * OK [CAPABILITY IMAP4REV1 X-NETSCAPE LOGIN-REFERRALS STARTTLS AUTH=LOGIN] localhost IMAP4rev1 2001.315 at Mon, 14 Jul 2003 08:52:43 +0100 (BST)^M
>
> (the CR at the end of the last line has been replaced with the two
> characters ^M for your safety and convenience.)

This looks fine.  Does it stop there?  It should continue with the
login.  Could you perhaps edebug `mail-source-fetch-imap' to see what
is going on?

Re: Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

On 14 jul 2003, jas@extundo.com wrote:

> Gaute B Strokkenes <gs234@cam.ac.uk> writes:
>
>>>> Inspection shows that (imap-error-text buf) returns nil.
>>>
>>> This should be fixed, although the problem generating the
>>> backtrace is probably still around.  Could you set `imap-log' and
>>> send the content of *imap-log* when this happens?
>>

[snip]

>> * OK [CAPABILITY IMAP4REV1 X-NETSCAPE LOGIN-REFERRALS STARTTLS AUTH=LOGIN] localhost IMAP4rev1 2001.315 at Mon, 14 Jul 2003 08:52:43 +0100 (BST)^M
>>
>> (the CR at the end of the last line has been replaced with the two
>> characters ^M for your safety and convenience.)
>
> This looks fine.  Does it stop there?

Yes, it stops there.

> It should continue with the login.  Could you perhaps edebug
> `mail-source-fetch-imap' to see what is going on?

Mmm.  I'd have to work out how edebug works first.  I'll do it if I
get round to it, I'm a bit busy right now.

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
Hold the MAYO & pass the COSMIC AWARENESS...

Re: Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

On 16 jul 2003, gs234@cam.ac.uk wrote:

> On 14 jul 2003, jas@extundo.com wrote:
>> This looks fine.  Does it stop there?
>
> Yes, it stops there.

I forgot to mention that _both_ my imap mail-sources stop working
after I've only interrupted one of them.

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
Here I am in the POSTERIOR OLFACTORY LOBULE but I don't see CARL SAGAN
 anywhere!!

Re: Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

On 16 jul 2003, gs234@cam.ac.uk wrote:

>> It should continue with the login.  Could you perhaps edebug
>> `mail-source-fetch-imap' to see what is going on?
>
> Mmm.  I'd have to work out how edebug works first.  I'll do it if I
> get round to it, I'm a bit busy right now.

It's taken me a while, but I've finally gotten round to this.
Consider the following snippet from mail-source-fetch-imap:

    (let* ((from (format "%s:%s:%s" server user port))
	   (found 0)
	   (buffer-name " *imap source*")
	   (buf (get-buffer-create (generate-new-buffer-name buffer-name)))
	   (mail-source-string (format "imap:%s:%s" server mailbox))
	   (imap-shell-program (or (list program) imap-shell-program))
	   remove)
      (if (and (imap-open server port stream authentication buffer-name)
	       (imap-authenticate
		user (or (cdr (assoc from mail-source-password-cache))
			 password) buf)
	       (imap-mailbox-select mailbox nil buf))

The problem appear to be that the " *imap source*" buffer is not
deleted when I interrupt a fetching session.  Then, the next time this
code is called, generate-new-buffer-name ensures that `buf' is a new
buffer called " *imap source*<1>".  However, when imap-open is called
it is passed buffer-name which is still set to " *imap source",
causing imap-open to operate on the wrong temporary buffer.

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
I have a TINY BOWL in my HEAD

Re: Problems with IMAP-over-ssl mail source.

[Gaute B Strokkenes]

On 25 aug 2003, biggaute@uwc.net wrote:

> Consider the following snippet from mail-source-fetch-imap:
>
> (let* ((from (format "%s:%s:%s" server user port))
> 	   (found 0)
> 	   (buffer-name " *imap source*")
> 	   (buf (get-buffer-create (generate-new-buffer-name buffer-name)))
> 	   (mail-source-string (format "imap:%s:%s" server mailbox))
> 	   (imap-shell-program (or (list program) imap-shell-program))
> 	   remove)
> (if (and (imap-open server port stream authentication buffer-name)
> 	       (imap-authenticate
> 		user (or (cdr (assoc from mail-source-password-cache))
> 			 password) buf)
> 	       (imap-mailbox-select mailbox nil buf))
>
> The problem appear to be that the " *imap source*" buffer is not
> deleted when I interrupt a fetching session.  Then, the next time
> this code is called, generate-new-buffer-name ensures that `buf' is
> a new buffer called " *imap source*<1>".  However, when imap-open is
> called it is passed buffer-name which is still set to " *imap
> source", causing imap-open to operate on the wrong temporary buffer.

And the obvious fix is:

Index: lisp/mail-source.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/mail-source.el,v
retrieving revision 6.38
diff -u -r6.38 mail-source.el
--- lisp/mail-source.el	14 Jul 2003 08:03:13 -0000	6.38
+++ lisp/mail-source.el	25 Aug 2003 20:51:52 -0000
@@ -968,12 +968,11 @@
   (mail-source-bind (imap source)
     (let* ((from (format "%s:%s:%s" server user port))
 	   (found 0)
-	   (buffer-name " *imap source*")
-	   (buf (get-buffer-create (generate-new-buffer-name buffer-name)))
+	   (buf (generate-new-buffer " *imap source*"))
 	   (mail-source-string (format "imap:%s:%s" server mailbox))
 	   (imap-shell-program (or (list program) imap-shell-program))
 	   remove)
-      (if (and (imap-open server port stream authentication buffer-name)
+      (if (and (imap-open server port stream authentication buf)
 	       (imap-authenticate
 		user (or (cdr (assoc from mail-source-password-cache))
 			 password) buf)

But I'm not sure that this is the best way to fix things.  Is it
correct to leave a " *imap source*" buffer lying about if
mail-source-fetch-imap is interupted?  Would it be better to reuse it
than to create a new one?

Lastly, this bug was introduced by Lars in mid-March:

 revision 6.33
 date: 2003/03/17 14:20:35;  author: larsi;  state: Exp;  lines: +8 -7
 * mail-source.el (mail-source-fetch-imap): Pass buffer-name to
 imap-open.

I've been unable to find discussion of why this change was made on the
ding list, though presumably there is some reason for it.  Lars?

-- 
Gaute Strokkenes                        http://www.srcf.ucam.org/~gs234/
The PINK SOCKS were ORIGINALLY from 1952!!
 But they went to MARS around 1953!!