pop mailboxes

pop mailboxes

[Daniel Simms]

hi all,

Well, I (finally!) started using gnus for email over the weekend and 
I have some problems, of course.  Whee!

Pop mail spool files, which begin with "^po:", get caught by ange-ftp
after the first *get-new-news.  It's only after the first fetching
since nnmail-get-new-mail loads ange-ftp's password reading function
halfway through the first pass.  I couldn't see how to cleanly fix
this, so I tried forcing ange-ftp to unload at the end of fetching
mail, but that didn't really work since ange-ftp has all sorts of
other non-gnus functions hooked from hell to breakfast.  I ended up
tweaking nnmail to also look for "^po;" and used that instead of
"po:".

While respooling old mail, I noticed that splitting into a
non-existant group isn't so useful.  Is there some way to get gnus to
auto-create non-existant groups?

I check mail from several different pop mailboxes, and wrapped
pop3-movemail so that I could explicitly specify maildrop, host, port,
authentication scheme, and password in the spool file variable, more
like vm does.  And then I was bored so I added some code to save the
passwords on sucessful authorization.  (I also added some flags to
allow leaving mail on the server, which isn't all that useful except
for some testing, since I haven't added the UIDL matching code yet.)
Does anybody want this?  Richard, Lars?


Anyway, thanks again to everybody who makes gnus work.

-- 
Daniel Simms           "You don't have to pretend to be interested in 
dsimms@alink.net        me you know.  I know I'm only a menial robot."
(408) 720-6161                           -- Marvin, _HhGttG_

Re: pop mailboxes

[Lars Magne Ingebrigtsen]

Daniel Simms <dsimms@alink.net> writes:

> Pop mail spool files, which begin with "^po:", get caught by ange-ftp
> after the first *get-new-news.  It's only after the first fetching
> since nnmail-get-new-mail loads ange-ftp's password reading function
> halfway through the first pass.

I don't see why ange-ftp should meddle with file names that start with
"po:".  What effects are you seeing?

> While respooling old mail, I noticed that splitting into a
> non-existant group isn't so useful.  Is there some way to get gnus to
> auto-create non-existant groups?

If you have the mail backend as a secondary server, `F' will subscribe
you to all new groups.  (New groups are created on the fly.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: pop mailboxes

[Daniel Simms]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2574 bytes --]

>>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  LMI> I don't see why ange-ftp should meddle with file names that
  LMI> start with "po:".  

Well it's *almost* how you specify remote files, ie /host:file (or
/user@host:file), no?

  LMI> What effects are you seeing?

ange-ftp tries to open an ftp session to "po", which is non-existant,
(and even if it was, wouldn't have my mail on it :)  Here's a trace
from the call in question.  (And before you freak out, the function I
use for nnmail-movemail-program does grok the ":" delimited pop
maildrop.)


;Signaling: (ftp-error "FTP Error: OPEN request failed: ftp: po: unknown host")
;  signal(ftp-error ("FTP Error: OPEN request failed: ftp: po: unknown host"))
;  ange-ftp-error("po" "dsimms" "OPEN request failed: ftp: po: unknown host")
;  ange-ftp-normal-login("po" "dsimms" "" "" #<process *ftp dsimms@po*>)
;  ange-ftp-get-process("po" "dsimms")
;  ange-ftp-host-type("po" "dsimms")
;  ange-ftp-expand-dir("po" "dsimms" "~")
;  ange-ftp-canonize-filename("/po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  ange-ftp-expand-file-name("/po:pop.alink.net:110:apop:n:dsimms%alink.net:*" nil)
;  apply(ange-ftp-expand-file-name ("/po:pop.alink.net:110:apop:n:dsimms%alink.net:*" nil))
;  ange-ftp-hook-function(expand-file-name "/po:pop.alink.net:110:apop:n:dsimms%alink.net:*" nil)
;  expand-file-name("/po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  ange-ftp-file-directory-p("/po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  apply(ange-ftp-file-directory-p "/po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  ange-ftp-hook-function(file-directory-p "/po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  file-directory-p("po:pop.alink.net:110:apop:n:dsimms%alink.net:*")
;  #[(file) "À	!ƒ\n
;  mapcar(#[(file) "À	!ƒ\n
;  nnmail-get-spool-files(nil)
;  nnmail-get-new-mail(nnmh nil "~/Mail/" nil)
;  nnmh-request-scan(nil "")
;  gnus-request-scan(nil (nnmh ""))
;  gnus-read-active-file()
;  gnus-group-get-new-news(nil)
;* call-interactively(gnus-group-get-new-news)

Like I said, I have no real idea why file-directory-p is hooked
differently by ange-ftp post-startup.   


  LMI> If you have the mail backend as a secondary server, `F' will
  LMI> subscribe you to all new groups.

coolness.  Well, hopefully I won't ever have to switch to gnus again,
y'know? :)

-- 
Daniel Simms                  "The best of all ways to lengthen our days 
dsimms@dsimms.com              is to steal a few hours from the night."  
(415) 269-7891                                        -Thomas Moore

Re: pop mailboxes

[Lars Magne Ingebrigtsen]

Daniel Simms <dsimms@dsimms.com> writes:

> Well it's *almost* how you specify remote files, ie /host:file (or
> /user@host:file), no?

[...]

> ;  file-directory-p("po:pop.alink.net:110:apop:n:dsimms%alink.net:*")

This isn't a valid value from `nnmail-spool-file'.  It should be just
"po:username". 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: pop mailboxes

[Daniel Simms]

>>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  >> ;
  >> file-directory-p("po:pop.alink.net:110:apop:n:dsimms%alink.net:*")

  LMI> This isn't a valid value from `nnmail-spool-file'.  It should
  LMI> be just "po:username".


Yes, yes*, it does the same thing with just po:luser too.  

* I also have a less creative nnmail-movemail-program:


(setq nnmail-movemail-program 'pop3-really-movemail)

(require 'pop3)
(defun pop3-split (regexp string)
  (let (list last-match)
    (store-match-data nil)
    (while (string-match regexp string (setq last-match (match-end 0)))
      (setq list (cons (substring string (or last-match 0) (match-beginning 0))list)))
    (setq list (cons (substring string (or last-match 0) nil) list))
    (nreverse list)))

(defvar pop3-pop-passwords nil
  "stupid list to keep track of passwords")

(defun pop3-really-movemail (pop-maildrop pop-crashbox) 
  "example pop-maildrop spec \"po;pop.alink.net;110;apop;n;dsimms;*\""
  (let* ((md-list (cdr (pop3-split ";" pop-maildrop)))
	 ;(md-list (pop3-split ";" (nth 1 (pop3-split ":" pop-maildrop))))
	 (pop3-save-password nil)
	 (pop3-mailhost (nth 0 md-list))
	 (pop3-port (nth 1 md-list))
	 (pop3-authentication-scheme (intern (nth 2 md-list)))
	 (pop3-leave-mail-on-server (string= "y" (nth 3 md-list)))
	 (pop3-maildrop (nth 4 md-list))
	 (pop3-password (nth 5 md-list))
	 (pop3-password-required t)
	 (pop3-timestamp nil)
	 (pop3-crashbox pop-crashbox))
    (cond
     ((or (string= "*" pop3-password) (string= "" pop3-password))
      (setq pop3-password (cadr (assoc pop-maildrop pop3-pop-passwords)))
      (setq pop3-save-password (not pop3-password))))
    (setq pop3-port (or (string-to-number pop3-port) pop3-port))
    (let ((retval (pop3-movemail pop3-crashbox)))
      (if (and retval pop3-save-password)
	  (setq pop3-pop-passwords 
		(cons (list pop-maildrop pop3-password)  pop3-pop-passwords)))
      retval)))


and you tweaked your pop3.el, a bit:

*** pop3.el.~1~	Sun Jan  5 00:49:42 1997
--- pop3.el	Tue Feb  4 15:53:25 1997
***************
*** 46,51 ****
--- 46,55 ----
    "*POP3 mailhost.")
  (defvar pop3-port 110
    "*POP3 port.")
+ (defvar pop3-leave-mail-on-server nil
+   "*POP3 DELE after RETR.")
+ (defvar pop3-save-password nil
+   "*POP3 save password after successful authorization.")
  
  (defvar pop3-password-required t
    "*Non-nil if a password is required when connecting to POP server.")
***************
*** 86,92 ****
        (save-excursion
  	(set-buffer crashbuf)
  	(append-to-file (point-min) (point-max) crashbox))
!       (pop3-dele process n)
        (setq n (+ 1 n)))
      (pop3-quit process)
      (kill-buffer crashbuf)
--- 90,96 ----
        (save-excursion
  	(set-buffer crashbuf)
  	(append-to-file (point-min) (point-max) crashbox))
!       (if (not pop3-leave-mail-on-server) (pop3-dele process n))
        (setq n (+ 1 n)))
      (pop3-quit process)
      (kill-buffer crashbuf)
***************
*** 250,257 ****
  	(progn
  	  (pop3-send-command process (format "PASS %s" pass))
  	  (let ((response (pop3-read-response process t)))
! 	    (if (not (and response (string-match "+OK" response)))
! 		(pop3-quit process)))))
      ))
  
  (defun pop3-apop (process user)
--- 254,262 ----
  	(progn
  	  (pop3-send-command process (format "PASS %s" pass))
  	  (let ((response (pop3-read-response process t)))
! 	    (if (and response (string-match "+OK" response))
! 		(progn (if pop3-save-password (setq pop3-password pass)) t)
! 	        (pop3-quit process)))))
      ))
  
  (defun pop3-apop (process user)
***************
*** 265,272 ****
  	(let ((hash (md5 (concat pop3-timestamp pass))))
  	  (pop3-send-command process (format "APOP %s %s" user hash))
  	  (let ((response (pop3-read-response process t)))
! 	    (if (not (and response (string-match "+OK" response)))
! 		(pop3-quit process)))))
      ))
  
  ;; TRANSACTION STATE
--- 270,278 ----
  	(let ((hash (md5 (concat pop3-timestamp pass))))
  	  (pop3-send-command process (format "APOP %s %s" user hash))
  	  (let ((response (pop3-read-response process t)))
! 	    (if (and response (string-match "+OK" response))
! 		(progn (if pop3-save-password (setq pop3-password pass)) t)
! 	        (pop3-quit process)))))
      ))
  
  ;; TRANSACTION STATE



-- 
Daniel Simms                              "If ignorance is bliss, then 
dsimms@alink.net                           knock the smile off my face"
(408) 720-6161                                        -"Freedom", RATM

Re: pop mailboxes

[Lars Magne Ingebrigtsen]

Daniel Simms <dsimms@alink.net> writes:

> Yes, yes*, it does the same thing with just po:luser too.  

(file-directory-p "po:hei")
=> nil

(require 'ange-ftp)
=> ange-ftp

(file-directory-p "po:hei")
=> nil

So I'm unable to reproduce this bug.  What Emacs version are you using?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: pop mailboxes

[Paul Franklin]

i can reproduce it iff the current directory is root:

cd /
emacs -q
M-:
y	(to deal with novice)
(file-directory-p "po:hei")

ange-ftp autoloads and I get prompted for a password

>>>>> Lars Magne Ingebrigtsen writes:

 > (file-directory-p "po:hei")
 > => nil

 > (require 'ange-ftp)
 > => ange-ftp

 > (file-directory-p "po:hei")
 > => nil

 > So I'm unable to reproduce this bug.  What Emacs version are you using?

Re: pop mailboxes

[Daniel Simms]

>>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  LMI> So I'm unable to reproduce this bug.  What Emacs version are
  LMI> you using?

It only works if the directory in question is /host:file, say if your
current directory is "/" (as Paul suggested), or if someone let
default-directory to "/".  

In the debugger before the culprit file-directory-p is entered, I
checked on default-directory and it is "/".  I can't figure out where
default-directory is set to "/" (otherwise I would just send you a
patch and not waste your time ...ok, I *would* send you a patch and
then waste your time on something else)).

 D> Entering:
 D> * file-directory-p("po:pop.alink.net:110:apop:n:dsimms.alink.net:*")
 D> * #[(file) "\300        !\203\n^@\302   !\207   C\207" [file-directory-p fi
 D> * mapcar(#[(file) "\300 !\203\n^@\302   !\207   C\207" [file-directory-p fi
 D> * byte-code("^H\205\334^@\301\n!\205)^@^H\303=\204^T^@\f\205)^@\305\n\306^N
 D> * nnmail-get-spool-files(nil)
 D>   nnmail-get-new-mail(nnmh nil "~/Mail/" nil)
 D>   nnmh-request-scan(nil "")
 D>   gnus-request-scan(nil (nnmh ""))
 D>   gnus-read-active-file()
 D>   gnus-group-get-new-news(nil)
 D> * call-interactively(gnus-group-get-new-news)

 D> Eval: default-directory
 D> => "/"


Whee!  

-- 
Daniel Simms                    There is more than one way to burn a book. 
dsimms@alink.net                And the world is full of people running 
(408) 720-6161                  about with lit matches.  -Ray Bradbury

Re: pop mailboxes

[Rich Pieri]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "DS" == Daniel Simms <dsimms@alink.net> writes:

DS> (setq nnmail-movemail-program 'pop3-really-movemail)

DS> (require 'pop3)

This will not work.  nnmail-movemail-program expects an external
program, not a function.

At this time, nnmail also does not use pop3.el.  I had not gotten around
to rewriting that section of nnmail before Red went into feature freeze.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3a
Charset: noconv

iQCVAwUBMvf5u56VRH7BJMxHAQGmwwP9E8JCifIsIrkxnnndrrTLmDhJu/gmzSyD
bhw+IFmVVkfJx7Fct+RY9PxByvG+HAKV/gby1bdYIL5XWOE7X9sLjtjhP2DtH+gv
SdlqSSkRGTkdaCndNQjf5olHrUqZlT6j7Ac2OF2jRMvnK+fd6Y/0rfmLLNOg2dp7
raMvhRW0mnE=
=wnZC
-----END PGP SIGNATURE-----
-- 
Rich Pieri <rich.pieri@prescienttech.com> | When not in use, Happy Fun Ball
Prescient Technologies, Inc.              | should be returned to its special
A Stone & Webster Company                 | container and kept under
I speak for myself, not PTI or SWEC       | refrigeration.

Re: pop mailboxes

[Daniel Simms]

>>>>> "RP" == Rich Pieri <rich.pieri@PrescientTech.com> writes:

 >>>> "DS" == Daniel Simms <dsimms@alink.net> writes:

  DS>> (setq nnmail-movemail-program 'pop3-really-movemail)

  DS>> (require 'pop3)

  RP> This will not work.  nnmail-movemail-program expects an external
  RP> program, not a function.

Really?  

>>>>> "I" == (gnus)Mail Backend Variables

 I> `nnmail-movemail-program'
 I>      This program is executed to move mail from the user's inbox to her
 I>      home directory.  The default is `movemail'.
 I> 
 I>      This can also be a function.  In that case, the function will be
 I>      called with two parameters -- the name of the inbox, and the file
 I>      to be moved to.

>>>>> "N" == ~/emacs/packages/gnus-5.4.9/lisp/nnmail.el 


 [[...]]

 N> ;; Function rewritten from rmail.el.
 N> (defun nnmail-move-inbox (inbox)

 [[...]]

 N>	  ;; If getting from mail spool directory, use movemail to move
 N>	  ;; rather than just renaming, so as to interlock with the
 N>	  ;; mailer.
 N>	  (unwind-protect
 N>	      (save-excursion
 N>		(setq errors (generate-new-buffer " *nnmail loss*"))
 N>		(buffer-disable-undo errors)
 N>		(let ((default-directory "/"))
 N>		  (if (nnheader-functionp nnmail-movemail-program)
 N>		      (funcall nnmail-movemail-program inbox tofile)
 N>		    (apply 
 N>		     'call-process
 N>		     (append
 N>		      (list
 N>		       (expand-file-name nnmail-movemail-program
 N>					 exec-directory)
 N>		       nil errors nil inbox tofile)
 N>		      (when nnmail-internal-password
 N>			(list (prin1-to-string nnmail-internal-password)))))))

(Sorry, I'm already two versions out of date, but one was released
*while* I was writing this email.)

  RP> At this time, nnmail also does not use pop3.el.  

Well, I had kind of noticed that, but figured that it was just an 
option.  Setting nnmail-movemail-program is about the only thing you
really need to do to make gnus really use pop3.el.

  RP> I had not gotten around to rewriting that section of nnmail
  RP> before Red went into feature freeze.

I didn't noticed anything on the wish list (or mailing list) about 
this.   What did you have in mind?  (This is more out of curiousity
than it is relevant to this particular thread.)


-- 
Daniel Simms      "A common mistake that people make when trying to design 
dsimms@alink.net   something completely foolproof [is] to underestimate the 
(408) 720-6161     ingenuity of complete fools" -Ford Prefect

Re: pop mailboxes

[Rich Pieri]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "DS" == Daniel Simms <dsimms@alink.net> writes:

I> This can also be a function.  In that case, the function will be
I> called with two parameters -- the name of the inbox, and the file
I> to be moved to.

This got added sometime between "now" and the last time I looked at
nnmail.  I have no idea if it works or not, but it will not work with
pop3-movemail since that function only expect one argument, the optional
crashbox to dump mail into; everything else is handled at this time by
variables.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3a
Charset: noconv

iQCVAwUBMvixvJ6VRH7BJMxHAQGeGQQAp6Q9hxkfLd62UmA0tLaY0p251Hn7h/FC
WhrcDmoMDFKrDPuE80Kg8SDlfnvF6RNZM0j+bGYKREfWoPC3JUX4HrQwmnW/jUYs
tdSEKjMB7OjdGyhUb15CBNbIyTTxIBuJmxc44NI17eqEA9X6Wbe1f4xLdnguf940
0Dr5UTvwL2Y=
=FwBk
-----END PGP SIGNATURE-----
-- 
Rich Pieri <rich.pieri@prescienttech.com> | Do not taunt Happy Fun Ball.
Prescient Technologies, Inc.              | 
A Stone & Webster Company                 | 
I speak for myself, not PTI or SWEC       | 

Re: pop mailboxes

[Paul Franklin]

>>>>> Daniel Simms writes:

 >>>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

 >   >> ;
 >   >> file-directory-p("po:pop.alink.net:110:apop:n:dsimms%alink.net:*")

 >   LMI> This isn't a valid value from `nnmail-spool-file'.  It should
 >   LMI> be just "po:username".


 > Yes, yes*, it does the same thing with just po:luser too.  

I took a look at the code, convinced that Gnus wouldn't have such an
error in it, since I use pop all of the time, and I periodically get
ange-ftp qutoloaded when I'm foolish enough to try to tab-complete in
the root directory.

Sure enough, there's a bug, at least in 5.4.10.  Sorry Lars.

There's another bug which this doesn't fix but I may fix sometime
tomorrow--it handles nnmail-spool-file set to
	'pop
but not
	'(pop "/some/other/file")

Tue Feb  4 20:26:20 1997  Paul Franklin  <paul@cs.washington.edu>

	* nnmail.el (nnmail-get-spool-files): Don't call file-directory-p
	on pop spool specifiers.

--- nnmail.el	1997/02/05 04:18:56	1.1
+++ nnmail.el	1997/02/05 04:23:31
@@ -1254,19 +1254,19 @@
 		'nconc
 		(mapcar 
 		 (lambda (file)
-		   (if (file-directory-p file)
+		   (if (and (not (string-match "^po:" file))
+			    (file-directory-p file))
 		       (nnheader-directory-regular-files file)
 		     (list file)))
 		 nnmail-spool-file))
 	       procmails))
-	     ((and (stringp nnmail-spool-file)
-		   (not (file-directory-p nnmail-spool-file)))
-	      (cons nnmail-spool-file procmails))
-	     ((and (stringp nnmail-spool-file)
-		   (file-directory-p nnmail-spool-file))
-	      (nconc
-	       (nnheader-directory-regular-files nnmail-spool-file)
-	       procmails))
+	     ((stringp nnmail-spool-file)
+	      (if (and (not (string-match "^po:" nnmail-spool-file))
+		       (file-directory-p nnmail-spool-file))
+		  (nconc 
+		   (nnheader-directory-regular-files nnmail-spool-file)
+		   procmails)
+		(cons nnmail-spool-file procmails)))
 	     ((eq nnmail-spool-file 'pop)
 	      (cons (format "po:%s" (user-login-name)) procmails))
 	     (t