maildir support in Pterodactyl Gnus

maildir support in Pterodactyl Gnus

[Jason R. Mastaler]

Is anyone using the maildir support in Pterodactyl Gnus?  I can't get
it to work right in v0.98.  After creating my ~/Maildir/, I specified
my mail-source in .gnus.

;; Get mail from a qmail maildir
 (setq mail-sources
      '((maildir)))

When I try and incorporate new mail with `g', I get the following
error message:

Mail source error ((error IO reading /users/jason/.emacs-mail-crash-box: Input/output error)). Continue? (yes or no)

I answer "no", and receive "Cannot get new mail".  Looking at
.emacs-mail-crash-box, the message appears to be stuck there.
However, if I then hit `g' a second time, the mail is properly
incorporated without any errors and the crash-box is cleared.  This
happens every time.  For the time being, I've switched back to pop3
which works fine, but I'd like really like to use maildir.  Any ideas?

Re: maildir support in Pterodactyl Gnus

[Shenghuo ZHU]

>>>>> "Jason" == Jason R Mastaler <jason@ahpcc.unm.edu> writes:

Jason> Is anyone using the maildir support in Pterodactyl Gnus?  I can't get
Jason> it to work right in v0.98.  After creating my ~/Maildir/, I specified
Jason> my mail-source in .gnus.

Jason> ;; Get mail from a qmail maildir
Jason>  (setq mail-sources
Jason>       '((maildir)))

Jason> When I try and incorporate new mail with `g', I get the following
Jason> error message:

Jason> Mail source error ((error IO reading /users/jason/.emacs-mail-crash-box: Input/output error)). Continue? (yes or no)

Jason> I answer "no", and receive "Cannot get new mail".  Looking at
Jason> .emacs-mail-crash-box, the message appears to be stuck there.
Jason> However, if I then hit `g' a second time, the mail is properly
Jason> incorporated without any errors and the crash-box is cleared.
Jason> This happens every time.  For the time being, I've switched
Jason> back to pop3 which works fine, but I'd like really like to use
Jason> maildir.  Any ideas?

(setq debug-on-signal t)
(setq debug-on-error t)

Then send the backtrace.

-- 
Shenghuo ZHU

Re: maildir support in Pterodactyl Gnus

[Jason R Mastaler]

[-- Attachment #1: Type: text/plain, Size: 129 bytes --]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> (setq debug-on-signal t)
> (setq debug-on-error t)
> 
> Then send the backtrace.


[-- Attachment #2: backtrace --]
[-- Type: text/plain, Size: 2277 bytes --]

Signaling: (error "IO error reading /users/jason/.emacs-mail-crash-box: Input/output error")
  insert-file-contents-internal("~/.emacs-mail-crash-box" nil nil nil nil raw-text used-codesys)
  byte-code("..." [run-hook-with-args insert-file-contents-access-hook filename visit coding-system-for-read run-special-hook-with-args insert-file-contents-pre-hook find-file-coding-system-for-read-from-filename buffer-file-coding-system-for-read no-conversion coding-system return-val find-coding-system message "Invalid coding-system (%s), using 'undecided" undecided insert-file-contents-internal beg end replace used-codesys] 9)
  insert-file-contents("~/.emacs-mail-crash-box" nil nil nil nil)
  mm-insert-file-contents("~/.emacs-mail-crash-box")
  nnmail-split-incoming("~/.emacs-mail-crash-box" nnml-save-mail nil nil nnml-active-number)
  (lambda (file orig-file) (nnmail-split-incoming file (quote nnml-save-mail) (quote nil) (nnmail-get-split-group orig-file source) (quote nnml-active-number)))("~/.emacs-mail-crash-box" "/users/jason/Maildir/new/942103606.16245.agave.alliance.unm.edu")
  mail-source-callback((lambda (file orig-file) (nnmail-split-incoming file (quote nnml-save-mail) (quote nil) (nnmail-get-split-group orig-file source) (quote nnml-active-number))) "/users/jason/Maildir/new/942103606.16245.agave.alliance.unm.edu")
  mail-source-fetch-maildir((maildir) (lambda (file orig-file) (nnmail-split-incoming file (quote nnml-save-mail) (quote nil) (nnmail-get-split-group orig-file source) (quote nnml-active-number))))
  funcall(mail-source-fetch-maildir (maildir) (lambda (file orig-file) (nnmail-split-incoming file (quote nnml-save-mail) (quote nil) (nnmail-get-split-group orig-file source) (quote nnml-active-number))))
  mail-source-fetch((maildir) (lambda (file orig-file) (nnmail-split-incoming file (quote nnml-save-mail) (quote nil) (nnmail-get-split-group orig-file source) (quote nnml-active-number))))
  nnmail-get-new-mail(nnml nnml-save-nov "~/Mail/" "list.redhat-announce")
  nnml-request-scan("list.redhat-announce" "")
  gnus-request-scan("nnml:list.redhat-announce" (nnml ""))
  gnus-activate-group("nnml:list.redhat-announce" scan)
  gnus-get-unread-articles(2)
  gnus-group-get-new-news(nil)
  call-interactively(gnus-group-get-new-news)

Re: maildir support in Pterodactyl Gnus

[Shenghuo ZHU]

>>>>> "Jason" == Jason R Mastaler <jason@4b.org> writes:

Jason> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> (setq debug-on-signal t)
>> (setq debug-on-error t)
>> 
>> Then send the backtrace.



Jason> Signaling: (error "IO error reading /users/jason/.emacs-mail-crash-box: Input/output error")
Jason>   insert-file-contents-internal("~/.emacs-mail-crash-box" nil nil nil nil raw-text used-codesys)
Jason>   byte-code("..." [run-hook-with-args insert-file-contents-access-hook filename visit coding-system-for-read run-special-hook-with-args insert-file-contents-pre-hook find-file-coding-system-for-read-from-filename buffer-file-coding-system-for-read no-conversion coding-system return-val find-coding-system message "Invalid coding-system (%s), using 'undecided" undecided insert-file-contents-internal beg end replace used-codesys] 9)
Jason>   insert-file-contents("~/.emacs-mail-crash-box" nil nil nil nil)
Jason>   mm-insert-file-contents("~/.emacs-mail-crash-box")

[...]

It's really weird. Generally, `insert-file-contents' would not cause
an I/O error. I guess the file is locked or maybe there is some
problems with your file system.

What are your OS and file system?

-- 
Shenghuo

Re: maildir support in Pterodactyl Gnus

[Jason R Mastaler]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> It's really weird. Generally, `insert-file-contents' would not cause
> an I/O error. I guess the file is locked or maybe there is some
> problems with your file system.

No locking is the beauty of maildir which works fine over NFS without
any locking.  You can safely read your mail over NFS if it's in
maildir format.

> What are your OS and file system?

The maildir is NFS mounted off a Linux 2.2.x server's ext2 filesystem.
The client is Linux 2.2.x as well.  Pretty common setup I'd imagine.

Re: maildir support in Pterodactyl Gnus

[Shenghuo ZHU]

>>>>> "Jason" == Jason R Mastaler <jason@4b.org> writes:

[...]

Jason> The maildir is NFS mounted off a Linux 2.2.x server's ext2
Jason> filesystem.  The client is Linux 2.2.x as well.  Pretty common
Jason> setup I'd imagine.

So your maildir ("~/Maildir/new/") and home dir ("~/") are not on the
same filesystem.  That would be a problem for 

        (rename-file "~/Maildir/new/" "~/.emacs-mail-crash-box") 

in mail-source-fetch-maildir. If you can confirm my guess, I can fix it.

-- 
Shenghuo

Re: maildir support in Pterodactyl Gnus

[Jason R Mastaler]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> So your maildir ("~/Maildir/new/") and home dir ("~/") are not on the
> same filesystem.  That would be a problem for 
> 
>         (rename-file "~/Maildir/new/" "~/.emacs-mail-crash-box") 
> 
> in mail-source-fetch-maildir. If you can confirm my guess, I can fix it.

They are both on the same filesystem, but my client workstation (juju)
which is running Gnus, is accessing them over NFS.

[jason@juju ~]$ df ~
Filesystem           1k-blocks      Used Available Use% Mounted on
rr:/u1/jason           6080483   3292458   2473027  57% /users/jason

[jason@juju ~]$ df ~/Maildir/new/
Filesystem           1k-blocks      Used Available Use% Mounted on
rr:/u1/jason           6080483   3292458   2473027  57% /users/jason

Re: maildir support in Pterodactyl Gnus

[Shenghuo ZHU]

>>>>> "Jason" == Jason R Mastaler <jason@4b.org> writes:

Jason> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> So your maildir ("~/Maildir/new/") and home dir ("~/") are not on the
>> same filesystem.  That would be a problem for 
>> 
>> (rename-file "~/Maildir/new/" "~/.emacs-mail-crash-box") 
>> 
>> in mail-source-fetch-maildir. If you can confirm my guess, I can fix it.

Jason> They are both on the same filesystem, but my client workstation (juju)
Jason> which is running Gnus, is accessing them over NFS.

[...]

I think it is a kind of NFS bug. I guess The time span between rename
and read is too short for NFS. Try the following code, see whether it
helps.

(defun mail-source-fetch-maildir (source callback)
  "Fetcher for maildir sources."
  (mail-source-bind (maildir source)
    (let ((found 0)
	  (mail-source-string (format "maildir:%s" path)))
      (dolist (file (directory-files path t))
	(when (and (file-regular-p file)
		   (not (mail-source-movemail file mail-source-crash-box)))
	  (incf found (mail-source-callback callback file))))
      found)))

-- 
Shenghuo

Re: maildir support in Pterodactyl Gnus

[Jason R Mastaler]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> I think it is a kind of NFS bug. I guess The time span between rename
> and read is too short for NFS. Try the following code, see whether it
> helps.

I think you are right.  I duplicated this setup on a local filesystem
and had no problems.  

> (defun mail-source-fetch-maildir (source callback)
>   "Fetcher for maildir sources."
>   (mail-source-bind (maildir source)
>     (let ((found 0)
> 	  (mail-source-string (format "maildir:%s" path)))
>       (dolist (file (directory-files path t))
> 	(when (and (file-regular-p file)
> 		   (not (mail-source-movemail file mail-source-crash-box)))
> 	  (incf found (mail-source-callback callback file))))
>       found)))

This code fixes the problem, but now when I incorporate new mail, the
contents of the maildir files are cleared (zero bytes), but they are
not being removed.

$ ls -l Maildir/new
total 0
-rw-------   1 jason    systaff         0 Nov  9 16:23 942189791.22909.agave.alliance.unm.edu
-rw-------   1 jason    systaff         0 Nov  9 16:25 942189909.22917.agave.alliance.unm.edu

Re: maildir support in Pterodactyl Gnus

[Shenghuo ZHU]

>>>>> "Jason" == Jason R Mastaler <jason@4b.org> writes:

Jason> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> I think it is a kind of NFS bug. I guess The time span between rename
>> and read is too short for NFS. Try the following code, see whether it
>> helps.

Jason> I think you are right.  I duplicated this setup on a local filesystem
Jason> and had no problems.  

>> (defun mail-source-fetch-maildir (source callback)
>> "Fetcher for maildir sources."
>> (mail-source-bind (maildir source)
>> (let ((found 0)
>> (mail-source-string (format "maildir:%s" path)))
>> (dolist (file (directory-files path t))
>> (when (and (file-regular-p file)
>> (not (mail-source-movemail file mail-source-crash-box)))
>> (incf found (mail-source-callback callback file))))
>> found)))

Jason> This code fixes the problem, but now when I incorporate new mail, the
Jason> contents of the maildir files are cleared (zero bytes), but they are
Jason> not being removed.

Jason> $ ls -l Maildir/new
Jason> total 0
Jason> -rw-------   1 jason    systaff         0 Nov  9 16:23 942189791.22909.agave.alliance.unm.edu
Jason> -rw-------   1 jason    systaff         0 Nov  9 16:25 942189909.22917.agave.alliance.unm.edu

That's a result of mail-source-movemail.  I create a new function and
add a new parameter for maildir in pGnus 0.99 (Now, it is still in
CVS).

After you upgrade to 0.99, try use

(maildir :function mail-source-movemail-and-remove)

instead of (maildir).

-- 
Shenghuo ZHU

Re: maildir support in Pterodactyl Gnus

[Jason R Mastaler]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> After you upgrade to 0.99, try use
> 
> (maildir :function mail-source-movemail-and-remove)
> 
> instead of (maildir).

Works great, thanks Shenghuo.