nnimap, gnus-agent and `/' as a subfolder separator

nnimap, gnus-agent and `/' as a subfolder separator

[Eric Knauel]

Hi,

The gnus-agent of Oort Gnus 0.17 seems to be confused when `/' is used
as a subfolder separator on IMAP servers. I'm not sure whether this is
a bug:

I subscribed some nnimap groups,
e.g. `nnimap+webmail.macnews.de:macnews/cocoa', accessing them in
`Plugged mode' works fine, so does fetching the articles with
gnus-agent `J s':

,----
| [knauel@albert cocoa] pwd
| /Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de/macnews/cocoa
| [knauel@albert cocoa] ls
| 1     2     3     cocoa
`----

However, if I try to access this group in `Unplugged mode' this will
happen:

,----
| Signaling: (error "Couldn't request group nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such directory)")
|   signal(error ("Couldn't request group nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such directory)"))
|   cerror("Couldn't request group %s: %s" "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)")
|   apply(cerror "Couldn't request group %s: %s" ("nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)"))
|   error("Couldn't request group %s: %s" "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)")
|   gnus-select-newsgroup("nnimap+webmail.macnews.de:macnews/cocoa" nil nil)
|   gnus-summary-read-group-1("nnimap+webmail.macnews.de:macnews/cocoa" nil t nil nil nil)
|   gnus-summary-read-group("nnimap+webmail.macnews.de:macnews/cocoa" nil t nil nil nil nil)
|   gnus-group-read-group(nil t)
|   gnus-group-select-group(nil)
|   gnus-topic-select-group(nil)
|   call-interactively(gnus-topic-select-group)
`----

I suppose that gnus searches for the mail files in the wrong directory
or gnus-agent creates the mail folder in the wrong directory. 

By looking at the function `nnmail-group-pathname' in nnmail.el I
found out that it returns
`/Users/knauel/.gnus/Mail/nnml/macnews_cocoa' in this case.  If I `ln
-s' that directory to the actual directory created by gnus-agent fetch
---it works fine.  

However, sometimes (I couldn't figure out why or when) gnus searches
these groups at
`/Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de', which is
quite correct, but it can't find the subfolder `macnews/cocoa' in this
directory.  It's looking for `macnews_cocoa' there, so if make another
link there, it'll work.  That's very confusing. ;-)

BTW I'm using (setq nnmail-use-long-file-names t) in my gnus.el.

-Eric
-- 
"Excuse me --- Di Du Du Duuuuh Di Dii --- Huh Weeeheeee" (Albert King)

Re: nnimap, gnus-agent and `/' as a subfolder separator

[Danny Siu]

Try this. it removes "/" from gnus-invalid-group-regexp

(setq gnus-invalid-group-regexp"[: `'\"]\\|^$")


Eric Knauel writes:

  Eric> Hi,

  Eric> The gnus-agent of Oort Gnus 0.17 seems to be confused when `/' is
  Eric> used as a subfolder separator on IMAP servers. I'm not sure whether
  Eric> this is a bug:

  Eric> I subscribed some nnimap groups,
  Eric> e.g. `nnimap+webmail.macnews.de:macnews/cocoa', accessing them in
  Eric> `Plugged mode' works fine, so does fetching the articles with
  Eric> gnus-agent `J s':

  Eric> ,---- | [knauel@albert cocoa] pwd |
  Eric> /Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de/macnews/cocoa
  Eric> | [knauel@albert cocoa] ls | 1 2 3 cocoa `----

  Eric> However, if I try to access this group in `Unplugged mode' this will
  Eric> happen:

  Eric> ,---- | Signaling: (error "Couldn't request group
  Eric> nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such
  Eric> directory)") | signal(error ("Couldn't request group
  Eric> nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such
  Eric> directory)")) | cerror("Couldn't request group %s: %s"
  Eric> "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such
  Eric> directory)") | apply(cerror "Couldn't request group %s: %s"
  Eric> ("nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such
  Eric> directory)")) | error("Couldn't request group %s: %s"
  Eric> "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such
  Eric> directory)") |
  Eric> gnus-select-newsgroup("nnimap+webmail.macnews.de:macnews/cocoa" nil
  Eric> nil) |
  Eric> gnus-summary-read-group-1("nnimap+webmail.macnews.de:macnews/cocoa"
  Eric> nil t nil nil nil) |
  Eric> gnus-summary-read-group("nnimap+webmail.macnews.de:macnews/cocoa"
  Eric> nil t nil nil nil nil) | gnus-group-read-group(nil t) |
  Eric> gnus-group-select-group(nil) | gnus-topic-select-group(nil) |
  Eric> call-interactively(gnus-topic-select-group) `----

  Eric> I suppose that gnus searches for the mail files in the wrong
  Eric> directory or gnus-agent creates the mail folder in the wrong
  Eric> directory.

  Eric> By looking at the function `nnmail-group-pathname' in nnmail.el I
  Eric> found out that it returns
  Eric> `/Users/knauel/.gnus/Mail/nnml/macnews_cocoa' in this case.  If I
  Eric> `ln -s' that directory to the actual directory created by gnus-agent
  Eric> fetch ---it works fine.

  Eric> However, sometimes (I couldn't figure out why or when) gnus searches
  Eric> these groups at
  Eric> `/Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de', which is
  Eric> quite correct, but it can't find the subfolder `macnews/cocoa' in
  Eric> this directory.  It's looking for `macnews_cocoa' there, so if make
  Eric> another link there, it'll work.  That's very confusing. ;-)

  Eric> BTW I'm using (setq nnmail-use-long-file-names t) in my gnus.el.

  Eric> -Eric -- "Excuse me --- Di Du Du Duuuuh Di Dii --- Huh Weeeheeee"
  Eric> (Albert King)



-- 
Danny Siu

Re: nnimap, gnus-agent and `/' as a subfolder separator

[Kevin Greiner]

Eric,
I'm assuming that you mentioned nnmail-use-long-file-names because you
debugged the code enough to localize the error.  Thanks.

Eric Knauel <knauel@informatik.uni-tuebingen.de> writes:

> Hi,
>
> The gnus-agent of Oort Gnus 0.17 seems to be confused when `/' is used
> as a subfolder separator on IMAP servers. I'm not sure whether this is
> a bug:
>
> I subscribed some nnimap groups,
> e.g. `nnimap+webmail.macnews.de:macnews/cocoa', accessing them in
> `Plugged mode' works fine, so does fetching the articles with
> gnus-agent `J s':
>
> ,----
> | [knauel@albert cocoa] pwd
> | /Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de/macnews/cocoa
> | [knauel@albert cocoa] ls
> | 1     2     3     cocoa
> `----

The function that I provide (see below) will not return this path.
You'll need to rename it to
/Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de/macnews_cocoa

> However, if I try to access this group in `Unplugged mode' this will
> happen:
>
> ,----
> | Signaling: (error "Couldn't request group nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such directory)")
> |   signal(error ("Couldn't request group nnimap+webmail.macnews.de:macnews/cocoa: Invalid group (no such directory)"))
> |   cerror("Couldn't request group %s: %s" "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)")
> |   apply(cerror "Couldn't request group %s: %s" ("nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)"))
> |   error("Couldn't request group %s: %s" "nnimap+webmail.macnews.de:macnews/cocoa" "Invalid group (no such directory)")
> |   gnus-select-newsgroup("nnimap+webmail.macnews.de:macnews/cocoa" nil nil)
> |   gnus-summary-read-group-1("nnimap+webmail.macnews.de:macnews/cocoa" nil t nil nil nil)
> |   gnus-summary-read-group("nnimap+webmail.macnews.de:macnews/cocoa" nil t nil nil nil nil)
> |   gnus-group-read-group(nil t)
> |   gnus-group-select-group(nil)
> |   gnus-topic-select-group(nil)
> |   call-interactively(gnus-topic-select-group)
> `----
>
> I suppose that gnus searches for the mail files in the wrong directory
> or gnus-agent creates the mail folder in the wrong directory. 


The agent should use a path similar to that used by the backend.  That
would appear to indicate that the agent should use 'macnews_cocoa'
rather than 'macnews/cocoa'.


> By looking at the function `nnmail-group-pathname' in nnmail.el I
> found out that it returns
> `/Users/knauel/.gnus/Mail/nnml/macnews_cocoa' in this case.  If I `ln
> -s' that directory to the actual directory created by gnus-agent fetch
> ---it works fine.  

You shouldn't link the nnml directory to the agent.  Doing so means
that both the nnml backend and the agent will try to manipulate the
same local files.  Both components believe that they have exclusive
control over these files so this could be trouble.

You've obviously read the code.  I could use a second opinion.  I've
concluded that nnmail-group-pathname doesn't use
nnmail-use-long-file-names as expected (at least as I would expected).
The problem is that nnmail-group-pathname first converts periods (.)
into underscores (_). It then checks nnmail-use-long-file-names and,
if it is null, converts periods (.) into slashes (/).  But what's the
point? By this time, no periods should remain in the group name.


> However, sometimes (I couldn't figure out why or when) gnus searches
> these groups at
> `/Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de', which is
> quite correct, but it can't find the subfolder `macnews/cocoa' in this
> directory.  It's looking for `macnews_cocoa' there, so if make another
> link there, it'll work.  That's very confusing. ;-)
>
> BTW I'm using (setq nnmail-use-long-file-names t) in my gnus.el.

Yes, nnmail-use-long-file-names appears to be the problem.

Let's see if this function fixes your problem.

(defun gnus-agent-group-path (group)
  "Translate GROUP into a file name."
  (nnheader-translate-file-chars
   (nnheader-replace-duplicate-chars-in-string
    (nnheader-replace-chars-in-string 
     (gnus-group-real-name group)
     ?/ ?_)
    ?. ?_)))

You can either replace the original in gnus-agent.el or simply eval
this form after loading gnus-agent.

Kevin

Re: nnimap, gnus-agent and `/' as a subfolder separator

[Eric Knauel]

Hi Kevin,

Kevin Greiner <kgreiner@xpediantsolutions.com> writes:

> I'm assuming that you mentioned nnmail-use-long-file-names because
> you debugged the code enough to localize the error.  Thanks.
>
> Eric Knauel <knauel@informatik.uni-tuebingen.de> writes:
>
> You've obviously read the code.  I could use a second opinion.  I've
> concluded that nnmail-group-pathname doesn't use
> nnmail-use-long-file-names as expected (at least as I would expected).
> The problem is that nnmail-group-pathname first converts periods (.)
> into underscores (_). It then checks nnmail-use-long-file-names and,
> if it is null, converts periods (.) into slashes (/).  But what's the
> point? By this time, no periods should remain in the group name.

Maybe it's this:

(defun nnmail-group-pathname (group dir &optional file)
  "Make file name for GROUP."
  (concat
   (let ((dir (file-name-as-directory (expand-file-name dir))))
     (setq group (nnheader-replace-duplicate-chars-in-string
                                   ^^^^^^^^^

This probably should be `nnheader-replace-chars-in-string'.

		  (nnheader-replace-chars-in-string group ?/ ?_)
		  ?. ?_))
     (setq group (nnheader-translate-file-chars group))
     ;; If this directory exists, we use it directly.
     (file-name-as-directory
      (if (or nnmail-use-long-file-names
	      (file-directory-p (concat dir group)))
	  (expand-file-name group dir)
	;; If not, we translate dots into slashes.
	(expand-file-name
	 (mm-encode-coding-string
	  (nnheader-replace-chars-in-string group ?. ?/)
	  nnmail-pathname-coding-system)
	 dir))))
   (or file "")))

With `nnheader-replace-duplicate-chars-in-string' the periods remain
in the group name:

(nnheader-replace-duplicate-chars-in-string "mail.uni.pu" ?. ?_)
=> "mail.uni.pu"

So the `(nnheader-replace-chars-in-string group ?. ?/)' in the
`nnmail-use-long-file-names' != nil case might be actually be a
hack. 

Using `dabbrev-expand' while coding Lisp is evil. ;-)

>> BTW I'm using (setq nnmail-use-long-file-names t) in my gnus.el.
>
> Yes, nnmail-use-long-file-names appears to be the problem.
>
> Let's see if this function fixes your problem.
>
> (defun gnus-agent-group-path (group)
>   "Translate GROUP into a file name."
>   (nnheader-translate-file-chars
>    (nnheader-replace-duplicate-chars-in-string
>     (nnheader-replace-chars-in-string 
>      (gnus-group-real-name group)
>      ?/ ?_)
>     ?. ?_)))
>
> You can either replace the original in gnus-agent.el or simply eval
> this form after loading gnus-agent.

That seems to solve the problem! I removed all directories and links
prefixed by "macnews/" in
`/Users/knauel/.gnus/News/agent/nnimap/webmail.macnews.de'. The next
`gnus-agent-fetch-session' created directories `macnews_xyz', which
open up properly. Thanks!

-Eric
-- 
"Excuse me --- Di Du Du Duuuuh Di Dii --- Huh Weeeheeee" (Albert King)