funny problems with nnmail-cache and split-fancy-with-parent

funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Recently, I decided I should start using
nnmail-split-fancy-with-parent again.  I added:

	(: nnmail-split-fancy-with-parent)

to my nnmail-split-fancy rules (at the top), and realized a couple
hours later that a bunch of messages were being stored in mail.misc.
I don't use mail.misc, but when I looked in the mail/misc folder there
were something like 100 messages there...presumably due to the
addition of this rule.

Why could that be?  taking a look at ~/.nnmail-cache a few lines look
like this (btw, this is with nnmail-split-fancy-with-parent OFF):

<OF5FE6F041.9810635D-ON86256BAD.00692342@norlight.com>	misc
<E8F83D6D2A6AD3118E0300902786A20502371482@NTEX>	misc
<3CD1933E.3030600@computer.org>	misc
<20020502193559.GA2395@watz.ch>	misc
<20020502213828.A11136@lucretia.debian.net>	misc
<20020502194229.02F623F489@hydrogen.ruby-lang.org>	misc

funny because nothing was actually being split there, but the cache
indicated that messages *were* being split there.

For example:

find . -name .overview | xargs grep OF5FE6F041.9810635D-ON86256BAD.00692342@norlight.com
./lists/debian/private/.overview [...]
 find . -name .overview | xargs grep  E8F83D6D2A6AD3118E0300902786A20502371482@NTEX
./lists/linuxppc/embedded/.overview [...]

[etc, etc -- none of those messages are being sent to mail.misc]

Anyone know what's going on here?

There's another problem here, I think.  About 100 messages were
sitting in mail/misc, but the group still indicated that there were 3
messages in it (3 very old messages).  M-g, C-u RET, etc didn't update
the number or let me see the messages.  not only that, but after
exiting Gnus and starting it back up all messages in mail.misc were
deleted (save for the original 3 which like to stick around for some
reason :)

Fortunately I had already skimmed the messages using less before this
happened and there was nothing important in them.

So, it seems I've stumbled across 2 problems -- the cache file
indicates an incorrect group name where a messages was filed, and when
nnmail-split-fancy-with-parent stored the new messages acording to the
cache file, the group information (low/high message numbers) wasn't
updated. (or something like that...I didn't get a chance to see what
happened).

Anyone use this stuff?  I'll start looking at the code to see what
went wrong in the meantime.

Thanks,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> Why could that be?  taking a look at ~/.nnmail-cache a few lines look
> like this (btw, this is with nnmail-split-fancy-with-parent OFF):
>
> <OF5FE6F041.9810635D-ON86256BAD.00692342@norlight.com>	misc
> <E8F83D6D2A6AD3118E0300902786A20502371482@NTEX>	misc
> <3CD1933E.3030600@computer.org>	misc
> <20020502193559.GA2395@watz.ch>	misc
> <20020502213828.A11136@lucretia.debian.net>	misc
> <20020502194229.02F623F489@hydrogen.ruby-lang.org>	misc
>
> funny because nothing was actually being split there, but the cache
> indicated that messages *were* being split there.

The cache indicates that the message was put in the group.  It is not
necessary to do this via splitting, `B m' also creates such an entry.

But misc != mail.misc.  Hm.  Maybe the code gets confused as to
what's your mail backend?  What does
(nnmail-cache-primary-mail-backend) eval to in your Gnus?

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> The cache indicates that the message was put in the group.  It is
> not necessary to do this via splitting, `B m' also creates such an
> entry.

Ah, I see...

> But misc != mail.misc.  Hm.  Maybe the code gets confused as to
> what's your mail backend?  What does
> (nnmail-cache-primary-mail-backend) eval to in your Gnus?

(nnml "")

as expected...where to look next?

how does Gnus determine what group a message was stored in?

Thanks for your help,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> how does Gnus determine what group a message was stored in?

You mean during splitting time?  It searches backwards from the end
of the msg id cache for the message id and then looks in that line
for the group name.

That means if there are several entries for the msgid (after moving
multiple times, for example), the last one wins.

But I don't understand why it uses a mail.misc group if the cache
file contains misc.

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> You mean during splitting time?  It searches backwards from the end
> of the msg id cache for the message id and then looks in that line
> for the group name.
>
> That means if there are several entries for the msgid (after moving
> multiple times, for example), the last one wins.
>
> But I don't understand why it uses a mail.misc group if the cache
> file contains misc.

Sorry, you're absolutely correct.  I was confused because I have a
mail.misc group and it threw me off.

The messages are actually being stored in the "misc" group.

I've also noticed that messages being stored into mail.inbox.alum are
recorded in the nnmail cache file as being stored in "alum"...which is
also incorrect.

Any ideas why this could be?

Thanks,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> I've also noticed that messages being stored into mail.inbox.alum are
> recorded in the nnmail cache file as being stored in "alum"...which is
> also incorrect.
>
> Any ideas why this could be?

Strange.  I don't know why that could be.  But you could trace the
relevant code...

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Strange.  I don't know why that could be.  But you could trace the
> relevant code...

Well, here's the problem.  In nnmail-cache-insert --

    (let ((g nil))
      (cond (grp
	     (setq g grp))
	    ((and (boundp 'group) group)
	     (setq g group))
	    ((and (boundp 'group-art-list) group-art-list
		  (listp group-art-list))
	     (setq g (caar group-art-list)))
	    ((and (boundp 'group-art) group-art (listp group-art))
	     (setq g (caar group-art)))
	    (t (setq g "")))

group is bound to "misc" (almost always).  (caar group-art) returns
the correct group name for splitting, but since this logic checks
group first, it gets the wrong value.

diff -u -r6.39 nnmail.el
--- lisp/nnmail.el	2002/04/29 20:08:18	6.39
+++ lisp/nnmail.el	2002/05/07 14:29:55
@@ -1469,13 +1469,13 @@
     (let ((g nil))
       (cond (grp
 	     (setq g grp))
+	    ((and (boundp 'group-art) group-art (listp group-art))
+	     (setq g (caar group-art)))
 	    ((and (boundp 'group) group)
 	     (setq g group))
 	    ((and (boundp 'group-art-list) group-art-list
 		  (listp group-art-list))
 	     (setq g (caar group-art-list)))
-	    ((and (boundp 'group-art) group-art (listp group-art))
-	     (setq g (caar group-art)))
 	    (t (setq g "")))
       (unless (gnus-buffer-live-p nnmail-cache-buffer)
 	(nnmail-cache-open))

Works for me, but after looking at this -- shouldn't there be a better
interface into nnmail-cache-insert?  We should go through and find all
the places which call it, which is only 10 places, and use the
optional argument for group name if possible.  checking for bound
symbols and using heuristics to get the group name seems a little
hackish to me ;)  (no offense to Kai!)

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Josh Huber <huber@alum.wpi.edu> writes:

> Works for me, but after looking at this -- shouldn't there be a
> better interface into nnmail-cache-insert?  We should go through and
> find all the places which call it, which is only 10 places, and use
> the optional argument for group name if possible.  checking for
> bound symbols and using heuristics to get the group name seems a
> little hackish to me ;) (no offense to Kai!)

Of course, I spoke too soon here.  The reason this was done became
apartent while looking at nnmail-check-duplication, which has no
knowledge of the group.

Perhaps something a little more generic for splitting could be done?
bind gnus-current-split-group or something like that?

Perhaps we should just ignore it ;)

What do you think about moving the priority of the symbol checking
(the diff in the parent).

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> Josh Huber <huber@alum.wpi.edu> writes:
>
>> Works for me, but after looking at this -- shouldn't there be a
>> better interface into nnmail-cache-insert?  We should go through and
>> find all the places which call it, which is only 10 places, and use
>> the optional argument for group name if possible.  checking for
>> bound symbols and using heuristics to get the group name seems a
>> little hackish to me ;) (no offense to Kai!)

No offense taken.  I now remember the feeling when writing the code.
It is not a pleasant memory :-)

> Of course, I spoke too soon here.  The reason this was done became
> apartent while looking at nnmail-check-duplication, which has no
> knowledge of the group.
>
> Perhaps something a little more generic for splitting could be done?
> bind gnus-current-split-group or something like that?

Yes.  Lars has said (years ago) that he just wants to define some
symbols which are bound and document them.

So why not do that now.  We need to find all places which call
nnmail-check-duplication, then frob them.  Alas, I now use nnimap
which doesn't use that code.  Any takers for testing it, at least?

> Perhaps we should just ignore it ;)
>
> What do you think about moving the priority of the symbol checking
> (the diff in the parent).

If it works for you, it's probably Good Stuff.

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> group is bound to "misc" (almost always).  (caar group-art) returns
> the correct group name for splitting, but since this logic checks
> group first, it gets the wrong value.
>
> diff -u -r6.39 nnmail.el
> --- lisp/nnmail.el	2002/04/29 20:08:18	6.39
> +++ lisp/nnmail.el	2002/05/07 14:29:55

I applied this patch (manually).  OK?

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I applied this patch (manually).  OK?

Oh, ok.  Thanks :)

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> No offense taken.  I now remember the feeling when writing the code.
> It is not a pleasant memory :-)

:)

> Yes.  Lars has said (years ago) that he just wants to define some
> symbols which are bound and document them.
>
> So why not do that now.  We need to find all places which call
> nnmail-check-duplication, then frob them.  Alas, I now use nnimap
> which doesn't use that code.  Any takers for testing it, at least?

Okay, I'll look into this since I use it.

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Raymond Scholz]

Josh Huber <huber@alum.wpi.edu> writes:

> diff -u -r6.39 nnmail.el
> --- lisp/nnmail.el	2002/04/29 20:08:18	6.39
> +++ lisp/nnmail.el	2002/05/07 14:29:55
> @@ -1469,13 +1469,13 @@
>      (let ((g nil))
>        (cond (grp
>  	     (setq g grp))
> +	    ((and (boundp 'group-art) group-art (listp group-art))
> +	     (setq g (caar group-art)))
>  	    ((and (boundp 'group) group)
>  	     (setq g group))
>  	    ((and (boundp 'group-art-list) group-art-list
>  		  (listp group-art-list))
>  	     (setq g (caar group-art-list)))
> -	    ((and (boundp 'group-art) group-art (listp group-art))
> -	     (setq g (caar group-art)))
>  	    (t (setq g "")))
>        (unless (gnus-buffer-live-p nnmail-cache-buffer)
>  	(nnmail-cache-open))
>
> Works for me, but after looking at this -- shouldn't there be a better
> interface into nnmail-cache-insert?  We should go through and find all
> the places which call it, which is only 10 places, and use the
> optional argument for group name if possible.  checking for bound
> symbols and using heuristics to get the group name seems a little
> hackish to me ;)  (no offense to Kai!)

Hi Josh!

As you may have read on gnus-bug <gg.87offmz7ul.fsf@mde1.zonix.de>,
your patch is causing problems with my setup (two Gcc headers with
different backends (nnfolder and nnml).

Kai asked me to bring up this issue on *ding* again.  I'm afraid I
cannot contribute a lot (apart from bug reports) because of my limited
knowlegde of what is going on in nnmail.el.

Here's the backtrace again:

,----
| Debugger entered--Lisp error: (wrong-type-argument listp "sent.mail.home.2002")
|   nnmail-cache-insert("<.87lmarby9j.fsf@mde1.zonix.de>")
|   nnml-request-accept-article("mail.other" "" t)
|   gnus-request-accept-article("nnml:mail.other" "nnml:" t t)
|   gnus-inews-do-gcc()
|   run-hooks(message-sent-hook)
|   message-send(nil)
|   message-send-and-exit(nil)
|   call-interactively(message-send-and-exit)
`----

Cheers, Ray
-- 
It's clear that whoever set up the font colorings for most programming
modes has seen too many Peter Max posters, or did more acid than I did
in the 60's.                     (Charles R. Martin in gnu.emacs.help)

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Raymond Scholz <ray-2002@zonix.de> writes:

> Kai asked me to bring up this issue on *ding* again.  I'm afraid I
> cannot contribute a lot (apart from bug reports) because of my
> limited knowlegde of what is going on in nnmail.el.

Okay, well.  The main problem is that how the group is being fetched
is pretty bogus.  But, perhaps we can narrow it down some.

You're gcc'ing to an nnml folder, which is the same as my setup.
(except in my case, it's always sent-mail or sent-news).

> Here's the backtrace again:
>
> ,----
> | Debugger entered--Lisp error: (wrong-type-argument listp "sent.mail.home.2002")
> |   nnmail-cache-insert("<.87lmarby9j.fsf@mde1.zonix.de>")
> |   nnml-request-accept-article("mail.other" "" t)
> |   gnus-request-accept-article("nnml:mail.other" "nnml:" t t)
> |   gnus-inews-do-gcc()
> |   run-hooks(message-sent-hook)
> |   message-send(nil)
> |   message-send-and-exit(nil)
> |   call-interactively(message-send-and-exit)
> `----

Can you M-x edebug-defun inside nnmail-cache-insert and trace it?
while you're in there can you eval group and (caar group-art)?

Also, a fix for you could be this:

Index: nnml.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnml.el,v
retrieving revision 6.39
diff -u -r6.39 nnml.el
--- nnml.el	2002/04/12 11:51:01	6.39
+++ nnml.el	2002/05/13 14:15:30
@@ -369,7 +369,7 @@
   (nnmail-check-syntax)
   (let (result)
     (when nnmail-cache-accepted-message-ids
-      (nnmail-cache-insert (nnmail-fetch-field "message-id")))
+      (nnmail-cache-insert (nnmail-fetch-field "message-id") group))
     (if (stringp group)
 	(and
 	 (nnmail-activate 'nnml)


Since the group is available, why not pass it into
nnmail-cache-insert?  Of course, this may not fix all cases.  The
proper fix is what Kai and I were discussing probably -- binding a
variable (say nnmail-current-group) which is used by
nnmail-cache-insert so as to remove the current mess.

The problem is we need to go through and find all the possible ways
which can can get to nnmail-cache-insert.  I'll probably take a look
at this later today.

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Raymond Scholz]

Josh Huber <huber@alum.wpi.edu> writes:

> Can you M-x edebug-defun inside nnmail-cache-insert and trace it?
> while you're in there can you eval group and (caar group-art)?

Oh, I accidently hit "cvs up" and I cannot reproduce the error.  This
means that your patch fixes the problem?!

Cheers, Ray
-- 
Sigmentation fault, core dumped.

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Raymond Scholz <ray-2002@zonix.de> writes:

> Oh, I accidently hit "cvs up" and I cannot reproduce the error.
> This means that your patch fixes the problem?!

Hmm, I didn't apply that patch yet, as I wanted to run with it in my
setup for a while before committing it.

Did you apply that small patch I sent before?

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Raymond Scholz]

Josh Huber <huber@alum.wpi.edu> writes:

> Hmm, I didn't apply that patch yet, as I wanted to run with it in my
> setup for a while before committing it.

You didn't apply <87vg9rn65j.fsf@alum.wpi.edu> ...

> Did you apply that small patch I sent before?

..., but you did (accidently?) apply <87661sqgsc.fsf@alum.wpi.edu>, as
seen in <E177Hl8-00016b-00@quimby.gnus.org> on gnus.cvslog.

Cheers, Ray
-- 
/* Halley */ (Halley's comment.)

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Raymond Scholz <ray-2002@zonix.de> writes:

> Josh Huber <huber@alum.wpi.edu> writes:
>
>> Hmm, I didn't apply that patch yet, as I wanted to run with it in my
>> setup for a while before committing it.
>
> You didn't apply <87vg9rn65j.fsf@alum.wpi.edu> ...

Right ;)

> ..., but you did (accidently?) apply <87661sqgsc.fsf@alum.wpi.edu>, as
> seen in <E177Hl8-00016b-00@quimby.gnus.org> on gnus.cvslog.

You're absolutely right.  I applied that one patch, and it was
intentional. :)

With that fix everything is working for you?  That's good.  Explicitly
passing in the group name will work much better, and I think I've got
all the cases where it's done.

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> With that fix everything is working for you?

Raymond?  Since I was the person who broke it, the least I can do is
to check whether Josh's hard work was successful...

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

I've been watching my .nnmail-cache file, and everything is sane for
me, but I'd like people who use different backends to test the
changes.  In particular, I'd like to make sure that IMAP works
correctly.

I only use nnml right now.  I've been meaning to switch over to an
IMAP setup, and I finally have some static IP addresses to house a
nice mail server, so that may happen soon.

Until then, I'd like people who use the split-with-parent, or even
just the duplication detection to make sure the ~/.nnmail-cache file
looks sane (wrt the group names).

Thanks,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> I've been watching my .nnmail-cache file, and everything is sane for
> me, but I'd like people who use different backends to test the
> changes.  In particular, I'd like to make sure that IMAP works
> correctly.

I'm using nnimap (mainly) and nnml (for just a few messages).
.nnmail-cache looks okay wrt the group names: most messages don't
mention a group, and some of them mention "inbox" or "spam".  Seems
fine, since these are the two nnml groups I use at the moment.

But a sizable number of lines have ^M at the end.  Hm.

Anything special I should look for?

Another things that's been on my mind for a while is that it might be
useful to change the .nnmail-cache format to include the server with
the group name.  It is in principle easy to do, but I don't know how
to do it in a backward-compatible manner.  Suggestions?

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I'm using nnimap (mainly) and nnml (for just a few messages).
> .nnmail-cache looks okay wrt the group names: most messages don't
> mention a group, and some of them mention "inbox" or "spam".  Seems
> fine, since these are the two nnml groups I use at the moment.

Should I assume you're not using splitting in Gnus to split the IMAP
articles?  (is this even possible?)

> But a sizable number of lines have ^M at the end.  Hm.
> Anything special I should look for?

That's pretty odd...are these lines with no group listed?  Is there a
\t after the msgid on those lines?  I'm not really seeing how that
could happen, unless the group name was somehow "^M" by accident...

> Another things that's been on my mind for a while is that it might
> be useful to change the .nnmail-cache format to include the server
> with the group name.  It is in principle easy to do, but I don't
> know how to do it in a backward-compatible manner.  Suggestions?

Doing this in a backward-compatible way is going to be hard, I agree.
Since nnmail-cache-fetch-group does this:

      (when (search-backward id nil t)
	(beginning-of-line)
	(skip-chars-forward "^\n\r\t")
	(unless (eolp)
	  (forward-char 1)
	  (buffer-substring (point)
			    (progn (end-of-line) (point)))))

It's difficult to get it to work...the first thing that comes to mind
is to add a second line including more information which will never
get matched as a messageid:

examplemsgid29sdf13499@foo.bar.com	mail.lists.foo
***	((group . "mail.lists.foo") (server . (nnml "")))

ttyl,

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>
>> I'm using nnimap (mainly) and nnml (for just a few messages).
>> .nnmail-cache looks okay wrt the group names: most messages don't
>> mention a group, and some of them mention "inbox" or "spam".  Seems
>> fine, since these are the two nnml groups I use at the moment.
>
> Should I assume you're not using splitting in Gnus to split the IMAP
> articles?  (is this even possible?)

I don't know what you should do :-)

But seriously, I use nnmail-split-methods for the nnml mails, whereas
I use server-side splitting (our Cyrus server supports Sieve) for
nnimap.  (Note that Gnus supports splitting for nnimap, too.  See the
variable nnimap-split-rule¹.)

>> But a sizable number of lines have ^M at the end.  Hm.
>> Anything special I should look for?
>
> That's pretty odd...are these lines with no group listed?  Is there a
> \t after the msgid on those lines?  I'm not really seeing how that
> could happen, unless the group name was somehow "^M" by accident...

Maybe my wording was unclear.  By ^M I mean character number 13, aka
CR.  I don't mean the two characters ^ and M.  I think ascii 13 is
not allowed in group names.

The ^M is at EOL.  So this smells of a DOSish file.  But of course I
have no idea where these characters come from.  Here is a typical ^M
inflicted line:

<vaf661ialiu.fsf@INBOX.auto.emacs.devel.tok.lucy.cs.uni-dortmund.de>

Hm.  Now that I look at the file it seems that most message ids are
from me.  Strange.  And all the ^M lines seem to come from messages I
sent from an nnimap group.  (I have magic in my .gnus file to put a
token in the msg id so that I can file followups in the same group as
their originals.)

>> Another things that's been on my mind for a while is that it might
>> be useful to change the .nnmail-cache format to include the server
>> with the group name.  It is in principle easy to do, but I don't
>> know how to do it in a backward-compatible manner.  Suggestions?
>
> Doing this in a backward-compatible way is going to be hard, I agree.
> Since nnmail-cache-fetch-group does this:
>
>       (when (search-backward id nil t)
> 	(beginning-of-line)
> 	(skip-chars-forward "^\n\r\t")
> 	(unless (eolp)
> 	  (forward-char 1)
> 	  (buffer-substring (point)
> 			    (progn (end-of-line) (point)))))
>
> It's difficult to get it to work...the first thing that comes to mind
> is to add a second line including more information which will never
> get matched as a messageid:
>
> examplemsgid29sdf13499@foo.bar.com	mail.lists.foo
> ***	((group . "mail.lists.foo") (server . (nnml "")))

Hm?  I think you're looking in the wrong place for the problems.  I
think it's entirely sufficient to put group names such as
nnml+private:foo.bar into the file.

The only problem is that the natural interpretation for `bare' group
names is that they come from the primary (native) server.  But this
is clearly not the case for the old group names.

One possibility would be to use the magic string `native' for the
primary server.  Then bare group names could be interpreted as
relative to the primary mail backend (as defined by the current
code).  And all other group names are fully qualified.

Another possibility would be to define a third field.  Lines with
only one field are old lines without group name.  Lines with two
fields are old lines with group name.  Lines with three fields
include the server, where an empty third field could mean the native
server.  Otherwise the third field could have a form such as nnml:foo
(like shown in the server buffer).

Maybe it would be useful for Gnus to define a certain standard as to
what server names should look like.  I'm a bit confused.  Sometimes,
it's nnml:foo, sometimes it's just foo.  People tried to explain to
me before, but I still don't understand.  Must be a thickhead ;-)

Does this rant make any sense at all?  I think I need some rest...

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> Maybe it would be useful for Gnus to define a certain standard as to
> what server names should look like.  I'm a bit confused.  Sometimes,
> it's nnml:foo, sometimes it's just foo.  People tried to explain to
> me before, but I still don't understand.  Must be a thickhead ;-)

A server has an "address", which is the "foo" part of "nnml:foo" or of
"nnml+foo:group.name".  A server may also have a "name", which is
"native" or the server's key in gnus-server-alist.  (Secondary servers
do not have "names".)  This is how the manual uses the terms "name"
and "address" - most of the time.  In common usage, "name" is often
used to refer to the "nnml:foo" string.  Sometimes the manual uses
"name" this way too.  I think the "nnml:foo" form is the more useful
identifier, since it's more informative and all servers are guaranteed
to have one.  Having an extra identifier doesn't provide any
additional functionality, and just causes confusion when the manual
calls it the "name".  I guess it would be a lot of work to get rid of
it, though.


paul

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> A server has an "address", which is the "foo" part of "nnml:foo" or of
> "nnml+foo:group.name".  A server may also have a "name", which is
> "native" or the server's key in gnus-server-alist.  (Secondary servers
> do not have "names".)

Ah.  It's clearer now.  So gnus-secondary-select-methods can be a list
of items, and each item can be a select method or a server name.

I think it would be clearer if Gnus accepted the "nnml:foo" strings
instead of just server names in gnus-secondary-select-methods.

(You can do stuff like (add-to-list 'gnus-secondary-select-methods
'(nnml "")), but you can also define a foreign server x and then do
(add-to-list 'gnus-secondary-select-methods "x").  At least rumor has
it that this is supposed to work.  I think this second method makes
it clearer that foreign and secondary servers are quite similar.)

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Maybe my wording was unclear.  By ^M I mean character number 13, aka
> CR.  I don't mean the two characters ^ and M.  I think ascii 13 is
> not allowed in group names.

Right, I understood -- sorry I was unclear.

> The ^M is at EOL.  So this smells of a DOSish file.  But of course I
> have no idea where these characters come from.  Here is a typical ^M
> inflicted line:
>
> <vaf661ialiu.fsf@INBOX.auto.emacs.devel.tok.lucy.cs.uni-dortmund.de>

Quite odd, eh?

> Hm.  Now that I look at the file it seems that most message ids are
> from me.  Strange.  And all the ^M lines seem to come from messages I
> sent from an nnimap group.  (I have magic in my .gnus file to put a
> token in the msg id so that I can file followups in the same group as
> their originals.)

This looks pretty strange, but I guess it's not somehow getting that
character as the group name, otherwise there would be a tab separating
the two fields.  Does anyone else see ^M's in their cache file?

> Hm?  I think you're looking in the wrong place for the problems.  I
> think it's entirely sufficient to put group names such as
> nnml+private:foo.bar into the file.

Right -- absolutely.  When you said backward compatibility, I thought
you meant allowing older Gnus to function with .nnmail-cache files
generated by the newer versions.

> The only problem is that the natural interpretation for `bare' group
> names is that they come from the primary (native) server.  But this
> is clearly not the case for the old group names.

Right...

> Another possibility would be to define a third field.  Lines with
> only one field are old lines without group name.  Lines with two
> fields are old lines with group name.  Lines with three fields
> include the server, where an empty third field could mean the native
> server.  Otherwise the third field could have a form such as
> nnml:foo (like shown in the server buffer).

This sounds like a step in the right direction.  I'll have to think
about it some more though.

I really think the differentiation between the primary server and the
secondary servers to be one of the more confusing/annoying things
about gnus.

Personally, requiring the prefix (i.e. "nnml:") before even the
primary server should be required, or at least used internally.  I'd
like it to be done it such a way that there was no distiction...just
gnus-server-list. (for example).

The user could then have the option to hide the prefix in the group
view, and use group names without the prefix.  If any ambiguities
occured (the same group from different servers) the different part
could be shown in the group buffer.

Of course, this would take a lot of work, and I'm sure there are many
problems lurking which would make this very difficult.

Thoughts?

-- 
Josh Huber

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Kai Großjohann]

Josh Huber <huber@alum.wpi.edu> writes:

> Right -- absolutely.  When you said backward compatibility, I thought
> you meant allowing older Gnus to function with .nnmail-cache files
> generated by the newer versions.

Old Gnusae only use search-backward to find the msg id, which is
enclosed in angle brackets <...>.  Since group names and server names
don't contain angle brackets, these versions are safe.

Current Gnusae use the group name, so this is indeed a problem.  Hm.
I'm not sure what to do about this.  Is there any sanity check in the
current code which we could abuse for backward compatibility?

kai
-- 
Silence is foo!

Re: funny problems with nnmail-cache and split-fancy-with-parent

[Raymond Scholz]

Josh Huber <huber@alum.wpi.edu> writes:

> Until then, I'd like people who use the split-with-parent, or even
> just the duplication detection to make sure the ~/.nnmail-cache file
> looks sane (wrt the group names).

Looks good to me (nnml, split-with-parent).

Cheers, Ray
-- 
Hfr fgebat rapelcgvba!