New mail appears to be old?!

New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 960 bytes --]

Hi all,

in the CVS version of GNUS I suddenly experience a rather unwanted and
annoying phenomenon: after getting new mail, only some part of the new
mail is marked as new/unread and makes the groups show up in the group
buffer as groups with new mail. 

This means I need to go through all buffers in order to find out
whether there is something new by looking for unfamiliar mail. This is
not good as it significantly increases the risk to miss new mail.

Does anyone else experience something similar? Although it may not be
correlated, I believe this behaviour may have begun when I started
using nndiary. 

If there is anything I can supply to help you guys find the bug,
please let me know.

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Simon Josefsson]

On Wed, 26 Sep 2001, Georg C. F. Greve wrote:

> in the CVS version of GNUS I suddenly experience a rather unwanted and
> annoying phenomenon: after getting new mail, only some part of the new
> mail is marked as new/unread and makes the groups show up in the group
> buffer as groups with new mail.

Does the group show up as having unread articles, but when you enter it,
only some of those new articles are displayed?  Or doesn't even the group
buffer reflect that you have new mail?

In the first case, either you may have corrupt active or nov files, or
there may be a bug in the new marks stuff.

In the latter case, I have no idea.  It seems very weird.  Maybe mail gets
corrupted somehow, and end up being concatenated to each other instead
of being filed as individual articles, or something like that.

What is your setup, nnml, nnfolder, mail-sources etc?

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 1362 bytes --]

 || On Wed, 26 Sep 2001 16:41:02 +0200 (CEST)
 || Simon Josefsson <jas@extundo.com> wrote: 

 >> in the CVS version of GNUS I suddenly experience a rather unwanted
 >> and annoying phenomenon: after getting new mail, only some part of
 >> the new mail is marked as new/unread and makes the groups show up
 >> in the group buffer as groups with new mail.

 sj> Does the group show up as having unread articles, but when you
 sj> enter it, only some of those new articles are displayed?  

No, sometimes groups (no recognizable pattern) do not show up having
new articles, although they clearly have messages I didn't see yet.

 sj> Or doesn't even the group buffer reflect that you have new mail?

Yes, sometimes, some groups do not reflect that they contain new mail.

 sj> In the first case, either you may have corrupt active or nov
 sj> files, or there may be a bug in the new marks stuff.

I'v tried regenerating the nov databases to no avail.

 sj> In the latter case, I have no idea.  It seems very weird.  Maybe
 sj> mail gets corrupted somehow, and end up being concatenated to
 sj> each other instead of being filed as individual articles, or
 sj> something like that.

Unlikely, I'm using nnml.

 sj> What is your setup, nnml, nnfolder, mail-sources etc?

Here is my cleaned .gnus file (got rid of the personal stuff):

[-- Attachment #1.2: gnus.el --]
[-- Type: application/emacs-lisp, Size: 24368 bytes --]

[-- Attachment #1.3: Type: text/plain, Size: 335 bytes --]


If there is anything else I can do to help tracking down the bug,
please let me know.

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Harry Putnam]

"Georg C. F. Greve" <greve@gnu.org> writes:

> If there is anything else I can do to help tracking down the bug,
> please let me know.

<long shot>
Do you see anything suspicious in News/cache/active?

Maybe the names of some groups with numbers after them, but no
actual articles from those groups in cache subdirectory of that name.

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 597 bytes --]

 || On 27 Sep 2001 16:20:16 +0200
 || reader@newsguy.com (Harry Putnam) wrote: 

 >> If there is anything else I can do to help tracking down the bug,
 >> please let me know.

 hp> <long shot>
 hp> Do you see anything suspicious in News/cache/active?

Not really. I'll try to stop using nndiary for now and see whether the
problem persists.

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 743 bytes --]

 || On 28 Sep 2001 16:40:16 +0200
 || greve@gnu.org ("Georg C. F. Greve") wrote: 

 gg> Not really. I'll try to stop using nndiary for now and see
 gg> whether the problem persists.

FYI: this seems to have helped although I'll keep an eye on it to see
whether this is just a coincidence.

The CVS version of nndiary I'm getting in the gnus/lisp directory
claims to be ";; Last Revision: Wed Aug  8 17:36:21 2001" which seems
rather old for something so new. Is this as it ought to be?

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Didier Verna]

"Georg C. F. Greve" <greve@gnu.org> wrote:

> The CVS version of nndiary I'm getting in the gnus/lisp directory
> claims to be ";; Last Revision: Wed Aug  8 17:36:21 2001" which seems
> rather old for something so new. Is this as it ought to be?

        Don't worry about that. It's an artefact of a package of mine that I
use to maintain file contents, but now that nndiary is in CVS and might be
touched by the other gnus developers, I'll probably remove this header.

-- 
Didier Verna, didier@lrde.epita.fr, http://www.lrde.epita.fr/~didier

EPITA / LRDE, 14-16 rue Voltaire   Tel.+33 (1) 53 14 59 47
94276 Le Kremlin-Bicêtre, France   Fax.+33 (1) 44 08 01 99   didier@xemacs.org

Re: New mail appears to be old?!

[Harry Putnam]

"Georg C. F. Greve" <greve@gnu.org> writes:

> Hi all,
>
> in the CVS version of GNUS I suddenly experience a rather unwanted and
> annoying phenomenon: after getting new mail, only some part of the new
> mail is marked as new/unread and makes the groups show up in the group
> buffer as groups with new mail. 
>
> This means I need to go through all buffers in order to find out
> whether there is something new by looking for unfamiliar mail. This is
> not good as it significantly increases the risk to miss new mail.
>
> Does anyone else experience something similar? Although it may not be
> correlated, I believe this behaviour may have begun when I started
> using nndiary. 
>
> If there is anything I can supply to help you guys find the bug,
> please let me know.

I did see such behaviour at one point.  But in my case all new mail
was marked as read.  It turned out to be a rogue entry in
~/News/cache/active for the nnml group I use to preview new mail.  I
removed that entry from ~/News/cache/active and the behavior stopped.
Later on (days) it reappeared in ~/News/cache/active and I removed
it again.  Never did see what caused that particular group to show up
in that active file.

Several cvs updates later I haven't seen the behavior now for a couple
weeks.

PS- I don't use nndiary and never did.

PPS- As a timely test for your current problem, you can view
the new mail files that gnus creates for you in ~/Mail/Incoming* with
G f (nndoc) to see what was actually collected in a retrieval by 
`G f Mail/MOST_RECENT_Incoming*'.  Unless you've turned it off with:   

  (setq mail-source-delete-incoming t)

Re: New mail appears to be old?!

[Dan Christensen]

Harry Putnam <reader@newsguy.com> writes:

> PPS- As a timely test for your current problem, you can view
> the new mail files that gnus creates for you in ~/Mail/Incoming* with
> G f (nndoc) to see what was actually collected in a retrieval by 
> `G f Mail/MOST_RECENT_Incoming*'.  

Hmm, it's too bad that `G D', which creates an nneething group,
assumes that each file is one message.  Wouldn't it be nice if
one could view a directory of mbox-style files as one newsgroup?

-- 
Dan Christensen
jdc+news@uwo.ca

Re: New mail appears to be old?!

[Harry Putnam]

Dan Christensen <jdc+news@uwo.ca> writes:

> Harry Putnam <reader@newsguy.com> writes:
>
>> PPS- As a timely test for your current problem, you can view
>> the new mail files that gnus creates for you in ~/Mail/Incoming* with
>> G f (nndoc) to see what was actually collected in a retrieval by 
>> `G f Mail/MOST_RECENT_Incoming*'.  
>
> Hmm, it's too bad that `G D', which creates an nneething group,
> assumes that each file is one message.  Wouldn't it be nice if
> one could view a directory of mbox-style files as one newsgroup?

I guess it would have to be a kind of Meta backend that encompassed
several backend formats.

Here is some food for thought along that line.  WIBNI we had functions
as part of gnus that could read a spool file like mbox and spit out
any of the other backend styles.  We do something like that already
with the spit rules or respool function but it is really clunky to use
unless you want to respool some non-main backend to your main backend.

I have developed awk scripting that does something like that, Peter
Acklam has posted perl scripting that handles some conversions.  With
those methods, gnus knows nothing of the work done and must be told
after the fact.

How about elisp that does it for us?  Something where you could aim
the function at a a directory containing a number of mbox files and
gnus would be smart enough to id the spools and convert them to your
choice. Creating new directories etc as needed.  Just as it now does
in a normal split or respool.

Only in this case one would be spared the need to go to the group,
mark the messages for respool and run `B r'.  Instead gnus would
query the user for a FROM backend name, a TO backend name and a list
of directories or files.  Then when you press <RET> gnus would do the
conversion enmasse.

Re: New mail appears to be old?!

[Simon Josefsson]

On Wed, 26 Sep 2001, Dan Christensen wrote:

> Harry Putnam <reader@newsguy.com> writes:
>
> > PPS- As a timely test for your current problem, you can view
> > the new mail files that gnus creates for you in ~/Mail/Incoming* with
> > G f (nndoc) to see what was actually collected in a retrieval by
> > `G f Mail/MOST_RECENT_Incoming*'.
>
> Hmm, it's too bad that `G D', which creates an nneething group,
> assumes that each file is one message.  Wouldn't it be nice if
> one could view a directory of mbox-style files as one newsgroup?

There is `G d' (gnus-group-make-directory-group).  Does it do anything
useful?

Re: New mail appears to be old?!

[Andreas Büsching]

Simon Josefsson <jas@extundo.com> wrote:

> On Wed, 26 Sep 2001, Dan Christensen wrote:
>
>> Harry Putnam <reader@newsguy.com> writes:
>>
>> > PPS- As a timely test for your current problem, you can view
>> > the new mail files that gnus creates for you in ~/Mail/Incoming* with
>> > G f (nndoc) to see what was actually collected in a retrieval by
>> > `G f Mail/MOST_RECENT_Incoming*'.
>>
>> Hmm, it's too bad that `G D', which creates an nneething group,
>> assumes that each file is one message.  Wouldn't it be nice if
>> one could view a directory of mbox-style files as one newsgroup?
>
> There is `G d' (gnus-group-make-directory-group).  Does it do anything
> useful?

Or you can try to set the variable nneething-include-files. For example,
if all your mbox-style files has a suffix ".mailbox" try to do the
following:

(setq nneething-include-files "\\.mailbox$")

crunchy

-- 
"I've got nothing to say but that's ok." -- John Lennon

Re: New mail appears to be old?!

[Dan Christensen]

Simon Josefsson <jas@extundo.com> writes:

> On Wed, 26 Sep 2001, Dan Christensen wrote:
>
>> Hmm, it's too bad that `G D', which creates an nneething group,
>> assumes that each file is one message.  Wouldn't it be nice if
>> one could view a directory of mbox-style files as one newsgroup?
>
> There is `G d' (gnus-group-make-directory-group).  Does it do anything
> useful?

Not on the directory of Incoming files.

Dan

Re: New mail appears to be old?!

[Karl Eichwalder]

"Georg C. F. Greve" <greve@gnu.org> writes:

> in the CVS version of GNUS I suddenly experience a rather unwanted and
> annoying phenomenon: after getting new mail, only some part of the new
> mail is marked as new/unread and makes the groups show up in the group
> buffer as groups with new mail. 

Happened to me, too (I don't use nndiary).  To cure the problem I moved
all article to a temp group, remove the old one and copied back the
article from the temp group.

I've some other mail groups where the '.' mark never goes away.  Is it
possible to restrict the '.' mark to news groups only?

IIRC, I reported both these problems earlier.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: New mail appears to be old?!

[Simon Josefsson]

Karl Eichwalder <keichwa@gmx.net> writes:

>> in the CVS version of GNUS I suddenly experience a rather unwanted and
>> annoying phenomenon: after getting new mail, only some part of the new
>> mail is marked as new/unread and makes the groups show up in the group
>> buffer as groups with new mail. 
>
> Happened to me, too (I don't use nndiary).  To cure the problem I moved
> all article to a temp group, remove the old one and copied back the
> article from the temp group.

Does it make the problem go away permanently for the group, or just
temporarily until it happens again?  If you M-g the group, and `G E'
the group, does the active range (third entry) match the high/low
article count in the group?  Is the active info ok?  (I.e. does M-x
nnml-generate-nov-databases RET solve it?)

> I've some other mail groups where the '.' mark never goes away.  

How do the `seen' mark list in `G E' look like?  Does the patch below
help?  (I wouldn't think so, but maybe.)

> Is it possible to restrict the '.' mark to news groups only?

I don't think so.. Lars?

Index: nnml.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnml.el,v
retrieving revision 6.27
diff -u -w -r6.27 nnml.el
--- nnml.el	2001/09/24 17:35:21	6.27
+++ nnml.el	2001/09/30 11:21:22
@@ -952,13 +952,14 @@
     (nnml-open-marks group server)
     ;; Update info using `nnml-marks'.
     (mapcar (lambda (pred)
+	      (unless (gnus-article-unpropagatable-p (cdr pred))
 	      (gnus-info-set-marks
 	       info
 	       (gnus-update-alist-soft
 		(cdr pred)
 		(cdr (assq (cdr pred) nnml-marks))
 		(gnus-info-marks info))
-	       t))
+		 t)))
 	    gnus-article-mark-lists)
     (let ((seen (cdr (assq 'read nnml-marks))))
       (gnus-info-set-read info
Index: nnfolder.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnfolder.el,v
retrieving revision 6.23
diff -u -w -r6.23 nnfolder.el
--- nnfolder.el	2001/09/24 17:35:21	6.23
+++ nnfolder.el	2001/09/30 11:21:22
@@ -1176,13 +1176,14 @@
     (nnfolder-open-marks group server)
     ;; Update info using `nnfolder-marks'.
     (mapcar (lambda (pred)
+	      (unless (gnus-article-unpropagatable-p (cdr pred))
 	      (gnus-info-set-marks
 	       info
 	       (gnus-update-alist-soft
 		(cdr pred)
 		(cdr (assq (cdr pred) nnfolder-marks))
 		(gnus-info-marks info))
-	       t))
+		 t)))
 	    gnus-article-mark-lists)
     (let ((seen (cdr (assq 'read nnfolder-marks))))
       (gnus-info-set-read info

Re: New mail appears to be old?!

[Karl Eichwalder]

Simon Josefsson <jas@extundo.com> writes:

> Does it make the problem go away permanently for the group, or just
> temporarily until it happens again?

It's permanently away.  IIRC, after remove the old group (G DEL) I exit
Emacs and 'rm -fr group.name' physically -- there was still a dot file
sitting around.  Sorry, I didn't check the active file.

I'll test the rest later.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: New mail appears to be old?!

[Kai Großjohann]

Karl Eichwalder <keichwa@gmx.net> writes:

> It's permanently away.  IIRC, after remove the old group (G DEL) I exit
> Emacs and 'rm -fr group.name' physically -- there was still a dot file
> sitting around.  Sorry, I didn't check the active file.

If you use C-u G DEL, then Gnus should delete the files, too.  Please
report a bug if that doesn't happen.

kai
-- 
Abort this operation?   [OK]  [Cancel]

Re: New mail appears to be old?!

[Karl Eichwalder]

Simon Josefsson <jas@extundo.com> writes:

> Does it make the problem go away permanently for the group, or just
> temporarily until it happens again?  If you M-g the group, and `G E'
> the group, does the active range (third entry) match the high/low
> article count in the group?  Is the active info ok?  (I.e. does M-x
> nnml-generate-nov-databases RET solve it?)

I could have sworn I ran nnml-generate-nov-databases before.  Now I did
it and things are looking good.  I'll try the patch when the problem
occurs again---crossed fingers.

Thanks for your help (and all your work on Gnus!).

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 803 bytes --]

 || On 30 Sep 2001 15:00:11 +0200
 || keichwa@gmx.net (Karl Eichwalder) wrote: 

 ke> I could have sworn I ran nnml-generate-nov-databases before.  

I did to no avail - will try your fix next.

 ke> Now I did it and things are looking good.  I'll try the patch
 ke> when the problem occurs again---crossed fingers.

I just had an idea. Are you (by any chance) using fancy splitting with
(: nnmail-split-fancy-with-parent)? I've just been thinking about
where such a problem could be caused and this was one of the places
that came to my mind.

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Kai Großjohann]

"Georg C. F. Greve" <greve@gnu.org> writes:

> I just had an idea. Are you (by any chance) using fancy splitting with
> (: nnmail-split-fancy-with-parent)? I've just been thinking about
> where such a problem could be caused and this was one of the places
> that came to my mind.

Are you saying that nnmail-split-fancy-with-parent has a bug?  Which
bug?  I want to fix it!

kai
-- 
Abort this operation?   [OK]  [Cancel]

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 1152 bytes --]

 || On Mon, 01 Oct 2001 13:54:17 +0200
 || Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote: 

 >> I just had an idea. Are you (by any chance) using fancy splitting
 >> with (: nnmail-split-fancy-with-parent)? I've just been thinking
 >> about where such a problem could be caused and this was one of the
 >> places that came to my mind.

 kg> Are you saying that nnmail-split-fancy-with-parent has a bug?
 kg> Which bug?  I want to fix it!

Whew. Now that I have just received DSL at home, my bug reports seem
to get treated much more enthusiastically and fast, it seems. *g*

In this case I'm not sure whether the bug really is in this function,
I'm just fishing in the mud for some similarity in the setup of Karl
and me so we might find the problem with non-displayed new mail that
way.

Unfortunately I don't know enough about the gnus interna to say how
likely this is. 

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]

Re: New mail appears to be old?!

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 1012 bytes --]

 || On Sun, 30 Sep 2001 06:37:27 +0200
 || Karl Eichwalder <keichwa@gmx.net> wrote: 

 >> in the CVS version of GNUS I suddenly experience a rather unwanted
 >> and annoying phenomenon: after getting new mail, only some part of
 >> the new mail is marked as new/unread and makes the groups show up
 >> in the group buffer as groups with new mail.

 ke> Happened to me, too (I don't use nndiary).  To cure the problem I
 ke> moved all article to a temp group, remove the old one and copied
 ke> back the article from the temp group.

Do you mean that you did this particularly for the group that had this
problem or did you do it for all groups? Did you remove the actual
group completely before copying it back or did you just make sure it
was empty?

Regards,
Georg

-- 
Georg C. F. Greve                                       <greve@gnu.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Brave GNU World	                           (http://brave-gnu-world.org)

[-- Attachment #2: Type: application/pgp-signature, Size: 268 bytes --]