has "incorporating new mail" bug been fixed yet?

has "incorporating new mail" bug been fixed yet?

[Randal L. Schwartz]

So where are we at?  Did the latest release fix it so that I can:

    M-g on a single nnml group
    2 M-g to get all nnml groups at level 2-1
    gnus-no-server to start at my level 2-1 groups

without the annoying dozens of "no mail" messages?

(I know, Randal, try it, and let us know...)

Well, just wanted to save the upgrade time if that's still known
broken. :)

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: has "incorporating new mail" bug been fixed yet?

[David S. Goldberg]

> So where are we at?  Did the latest release fix it so that I can:

>     M-g on a single nnml group

Yes.

>     2 M-g to get all nnml groups at level 2-1

Yes.

>     gnus-no-server to start at my level 2-1 groups

Yes.

> without the annoying dozens of "no mail" messages?

No :-(

My workaround: nnml incorporation works without all the "no mail"
messages if you run gnus and use 'g'.  In fact, I've found that to be
so fast now that I don't mind the connection to my primary nntp
server.  Of course that nntp server is only three hops away on a
mostly 100baseT switched network.  I've added this advice

(defadvice gnus-group-list-groups (around set-default-list-level activate)
  ad-do-it
  (if (ad-get-arg 0) (setq gnus-group-default-list-level (ad-get-arg 0))))

to ensure that when I do a '2 l' (or '1 l') only the groups on that
level are displayed and subsequent calls to 'l' don't bring in all the
higher ones.

I don't fully understand the lisp needed to fix this.  I *think* that
nnml-request-scan (and probably others, too) needs to be made
specifically aware of a directory mail-source but I haven't had time
to try to actually write code to make it do so.
-- 
Dave Goldberg
Post: The Mitre Corporation\MS B325\202 Burlington Rd.\Bedford, MA 01730
Phone: 781-271-3887
Email: dsg@mitre.org

Re: has "incorporating new mail" bug been fixed yet?

[Bud Rogers]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

>     M-g on a single nnml group

I don't ordinarily use that, but I just tried it in this group and it
appeared to work.

>     2 M-g to get all nnml groups at level 2-1

I use 2g to get all nnml groups at level 2-1.  If there are any annoying
messages they go by too fast to see.

HTH

-- 
Bud Rogers <budr@sirinet.net>	http://www.sirinet.net/~budr/zamm.html

Re: has "incorporating new mail" bug been fixed yet?

[Randal L. Schwartz]

>>>>> "David" == David S Goldberg <dsg@mitre.org> writes:

>> So where are we at?  Did the latest release fix it so that I can:

>> M-g on a single nnml group

David> Yes.

>> 2 M-g to get all nnml groups at level 2-1

David> Yes.

>> gnus-no-server to start at my level 2-1 groups

David> Yes.

>> without the annoying dozens of "no mail" messages?

David> No :-(

Ahh, so it's still a bug.  Does Lars know it's still a bug? :)

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: has "incorporating new mail" bug been fixed yet?

[Randal L. Schwartz]

>>>>> "Bud" == Bud Rogers <budr@sirinet.net> writes:

Bud> merlyn@stonehenge.com (Randal L. Schwartz) writes:
>> M-g on a single nnml group

Bud> I don't ordinarily use that, but I just tried it in this group and it
Bud> appeared to work.

>> 2 M-g to get all nnml groups at level 2-1

Bud> I use 2g to get all nnml groups at level 2-1.  If there are any annoying
Bud> messages they go by too fast to see.

Do you have a directory source?  I use procmail-like software to
presort my mail, so I have (mumble) 50-some .spool files in my
.incoming directory.  And I see 50-some "new messages" messages each
time.  It probably would be OK if GNUS deleted the files, but I think
we'd have a locking staredown problem at that point.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: has "incorporating new mail" bug been fixed yet?

[Bud Rogers]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

> Do you have a directory source?  I use procmail-like software to
> presort my mail, so I have (mumble) 50-some .spool files in my
> .incoming directory.  And I see 50-some "new messages" messages each
> time.  It probably would be OK if GNUS deleted the files, but I think
> we'd have a locking staredown problem at that point.

No, I use nnml-split-fancy.  Sorry.

-- 
Bud Rogers <budr@sirinet.net>	http://www.sirinet.net/~budr/zamm.html

Re: has "incorporating new mail" bug been fixed yet?

[David S. Goldberg]

> Do you have a directory source?  I use procmail-like software to
> presort my mail, so I have (mumble) 50-some .spool files in my
> ..incoming directory.  And I see 50-some "new messages" messages
> each time.  It probably would be OK if GNUS deleted the files, but I
> think we'd have a locking staredown problem at that point.

Even if GNUS did delete the empty files, it wouldn't make a
difference.

As you note, the problem is due to using a directory source, but it's
not the case that the existence of 50-some .spool files causes the
problem.  The problem is that you've got 50-some nnml groups at levels
1 and 2 and the directory is being searched for a file for each one,
whether or not such a file exists.  This is apparently (to me, based
on the explanation I only vaguely recall from Simon) due to nnml not
having a particular feature in the nnml-request-scan function.  I've
tried to look at what Simon did with nnimap-request-scan, but I just
plain don't get the code.  This is, in my opinion, a side effect of
the design of mail-source versus the old procmail specific code.  I am
quite certain that Lars is aware of this, I'm not so certain he
considers it a bug.
-- 
Dave Goldberg
Post: The Mitre Corporation\MS B325\202 Burlington Rd.\Bedford, MA 01730
Phone: 781-271-3887
Email: dsg@mitre.org

Re: has "incorporating new mail" bug been fixed yet?

[Randal L. Schwartz]

>>>>> "David" == David S Goldberg <dsg@mitre.org> writes:

David> As you note, the problem is due to using a directory source, but it's
David> not the case that the existence of 50-some .spool files causes the
David> problem.  The problem is that you've got 50-some nnml groups at levels
David> 1 and 2 and the directory is being searched for a file for each one,
David> whether or not such a file exists.  This is apparently (to me, based
David> on the explanation I only vaguely recall from Simon) due to nnml not
David> having a particular feature in the nnml-request-scan function.  I've
David> tried to look at what Simon did with nnimap-request-scan, but I just
David> plain don't get the code.  This is, in my opinion, a side effect of
David> the design of mail-source versus the old procmail specific code.  I am
David> quite certain that Lars is aware of this, I'm not so certain he
David> considers it a bug.

Well, when qgnus issues only one message, but pgnus issues hundreds of
messages, I can't see how that would not be considered a "bug" or at
least a degradation in performance.

I know, "patches welcome".  Maybe I'll have to dust off my elisp
reading skills.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: has "incorporating new mail" bug been fixed yet?

[Simon Josefsson]

David S. Goldberg <dsg@mitre.org> writes:

> As you note, the problem is due to using a directory source, but it's
> not the case that the existence of 50-some .spool files causes the
> problem.  The problem is that you've got 50-some nnml groups at levels
> 1 and 2 and the directory is being searched for a file for each one,
> whether or not such a file exists.

Right.

> This is apparently (to me, based on the explanation I only vaguely
> recall from Simon) due to nnml not having a particular feature in
> the nnml-request-scan function.

Actually, it's because nnml don't have `nnml-retrieve-groups', but
that's not relevant -- writing a `nnfoo-retrieve-groups' is only
useful if the backend is slow on doing `nnfoo-request-group' which
isn't the case for nnml (it was for nnimap though), and it's not
what's causing theese messages.

The annoying messages are a result of how the `directory' mail-source
work.

"Normal" mail-sources (that is, all other mail-sources than
`directory') get queried once when you fetch new mail. For `directory'
mail-sources, there's a hack in `nnmail-get-new-mail' which change the
mail-source for every group you fetch new mail in. I haven't really
used directory mail-sources, but I believe the hack is there to make
directory mail-sources work as expected, which would be to make mail
in file "foo" should end up in nnml group "foo" and nowhere else.

This means that instead of having 1 mail-source for your n nnml groups
which one could be led into believing when looking at `mail-sources'
you have in fact n mail-sources, one for each of your n nnml groups,
and the thing you write in your `mail-sources' is a template that is
expanded to the correct mail-source at run-time.

When you do `1 g' to get new mail for some of your nnml groups, you'll
get one message for every mail-source that is being queried. If you
have 10 nnml groups on level 1, there will be 10 different
mail-sources queried for new mail. And you'll get 10 messages about
there being none (or some, or whatever) mail fetched from that
mail-source.

I believe this discussion came up recently, just after my speed-up
patch for scanning mail-sources only once. Since that patch really
shouldn't change any underlaying behaviour, something might be wrong
with it. I reverted to PGnus 0.99 (which is before my patch was
introduced) and I was able to reproduce the behaviour Randal
describes. Randal (and other directory mail-source users), when did
you start seeing this behaviour? Recently?  Was it introduced with
PGnus 0.99?

If not, I think this is how it's supposed to work. You get one message
for each of your mail-source, and that will help you to debug things
if one of the n fetches failed.

This (untested) patch change the verbosity level on some of the
related messages, it should make the "annoying" messages go away. I
don't think this should be commited as-is (or at all), maybe the
verbosity level should be different for `directory' mail-sources and
other mail-sources? That isn't very clean either.

Index: nnmail.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnmail.el,v
retrieving revision 5.44
diff -w -u -u -w -r5.44 nnmail.el
--- nnmail.el	1999/12/07 05:09:43	5.44
+++ nnmail.el	1999/12/25 01:09:11
@@ -1444,7 +1444,7 @@
       ;; The we go through all the existing mail source specification
       ;; and fetch the mail from each.
       (while (setq source (pop fetching-sources))
-	(nnheader-message 4 "%s: Reading incoming mail from %s..."
+	(nnheader-message 8 "%s: Reading incoming mail from %s..."
 			  method (car source))
 	(when (setq new
 		    (mail-source-fetch
@@ -1458,19 +1458,18 @@
 	  (incf total new)
 	  (incf i)))
       ;; If we did indeed read any incoming spools, we save all info.
-      (if (zerop total)
-	  (nnheader-message 4 "%s: Reading incoming mail (no new mail)...done"
-			    method (car source))
+      (unless (zerop total)
 	(nnmail-save-active
 	 (nnmail-get-value "%s-group-alist" method)
 	 (nnmail-get-value "%s-active-file" method))
 	(when exit-func
 	  (funcall exit-func))
 	(run-hooks 'nnmail-read-incoming-hook)
-	(nnheader-message 4 "%s: Reading incoming mail (%d new)...done" method
-			  total))
+	(nnheader-message 4 "%s: Read %d new mail from %s." method total
+			  (car source)))
       ;; Close the message-id cache.
       (nnmail-cache-close)
+      (nnheader-message 8 "%s: Reading incoming mail...done" method total)
       ;; Allow the user to hook.
       (run-hooks 'nnmail-post-get-new-mail-hook))))