gnus-group-get-new-news broken again

gnus-group-get-new-news broken again

[Bruce Stephens]

I have mail groups at level 1, and news groups at level 3.  I use C-u
2 g to get new email messages.  

A few days ago, this worked, but now it doesn't (it doesn't get any
new messages).  I'm using CVS gnus, and this coincides approximately
with the 0.99->5.8.2->5.8.3 change, I guess.

Re: gnus-group-get-new-news broken again

[Michael Harnois]

On 04 Dec 1999 14:24:12 +0000, Bruce Stephens <bruce@cenderis.demon.co.uk> said:

    > I have mail groups at level 1, and news groups at level 3.  I
    > use C-u 2 g to get new email messages.

    > A few days ago, this worked, but now it doesn't (it doesn't get
    > any new messages).  I'm using CVS gnus, and this coincides
    > approximately with the 0.99->5.8.2->5.8.3 change, I guess.

I reported it three days ago, so it's at least that old...

-- 
Michael D. Harnois, Redeemer Lutheran Church, Washburn, IA 
mharnois@willinet.net                      aa0bt@aa0bt.ampr.org 
 Perish discretion when it interferes with duty.
      -- Hannah More

Re: gnus-group-get-new-news broken again

[Shenghuo ZHU]

>>>>> "Michael" == Michael Harnois <mharnois@willinet.net> writes:

Michael> On 04 Dec 1999 14:24:12 +0000, Bruce Stephens <bruce@cenderis.demon.co.uk> said:
>> I have mail groups at level 1, and news groups at level 3.  I
>> use C-u 2 g to get new email messages.

>> A few days ago, this worked, but now it doesn't (it doesn't get
>> any new messages).  I'm using CVS gnus, and this coincides
>> approximately with the 0.99->5.8.2->5.8.3 change, I guess.

Michael> I reported it three days ago, so it's at least that old...

Three days ago, 0.99 is released. Since Bruce uses CVS, it could
happen in 0.99pre.

However, it works for me.

-- 
Shenghuo

Re: gnus-group-get-new-news broken again

[Michael Harnois]

On 04 Dec 1999 12:17:55 -0500, Shenghuo ZHU <zsh@cs.rochester.edu> said:

    > Three days ago, 0.99 is released. Since Bruce uses CVS, it could
    > happen in 0.99pre.

I use CVS as well. The problem coincided with the release of the new
version. 

-- 
Michael D. Harnois, Redeemer Lutheran Church, Washburn, IA 
mharnois@willinet.net                      aa0bt@aa0bt.ampr.org 
 Perish discretion when it interferes with duty.
      -- Hannah More

Re: gnus-group-get-new-news broken again

[Simon Josefsson]

Michael Harnois <mharnois@willinet.net> writes:

>     > I have mail groups at level 1, and news groups at level 3.  I
>     > use C-u 2 g to get new email messages.
> 
>     > A few days ago, this worked, but now it doesn't (it doesn't get
>     > any new messages).  I'm using CVS gnus, and this coincides
>     > approximately with the 0.99->5.8.2->5.8.3 change, I guess.
> 
> I reported it three days ago, so it's at least that old...

Hm, does it go away if you remove my speedup-patch, by applying this
patch below?  Commited some days ago, so it might be it.

If so, what does the select method for your level 1 group look like?
(`G e' in the group buffer, on the group)

--- gnus-start.el	Sun Dec  5 01:34:31 1999
+++ gnus-start.el.~5.41~	Sun Dec  5 01:32:59 1999
@@ -1550,10 +1542,7 @@
 		  (setcdr (assoc method retrievegroups)
 			  (cons group (cdr (assoc method retrievegroups))))
 		(push (list method group) retrievegroups))
-	    (if (member method scanned-methods)
-		(setq active (gnus-activate-group group))
-	      (setq active (gnus-activate-group group 'scan))
-	      (push method scanned-methods))
+	  (setq active (gnus-activate-group group 'scan))
 	    (inline (gnus-close-group group))))))
 
       ;; Get the number of unread articles in the group.

Re: gnus-group-get-new-news broken again

[Shenghuo ZHU]

>>>>> "Simon" == Simon Josefsson <jas@pdc.kth.se> writes:

Simon> Michael Harnois <mharnois@willinet.net> writes:
>> > I have mail groups at level 1, and news groups at level 3.  I
>> > use C-u 2 g to get new email messages.
>> 
>> > A few days ago, this worked, but now it doesn't (it doesn't get
>> > any new messages).  I'm using CVS gnus, and this coincides
>> > approximately with the 0.99->5.8.2->5.8.3 change, I guess.
>> 
>> I reported it three days ago, so it's at least that old...

Simon> Hm, does it go away if you remove my speedup-patch, by applying this
Simon> patch below?  Commited some days ago, so it might be it.

Simon> If so, what does the select method for your level 1 group look like?
Simon> (`G e' in the group buffer, on the group)

Simon> --- gnus-start.el	Sun Dec  5 01:34:31 1999
Simon> +++ gnus-start.el.~5.41~	Sun Dec  5 01:32:59 1999
Simon> @@ -1550,10 +1542,7 @@
Simon>  		  (setcdr (assoc method retrievegroups)
Simon>  			  (cons group (cdr (assoc method retrievegroups))))
Simon>  		(push (list method group) retrievegroups))
Simon> -	    (if (member method scanned-methods)
Simon> -		(setq active (gnus-activate-group group))
Simon> -	      (setq active (gnus-activate-group group 'scan))
Simon> -	      (push method scanned-methods))
Simon> +	  (setq active (gnus-activate-group group 'scan))
Simon>  	    (inline (gnus-close-group group))))))
 
Simon>        ;; Get the number of unread articles in the group.


I think the patch speed up most mail sources correctly except
directory, since there is a hack code in nnmail-get-new-mail
(nnmail.el:1426), which only fetch the contents of a single group's
spool file. With the patch, `2 g' fetching for the first group
inhibits fetching from other spool files.  For those users who do not
use directory as a mail source, this patch would not be a problem.

-- 
Shenghuo

Re: gnus-group-get-new-news broken again

[Simon Josefsson]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> I think the patch speed up most mail sources correctly except
> directory, since there is a hack code in nnmail-get-new-mail
> (nnmail.el:1426), which only fetch the contents of a single group's
> spool file. With the patch, `2 g' fetching for the first group
> inhibits fetching from other spool files.  For those users who do not
> use directory as a mail source, this patch would not be a problem.

Ouch. This patch add the hack to the scanning part too, but this is
getting quite messy. It wouldn't work if `mail-sources' can be a group
parameter (or non-global in any way), but I'm not sure if it can be,
so...  Whatdoyouthink, should I apply this or remove my earlier
speed-up patch?

Index: gnus-start.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/gnus-start.el,v
retrieving revision 5.46
diff -w -u -u -w -r5.46 gnus-start.el
--- gnus-start.el	1999/12/05 03:00:20	5.46
+++ gnus-start.el	1999/12/05 15:08:44
@@ -1550,7 +1550,15 @@
 		  (setcdr (assoc method retrievegroups)
 			  (cons group (cdr (assoc method retrievegroups))))
 		(push (list method group) retrievegroups))
-	    (if (member method scanned-methods)
+	    ;; hack: `nnmail-get-new-mail' changes the mail-source depending
+	    ;; on the group, so we must perform a scan for every group
+	    ;; if the users has any directory mail sources.
+	    (if (and (null (assq 'directory
+				 (or mail-sources
+				     (if (listp nnmail-spool-file) 
+					 nnmail-spool-file
+				       (list nnmail-spool-file)))))
+		     (member method scanned-methods))
 		(setq active (gnus-activate-group group))
 	      (setq active (gnus-activate-group group 'scan))
 	      (push method scanned-methods))

Re: gnus-group-get-new-news broken again

[Shenghuo ZHU]

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]

>>>>> "Simon" == Simon Josefsson <jas@pdc.kth.se> writes:

Simon> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> I think the patch speed up most mail sources correctly except
>> directory, since there is a hack code in nnmail-get-new-mail
>> (nnmail.el:1426), which only fetch the contents of a single group's
>> spool file. With the patch, `2 g' fetching for the first group
>> inhibits fetching from other spool files.  For those users who do not
>> use directory as a mail source, this patch would not be a problem.

Simon> Ouch. This patch add the hack to the scanning part too, but
Simon> this is getting quite messy. It wouldn't work if `mail-sources'
Simon> can be a group parameter (or non-global in any way), but I'm
Simon> not sure if it can be, so...  Whatdoyouthink, should I apply
Simon> this or remove my earlier speed-up patch?

[...]

Speed up patch is good for most cases, at least people like the
speed.

Actually, Lars had put some codes to avoid multiply fetching a mail
source, see `nnmail-fetched-sources'.  So, the message "%s: Reading
incoming mail (no new mail)...done" does not mean that Gnus fetches
those mail sources again, while "%s: Reading incoming mail from %s..." 
means it fetches. But the overhead, such as nnmail-active and
nnmail-cache-open, may cost some.  So, I am thinking to speed up
nnmail-get-new-mail by checking sources before actually doing
something, so that Gnus still scan backend without fetch mail for most
cases (except directory).  See the attached patch. What do you think?

Now, let's return to the beginning of this thread. I am not sure
whether Bruce and Michael use directory. If not, Simon's speed up
patch wouldn't cause trouble generally.

-- 
Shenghuo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnmail-5.43zsh.diff --]
[-- Type: text/x-patch, Size: 2741 bytes --]

jIndex: nnmail.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnmail.el,v
retrieving revision 5.43
diff -u -r5.43 nnmail.el
--- nnmail.el	1999/12/04 01:11:10	5.43
+++ nnmail.el	1999/12/05 16:25:29
@@ -1393,6 +1393,7 @@
   (let* ((sources (or mail-sources
 		      (if (listp nnmail-spool-file) nnmail-spool-file
 			(list nnmail-spool-file))))
+	 fetching-sources
 	 (group-in group)
 	 (i 0)
 	 (new 0)
@@ -1400,14 +1401,6 @@
 	 incoming incomings source)
     (when (and (nnmail-get-value "%s-get-new-mail" method)
 	       nnmail-spool-file)
-      ;; We first activate all the groups.
-      (nnmail-activate method)
-      ;; Allow the user to hook.
-      (run-hooks 'nnmail-pre-get-new-mail-hook)
-      ;; Open the message-id cache.
-      (nnmail-cache-open)
-      ;; The we go through all the existing mail source specification
-      ;; and fetch the mail from each.
       (while (setq source (pop sources))
 	;; Be compatible with old values.
 	(cond
@@ -1439,21 +1432,31 @@
 	(when nnmail-fetched-sources
 	  (if (member source nnmail-fetched-sources)
 	      (setq source nil)
-	    (push source nnmail-fetched-sources)))
-	(when source
-	  (nnheader-message 4 "%s: Reading incoming mail from %s..."
-			    method (car source))
-	  (when (setq new
-		      (mail-source-fetch
-		       source
-		       `(lambda (file orig-file)
-			  (nnmail-split-incoming
-			   file ',(intern (format "%s-save-mail" method))
-			   ',spool-func
-			   (nnmail-get-split-group orig-file source)
-			   ',(intern (format "%s-active-number" method))))))
-	    (incf total new)
-	    (incf i))))
+	    (push source nnmail-fetched-sources)
+	    (push source fetching-sources)))))
+    (when fetching-sources
+      ;; We first activate all the groups.
+      (nnmail-activate method)
+      ;; Allow the user to hook.
+      (run-hooks 'nnmail-pre-get-new-mail-hook)
+      ;; Open the message-id cache.
+      (nnmail-cache-open)
+      ;; The we go through all the existing mail source specification
+      ;; and fetch the mail from each.
+      (while (setq source (pop fetching-sources))
+	(nnheader-message 4 "%s: Reading incoming mail from %s..."
+			  method (car source))
+	(when (setq new
+		    (mail-source-fetch
+		     source
+		     `(lambda (file orig-file)
+			(nnmail-split-incoming
+			 file ',(intern (format "%s-save-mail" method))
+			 ',spool-func
+			 (nnmail-get-split-group orig-file source)
+			 ',(intern (format "%s-active-number" method))))))
+	  (incf total new)
+	  (incf i)))
       ;; If we did indeed read any incoming spools, we save all info.
       (if (zerop total)
 	  (nnheader-message 4 "%s: Reading incoming mail (no new mail)...done"

Re: gnus-group-get-new-news broken again

[Michael Harnois]

On 05 Dec 1999 11:24:17 -0500, Shenghuo ZHU <zsh@cs.rochester.edu> said:

    > Now, let's return to the beginning of this thread. I am not sure
    > whether Bruce and Michael use directory. If not, Simon's speed
    > up patch wouldn't cause trouble generally.

I do, because I use procmail for preprocessing.

-- 
Michael D. Harnois, Redeemer Lutheran Church, Washburn, IA 
mharnois@willinet.net                      aa0bt@aa0bt.ampr.org 
 If they can get you asking the wrong questions, 
 they don't have to worry about the answers.
      -- Thomas Pynchon

Re: gnus-group-get-new-news broken again

[Bruce Stephens]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

[...]

> Now, let's return to the beginning of this thread. I am not sure
> whether Bruce and Michael use directory. If not, Simon's speed up
> patch wouldn't cause trouble generally.

I do, and if I remove the patch (using the patch Simon just posted),
the problem goes away.  So that suggests that the patch doesn't work
with directory.