ahh... it's the problem noted a few months back

ahh... it's the problem noted a few months back

[Randal L. Schwartz]

when I say "3 g" in the Group buffer, I get a message for *every*
nnml: group in my active.  Ugh.  This is annoying.  Make it stop,
mommy.  qgnus didn't do that.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: ahh... it's the problem noted a few months back

[David S. Goldberg]

Yes, this is the one thing I really dislike about the new mail-source
stuff.  I guess it works OK for people who use gnus' internal
splitting since the only people I see complaining use something
external (e.g. procmail or whatever).  The only workaround I know of
is to set those nnml groups that actually expect to receive new mail
to a lower level and then call 'g' with that level.  This works, but
I've found other reasons for it to be less than satisfying; in
particular the only way I can find to activate all the other groups
(so I can get completion on their names, if nothing else) is to call
'g' with the higher level and that takes a long time due to the
attempt to fetch new mail that is guaranteed not to be there.  It also
means that I've got a difficult choice of whether to put those groups
at a level higher or lower than nntp groups.  I've chosen lower which
means when I want to read nntp groups (via 4 g in my case) I have to
wait for nnml to try to fetch new mail for all the higher level
groups.  I live with this because I don't read nntp groups all that
often.  I think someone once suggested creating a separate nnml server
for the non-mail-receiving groups, but that just seems like too much
of a hassle.
-- 
Dave Goldberg
Post: The Mitre Corporation\MS B325\202 Burlington Rd.\Bedford, MA 01730
Phone: 781-271-3887
Email: dsg@mitre.org

Re: ahh... it's the problem noted a few months back

[Steinar Bang]

>>>>> dsg@mitre.org (David S. Goldberg):

> Yes, this is the one thing I really dislike about the new
> mail-source stuff.  I guess it works OK for people who use gnus'
> internal splitting since the only people I see complaining use
> something external (e.g. procmail or whatever).

It seems to be working OK for the nnimap users, with splitting on the
server side.  At least I haven't seen any complaints about this
behaviour on the nnimap list.

Re: ahh... it's the problem noted a few months back

[Kai Großjohann]

Steinar Bang <sb@metis.no> writes:

> It seems to be working OK for the nnimap users, with splitting on the
> server side.  At least I haven't seen any complaints about this
> behaviour on the nnimap list.

nnimap does not use the normal splitting at all.  There, the messages
are already in the correct group when Gnus sees them.

With procmail splitting, Gnus still has to read a file
~/spoo/foo.bar.incoming in order to put all mails in it into the group
foo.bar. 

kai
-- 
This gubblick contains many nonsklarkish English flutzpahs,
but the overall pluggandisp can be glorked from context. -- David Moser

Re: ahh... it's the problem noted a few months back

[David S. Goldberg]

I don't do server side splitting in nnimap (I use nnimap-split-rules)
but I do notice that even for nnimap, all groups subscribed at a
particular level get checked (e.g. I see the "updating info for ..."
message, even for groups that never get mails split to them).  I don't
have that many nnimap groups (yet) so I find that to be less of a
bother than what's going on with nnml.  I'd really like the old, pre
mail-sources, behavior (e.g. nnmail-use-procmail or whatever it was
called) in which the mail-sources directory (or files, whatver) was
checked and then only those groups for which new mail had been
incorporated got updated.  The current behavior, in which each group
is checked to see if a mail source is available takes a long time when
the number of groups gets large.
-- 
Dave Goldberg
Post: The Mitre Corporation\MS B325\202 Burlington Rd.\Bedford, MA 01730
Phone: 781-271-3887
Email: dsg@mitre.org

Re: ahh... it's the problem noted a few months back

[Kai Großjohann]

David S. Goldberg <dsg@mitre.org> writes:

> I don't do server side splitting in nnimap (I use nnimap-split-rules)
> but I do notice that even for nnimap, all groups subscribed at a
> particular level get checked (e.g. I see the "updating info for ..."
> message, even for groups that never get mails split to them).

That's bad.  If you can make it show `checking mailbox foo', then this
will be about an order of magnitude faster than `updating info for
foo'.  Really!  (Well, maybe it's only a factor of five.)

I used to have `updating info for...' as well, but I managed to make
it show `checking mailbox ...' by fiddling with the select method.
And with the level, I think.

M-x gnus RET would show `checking', whereas M-x gnus-no-server RET and
C-u 4 2 M-x gnus RET would show `updating'.  I think.

Simon knows more, for sure.

kai
-- 
This gubblick contains many nonsklarkish English flutzpahs,
but the overall pluggandisp can be glorked from context. -- David Moser

Re: ahh... it's the problem noted a few months back

[Tibor Simko]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 1283 bytes --]

>>>>> "DSG" == David S Goldberg <dsg@mitre.org> writes:

    DSG> I do notice that even for nnimap, all groups subscribed at a
    DSG> particular level get checked (e.g. I see the "updating info
    DSG> for ..."  message, even for groups that never get mails split
    DSG> to them).  

>>>>> "KG" == Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

    KG> I used to have `updating info for...' as well, but I managed
    KG> to make it show `checking mailbox ...' by fiddling with the
    KG> select method.  And with the level, I think. [...]  Simon
    KG> knows more, for sure.

Simon has suggested this workaround: 

>>>>> "SJ" == Simon Josefsson <jas@pdc.kth.se> writes:

    SJ> This turned out to be due to Gnus believing the method was
    SJ> something like
    SJ>
    SJ> (nnimap "")
    SJ>
    SJ> when you did have
    SJ>
    SJ> (nnimap "" (nnimap-address "foo"))
    SJ>
    SJ> in your `gnus-secondary-select-method'.
    SJ>
    SJ> As a workaround, edit the group info with `G E' in the group
    SJ> menu to turn the first method into the latter.

Worked fine for me, but only for `g'.  It did not work for `2 g' style
of getting new email.  Simon said something like that it is a
"feature" difficult to get rid of, unfortunately.

cheers,

-TS

Re: ahh... it's the problem noted a few months back

[David S. Goldberg]

>>>>> Tibor Simko writes:
> Simon has suggested this workaround:
>[...]

Nope, that wasn't it in my case.  The select method in the group
properties looks right.  In my case it's because I've always preferred
to start with gnus-no-server and have since gone to setting
gnus-group-use-permanent-levels.

Thanks,
-- 
Dave Goldberg
Post: The Mitre Corporation\MS B325\202 Burlington Rd.\Bedford, MA 01730
Phone: 781-271-3887
Email: dsg@mitre.org

Re: ahh... it's the problem noted a few months back

[Simon Josefsson]

David S. Goldberg <dsg@mitre.org> writes:

> Nope, that wasn't it in my case.  The select method in the group
> properties looks right.  In my case it's because I've always preferred
> to start with gnus-no-server and have since gone to setting
> gnus-group-use-permanent-levels.

Yes, like Tibor mentioned my hint doesn't help the `2 g' cases. Some
time ago I traced this into `gnus-get-unread-articles', it falls
through to the 'update-info' call below. It shouldn't. I don't
understand the purpose of this code, so it's hard to come up with a
fix. Perhaps another cond case in the foreign-level could solve this,
but I'm only guessing here...

(defun gnus-get-unread-articles (&optional level)
  (let* (...
	 (level (or level gnus-activate-level (1+ gnus-level-subscribed)))
	 (foreign-level
	  (min
	   (cond ((and gnus-activate-foreign-newsgroups
		       (not (numberp gnus-activate-foreign-newsgroups)))
		  (1+ gnus-level-subscribed))
		 ((numberp gnus-activate-foreign-newsgroups)
		  gnus-activate-foreign-newsgroups)
		 (t 0))
	   level))
...
	  ;; These groups are foreign.  Check the level.
	  (when (<= (gnus-info-level info) foreign-level)
	    (setq active (gnus-activate-group group 'scan))
	    ;; Let the Gnus agent save the active file.
	    (when (and gnus-agent gnus-plugged active)
	      (gnus-agent-save-group-info
	       method (gnus-group-real-name group) active))
	    (unless (inline (gnus-virtual-group-p group))
	      (inline (gnus-close-group group)))
	    (when (fboundp (intern (concat (symbol-name (car method))
					   "-request-update-info")))
	      (inline (gnus-request-update-info info method))))

Re: ahh... it's the problem noted a few months back

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@pdc.kth.se> writes:

> Yes, like Tibor mentioned my hint doesn't help the `2 g' cases. Some
> time ago I traced this into `gnus-get-unread-articles', it falls
> through to the 'update-info' call below. It shouldn't. I don't
> understand the purpose of this code, so it's hard to come up with a
> fix. Perhaps another cond case in the foreign-level could solve this,
> but I'm only guessing here...

Well, the purpose of the code is to update all the groups that should
be updated.  :-)  All non-native, non-secondary groups get checked
individually.

But when Gnus mistakenly takes some groups as foreign instead of
secondary, you get all these gazillion "Reading new mail..."
messages.  So the question is -- why doesn't Gnus realize that these
groups are secondary?

I think.  Could somebody who sees this edebug through this function
and see which `(gnus-activate-group group 'scan)' is doing the dirty
deed?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ahh... it's the problem noted a few months back

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Well, the purpose of the code is to update all the groups that should
> be updated.  :-)  All non-native, non-secondary groups get checked
> individually.
> 
> But when Gnus mistakenly takes some groups as foreign instead of
> secondary, you get all these gazillion "Reading new mail..."
> messages.  So the question is -- why doesn't Gnus realize that these
> groups are secondary?
> 
> I think.  Could somebody who sees this edebug through this function
> and see which `(gnus-activate-group group 'scan)' is doing the dirty
> deed?

I traced this in more detail now, and I think this is what's
happening. When one type '<number> g' Gnus don't call
`gnus-read-active-file' but let `gnus-get-unread-articles' get the
active info:

        (cond
        ...
	 ;; Activate groups.
	 ((and (not active) (not gnus-read-active-file))
	  (setq active (gnus-activate-group group 'scan))
	  (inline (gnus-close-group group)))))

Aside from starting the splitting procedure once for all requested
groups, entering/closing a group is not fast in IMAP.

Idea: If the backend support `nnfoo-retrieve-groups' it would be
faster to request ONE scan, and use `gnus-retrieve-groups' to gather
active info on the groups that fall through the cond case. If the
backend doesn't support `nnfoo-retrieve-groups' I'm not sure which is
faster, so in that case it might be safer to leave it as is.

Re: ahh... it's the problem noted a few months back

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@pdc.kth.se> writes:

> Idea: If the backend support `nnfoo-retrieve-groups' it would be
> faster to request ONE scan, and use `gnus-retrieve-groups' to gather
> active info on the groups that fall through the cond case. If the
> backend doesn't support `nnfoo-retrieve-groups' I'm not sure which is
> faster, so in that case it might be safer to leave it as is.

Yes, that sounds like a good idea.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ahh... it's the problem noted a few months back

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> > Idea: If the backend support `nnfoo-retrieve-groups' it would be
> > faster to request ONE scan, and use `gnus-retrieve-groups' to gather
> > active info on the groups that fall through the cond case. If the
> > backend doesn't support `nnfoo-retrieve-groups' I'm not sure which is
> > faster, so in that case it might be safer to leave it as is.
> 
> Yes, that sounds like a good idea.

This patch implement it. The active file reading stuff was copied from
`gnus-read-active-file-1', perhaps it should be isolated in a separate
function.

Commit it?

1999-11-12  Simon Josefsson  <jas@pdc.kth.se>

	* gnus-start.el (gnus-get-unread-articles): Use
	nnfoo-retrieve-groups to find new news, if available.

Index: gnus-start.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/gnus-start.el,v
retrieving revision 5.38
diff -w -u -r5.38 gnus-start.el
--- gnus-start.el	1999/11/12 19:34:16	5.38
+++ gnus-start.el	1999/11/12 22:18:49
@@ -1498,7 +1498,7 @@
 		  gnus-activate-foreign-newsgroups)
 		 (t 0))
 	   level))
-	 info group active method)
+	 info group active method retrievegroups)
     (gnus-message 5 "Checking new news...")
 
     (while newsrc
@@ -1535,8 +1535,15 @@
 	  (setq active 'ignore))
 	 ;; Activate groups.
 	 ((not gnus-read-active-file)
+	  (if (gnus-check-backend-function 'retrieve-groups group)
+	      ;; if server support nnfoo-retrieve-groups we push
+	      ;; the group onto retrievegroups for later checking
+	      (if (assoc method retrievegroups)
+		  (setcdr (assoc method retrievegroups)
+			  (cons group (cdr (assoc method retrievegroups))))
+		(push (list method group) retrievegroups))
 	  (setq active (gnus-activate-group group 'scan))
-	  (inline (gnus-close-group group)))))
+	    (inline (gnus-close-group group))))))
 
       ;; Get the number of unread articles in the group.
       (cond
@@ -1550,6 +1557,37 @@
 	;; unread articles and stuff.
 	(gnus-set-active group nil)
 	(setcar (gnus-gethash group gnus-newsrc-hashtb) t))))
+
+    ;; when server supports nnfoo-retrieve-groups we use it to find
+    ;; out if there's any new news in groups specified by retrievegroups
+    (while retrievegroups
+      (let* ((mg (pop retrievegroups))
+	     (method (car mg))
+	     (groups (cdr mg))
+	     list-type)
+	(gnus-request-scan nil method)
+	(save-excursion
+	  (set-buffer nntp-server-buffer)
+	  (gnus-check-server method)
+	  (setq list-type (gnus-retrieve-groups groups method))
+	  (cond ((not list-type)
+		 (gnus-error
+		  1.2 "Cannot read partial active file from %s server."
+		  (car method)))
+		((eq list-type 'active)
+		 (gnus-active-to-gnus-format method gnus-active-hashtb nil t))
+		(t
+		 (gnus-groups-to-gnus-format method gnus-active-hashtb t))))
+	(dolist (group groups)
+	  (cond
+	   ((setq active (gnus-active (gnus-info-group
+				       (setq info (gnus-get-info group)))))
+	    (inline (gnus-get-unread-articles-in-group info active t)))
+	   (t
+	    ;; The group couldn't be reached, so we nix out the number of
+	    ;; unread articles and stuff.
+	    (gnus-set-active group nil)
+	    (setcar (gnus-gethash group gnus-newsrc-hashtb) t))))))
 
     (gnus-message 5 "Checking new news...done")))
 

Re: ahh... it's the problem noted a few months back

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@pdc.kth.se> writes:

> This patch implement it. The active file reading stuff was copied from
> `gnus-read-active-file-1', perhaps it should be isolated in a separate
> function.

Yes, probably.

> Commit it?

Please do.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ahh... it's the problem noted a few months back

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> > This patch implement it. The active file reading stuff was copied from
> > `gnus-read-active-file-1', perhaps it should be isolated in a separate
> > function.
> 
> Yes, probably.

I created `gnus-read-active-file-2'.

> > Commit it?
> 
> Please do.

Done.  Aah... `1 g' is really fast now. At last.

1999-11-13  Simon Josefsson  <jas@pdc.kth.se>

	* gnus-start.el (gnus-get-unread-articles): Use
	nnfoo-retrieve-groups to find new news, if available.
	(gnus-read-active-file-2): New function.
	(gnus-get-unread-articles): Use it.
	(gnus-read-active-file-1): Ditto.

Re: ahh... it's the problem noted a few months back

[Randal L. Schwartz]

>>>>> "Simon" == Simon Josefsson <jas@pdc.kth.se> writes:

>> 
>> Please do.

Simon> Done.  Aah... `1 g' is really fast now. At last.

Oooh... Oooh!  does that mean I just need to wait for Mr. Gnus to push
a new version...

I can't wait!

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: ahh... it's the problem noted a few months back

[Tibor Simko]

>>>>> "SJ" == Simon Josefsson <jas@pdc.kth.se> writes:

    SJ> Done.  Aah... `1 g' is really fast now. At last.

It really is!  Kudos and thanks,

-TS