From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/66372 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: split-fancy and gnus-registry confusion Date: Wed, 27 Feb 2008 16:40:29 -0600 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <86y796owf6.fsf@lifelogs.com> References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1204151968 3548 80.91.229.12 (27 Feb 2008 22:39:28 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 27 Feb 2008 22:39:28 +0000 (UTC) Cc: ding@gnus.org To: Norman Walsh , Jake Colman Original-X-From: ding-owner+M14863@lists.math.uh.edu Wed Feb 27 23:39:52 2008 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1JUUwK-0006v1-5G for ding-account@gmane.org; Wed, 27 Feb 2008 23:39:52 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1JUUvc-0002cD-1R; Wed, 27 Feb 2008 16:39:08 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JUUva-0002bt-98 for ding@lists.math.uh.edu; Wed, 27 Feb 2008 16:39:06 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1JUUvT-0006tV-B6 for ding@lists.math.uh.edu; Wed, 27 Feb 2008 16:39:06 -0600 Original-Received: from mail.blockstar.com ([170.224.69.95]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1JUUvY-0007Eb-00 for ; Wed, 27 Feb 2008 23:39:04 +0100 Original-Received: from tzlatanov-ubuntu-desktop.jumptrading.com (unknown [38.98.147.130]) by mail.blockstar.com (Postfix) with ESMTP id ED4363E8081; Wed, 27 Feb 2008 14:39:32 -0800 (PST) X-Hashcash: 1:20:080227:colman@ppllc.com::At5Ubf4rLfZKxykL:08Bdo X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" X-Hashcash: 1:20:080227:ding@gnus.org::G5rya7drcgEx6H1f:00000cbM X-Hashcash: 1:20:080227:ndw@nwalsh.com::KefJrU43t/paODMU:0009BON In-Reply-To: (Norman Walsh's message of "Thu, 21 Feb 2008 12:09:24 -0500") User-Agent: Gnus/5.110007 (No Gnus v0.7) Emacs/23.0.50 (gnu/linux) X-Spam-Score: -2.6 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:66372 Archived-At: --=-=-= On Thu, 21 Feb 2008 12:09:24 -0500 Norman Walsh wrote: NW> I'm completely confused by something. NW> I'm using nnmail-split-fancy and gnus-registry-split-fancy-with-parent: ... On Tue, 19 Feb 2008 18:41:19 -0500 Jake Colman wrote: Jake> I just started seeing this come up when getting messages from my IMAP Jake> server: Jake> "gnus-registry-split-fancy-with-parent: too many extra matches for " Jake> This just started today and I'm seeing several messages like this. Jake> What's going on?! The attached patch against today's CVS will give you proper logging, plus I reworked the function gnus-registry-split-fancy-with-parent to be much nicer. Make sure you have gnus-verbose turned up to 9, then try this out and let me know if the logging helped, and if things generally worked OK. I am reworking both logging and splitting, so comment on both please. Anyone else interested in testing, please give it a shot. Ted --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=registry.logging.patch Index: gnus-registry.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-registry.el,v retrieving revision 7.47 diff -a -u -r7.47 gnus-registry.el --- gnus-registry.el 26 Feb 2008 18:16:38 -0000 7.47 +++ gnus-registry.el 27 Feb 2008 22:34:59 -0000 @@ -425,119 +425,150 @@ For a message to be split, it looks for the parent message in the References or In-Reply-To header and then looks in the registry to see which group that message was put in. This group is -returned, unless it matches one of the entries in -gnus-registry-unfollowed-groups or -nnmail-split-fancy-with-parent-ignore-groups. +returned, unless `gnus-registry-follow-group-p' return nil for +that group. See the Info node `(gnus)Fancy Mail Splitting' for more details." - (let* ((refstr (or (message-fetch-field "references") "")) ; guarantee string - (reply-to (message-fetch-field "in-reply-to")) ; grab reply-to + (let* ((refstr (or (message-fetch-field "references") "")) ; guaranteed + (reply-to (message-fetch-field "in-reply-to")) ; may be nil ;; now, if reply-to is valid, append it to the References (refstr (if reply-to (concat refstr " " reply-to) refstr)) - (nnmail-split-fancy-with-parent-ignore-groups - (if (listp nnmail-split-fancy-with-parent-ignore-groups) - nnmail-split-fancy-with-parent-ignore-groups - (list nnmail-split-fancy-with-parent-ignore-groups))) - res) - ;; the references string must be valid and parse to valid references - (if (and refstr (gnus-extract-references refstr)) - (dolist (reference (nreverse (gnus-extract-references refstr))) - (setq res (or (gnus-registry-fetch-group reference) res)) - (when (or (gnus-registry-grep-in-list - res - gnus-registry-unfollowed-groups) - (gnus-registry-grep-in-list - res - nnmail-split-fancy-with-parent-ignore-groups)) - (setq res nil))) - - ;; else: there were no references, now try the extra tracking - (let ((sender (gnus-string-remove-all-properties(message-fetch-field "from"))) - (subject (gnus-string-remove-all-properties - (gnus-registry-simplify-subject - (message-fetch-field "subject")))) - (single-match t)) - (when (and single-match - (gnus-registry-track-sender-p) - sender) - (maphash - (lambda (key value) - (let ((this-sender (cdr - (gnus-registry-fetch-extra key 'sender)))) - (when (and single-match - this-sender - (equal sender this-sender)) - ;; too many matches, bail - (unless (equal res (gnus-registry-fetch-group key)) - (setq single-match nil)) - (setq res (gnus-registry-fetch-group key)) - (when (and sender res) - (gnus-message - ;; raise level of messaging if gnus-registry-track-extra - (if gnus-registry-track-extra 7 9) - "%s (extra tracking) traced sender %s to group %s" - "gnus-registry-split-fancy-with-parent" - sender - res))))) - gnus-registry-hashtb)) - (when (and single-match - (gnus-registry-track-subject-p) - subject - (< gnus-registry-minimum-subject-length (length subject))) - (maphash - (lambda (key value) - (let ((this-subject (cdr - (gnus-registry-fetch-extra key 'subject)))) - (when (and single-match - this-subject - (equal subject this-subject)) - ;; too many matches, bail - (unless (equal res (gnus-registry-fetch-group key)) - (setq single-match nil)) - (setq res (gnus-registry-fetch-group key)) - (when (and subject res) - (gnus-message - ;; raise level of messaging if gnus-registry-track-extra - (if gnus-registry-track-extra 7 9) - "%s (extra tracking) traced subject %s to group %s" - "gnus-registry-split-fancy-with-parent" - subject - res))))) - gnus-registry-hashtb)) - (unless single-match - (gnus-message - 3 - "gnus-registry-split-fancy-with-parent: too many extra matches for %s" - refstr) - (setq res nil)))) - (when (and refstr res) - (gnus-message - 5 - "gnus-registry-split-fancy-with-parent traced %s to group %s" - refstr res)) - - (when (and res gnus-registry-use-long-group-names) - (let ((m1 (gnus-find-method-for-group res)) - (m2 (or gnus-command-method - (gnus-find-method-for-group gnus-newsgroup-name))) - (short-res (gnus-group-short-name res))) - (if (gnus-methods-equal-p m1 m2) - (progn + ;; these may not be used, but the code is cleaner having them up here + (sender (gnus-string-remove-all-properties + (message-fetch-field "from"))) + (subject (gnus-string-remove-all-properties + (gnus-registry-simplify-subject + (message-fetch-field "subject")))) + + (nnmail-split-fancy-with-parent-ignore-groups + (if (listp nnmail-split-fancy-with-parent-ignore-groups) + nnmail-split-fancy-with-parent-ignore-groups + (list nnmail-split-fancy-with-parent-ignore-groups))) + (log-agent "gnus-registry-split-fancy-with-parent") + found) + + ;; this is a big if-else statement. it uses + ;; gnus-registry-post-process-groups to filter the results after + ;; every step. + (cond + ;; the references string must be valid and parse to valid references + ((and refstr (gnus-extract-references refstr)) + (dolist (reference (nreverse (gnus-extract-references refstr))) + (gnus-message + 9 + "%s is looking for matches for reference %s from [%s]" + log-agent reference refstr) + (dolist (group (gnus-registry-fetch-groups reference)) + (when (and group (gnus-registry-follow-group-p group)) (gnus-message - 9 - "gnus-registry-split-fancy-with-parent stripped group %s to %s" - res - short-res) - (setq res short-res)) - ;; else... + 7 + "%s traced the reference %s from [%s] to group %s" + log-agent reference refstr group) + (push group found)))) + ;; filter the found groups and return them + (setq found (gnus-registry-post-process-groups "references" refstr found))) + + ;; else: there were no matches, now try the extra tracking by sender + ((and (gnus-registry-track-sender-p) + sender) + (maphash + (lambda (key value) + (let ((this-sender (cdr + (gnus-registry-fetch-extra key 'sender))) + matches) + (when (and this-sender + (equal sender this-sender)) + (setq found (append (gnus-registry-fetch-groups key) found)) + (push key matches) + (gnus-message + ;; raise level of messaging if gnus-registry-track-extra + (if gnus-registry-track-extra 7 9) + "%s (extra tracking) traced sender %s to groups %s (keys %s)" + log-agent sender found matches)))) + gnus-registry-hashtb) + ;; filter the found groups and return them + (setq found (gnus-registry-post-process-groups "sender" sender found))) + + ;; else: there were no matches, now try the extra tracking by subject + ((and (gnus-registry-track-subject-p) + subject + (< gnus-registry-minimum-subject-length (length subject))) + (maphash + (lambda (key value) + (let ((this-subject (cdr + (gnus-registry-fetch-extra key 'subject))) + matches) + (when (and this-subject + (equal subject this-subject)) + (setq found (append (gnus-registry-fetch-groups key) found)) + (push key matches) + (gnus-message + ;; raise level of messaging if gnus-registry-track-extra + (if gnus-registry-track-extra 7 9) + "%s (extra tracking) traced subject %s to groups %s (keys %s)" + log-agent subject found matches)))) + gnus-registry-hashtb) + ;; filter the found groups and return them + (setq found (gnus-registry-post-process-groups "subject" subject found)))))) + +(defun gnus-registry-post-process-groups (mode key groups) + "Modifies GROUPS obtained by searching by MODE for KEY to determine which ones to follow. + +MODE can be 'subject' or 'sender' for example. The KEY is the +value by which MODE was searched. + +Transforms each group name to the equivalent short name. + +Checks if the current Gnus method (from `gnus-command-method' or +from `gnus-newsgroup-name') is the same as the group's method. +This is not possible if gnus-registry-use-long-group-names is +false. Foreign methods are not supported so they are rejected. + +Reduces the list to a single group, or complains if that's not +possible." + (let ((log-agent "gnus-registry-post-process-group") + out) + (if gnus-registry-use-long-group-names + (dolist (group groups) + (let ((m1 (gnus-find-method-for-group group)) + (m2 (or gnus-command-method + (gnus-find-method-for-group gnus-newsgroup-name))) + (short-name (gnus-group-short-name group))) + (if (gnus-methods-equal-p m1 m2) + (progn + ;; this is REALLY just for debugging + (gnus-message + 10 + "%s stripped group %s to %s" + log-agent group short-name) + (unless (member short-name out) + (push short-name out))) + ;; else... + (gnus-message + 7 + "%s ignored foreign group %s" + log-agent group)))) + (setq out groups)) + (when (cdr-safe out) (gnus-message - 7 - "gnus-registry-split-fancy-with-parent ignored foreign group %s" - res) - (setq res nil)))) - res)) + 5 + "%s: too many extra matches (%s) for %s %s. Returning none." + log-agent out mode key) + (setq out nil)) + out)) + +(defun gnus-registry-follow-group-p (group) + "Determines if a group name should be followed. +Consults `gnus-registry-unfollowed-groups' and +`nnmail-split-fancy-with-parent-ignore-groups'." + (not (or (gnus-registry-grep-in-list + group + gnus-registry-unfollowed-groups) + (gnus-registry-grep-in-list + group + nnmail-split-fancy-with-parent-ignore-groups)))) (defun gnus-registry-wash-for-keywords (&optional force) (interactive) --=-=-=--