new NOV stuff in p0.40

new NOV stuff in p0.40

[Karl Kleinpaste]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3354 bytes --]

I've been trying out the new 0.40, and I'm confused.  I looked through
the code to see what's being done about .overview, but I'm failing to
get a grip on what's happening.

First, why are there 2 new variables for NOV control,
gnus-extra-headers and nnmail-extra-headers?  When might one
conceivably want them to be different?  Do I at least understand
correctly that nnmail-e-h is used when *storing* mail messages while
gnus-e-h is used when *receiving* XOVER response?

I'm using:

(setq gnus-extra-headers '(Keywords To Newsgroups)
      nnmail-extra-headers '(Keywords To Newsgroups)
      gnus-ignored-from-addresses "karl@charcoal\\.com\\|karl@jprc\\.com")

Second, I have set gnus-ignored-from-addresses to several variations
on my addresses, as you see above, but yet *Summary* buffers are still
showing myself rather than where the messages were going.  I'm testing
by sending mail to myself under aliases, so the message ends up both
Gcc'd and delivered into one nnml group.  Sample .overview lines from
such a pair:

327	NOV tst	Karl Kleinpaste <karl@charcoal.com>	25 Oct 1998 18:37:14 -500	<vxkaf2kfc2d.fsf@mesquite.charcoal.com>		5	1	Xref: mesquite.charcoal.com personal.karl:327	To: usenet@charcoal.com	Keywords: basic,testing	
328	NOV tst	Karl Kleinpaste <karl@charcoal.com>	25 Oct 1998 18:37:14 -500	<vxkaf2kfc2d.fsf@mesquite.charcoal.com>		6	2	Xref: mesquite.charcoal.com personal.karl:328	To: usenet@charcoal.com	Keywords: basic,testing	

And *Summary* shows:

  :                          : NOV tst
R +    [   1: Karl Kleinpaste     ] 
  +    [   2: Karl Kleinpaste     ] 

How do I get *Summary* to stop showing "me" and start showing
"usenet@..."?  I thought the setq above would give it to me, but
apparently not so.  I'm sorely tempted to re-deploy last week's little
patch for nnheader.el which led to Lars' inspiration for NOV re-work,
but I'd rather get it "right" using the new tools.

Third, of far more importance to me is the fact that I've lost
something else: I did a bit of hackery a couple years ago to give
myself general scorability on Keywords headers.  My INN installations
include server-side auto-generation of Keywords by grinding article
bodies in search of computationally interesting words, creating
Keywords content on the fly for overviews, whether there's such a
header in the article or not.  My overviews are ~20% larger than
without this capability, with examples now looking like this:

11676	slrn, can't disable color mode	mjc@c1000907-b.sttls1.wa.home.com (Mark Crosland)	Sun, 25 Oct 1998 22:43:03 GMT	<slrn7377gn.7vk.mjc@c1000907-b.sttls1.wa.home.com>		760	5	Xref: news.jprc.com news.software.readers:11676 	Keywords: ,ÿ,slrn,thought,colors,coming,newsrc,color,mode,seem,stop,use

This lets me do some extremely useful content-scoring without
requiring full-body searches, using a few additional entries in the
scoring keymaps and -- up to now -- a couple new macros like
mail-header-keywords (which no longer work, of course, in this brave
new world of generalized NOV handling).  Example use: In
news.software.readers, I score up "gnus" in keywords highly, which
catches a surprisingly large number of references to Our Favorite
Newsreader.  Is there any way to (re)gain this kind of score
capability in a generalized manner, without hacking up per-header
functions?

dazed and confused,
--karl

Re: new NOV stuff in p0.40

[Lars Magne Ingebrigtsen]

Karl Kleinpaste <karl@jprc.com> writes:

> First, why are there 2 new variables for NOV control,
> gnus-extra-headers and nnmail-extra-headers?  When might one
> conceivably want them to be different?

If one is interested only in seeing the To header in the nnfolder
groups, then setting the nnmail-extra-headers variabel would be
counter-productive, since nnfolder doesn't do NOV headers.

> Do I at least understand correctly that nnmail-e-h is used when
> *storing* mail messages

Yes.

> while gnus-e-h is used when *receiving* XOVER response?

Almost.  It's used when receiving headers, whether they be HEAD
headers or NOV headers.

>       gnus-ignored-from-addresses "karl@charcoal\\.com\\|karl@jprc\\.com")
> 
> Second, I have set gnus-ignored-from-addresses to several variations
> on my addresses, as you see above, but yet *Summary* buffers are still
> showing myself rather than where the messages were going.

You have to use the %f spec to get to-or-newsgroup-or-from:

(setq gnus-summary-line-format
      "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n")

> Example use: In news.software.readers, I score up "gnus" in keywords
> highly, which catches a surprisingly large number of references to
> Our Favorite Newsreader.  Is there any way to (re)gain this kind of
> score capability in a generalized manner, without hacking up
> per-header functions?

Adding this should be easy, but what should the score file syntax be?

For those who haven't followed the ChangeLogs, Gnus header vectors now
have an additional slot, which contains an alist of header/value
pairs.  This slot is called the "extra" slot.

To score on From we have

("from"
 ("Lars" -2000 nil s))

How about

("extra/keywords"
 ("gnus" 200 nil s))

That is, elements that are strings that begin with "extra" refer to
this slot, and the part after the slash refers to the header?

Or perhaps

("extra"
 ("keywords"
  ("gnus" 200 nil s))
 ("to"
  ("Lars" -200 nil s)))

I think this might actually be easier to do.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new NOV stuff in p0.40

[Karl Kleinpaste]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> If one is interested only in seeing the To header in the nnfolder
> groups, then setting the nnmail-extra-headers variabel would be
> counter-productive, since nnfolder doesn't do NOV headers.

OK.

>> while gnus-e-h is used when *receiving* XOVER response?

> Almost.  It's used when receiving headers, whether they be HEAD
> headers or NOV headers.

OK again.

> You have to use the %f spec to get to-or-newsgroup-or-from:
> (setq gnus-summary-line-format
>       "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n")

Ah, that's what I missed.  I hack my summary format, using
gnus-select-group-hook, and didn't notice that.  Thanx for the clue.

> Adding this should be easy, but what should the score file syntax be?

I believe I favor the 2nd suggested form, using sublists under "extra".

Since you consider this easy :-), I believe I'll wait a day or three
to see if you come up with something; if not, I'll see if I can get my
elisp karma worked up to produce something sensible.

--karl

Re: new NOV stuff in p0.40

[Karl Kleinpaste]

>> You have to use the %f spec to get to-or-newsgroup-or-from:
>> (setq gnus-summary-line-format
>>       "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n")

> Ah, that's what I missed.  I hack my summary format, using
> gnus-select-group-hook, and didn't notice that.  Thanx for the clue.

This doesn't appear to be working, or I'm misusing it somehow.

Same scenario as before, self-mailing to an alias.  Gcc'd and
delivered copies' .overview lines:

329	'nother test	Karl Kleinpaste <karl@charcoal.com>	25 Oct 1998 20:49:30 -500	<vxkzpakozx1.fsf@mesquite.charcoal.com>		12	1	Xref: mesquite.charcoal.com personal.karl:329	Keywords: basic,testing,hmmm	To: usenet@charcoal.com	
330	'nother test	Karl Kleinpaste <karl@charcoal.com>	25 Oct 1998 20:49:30 -500	<vxkzpakozx1.fsf@mesquite.charcoal.com>		13	2	Xref: mesquite.charcoal.com personal.karl:330	Keywords: basic,testing,hmmm	To: usenet@charcoal.com	

Asking about the format variable rewards me with:

	`gnus-summary-line-format' is a variable declared in Lisp.

	Value: "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n"
	This value is specific to the current buffer.
	Its default-value is "%U%R%z%I%(%[%4L: %-20,20n%]%) %s\n"

(The doc string needs to be updated to reflect new default, BTW.)

Address string seems to be set properly:

(setq gnus-ignored-from-addresses "karl@charcoal\\.com\\|karl@jprc\\.com")

But it still displays my own name:

  :                          : 'nother test
R +    [   1: Karl Kleinpaste     ] 
R +    [   2: Karl Kleinpaste     ] 

It's hook-selected:

(setq gnus-select-group-hook
      (function
       (lambda ()
	 ;; Ignore From: in summary for certain groups.
	 (cond ((or	(string-match "clari." gnus-newsgroup-name))
		(setq gnus-summary-line-format
		      "%U%R%z%I%(%[%4L: %-6,6n%]%)   %s\n"))
	       ((or (string-match "comp.sources" gnus-newsgroup-name)
		    (string-match "dow-jones" gnus-newsgroup-name)
		    (string-match "comp.internet.net-happening" gnus-newsgroup-name))
		(setq gnus-summary-line-format
		      "%U%R%z%I%(%[%4L%]%)   %s\n"))
	       (t
		(setq gnus-summary-line-format
		      "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n")))	; p0.40+ %f from/to/ng.
	 ;; Put full newsgroup name in summary mode line for nnir.
	 (cond ((string-match "nnir:" gnus-newsgroup-name)
		(setq gnus-summary-mode-line-format
		      "Gnus: %G [%A] %Z"))
	       (t
		(setq gnus-summary-mode-line-format
		      "Gnus: %g [%A] %Z")))
	 )))

Ideas?

Re: new NOV stuff in p0.40

[Lars Magne Ingebrigtsen]

Karl Kleinpaste <karl@jprc.com> writes:

> This doesn't appear to be working, or I'm misusing it somehow.

First, check that you really get the extra headers:

(mail-header-extra (gnus-summary-article-header))

on one of the summary lines that should have To headers.  If you get
nil, then something is wrong.

> It's hook-selected:
> 
> (setq gnus-select-group-hook

And if it's not nil, try to set this to nil, and set the summary line
format variable globally.  (Just to make sure what we're debugging
here.  :-)

Oops.  I just found a bug in the NOV parsing code.  Fix in 0.41.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new NOV stuff in p0.40

[Kai Grossjohann]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > Karl Kleinpaste <karl@jprc.com> writes:
  > 
  > > This doesn't appear to be working, or I'm misusing it somehow.
  > 
  > First, check that you really get the extra headers:
  > 
  > (mail-header-extra (gnus-summary-article-header))

I get nil, even though the nnml .overview file of that group contains
the To header.  I've reported this to Lars before, and he couldn't
reproduce the bug.  Maybe you could all try this out and see what you
get, maybe Lars will be seeing a pattern then?

You need to set gnus-extra-headers and nnmail-extra-headers, I think,
and then new .overview lines should have the extra headers.  Or, you
can type M-x nnml-generate-nov-databases RET to retroactively generate
these extra headers.

kai
-- 
Life is hard and then you die.

Re: new NOV stuff in p0.40

[Lars Magne Ingebrigtsen]

Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

> I get nil, even though the nnml .overview file of that group contains
> the To header.  I've reported this to Lars before, and he couldn't
> reproduce the bug.  Maybe you could all try this out and see what you
> get, maybe Lars will be seeing a pattern then?

Well, like I said at the end of the message, I did find a bug in the
NOV code.

I don't believe in editing beginnings of articles just because I
figure something out while writing them.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new NOV stuff in p0.40

[Kai Grossjohann]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > Well, like I said at the end of the message, I did find a bug in the
  > NOV code.

Sorry, I overlooked that.
  
kai
-- 
Life is hard and then you die.

Re: new NOV stuff in p0.40

[Lars Magne Ingebrigtsen]

Karl Kleinpaste <karl@jprc.com> writes:

> I believe I favor the 2nd suggested form, using sublists under "extra".
> 
> Since you consider this easy :-), I believe I'll wait a day or three
> to see if you come up with something; if not, I'll see if I can get my
> elisp karma worked up to produce something sensible.

I think I'll leave it to you.  :-)

(I think the right approach would be to add a gnus-score-extra method
to `gnus-header-index', and write a new scoring function; perhaps
based on gnus-score-string.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new NOV stuff in p0.40

[Karl Kleinpaste]

Karl Kleinpaste <karl@jprc.com> writes:
>> Since you consider this easy :-), I believe I'll wait a day or three
>> to see if you come up with something; if not, I'll see if I can get my
>> elisp karma worked up to produce something sensible.

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> I think I'll leave it to you.  :-)

Well, you asked for it.

Here's a pile of updated code for p0.40.  Even though you recommended
against it, I thought I could get away with using gnus-score-string
directly, and I think I nearly succeeded. :-)  Unfortunately, there
may be a couple of sacrifices that were made in the attempt.

What's in the patch below:

1. Minor additions to tables in gnus-{cus,sum}.el.
2. A serious bugfix to gnus-nov-parse-extra in gnus-sum.el; this is
probably the bug you noticed a few evenings ago, and the fix is pretty 
obvious; I'd be surprised if it varies at all from what you now have.
3. Lots o' stuff (most of the patch) in gnus-score.el:
- Additions to doc strings & tables.
- Querying user for what extra header to score.
- Passing an extra arg to gnus-summary-score-entry and
  gnus-summary-score-effect.
- Hacking out a specific extra header in gnus-summary-header.
- Handling extra cases in gnus-score-string because the extras aren't
  a string.  There is dark evil here.  I really didn't know what I was 
  getting into until I got to this point and found myself facing what
  appeared to me to be a backward-ordered matching structure; this is
  probably the big argument in favor of not having used
  gnus-score-string in the 1st place...but frankly, I was too far in
  to let it go, because by then I'd slaughtered my working 0.40 tree
  and had to make it work before I could let myself get some sleep. :-)

I'm posting this to the list in the hopes that some folks out there
will be interested in taking a look (and perhaps telling me how
foolish I was for not taking that original suggestion to create
gnus-score-extra rather than retrofitting gnus-score-string the way
I've done here -- my elisp karma is not as strong as perhaps it ought
to be).

What these changes produce is scorefile behavior which uses elements
containing a 5th field, e.g.,
	(("extra"
	  ("gnus" nil nil s "Keywords")))
There is an add'l query during e.g. `I e s p' in which Gnus now asks
what header it is that you want scored.  This does a completing read
against the symbols in gnus-extra-headers, on the theory that you
probably should be concerned only with scoring extra headers of which
you have awareness anyhow.

Please give it a try.  I'm wide open to critique, addition, change, or
utter rewrite.

In a day or so, I'll send a module I've been using for a couple years 
to auto-generate Keywords within Gnus at posting time, too, so that
people who can score on Keywords have something useful to chew on.

--karl

--- gnus-cus.el.orig	Thu Sep 24 14:32:01 1998
+++ gnus-cus.el	Tue Oct 27 16:53:41 1998
@@ -581,12 +581,13 @@
 			 `(checklist :inline t
 				     :greedy t
 				     (gnus-score-string :tag "From")
 				     (gnus-score-string :tag "Subject")
 				     (gnus-score-string :tag "References")
 				     (gnus-score-string :tag "Xref")
+				     (gnus-score-string :tag "Extra")
 				     (gnus-score-string :tag "Message-ID")
 				     (gnus-score-integer :tag "Lines")
 				     (gnus-score-integer :tag "Chars")
 				     (gnus-score-date :tag "Date")
 				     (gnus-score-string :tag "Head"
 							:doc "\
--- gnus-sum.el.orig	Sun Oct 25 17:13:17 1998
+++ gnus-sum.el	Tue Oct 27 18:18:45 1998
@@ -1818,12 +1818,13 @@
       nil
     (let ((headers '(("author" "from" string)
 		     ("subject" "subject" string)
 		     ("article body" "body" string)
 		     ("article head" "head" string)
 		     ("xref" "xref" string)
+		     ("extra header" "extra" string)
 		     ("lines" "lines" number)
 		     ("followups to author" "followup" string)))
 	  (types '((number ("less than" <)
 			   ("greater than" >)
 			   ("equal" =))
 		   (string ("substring" s)
@@ -3125,14 +3126,14 @@
   '(buffer-substring (point) (if (gnus-nov-skip-field) (1- (point)) eol)))
 
 (defmacro gnus-nov-parse-extra ()
   '(let (out string)
      (while (not (memq (char-after) '(?\n nil)))
        (setq string (gnus-nov-field))
-       (when (string-match "^\\([^ :]\\): " string)
-	 (push (cons (intern (match-string 1))
+       (when (string-match "^\\([^ :]+\\): " string)
+	 (push (cons (intern (match-string 1 string))
 		     (substring string (match-end 0)))
 	       out)))
      out))
 
 ;; This function has to be called with point after the article number
 ;; on the beginning of the line.
--- gnus-score.el.orig	Sat Oct 10 20:32:03 1998
+++ gnus-score.el	Wed Oct 28 00:36:26 1998
@@ -305,12 +305,13 @@
  s: subject
  b: body
  h: head
  i: message-id
  t: references
  x: xref
+ e: `extra' (non-standard overview)
  l: lines
  d: date
  f: followup
 
 If nil, the user will be asked for a header."
   :group 'gnus-score-default
@@ -318,12 +319,13 @@
 		 (const :tag "subject" s)
 		 (const :tag "body" b)
 		 (const :tag "head" h)
 		 (const :tag "message-id" i)
 		 (const :tag "references" t)
 		 (const :tag "xref" x)
+		 (const :tag "extra" e)
 		 (const :tag "lines" l)
 		 (const :tag "date" d)
 		 (const :tag "followup" f)
 		 (const :tag "ask" nil)))
 
 (defcustom gnus-score-default-type nil
@@ -441,12 +443,13 @@
     ("date" 3 gnus-score-date)
     ("message-id" 4 gnus-score-string)
     ("references" 5 gnus-score-string)
     ("chars" 6 gnus-score-integer)
     ("lines" 7 gnus-score-integer)
     ("xref" 8 gnus-score-string)
+    ("extra" 9 gnus-score-string)
     ("head" -1 gnus-score-body)
     ("body" -1 gnus-score-body)
     ("all" -1 gnus-score-body)
     ("followup" 2 gnus-score-followup)
     ("thread" 5 gnus-score-thread)))
 
@@ -499,12 +502,13 @@
 	    (?s "subject" nil nil string)
 	    (?b "body" "" nil body-string)
 	    (?h "head" "" nil body-string)
 	    (?i "message-id" nil t string)
 	    (?r "references" "message-id" nil string)
 	    (?x "xref" nil nil string)
+	    (?e "extra" nil nil string)
 	    (?l "lines" nil nil number)
 	    (?d "date" nil nil date)
 	    (?f "followup" nil nil string)
 	    (?t "thread" "message-id" nil string)))
 	 (char-to-type
 	  '((?s s "substring" string)
@@ -527,13 +531,13 @@
 	 (hchar (and gnus-score-default-header
 		     (aref (symbol-name gnus-score-default-header) 0)))
 	 (tchar (and gnus-score-default-type
 		     (aref (symbol-name gnus-score-default-type) 0)))
 	 (pchar (and gnus-score-default-duration
 		     (aref (symbol-name gnus-score-default-duration) 0)))
-	 entry temporary type match)
+	 entry temporary type match extra)
 
     (unwind-protect
 	(progn
 
 	  ;; First we read the header to score.
 	  (while (not hchar)
@@ -619,15 +623,32 @@
 	    (if mimic
 		(error "%c %c %c %c" prefix hchar tchar pchar)
 	      (error "Illegal match duration"))))
       ;; Always kill the score help buffer.
       (gnus-score-kill-help-buffer))
 
+    ;; If scoring an extra (non-standard overview) header,
+    ;; we must find out which header is in question.
+    (setq extra
+	  (and gnus-extra-headers
+	       (equal (nth 1 entry) "extra")
+	       (intern					; need symbol
+		(gnus-completing-read
+		 (symbol-name (car gnus-extra-headers))	; default response
+		 "Score extra header:"			; prompt
+		 (mapcar (lambda (x)			; completion list
+			   (cons (symbol-name x) x))
+			 gnus-extra-headers)
+		 nil					; no completion limit
+		 t))))					; require match
+    ;; extra is now nil or a symbol.
+
     ;; We have all the data, so we enter this score.
     (setq match (if (string= (nth 2 entry) "") ""
-		  (gnus-summary-header (or (nth 2 entry) (nth 1 entry)))))
+		  (gnus-summary-header (or (nth 2 entry) (nth 1 entry))
+				       nil extra)))
 
     ;; Modify the match, perhaps.
     (cond
      ((equal (nth 1 entry) "xref")
       (when (string-match "^Xref: *" match)
 	(setq match (substring match (match-end 0))))
@@ -657,13 +678,15 @@
      match				; Match
      type				; Type
      (if (eq score 's) nil score)	; Score
      (if (eq temporary 'perm)		; Temp
 	 nil
        temporary)
-     (not (nth 3 entry)))		; Prompt
+     (not (nth 3 entry))		; Prompt
+     nil				; not silent 
+     extra)				; non-standard overview.
 
     (when (eq symp 'a)
       ;; We change the score file back to the previous one.
       (save-excursion
 	(set-buffer gnus-summary-buffer)
 	(gnus-score-load-file current-score-file)))))
@@ -706,20 +729,22 @@
     (split-window)
     (pop-to-buffer "*Score Help*")
     (let ((window-min-height 1))
       (shrink-window-if-larger-than-buffer))
     (select-window (get-buffer-window gnus-summary-buffer))))
 
-(defun gnus-summary-header (header &optional no-err)
+(defun gnus-summary-header (header &optional no-err extra)
   ;; Return HEADER for current articles, or error.
   (let ((article (gnus-summary-article-number))
 	headers)
     (if article
 	(if (and (setq headers (gnus-summary-article-header article))
 		 (vectorp headers))
-	    (aref headers (nth 1 (assoc header gnus-header-index)))
+	    (if extra			; `header' must be "extra"
+		(or (cdr (assq extra (mail-header-extra headers))) "")
+	      (aref headers (nth 1 (assoc header gnus-header-index))))
 	  (if no-err
 	      nil
 	    (error "Pseudo-articles can't be scored")))
       (if no-err
 	  (error "No article on current line")
 	nil))))
@@ -739,22 +764,23 @@
   (cdr (assoc symbol
 	      (or alist
 		  gnus-score-alist
 		  (gnus-newsgroup-score-alist)))))
 
 (defun gnus-summary-score-entry (header match type score date
-					&optional prompt silent)
+					&optional prompt silent extra)
   (interactive)
   "Enter score file entry.
 HEADER is the header being scored.
 MATCH is the string we are looking for.
 TYPE is the match type: substring, regexp, exact, fuzzy.
 SCORE is the score to add.
 DATE is the expire date, or nil for no expire, or 'now for immediate expire.
 If optional argument `PROMPT' is non-nil, allow user to edit match.
-If optional argument `SILENT' is nil, show effect of score entry."
+If optional argument `SILENT' is nil, show effect of score entry.
+If optional argument `EXTRA' is non-nil, it's a non-standard overview header."
   ;; Regexp is the default type.
   (when (eq type t)
     (setq type 'r))
   ;; Simplify matches...
   (cond ((or (eq type 'r) (eq type 's) (eq type nil))
 	 (setq match (if match (gnus-simplify-subject-re match) "")))
@@ -789,12 +815,17 @@
       (when (= score gnus-score-interactive-default-score)
 	(setq score nil))
       (let ((old (gnus-score-get header))
 	    elem)
 	(setq new
 	      (cond
+	       (extra
+		(list match score
+		      (and date (if (numberp date) date
+				  (date-to-day date)))
+		      type (symbol-name extra)))
 	       (type
 		(list match score
 		      (and date (if (numberp date) date
 				  (date-to-day date)))
 		      type))
 	       (date (list match score (date-to-day date)))
@@ -819,24 +850,25 @@
 
     ;; Score the current buffer.
     (unless silent
       (if (and (>= (nth 1 (assoc header gnus-header-index)) 0)
 	       (eq (nth 2 (assoc header gnus-header-index))
 		   'gnus-score-string))
-	  (gnus-summary-score-effect header match type score)
+	  (gnus-summary-score-effect header match type score extra)
 	(gnus-summary-rescore)))
 
     ;; Return the new scoring rule.
     new))
 
-(defun gnus-summary-score-effect (header match type score)
+(defun gnus-summary-score-effect (header match type score extra)
   "Simulate the effect of a score file entry.
 HEADER is the header being scored.
 MATCH is the string we are looking for.
 TYPE is the score type.
-SCORE is the score to add."
+SCORE is the score to add.
+EXTRA is the possible non-standard header."
   (interactive (list (completing-read "Header: "
 				      gnus-header-index
 				      (lambda (x) (fboundp (nth 2 x)))
 				      t)
 		     (read-string "Match: ")
 		     (y-or-n-p "Use regexp match? ")
@@ -851,13 +883,13 @@
 			 match)
 			((eq type 'e)
 			 (concat "\\`" (regexp-quote match) "\\'"))
 			(t
 			 (regexp-quote match)))))
       (while (not (eobp))
-	(let ((content (gnus-summary-header header 'noerr))
+	(let ((content (gnus-summary-header header 'noerr extra))
 	      (case-fold-search t))
 	  (and content
 	       (when (if (eq type 'f)
 			 (string-equal (gnus-simplify-subject-fuzzy content)
 				       regexp)
 		       (string-match regexp content))
@@ -1863,18 +1895,29 @@
     ;; Sorting the articles costs os O(N*log N) but will allow us to
     ;; only match with each unique header.  Thus the actual matching
     ;; will be O(M*U) where M is the number of strings to match with,
     ;; and U is the number of unique headers.  It is assumed (but
     ;; untested) this will be a net win because of the large constant
     ;; factor involved with string matching.
-    (setq gnus-scores-articles (sort gnus-scores-articles 'gnus-score-string<)
+    (setq gnus-scores-articles
+	  ;; We cannot string-sort the extra headers list.  *sigh*
+	  (if (= gnus-score-index 9)
+	      gnus-scores-articles
+	    (sort gnus-scores-articles 'gnus-score-string<))
 	  articles gnus-scores-articles)
 
     (erase-buffer)
     (while (setq art (pop articles))
       (setq this (aref (car art) gnus-score-index))
+
+      ;; If we're working with non-standard headers, we are stuck
+      ;; with working on them as a group.  What a hassle.
+      ;; Just wait 'til you see what horrors we commit against `match'...
+      (if (= gnus-score-index 9)
+	  (setq this (prin1-to-string this)))	; ick.
+
       (if simplify
 	  (setq this (gnus-map-function gnus-simplify-subject-functions this)))
       (if (equal last this)
 	  ;; O(N*H) cons-cells used here, where H is the number of
 	  ;; headers.
 	  (push art alike)
@@ -1899,12 +1942,13 @@
 	    entries (assoc header alist))
       (while (cdr entries)		;First entry is the header index.
 	(let* ((kill (cadr entries))
 	       (type (or (nth 3 kill) 's))
 	       (score (or (nth 1 kill) gnus-score-interactive-default-score))
 	       (date (nth 2 kill))
+	       (extra (nth 4 kill))	; non-standard header; string.
 	       (found nil)
 	       (mt (aref (symbol-name type) 0))
 	       (case-fold-search (not (memq mt '(?R ?S ?E ?F))))
 	       (dmt (downcase mt))
 					; Assume user already simplified regexp and fuzzies
 	       (match (if (and simplify (not (memq dmt '(?f ?r))))
@@ -1914,12 +1958,18 @@
                         (nth 0 kill)))
 	       (search-func
 		(cond ((= dmt ?r) 're-search-forward)
 		      ((or (= dmt ?e) (= dmt ?s) (= dmt ?f)) 'search-forward)
 		      ((= dmt ?w) nil)
 		      (t (error "Illegal match type: %s" type)))))
+
+	  ;; Evil hackery to make match usable in non-standard headers.
+	  (when extra
+	    (setq match (concat "[ (](" extra " \\. \".*" match ".*\")[ )]")
+		  search-func 're-search-forward))	; XXX danger?!?
+
 	  (cond
 	   ;; Fuzzy matches.  We save these for later.
 	   ((= dmt ?f)
 	    (push (cons entries alist) fuzzies)
 	    (setq entries (cdr entries)))
 	   ;; Word matches.  Save these for even later.

Re: new NOV stuff in p0.40

[Karl Kleinpaste]

The match wrapper wasn't quite sufficiently precise.  One-line change.

--- gnus-score.el.~1~	Wed Oct 28 00:36:26 1998
+++ gnus-score.el	Wed Oct 28 08:19:57 1998
@@ -1964,7 +1964,7 @@
 
 	  ;; Evil hackery to make match usable in non-standard headers.
 	  (when extra
-	    (setq match (concat "[ (](" extra " \\. \".*" match ".*\")[ )]")
+	    (setq match (concat "[ (](" extra " \\. \"[^)]*" match "[^(]*\")[ )]")
 		  search-func 're-search-forward))	; XXX danger?!?
 
 	  (cond