Splitting mail -- XEmacs 21.4 vs 21.5

Splitting mail -- XEmacs 21.4 vs 21.5

[Steve Youngs]

[-- Attachment #1: Type: text/plain, Size: 2942 bytes --]

Now, you'd think that the exact same Gnus mail splitting rules would
split mail into the same groups regardless of XEmacs version.  But
you'd be wrong!

Here are my split rules:

(setq nnmail-split-methods 'nnmail-split-fancy
      nnmail-message-id-cache-length 5000
      nnmail-crosspost nil
      nnmail-split-fancy
      '(| 
	;; Tiffany is highest priority.
	(any "ms_tamber@hotmail\\.com" "private.Tiffany")
	;; SPAM...
	;; Most people put their spam splits after their mailing
	;; lists and stuff.  I think this is the wrong way to do it,
	;; because you'd still get heaps of spam in your mailing list
	;; groups.
	("X-Spam-Flag" "Yes" "SPAM.spamassassin")
 	("Content-Type" content-spam "SPAM.content")
  	("Content-Transfer-Encoding" encoding-spam "SPAM.encoding")
  	("Subject" subject-spam "SPAM.subjects")
  	(from author-spam "SPAM.authors")
  	("Received" domain-spam "SPAM.domains")
	("Keywords" keyword-spam "SPAM.keywords")
	;; XEmacs stuff I want to disappear.
	(to "xemacs-\\(people\\|request\\|users\\|webmaint\\|winnt\\|nt\\)+@xemacs\\.org"
	    "SPAM.xemacs")
	(to "youngs@xemacs\\.org" "SPAM.xemacs")
	;; Bounces
	(from mail "returned.mail")
	;; Log files
	("Subject" 
	 "\\(Logs:.*\\|Mail Stats\\|eicq Daily Usenet\\|var/log/.*\\)" "private.logs")
	;; Mailing lists 
	(any "\\(eicq\\|mh-e\\|bbdb\\|tramp\\|xemacs\\)-\\b\\(\\w+\\)@\\(\\(lists\\.\\)?\\(sf\\|sourceforge\\)\\.net\\|\\(mail\\.freesoftware\\.fsf\\|gnu\\|nongnu\\|xemacs\\)\\.org\\)"
	     "\\1.\\2")
	(from "noreply@sourceforge\\.net" "eicq.admin")
	(from "mailman-owner@lists\\.\\(sf\\|sourceforge\\)\\.net"
	      "eicq.admin")
	(any "ding@\\(hpc\\.uh\\.edu\\|gnus\\.org\\)" "gnus.ding")
	(to "cvslog@quimby\\.gnus\\.org" "gnus.cvs")
	(from "freshmeat-news" "freshmeat.news")
	(from "Administrator@jobnet\\.com\\.au" "employment")
	(from "subscriber\\.email@jobs\\.jobserve\\.com" "employment")
	(any "icq-devel" "icq.devel")
	(any "sql-ledger-users" "sql.ledger")
	(any "inn-workers" "inn.workers")
	(any "linux-kernel@vger\\.kernel\\.org" "linux.kernel")
	;; To me, personally
	(to 
	 "steve$\\|steve@\\(localhost\\|eicq\\.dnsalias\\.org\\)"
	 "private.local")
	(to "\\(sr\\)?youngs\\(_steve\\)?@\\(bigpond\\.net\\.au\\|tux\\.org\\|users\\.sourceforge\\.net\\|yahoo\\.co\\.uk\\)" "private.net")
	;; Catch all
	"INBOX"))

In XEmacs 21.4.15 everything goes where you'd expect it to go.

In XEmacs 21.5.16 (+CVS-20040227) everything goes to INBOX.

Those settings above used to work for me in 21.5, but unfortunately it
has been so long since I last used 21.5 that I can't tell you when it
last worked.  Sorry. :-(

I wonder what's gone a-drift in 21.5?  Nothing in the ChangeLogs
really jumps out at me.

-- 
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
|        XEmacs - The only _______ you'll ever need.       |
|          Fill in the blank, yes, it's THAT good!         |
|------------------------------<sryoungs@bigpond.net.au>---|

[-- Attachment #2: Type: application/pgp-signature, Size: 256 bytes --]

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Katsumi Yamaoka]

>>>>> In <microsoft-free.87u119i9zm.fsf@eicq.dnsalias.org>
>>>>>	Steve Youngs <sryoungs@bigpond.net.au> wrote:

SY> (setq nnmail-split-methods 'nnmail-split-fancy
SY>       nnmail-message-id-cache-length 5000
SY>       nnmail-crosspost nil
SY>       nnmail-split-fancy

[...]

SY> 	"INBOX"))

SY> In XEmacs 21.4.15 everything goes where you'd expect it to go.
SY> In XEmacs 21.5.16 (+CVS-20040227) everything goes to INBOX.

You may be able to solve the problem by setting the
`nnmail-split-fancy-match-partial-words' variable as t.  See the
doc-string for it.  There is an XEmacs 21.5 bug which is probably
related to the following thread:

http://thread.gmane.org/sdd685fw0w.fsf@wes.hardakers.net

The test code I used is below:

(let ((nnmail-split-fancy '(...your customization...))
      (nnmail-split-fancy-match-partial-words nil))
  (with-temp-buffer
    (insert "\
From: ms_tamber@hotmail.com
To: Steve Youngs <sryoungs@bigpond.net.au>

Hello world!
")
    (nnmail-split-fancy)))
 => ("INBOX")
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Katsumi Yamaoka]

>>>>> In <b9yn071w5gr.fsf@jpl.org> Katsumi Yamaoka wrote:

> You may be able to solve the problem by setting the
> `nnmail-split-fancy-match-partial-words' variable as t.

Oops, sorry.  Gnus should be built with XEmacs 21.5 if it is
used with XEmacs 21.5.  However, I tested it using Gnus built
with XEmacs 21.4.  The problem is surely a syntax-table bug in
XEmacs 21.5, and it can be solved by building Gnus with XEmacs
21.5.  Here are test cases:

(require 'nnmail)
(with-syntax-table nnmail-split-fancy-syntax-table
  (string-match "\\<ms_tamber@hotmail\\.com\\>"
		"ms_tamber@hotmail.com"))
 => nil

(ignore-errors (load "/SOMEWHERE/gnus/lisp/dgnushack.el"))
(with-syntax-table nnmail-split-fancy-syntax-table
  (string-match "\\<ms_tamber@hotmail\\.com\\>"
		"ms_tamber@hotmail.com"))
 => 0

See also:

> http://thread.gmane.org/sdd685fw0w.fsf@wes.hardakers.net

>>>>> In <microsoft-free.87u119i9zm.fsf@eicq.dnsalias.org>
>>>>>	Steve Youngs <sryoungs@bigpond.net.au> wrote:

SY> Those settings above used to work for me in 21.5, but
SY> unfortunately it has been so long since I last used 21.5 that I
SY> can't tell you when it last worked.  Sorry. :-(

The bug came to light by Gnus on January 7, 2004.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Karl Pflästerer]

On  1 Mar 2004, Steve Youngs <- sryoungs@bigpond.net.au wrote:

> Now, you'd think that the exact same Gnus mail splitting rules would
> split mail into the same groups regardless of XEmacs version.  But
> you'd be wrong!
[...]
> In XEmacs 21.4.15 everything goes where you'd expect it to go.

> In XEmacs 21.5.16 (+CVS-20040227) everything goes to INBOX.
[...]
> I wonder what's gone a-drift in 21.5?  Nothing in the ChangeLogs
> really jumps out at me.

I had the same problem and use the following in my .gnus.el as
workaround.

(setq nnmail-split-fancy-syntax-table
  (let ((table (copy-syntax-table (standard-syntax-table))))
    ;; support the %-hack
    (modify-syntax-entry ?\% "." table)
    table))
;  "Syntax table used by `nnmail-split-fancy'.")

Gnus uses in the original code:

,------------------[ nnmail.el ]-----------------.
| (defvar nnmail-split-fancy-syntax-table         |
|   (let ((table (make-syntax-table)))            |
|     ;; support the %-hack                       |
|     (modify-syntax-entry ?\% "." table)         |
|     table)                                      |
|   "Syntax table used by `nnmail-split-fancy'.") |
`------------------------------------------------´

`make-syntax-table' seems to be broken in XEmacs 21.5.  That's seems
also to be related to a problem with syntax-table; cf.
<b9yfzd1tgvd.fsf@jpl.org>

See also the posting <v9brnotsm2.fsf@marauder.physik.uni-ulm.de> from
Reiner Steib which he crossposted also to XEmacs-beta (but it seems it
never eached the list).  Perhaps he should post it again or someone
should forward it?


   KP

-- 
    'Twas brillig, and the slithy toves
        Did gyre and gimble in the wabe;
    All mimsy were the borogoves,
         And the mome raths outgrabe.   "Lewis Carroll" "Jabberwocky"

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Steve Youngs]

[-- Attachment #1: Type: text/plain, Size: 3376 bytes --]

* Katsumi Yamaoka <yamaoka@jpl.org> writes:

  >>>>>> In <b9yn071w5gr.fsf@jpl.org> Katsumi Yamaoka wrote:
  >> You may be able to solve the problem by setting the
  >> `nnmail-split-fancy-match-partial-words' variable as t.

  > Oops, sorry.  Gnus should be built with XEmacs 21.5 if it is
  > used with XEmacs 21.5.  

Yes, building Gnus with XEmacs 21.5 gets around it, and _that_ Gnus
seems to work OK in XEmacs 21.4 too.

  > See also:

  >> http://thread.gmane.org/sdd685fw0w.fsf@wes.hardakers.net

Seems that we've (XEmacs.org) screwed things royally.  Who's big on
syntax tables that could take a look at this?

  > The bug came to light by Gnus on January 7, 2004.

The only things that I could find that might be remotely related
are... 

2003-02-28  Ben Wing  <ben@xemacs.org>

  [...]

	* chartab.c (fill_char_table):
	* chartab.c (put_char_table):
	* lrecord.h:
	Fix crash due to attempt to free objects across dump/undump.

This patch is at:

<http://list-archive.xemacs.org/xemacs-patches/200303/msg00000.html>


And...

2003-02-14  Ben Wing  <ben@xemacs.org>

	* buffer.c:
	* syntax.c:
	Move syntax table description from buffer.c to syntax.c.
	
	* chartab.c:
	* chartab.c (set_char_table_dirty):
	* chartab.c (fill_char_table):
	* chartab.c (Fmake_char_table):
	* chartab.c (Fcopy_char_table):
	* chartab.c (get_range_char_table_1):
	* chartab.c (get_range_char_table):
	* chartab.c (put_char_table):
	* chartab.h:
	* chartab.h (struct Lisp_Char_Table):
	Free extra char table entries to avoid excessive garbage.
	Add flags for dirty and mirror_table_p to char tables.
	Add a back pointer from mirror tables to the original syntax table.
	When modifying a syntax table, don't update the mirror table right
	away, just mark as dirty.
	Add various asserts to make sure we are dealing with the right type
	of table (mirror or non-mirror).
	
	* font-lock.c:
	* font-lock.c (find_context):
	* syntax.c (init_syntax_cache):
	* syntax.c (setup_syntax_cache):
	* syntax.c (mark_buffer_syntax_cache):
	* syntax.c (init_buffer_syntax_cache):
	* syntax.c (syntax_cache_table_was_changed):
	* syntax.c (update_syntax_cache):
	* syntax.c (scan_lists):
	* syntax.c (scan_sexps_forward):
	* syntax.c (copy_if_not_already_present):
	* syntax.c (update_just_this_syntax_table):
	* syntax.c (update_syntax_table):
	* syntax.c (vars_of_syntax):
	* syntax.h:
	* syntax.h (struct syntax_cache):
	Add entry to syntax caches for the non-mirror table.  Set it
	appropriately when initializing the syntax table.  Use it, not
	the mirror table, for calls to syntax_match().
	Don't create a bogus float each time, just once at startup.
	Add some asserts, as in chartab.c.

	* syntax.h (SYNTAX_CODE_FROM_CACHE):
	When retrieving the syntax code, check the dirty flag and update
	the mirror tables as appropriate.
	Add some asserts, as above.

This one is here:
	
<http://list-archive.xemacs.org/xemacs-patches/200302/msg00150.html>

Thanks very much for getting back to me on this, Katsumi, I really
appreciate it.  BTW, do you know enough about these kind of things to
take a look at these two patches and perhaps offer some suggestions?

-- 
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
|        XEmacs - The only _______ you'll ever need.       |
|          Fill in the blank, yes, it's THAT good!         |
|------------------------------<sryoungs@bigpond.net.au>---|

[-- Attachment #2: Type: application/pgp-signature, Size: 256 bytes --]

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Stephen J. Turnbull]

>>>>> "SY" == Steve Youngs <sryoungs@bigpond.net.au> writes:

    SY> Now, you'd think that the exact same Gnus mail splitting rules
    SY> would split mail into the same groups regardless of XEmacs
    SY> version.

No sane person expects anything related to Gnus to be deterministic.

    SY> Those settings above used to work for me in 21.5, but
    SY> unfortunately it has been so long since I last used 21.5 that
    SY> I can't tell you when it last worked.  Sorry. :-(

Before your last upgrade of Gnus, obviously.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Stephen J. Turnbull]

>>>>> "KY" == Katsumi Yamaoka <yamaoka@jpl.org> writes:

    KY> The problem is surely a syntax-table bug in XEmacs 21.5,

So file a report.  You know the drill: specify the behavior of the
function that is failing, cite the portion of the Lisp Reference which
gives the specification, explain what is actually happening that
varies from it.

"Gnus doesn't work, so it's a syntax table bug in XEmacs" is not worth
investigating for me, and the people who do know Gnus or syntax tables
well don't seem to be paying attention at the moment.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: Splitting mail -- XEmacs 21.4 vs 21.5

[Katsumi Yamaoka]

>>>>> In <87smgqirw7.fsf@tleepslib.sk.tsukuba.ac.jp>
>>>>>	"Stephen J. Turnbull" <stephen@xemacs.org> wrote:

>>>>>> "KY" == Katsumi Yamaoka <yamaoka@jpl.org> writes:

>     KY> The problem is surely a syntax-table bug in XEmacs 21.5,

> So file a report.  You know the drill: specify the behavior of the
> function that is failing, cite the portion of the Lisp Reference which
> gives the specification, explain what is actually happening that
> varies from it.

> "Gnus doesn't work, so it's a syntax table bug in XEmacs" is not worth
> investigating for me, and the people who do know Gnus or syntax tables
> well don't seem to be paying attention at the moment.

I don't know what's happening in XEmacs 21.5, but I simply
groped how to make Gnus work correctly.  The reason I said it
is a syntax-table bug is because the copy-syntax-table function
doesn't copy modified syntax entries as I wrote a comment in
the dgnushack,el file:

  ;; The original `with-syntax-table' uses `copy-syntax-table' which
  ;; doesn't seem to copy modified syntax entries in XEmacs 21.5.

(let ((table (copy-syntax-table emacs-lisp-mode-syntax-table)))
  (modify-syntax-entry ?= " " table)
  (with-temp-buffer
    (with-syntax-table table
      (insert "foo=bar")
      (goto-char (point-min))
      (forward-sexp 1)
      (buffer-substring (point) (point-max)))))
 => ""

And I've redefined the with-syntax-table macro there so that it
doesn't use copy-syntax-table when building Gnus.  Since it is
no more than a makeshift, I am willing to remove it if there is
a better way.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>