scoring on subject

scoring on subject

[Dirk Meyer]

Hi,

I want to filter some spam from Japan in newsgroups. Even if it's not
spam, I can't read it. I could score on the content-type:

("head"
  ("Content-Type: text/plain; charset=\"ISO-2022-JP\"" -1000 nil S))

That works, but because of the "head" scoring becomes _very_
slow. Scoring on the subject is easy. As example, a subject like this: 

Subject: =?iso-2022-jp?b?GyRCN2MwQhsoQlBDGyRCJT0lVSVIRExITiEjNHw0VjhCGyhC?=
	=?iso-2022-jp?b?GyRCRGokSSRsJEckYhsoQjUbJEJLXCRHGyhCMTUwMDA=?=
	=?iso-2022-jp?b?GyRCISobKEI=?=

can be detected as spam by scanning for iso-2022-jp. But Gnus
translated it into the correct character set, so scoring doesn't work
anymore. How can I score on the subject as it is in the mail?


Dischi

-- 
panic ("No CPUs found.  System halted.\n");
	2.4.3 linux/arch/parisc/kernel/setup.c

Re: scoring on subject

[Ted Zlatanov]

On Thu, 17 Jun 2004, dischi@tzi.de wrote:

> Subject: =?iso-2022-jp?b?GyRCN2MwQhsoQlBDGyRCJT0lVSVIRExITiEjNHw0VjhCGyhC?=
> 	=?iso-2022-jp?b?GyRCRGokSSRsJEckYhsoQjUbJEJLXCRHGyhCMTUwMDA=?=
> 	=?iso-2022-jp?b?GyRCISobKEI=?=
> 
> can be detected as spam by scanning for iso-2022-jp. But Gnus
> translated it into the correct character set, so scoring doesn't work
> anymore. How can I score on the subject as it is in the mail?

Maybe what you really need is to find out the character set of a
subject string, and that should be the scoring test
(subject-encoding)?  I think Gnus should keep presenting the subject
string in its decoded form, because the only thing you really need out
of the example above is the encoding.  The rest is useless without
decoding.

Ted

Re: scoring on subject

[Jesper Harder]

Dirk Meyer <dischi@tzi.de> writes:

> Subject: =?iso-2022-jp?b?GyRCN2MwQhsoQlBDGyRCJT0lVSVIRExITiEjNHw0VjhCGyhC?=
> 	=?iso-2022-jp?b?GyRCRGokSSRsJEckYhsoQjUbJEJLXCRHGyhCMTUwMDA=?=
> 	=?iso-2022-jp?b?GyRCISobKEI=?=
>
> can be detected as spam by scanning for iso-2022-jp. But Gnus
> translated it into the correct character set, so scoring doesn't work
> anymore. How can I score on the subject as it is in the mail?

Toggle `nnmail-mail-splitting-decodes'.  Or maybe better use regexp
categories, e.g. \cj matches Japanese.

-- 
Jesper Harder                                <http://purl.org/harder/>

Re: scoring on subject

[Katsumi Yamaoka]

>>>>> In <m3llim53z4.fsf@defun.localdomain>
>>>>>	Jesper Harder <harder@ifa.au.dk> wrote:

> Dirk Meyer <dischi@tzi.de> writes:

>> Subject: =?iso-2022-jp?b?GyRCN2MwQhsoQlBDGyRCJT0lVSVIRExITiEjNHw0VjhCGyhC?=
>> 	=?iso-2022-jp?b?GyRCRGokSSRsJEckYhsoQjUbJEJLXCRHGyhCMTUwMDA=?=
>> 	=?iso-2022-jp?b?GyRCISobKEI=?=

That's really a spam, selling illegally copied softwares.

> Toggle `nnmail-mail-splitting-decodes'.  Or maybe better use regexp
> categories, e.g. \cj matches Japanese.

And also \cc, \ch and \cy can be used for Chinese, Korean and
Cyrillic respectively (note that \ should be quoted as "\\").
You can find them in the international/characters.el file.

BTW, since I cannot read those languages similarly, I'm using
the following procmail filters:

:0
* ^Content-Type:.+charset=\"?(\
default_charset\
|euc-kr\
|gb2312(_charset)?\
|ks_c_5601-1987\
|windows-125[12]\
)\"?
/dev/null

:0
* (^From:|^Subject:).*=\?(\
big5\
|euc-kr\
|gb2312\
|ks_c_5601-1987\
|windows-125[12]\
)\?
/dev/null
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: scoring on subject

[Dirk Meyer]

Katsumi Yamaoka wrote:
>>>>>> In <m3llim53z4.fsf@defun.localdomain>
>>>>>>	Jesper Harder <harder@ifa.au.dk> wrote:
>
>> Dirk Meyer <dischi@tzi.de> writes:
>
>>> Subject: =?iso-2022-jp?b?GyRCN2MwQhsoQlBDGyRCJT0lVSVIRExITiEjNHw0VjhCGyhC?=
>>> 	=?iso-2022-jp?b?GyRCRGokSSRsJEckYhsoQjUbJEJLXCRHGyhCMTUwMDA=?=
>>> 	=?iso-2022-jp?b?GyRCISobKEI=?=
>
> That's really a spam, selling illegally copied softwares.
>
>> Toggle `nnmail-mail-splitting-decodes'.  Or maybe better use regexp
>> categories, e.g. \cj matches Japanese.
>
> And also \cc, \ch and \cy can be used for Chinese, Korean and
> Cyrillic respectively (note that \ should be quoted as "\\").
> You can find them in the international/characters.el file.

Thanks, that's what I was looking for. Works great. 

> BTW, since I cannot read those languages similarly, I'm using
> the following procmail filters:

I have procmail filters, too, but I need some _fast_ filter for
newsgroups. 


Dischi

-- 
Monday is an awful way to spend 1/7th of your life.