Character of Death locks one out of all newsgroups

Character of Death locks one out of all newsgroups

[jidanni]

One bad character locks me out of all newsgroups... until when, one
doesn't know. Great denial of service attack.

Debugger entered--Lisp error: (error "Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'")
  signal(error ("Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'"))
  error("Can't rfc2047-encode `%s'" "\x2563d\x2587c \x256cf\x7ecf")
  rfc2047-encode(5 10)
  rfc2047-encode-region(1 12)
  nnrss-mime-encode-string("Re: \x2563d\x2587c亲\x256cf经历 3")
  nnrss-check-group("BDSM \x25322\x25cf0\x25d37\x268a2" "")
  nnrss-request-group("BDSM 一般討論" "" nil)
  gnus-activate-group("nnrss:BDSM 一般討論" scan)
  gnus-get-unread-articles(nil)
  gnus-group-get-new-news()

Input was http://www.bdsm.com.tw/phorum/rss.php?1

If you can't rfc2047-encode something, then put a "bad character
character" in its place.

Firefox shows the line as
Re: 我的亲身经历 3
That would be still better that bombing out too.

Anyway, I had to use
C-k runs the command gnus-group-kill-group
to be able to use
g runs the command gnus-group-get-new-news
ever again. Else one must use many
M-g (translated from <escape> g) runs the command gnus-group-get-new-news-this-group
If I dare use
c runs the command gnus-group-catchup-current
I just get

Debugger entered--Lisp error: (error "Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'")
  signal(error ("Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'"))
  error("Can't rfc2047-encode `%s'" "\x2563d\x2587c \x256cf\x7ecf")
  rfc2047-encode(5 10)
  rfc2047-encode-region(1 12)
  nnrss-mime-encode-string("Re: \x2563d\x2587c&#20146;\x256cf&#32463;&#21382; 3")
  nnrss-check-group("BDSM \x25322\x25cf0\x25d37\x268a2" "")
  nnrss-request-group("BDSM 一般討論" "" nil)
  gnus-activate-group("nnrss:BDSM 一般討論")
  gnus-sequence-of-unread-articles("nnrss:BDSM 一般討論")
  gnus-group-catchup("nnrss:BDSM 一般討論" nil)
  gnus-group-catchup-current(nil)
  call-interactively(gnus-group-catchup-current)

Who knows why it must encode things just to mark them as read.

I wonder how I ever can read this group again?

Re: Character of Death locks one out of all newsgroups

[jidanni]

One needs to unconnect to the Internet, start gnus again, unsubscribe
the group, and wait several weeks for that thread with the poison
character to die off RSS (monitoring via e.g., firefox), or else one
will not be able to read ANY mail or news groups.

Re: Character of Death locks one out of all newsgroups

[Reiner Steib]

[-- Attachment #1: Type: text/plain, Size: 913 bytes --]

On Fri, Dec 26 2008, jidanni@jidanni.org wrote:

> One bad character locks me out of all newsgroups... until when, one
> doesn't know. Great denial of service attack.
>
> Debugger entered--Lisp error: (error "Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'")
>   signal(error ("Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'"))
>   error("Can't rfc2047-encode `%s'" "\x2563d\x2587c \x256cf\x7ecf")
>   rfc2047-encode(5 10)
>   rfc2047-encode-region(1 12)
>   nnrss-mime-encode-string("Re: \x2563d\x2587c亲\x256cf经历 3")
>   nnrss-check-group("BDSM \x25322\x25cf0\x25d37\x268a2" "")
>   nnrss-request-group("BDSM 一般討論" "" nil)
>   gnus-activate-group("nnrss:BDSM 一般討論" scan)
>   gnus-get-unread-articles(nil)
>   gnus-group-get-new-news()
>
> Input was http://www.bdsm.com.tw/phorum/rss.php?1

A minimal recipe to reproduce this in Emacs 22 is this:

[-- Attachment #2: Type: text/plain, Size: 101 bytes --]


(let ((rfc2047-encoding-type 'mime)
	     rfc2047-encode-max-chars)
	 (rfc2047-encode-string "身

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Type: text/plain; charset=shift_jis, Size: 46 bytes --]

ïn"))

(It works fine with Emacs 23 (CVS))

[-- Attachment #4: Type: text/plain, Size: 398 bytes --]


,----
|       character: 身 (153295, #o453317, #x256cf, U+8EAB)
|         charset: chinese-big5-1
| 		 (Frequently used part (A141-C67F) of Big5 (Chinese traditional).)
|      code point: #x2D #x4F
|          syntax: w 	which means: word
|        category: C:Chinese (Han) characters of 2-byte character sets c:Chinese
| 		 |:While filling, we can break a line at this character.
`----

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #5: Type: text/plain; charset=shift_jis, Size: 305 bytes --]


,----
|   character: ïn (32463, #o77317, #x7ecf)
|     charset: japanese-jisx0208-1978
| 	     (JISX0208.1978 Japanese Kanji (so called "old JIS"): ISO-IR-42.)
|  code point: #x7D #x4F
|      syntax: w 	which means: word
`----

rfc2047-encode fails because mime-charset is '(iso-2022-jp big5):

[-- Attachment #6: Type: text/plain, Size: 42 bytes --]


ELISP> (with-temp-buffer
	 (insert "身

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #7: Type: text/plain; charset=shift_jis, Size: 2310 bytes --]

ïn")
	 (mm-find-mime-charset-region (point-min) (point-max)))
(iso-2022-jp big5)

> If you can't rfc2047-encode something, then put a "bad character
> character" in its place.

This message (which contains the bad characters) will be split into
parts with utf-8 and shift_jis, see "A message part needs to be split
into %d charset parts." in `mml-parse-1'.

Maybe nnrss could do something similar, e.g. return
"=?utf-8?Q?=E8=BA=AB?= =?shift_jis?Q?=EFn?="?  I know next to nothing
about RSS, so I'm not sure.  Anyone?

> Firefox shows the line as
> Re: ‰ä“I亲g经历 3
> That would be still better that bombing out too.

> Anyway, I had to use
> C-k runs the command gnus-group-kill-group
> to be able to use
> g runs the command gnus-group-get-new-news
> ever again. Else one must use many
> M-g (translated from <escape> g) runs the command gnus-group-get-new-news-this-group
> If I dare use
> c runs the command gnus-group-catchup-current
> I just get
>
> Debugger entered--Lisp error: (error "Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'")
>   signal(error ("Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'"))
>   error("Can't rfc2047-encode `%s'" "\x2563d\x2587c \x256cf\x7ecf")
>   rfc2047-encode(5 10)
>   rfc2047-encode-region(1 12)
>   nnrss-mime-encode-string("Re: \x2563d\x2587c&#20146;\x256cf&#32463;&#21382; 3")
>   nnrss-check-group("BDSM \x25322\x25cf0\x25d37\x268a2" "")
>   nnrss-request-group("BDSM ˆê”Ê“¢˜_" "" nil)
>   gnus-activate-group("nnrss:BDSM ˆê”Ê“¢˜_")
>   gnus-sequence-of-unread-articles("nnrss:BDSM ˆê”Ê“¢˜_")
>   gnus-group-catchup("nnrss:BDSM ˆê”Ê“¢˜_" nil)
>   gnus-group-catchup-current(nil)
>   call-interactively(gnus-group-catchup-current)
>
> Who knows why it must encode things just to mark them as read.
>
> I wonder how I ever can read this group again?

On Mon, Dec 29 2008, jidanni@jidanni.org wrote:

> One needs to unconnect to the Internet, start gnus again, unsubscribe
> the group, and wait several weeks for that thread with the poison
> character to die off RSS (monitoring via e.g., firefox), or else one
> will not be able to read ANY mail or news groups.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Character of Death locks one out of all newsgroups

[jidanni]

It seems the problem is that the former is called, instead of the latter:
nnrss-mime-encode-string("Re: \x2563d\x2587c亲\x256cf经&#21=
nnrss-mime-encode-string("Re: \x2563d\x2587c亲\x256cf经历 3")

I.e., the correct rendering would have the ugly &s in it, as it is the
posters fault. Instead nnrss-check-group tries too hard: it first runs a
decoder over the message when it shouldn't. Otherwise the decoder gets
run twice, when it should only be run once.

So nnrss-mime-encode-string is OK, it was just being handed an overly
filtered string mangled somewhere by nnrss-check-group before it call
nnrss-mime-encode-string.

In any case, if one day there are really bad characters (and not just
ones created inside nnrss.el), "????" should be shown, and the error
caught, instead of locking one out of all newsgroups.

(Reiner, your reply got split into many mime parts (as would be expected.)

Re: Character of Death locks one out of all newsgroups

[Katsumi Yamaoka]

>>>>> jidanni@jidanni.org wrote:

> One bad character locks me out of all newsgroups... until when, one
> doesn't know. Great denial of service attack.

> Debugger entered--Lisp error: (error "Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'")
>   signal(error ("Can't rfc2047-encode `\x2563d\x2587c \x256cf\x7ecf'"))
>   error("Can't rfc2047-encode `%s'" "\x2563d\x2587c \x256cf\x7ecf")
>   rfc2047-encode(5 10)
>   rfc2047-encode-region(1 12)
>   nnrss-mime-encode-string("Re: \x2563d\x2587c亲\x256cf经历 3")
>   nnrss-check-group("BDSM \x25322\x25cf0\x25d37\x268a2" "")
>   nnrss-request-group("BDSM 一般討論" "" nil)
>   gnus-activate-group("nnrss:BDSM 一般討論" scan)
>   gnus-get-unread-articles(nil)
>   gnus-group-get-new-news()

I discovered this is due to the `mm-url-decode-entities' function
that uses `char-to-string' to convert entities to human-readable
characters.  When decoding the entity "经", it runs:

(char-to-string 32463)

That's ok in Emacs 23, however to make it work also with Emacs 22
it should be:

(char-to-string (decode-char 'ucs 32463))

Because of this, funny characters are passed to the rfc2047 encoder
in Emacs 22.

For XEmacs with mule, we will have to do like emacs-w3m does, but
there seems to be no way to make it work with Emacs 21.  Anyway we
will need time to fix this problem (see `w3m-ucs-to-char' provided
in w3m-ems.el and w3m-xmas.el and `w3m-decode-entities' in w3m.el.)

A workaround:
--8<---------------cut here---------------start------------->8---
(eval-after-load "mm-url"
  '(progn
     (require 'w3m)
     (defalias 'mm-url-decode-entities `w3m-decode-entities)))
--8<---------------cut here---------------end--------------->8---

I verified it works with Emacs 22.1, 22.2, 22.3, XEmacs 21.4.22,
21.5-b28, and SXEmacs 22.1.10 as follows:

(rfc2047-decode-string
 (nnrss-mime-encode-string "Re: 我的&#20146;身&#32463;&#21382; 3"))
 => "Re: 我的亲身经历 3"

Regards,

Re: Character of Death locks one out of all newsgroups

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

>>>>> Katsumi Yamaoka wrote:
> Anyway we will need time to fix this problem (see `w3m-ucs-to-char'
> provided in w3m-ems.el and w3m-xmas.el and `w3m-decode-entities' in
> w3m.el.)

I made a patch for mm-util.el and mm-url.el, however I'm not quite
sure it works in your Emacs environment (locale?, language?, ...).
So could you try loading the attached file and reading the nnrss
group in question?  If it works, I'll add them to No Gnus.


[-- Attachment #2: LOADME --]
[-- Type: application/emacs-lisp, Size: 1902 bytes --]

Re: Character of Death locks one out of all newsgroups

[jidanni]

>>>>> "KY" == Katsumi Yamaoka <yamaoka@jpl.org> writes:
KY> So could you try loading the attached file and reading the nnrss
KY> group in question?  If it works, I'll add them to No Gnus.
OK, if I do
$ emacs -l LOADME.el
then I can use gnus again. If I don't then I am locked out. Therefore
your LOADME.el fixes the problem, and I will load it everyday until
Debian experimental or Debian sid upgrades to emacs 23. Thanks!

Re: Character of Death locks one out of all newsgroups

[Katsumi Yamaoka]

>>>>> jidanni@jidanni.org wrote:
>>>>>> "KY" == Katsumi Yamaoka <yamaoka@jpl.org> writes:
KY> So could you try loading the attached file and reading the nnrss
KY> group in question?  If it works, I'll add them to No Gnus.
> OK, if I do
> $ emacs -l LOADME.el
> then I can use gnus again. If I don't then I am locked out. Therefore
> your LOADME.el fixes the problem, and I will load it everyday until
> Debian experimental or Debian sid upgrades to emacs 23. Thanks!

Thank you for the test.  I've installed the changes with a slight
modification: http://article.gmane.org/gmane.emacs.gnus.commits/6122