decoded authors names in agent/.overview

decoded authors names in agent/.overview

[Katsumi Yamaoka]

Hi,

A Japanese user who recently updated Emacs and Gnus into 22.1 and
v5.11 reported[1] that authors' names that appear in summary lines
are broken when reading Japanese newsgroups, e.g., fj.soc.politics.
I also reproduced this problem today only once.  However, it never
arises after I ran `gnus-agent-expire', and I didn't find the way
to reproduce it so far.  In the summary buffer, they looked like:

O  [  29: GON                    ] ...Japanese subject...
O  [   7: Fatman                 ]
O  [  16: ?????? sakaei          ]
O      [  22: ?????? sakaei          ]
O  [   1: ????????????               ]

Those question marks are exactly the ones I saw, though they had
to be Japanese names.  I checked those articles and NOV's in the
nntp server and confirmed they all were correctly RFC2047-encoded
Japanese names.  But I finally found the real cause.  It is that
decoded authors' names were saved in
the ~/News/agant/nntp/SERVER/NEWS/GROUP/NAME/.overview file.  Of
course they are RFC2047-encoded ones normally, though.

Does anyone know what does this, when, how?  Anyway I'm going to
look into it further, next week.

Regards,

[1] <86wsyf0wbc.fsf@bsd2.4bn.ne.jp> in fj.news.reader.gnus

Re: decoded authors names in agent/.overview

[Katsumi Yamaoka]

>>>>> In <b4mtztir5kb.fsf@jpl.org> Katsumi Yamaoka wrote:

> A Japanese user who recently updated Emacs and Gnus into 22.1 and
> v5.11 reported[1] that authors' names that appear in summary lines
> are broken when reading Japanese newsgroups, e.g., fj.soc.politics.
> I also reproduced this problem today only once.  However, it never
> arises after I ran `gnus-agent-expire', and I didn't find the way
> to reproduce it so far.  In the summary buffer, they looked like:

> O  [  29: GON                    ] ...Japanese subject...
> O  [   7: Fatman                 ]
> O  [  16: ?????? sakaei          ]
> O      [  22: ?????? sakaei          ]
> O  [   1: ????????????               ]

> Those question marks are exactly the ones I saw, though they had
> to be Japanese names.  I checked those articles and NOV's in the
> nntp server and confirmed they all were correctly RFC2047-encoded
> Japanese names.  But I finally found the real cause.  It is that
> decoded authors' names were saved in
> the ~/News/agant/nntp/SERVER/NEWS/GROUP/NAME/.overview file.  Of
> course they are RFC2047-encoded ones normally, though.

I've solved this problem and installed the fix in both the trunk
and the v5-10 branch.  There were two causes that aroused this:

1. When a user requests reading articles in a certain newsgroup
   for the first time after starting Gnus, Gnus tries sending an
   XOVER command to the news server.  If those articles have been
   all expired in the server, the server returns a response
   without NOV lines as follows:

--8<---------------cut here---------------start------------->8---
GROUP gnu.emacs.gnus
211 919 56683 58329 gnu.emacs.gnus
XOVER 1-10
224 1-10 fields follow
.
--8<---------------cut here---------------end--------------->8---

   In that case, Gnus decides the XOVER command is not available
   in the server, and will use the HEAD command hereafter.  It
   likely happens if a user visits a newsgroup after a long
   interval and requests a first few unread articles.  It lowers
   the performance anyway.

2. When the Agent fetches headers using the HEAD command, none of
   RFC2047-encoded words in headers should never be decoded.  For
   that purpose, the Agent binds the variable
   `gnus-decode-encoded-word-function' to `identity'.  However, I
   forgot to bind the `gnus-decode-encoded-address-function'
   variable as well when I introduced it last year.

The long and short of it is that the culprit is me.  Sorry.