Re: nntp servers with multibyte group names?

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Russ Allbery <eagle@eyrie.org> writes:

> Adam Sjøgren <asjo@koldfront.dk> writes:
>
>> According to RFC 3977 (Network News Transfer Protocol (NNTP)):
>
>>   "o  Although this specification allows UTF-8 for newsgroup names, they
>>       SHOULD be restricted to US-ASCII until a successor to RFC 1036
>>       [RFC1036] standardises another approach. 8-bit encodings SHOULD
>>       NOT be used because they are likely to cause interoperability
>>       problems."
>
>>      - https://tools.ietf.org/html/rfc3977#section-10
>
> For a bit of background here, non-ASCII newsgroup names mostly work, and
> are even used in some areas, but we saw a few instances of strange
> behavior in some experiments.  However, putting raw UTF-8 directly into
> the Newsgroups header breaks compatibility with RFC 5322 (email), which
> prohibits non-ASCII characters in headers.
>
> Email would say that you should MIME-encode those names, but that will
> definitely break all Usenet software, which assumes that Newsgroups are
> byte strings that don't require any further interpretation.  (And some of
> the encoding characters are invalid in newsgroup names, I believe.)
>
> We weren't able to find a good reconciliation of that conflict before the
> IETF working group ran out of steam.
>
> So you can probably just use raw UTF-8 directly in newsgroup names with a
> local server, but expect some strangeness with some clients, and you are
> (for whatever it's worth) breaking compatibility with the email standards
> by doing so.

Thanks very much for this, Russ -- this is good background.

Since Gnus is only a client, we're not in a position to decide about the
encoding of NNTP group names (thankfully), we only need to decide how to
accept and handle such names as we receive from a server. As Stefan
Monnier pointed out in answer to a separate question on emacs.help, the
NNTP protocol will likely speak several different text encodings, so
Gnus should still be running the network connection in binary mode. I'm
going to leave the majority of the code as-is, and make the smallest
change to group-name decoding I can.

Thanks again,
Eric