posting-charset abolition

posting-charset abolition

[Katsumi Yamaoka]

Hi,

I'm planning to make a proposal to abolish the group parameter
posting-charset-alist and the variable message-posting-charset.
I write a story from the beginning:

Gnus always puts the ``Content-Transfer-Encoding: 8bit'' header
into news articles to be sent by default.  Since Japanese news
articles use the iso-2022-jp charset normally, putting CTE: 8bit
is nonsense (iso-2022-jp never uses 8-bit data).  It is caused by
the default value for the posting-charset-alist group parameter,
which contains the element (message-this-is-news nil t).  See
the docstring for the variable gnus-group-posting-charset-alist
for details.

By default, the iso-8859-1 charset is used for posting news
articles to Norwegian or French newsgroups.  However, please
consider.  Though it is unlikely to post a Japanese article to a
Norwegian group, a posting-charset should be determined by
article contents rather than a newsgroup name.  Even if there is
a candidate of two or more charsets for article contents, we can
decide the priorities by the option mm-coding-system-priorities.

I am anxious about the following info description:

     When posting, `gnus-group-posting-charset-alist' is used to
  determine which charsets should not be encoded using the MIME
  encodings.  For instance, some hierarchies discourage using
  quoted-printable header encoding.

Do such newsgroups exist even now?

By the way, now I'm using the following code not to add the
CTE: 8bit header to news articles:

(eval-after-load "gnus-msg"
  '(if (boundp 'gnus-group-posting-charset-alist)
       (let ((news (assq 'message-this-is-news
			 gnus-group-posting-charset-alist)))
	 (if news
	     (setcdr news '(nil nil))))))
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Katsumi Yamaoka]

>>>>> In <yotlof9c9ier.fsf@jpl.org>
>>>>>	Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> I'm planning to make a proposal to abolish the group parameter
> posting-charset-alist and the variable message-posting-charset.

I decided not to propose abolition now, since I thought it is
probably required in a certain world.  Instead, I will consider
the way to prevent Gnus from adding ``CTE: 8bit'' to 7-bit
charsets.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Ted Zlatanov]

On Wed, 30 Oct 2002, yamaoka@jpl.org wrote:
> Gnus always puts the ``Content-Transfer-Encoding: 8bit'' header
> into news articles to be sent by default.  Since Japanese news
> articles use the iso-2022-jp charset normally, putting CTE: 8bit
> is nonsense (iso-2022-jp never uses 8-bit data).

Forgive my ignorance, but I don't understand why CTE: 8bit needs to be
removed for 7bit encodings.  Does the 7bit CTE save bits or processing
time when the message is delivered (my understanding is that this used
to be the case long ago, but today very few active MTAs use 7bit)?
Does the 8bit CTE somehow distort any of the current 7bit encodings?
If the answer to both questions is no, maybe it's best to leave the
CTE at 8bit.

Ted

Re: posting-charset abolition

[Katsumi Yamaoka]

>>>>> In <m3of9ahmf2.fsf@heechee.beld.net>
>>>>>	Ted Zlatanov <tzz@lifelogs.com> wrote:

> On Wed, 30 Oct 2002, yamaoka@jpl.org wrote:

>> Gnus always puts the ``Content-Transfer-Encoding: 8bit'' header

> Forgive my ignorance, but I don't understand why CTE: 8bit needs to be
> removed for 7bit encodings.  Does the 7bit CTE save bits or processing
> time when the message is delivered (my understanding is that this used
> to be the case long ago, but today very few active MTAs use 7bit)?
> Does the 8bit CTE somehow distort any of the current 7bit encodings?
> If the answer to both questions is no, maybe it's best to leave the
> CTE at 8bit.

You are right.  Putting CTE:8 to 7-bit articles is absolutely
harmless.  However, it is closely reminded of vicious MUAs
rampant all over the world.  Possibly they became better than
ancient times, though.  (In old days, Japanese mails were in a
wretched condition.)  Although this is no more than a personal
matter of taste, it is worthwhile to delete an excessive header,
isn't it?

rfc1521.txt:
  ..."Content-Transfer-Encoding: 7BIT" is assumed if the
  Content-Transfer-Encoding header field is not present.

So, I want to do that if it is not a burden on Gnus.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Ted Zlatanov]

On Thu, 31 Oct 2002, yamaoka@jpl.org wrote:
> You are right.  Putting CTE:8 to 7-bit articles is absolutely
> harmless.  However, it is closely reminded of vicious MUAs
> rampant all over the world.  Possibly they became better than
> ancient times, though.  (In old days, Japanese mails were in a
> wretched condition.)  Although this is no more than a personal
> matter of taste, it is worthwhile to delete an excessive header,
> isn't it?
> 
> rfc1521.txt:
>   ..."Content-Transfer-Encoding: 7BIT" is assumed if the
>   Content-Transfer-Encoding header field is not present.
> 
> So, I want to do that if it is not a burden on Gnus.

I think the simplest programmatic solution is to scan the message
buffer right before sending, when it's already encoded, and add the
CTE: 8bit header if any 8-bit characters are detected.  But I still
think this is unnecessary work for other than aesthetic reasons, as
very few MTAs today do 7bit CTE, and setting the header to 8bit is
harmless to 7bit data.

I don't understand the statement about "vicious MUAs rampant."

-- 
Ted Zlatanov

Re: posting-charset abolition

[Katsumi Yamaoka]

>>>>> In <4n3cqmy93k.fsf@benko.bwh.harvard.edu>
>>>>>	Ted Zlatanov <tzz@lifelogs.com> wrote:

> I think the simplest programmatic solution is to scan the message
> buffer right before sending, when it's already encoded, and add the
> CTE: 8bit header if any 8-bit characters are detected.

Yes.  It can be done by using message-header-hook or something.
However, though I haven't seen the source code in details yet, I
think there is a more smart solution.  Because, even if I use
the following codes to remove CTE:8, CTE is added if necessary.

(eval-after-load "gnus-msg"
  '(if (boundp 'gnus-group-posting-charset-alist)
       (let ((news (assq 'message-this-is-news
			 gnus-group-posting-charset-alist)))
	 (if news
	     (setcdr news '(nil nil))))))

> But I still
> think this is unnecessary work for other than aesthetic reasons, as
> very few MTAs today do 7bit CTE, and setting the header to 8bit is
> harmless to 7bit data.

To be sure, it is right.  However, would you allow me if it is
achieved with a trifling change?

> I don't understand the statement about "vicious MUAs rampant."

Possibly my talk was too acrimoniously.  Never mind please.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Ted Zlatanov]

On Fri, 01 Nov 2002, yamaoka@jpl.org wrote:
> To be sure, it is right.  However, would you allow me if it is
> achieved with a trifling change?

I'm OK with the change.  Go wild :)

Ted

Re: posting-charset abolition

[Jesper Harder]

[-- Attachment #1: Type: text/plain, Size: 682 bytes --]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Yes.  It can be done by using message-header-hook or something.
> However, though I haven't seen the source code in details yet, I
> think there is a more smart solution.  Because, even if I use
> the following codes to remove CTE:8, CTE is added if necessary.
>
> (eval-after-load "gnus-msg"
>   '(if (boundp 'gnus-group-posting-charset-alist)
>        (let ((news (assq 'message-this-is-news
> 			 gnus-group-posting-charset-alist)))
> 	 (if news
> 	     (setcdr news '(nil nil))))))

I think this is a kludge.  Shouldn't we rather fix `mm-body-encoding' to
return the proper CTE for 7bit charsets?

What do you think about this fix?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: mm-bodies.el.diff --]
[-- Type: text/x-patch, Size: 419 bytes --]

--- /home/harder/gnus/lisp/mm-bodies.el	Wed Aug  7 16:57:09 2002
+++ mm-bodies.el	Fri Nov  1 07:35:07 2002
@@ -143,6 +143,7 @@
       bits)
      ((and (not mm-use-ultra-safe-encoding)
 	   (not longp)
+	   (not (eq '7bit (cdr (assq charset mm-body-charset-encoding-alist))))
 	   (or (eq t (cdr message-posting-charset))
 	       (memq charset (cdr message-posting-charset))
 	       (eq charset mail-parse-charset)))

Re: posting-charset abolition

[Katsumi Yamaoka]

>>>>> In <m37kfxsryu.fsf@defun.localdomain>
>>>>>	Jesper Harder <harder@ifa.au.dk> wrote:

> Katsumi Yamaoka <yamaoka@jpl.org> writes:

>> (eval-after-load "gnus-msg"
>>   '(if (boundp 'gnus-group-posting-charset-alist)
>>        (let ((news (assq 'message-this-is-news
>> 			 gnus-group-posting-charset-alist)))
>> 	 (if news
>> 	     (setcdr news '(nil nil))))))

> I think this is a kludge.  Shouldn't we rather fix `mm-body-encoding' to
> return the proper CTE for 7bit charsets?

> What do you think about this fix?

Wow!  That's just the thing, maybe.  Since I didn't have time to
look at a code recently, I will confirm your change later and
use it.  Thanks a lot!
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Katsumi Yamaoka]

>>>>> In <yotl3cql7lc5.fsf@jpl.org>
>>>>>	Katsumi Yamaoka <yamaoka@jpl.org> wrote:

>>>>>> In <m37kfxsryu.fsf@defun.localdomain>
>>>>>>	Jesper Harder <harder@ifa.au.dk> wrote:

>> What do you think about this fix?

> Wow!  That's just the thing, maybe.  Since I didn't have time to
> look at a code recently, I will confirm your change later and
> use it.  Thanks a lot!

I've confirmed that it is perfect.  I've just committed it.

However, the variable gnus-group-posting-charset-alist became
important in another viewpoint in Japanese news articles.

Hey Japanese folks, look at this!

In GNUS v3 or v4 era, posting 8-bit news articles was considered
to be evil in Japanese newsgroups because newsreaders or news
servers were not supporting it.  Though it won't be in such a
condition today, 8-bit news articles still tend to be base64
encoded.  If you don't desire to send 8-bit news articles and
there is an opportunity of writing katakana-jisx0201 (may be
called as hankaku-katakana) or any other special characters, the
following setting is also effective even now:

(eval-after-load "gnus-msg"
  '(if (boundp 'gnus-group-posting-charset-alist)
       (let ((news (assq 'message-this-is-news
			 gnus-group-posting-charset-alist)))
	 (if news
	     (setcdr news '(nil nil))))))
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: posting-charset abolition

[Jesper Harder]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Hey Japanese folks, look at this!
>
> In GNUS v3 or v4 era, posting 8-bit news articles was considered
> to be evil in Japanese newsgroups because newsreaders or news
> servers were not supporting it.  Though it won't be in such a
> condition today, 8-bit news articles still tend to be base64
> encoded.  If you don't desire to send 8-bit news articles and
> there is an opportunity of writing katakana-jisx0201 (may be
> called as hankaku-katakana) or any other special characters, the
> following setting is also effective even now:
>
> (eval-after-load "gnus-msg"
>   '(if (boundp 'gnus-group-posting-charset-alist)
>        (let ((news (assq 'message-this-is-news
> 			 gnus-group-posting-charset-alist)))
> 	 (if news
> 	     (setcdr news '(nil nil))))))

I don't grok Japanese charsets.  But if you think it's a good setting,
wouldn't it be better to add this rule to
`gnus-group-posting-charset-alist' for the Japanese news groups?

Re: posting-charset abolition

[Katsumi]

>>>>> In <m3k7jx4blj.fsf@defun.localdomain>
>>>>>	Jesper Harder <harder@ifa.au.dk> wrote:

> Katsumi Yamaoka <yamaoka@jpl.org> writes:

>> (eval-after-load "gnus-msg"
>>   '(if (boundp 'gnus-group-posting-charset-alist)
>>        (let ((news (assq 'message-this-is-news
>> 			 gnus-group-posting-charset-alist)))
>> 	 (if news
>> 	     (setcdr news '(nil nil))))))

> I don't grok Japanese charsets.  But if you think it's a good
> setting, wouldn't it be better to add this rule to
> `gnus-group-posting-charset-alist' for the Japanese news groups?

Thank you for the suggestion.  However, I don't want to concern
myself with that variable no more.  As I wrote in the beginning
of this thread, I am suspecting the existence of the posting-
charset itself.  When I am motivated again someday, I might do
it. :)
-- 
Katsumi@Yamaoka.cc