de .* and gnus-group-posting-charset-alist

de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

A discussion at de.comp.gnu showed that following change would be
welcome; I don't know what the reason was to set it di "nil" for the de
hierarchy?

Index: gnus-msg.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/gnus-msg.el,v
retrieving revision 5.47
diff -u -r5.47 gnus-msg.el
--- gnus-msg.el	1999/12/04 08:26:03	5.47
+++ gnus-msg.el	1999/12/07 20:44:31
@@ -102,8 +102,8 @@
 
 (defcustom gnus-group-posting-charset-alist
   '(("^no\\." iso-8859-1)
+    ("^de\\." iso-8859-1)
     (message-this-is-mail nil)
-    ("^de\\." nil)
     (".*" iso-8859-1)
     (message-this-is-news iso-8859-1))
   "Alist of regexps (to match group names) and default charsets to be unencoded when posting."

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Karl EICHWALDER <ke@gnu.franken.de> writes:

>  (defcustom gnus-group-posting-charset-alist
>    '(("^no\\." iso-8859-1)
> +    ("^de\\." iso-8859-1)
>      (message-this-is-mail nil)
> -    ("^de\\." nil)

This seems to be very reasonable; I've committed it to the CVS.

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Karl EICHWALDER <ke@gnu.franken.de> writes:

> A discussion at de.comp.gnu showed that following change would be
> welcome; I don't know what the reason was to set it di "nil" for the de
> hierarchy?

I discussed with the guys in de.comm.software.newsreader, and they
explained to me that Latin 1 chars in headers must be escaped -- I
noticed this because I have a Latin 1 char in my name, and this
character was posted as-is rather than the following:

Kai =?iso-8859-1?q?Gro=DFjohann?=

The people over there also said that it is okay to have Latin 1
characters in the body of a posting, provided the MIME headers are
right.  But there are no MIME headers which apply to the header of a
posting, hence it is necessary to always encode non-ASCII characters
in headers.

So, if you tell Gnus to post Latin 1 characters as-is in de.ALL,
people who post in de.comm.software.newsreader will be flamed :-/

I think we have the following options:

* Tell Gnus to always encoding Latin 1 chars in de.ALL.  This gives us
  correct postings with possibly superfluous encoding.

* Tell Gnus to never encode Latin 1 chars in de.ALL.  This gives
  problems for people with Latin 1 characters in their name.

* Like previous, but forbid use of non-ASCII characters in headers.

I think the second option is out of the question, and I don't like the
last option, either.  My name is Großjohann and not Grossjohann!

What do you think?

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

|   I discussed with the guys in de.comm.software.newsreader, and they
|   explained to me that Latin 1 chars in headers must be escaped

Yes, of course.  gnus-msg.el and/or message.el has to take care about
it.  AFAIK, encoding header lines is independend from the encoding of
the body.

|   * Tell Gnus to always encoding Latin 1 chars in de.ALL.  This gives us
|     correct postings with possibly superfluous encoding.
|   
|   * Tell Gnus to never encode Latin 1 chars in de.ALL.  This gives
|     problems for people with Latin 1 characters in their name.
|   
|   * Like previous, but forbid use of non-ASCII characters in headers.

Please, implement the 4th option:

    * Message bodies: Tell Gnus to never encode Latin 1 chars in de.ALL.
      Header lines: Encode non-ASCII characters.

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Karl EICHWALDER <ke@gnu.franken.de> writes:

> |   I discussed with the guys in de.comm.software.newsreader, and they
> |   explained to me that Latin 1 chars in headers must be escaped

How could I forget that discussion?  Nevertheless, I think that using
8-bit characters in the header is better than using quoted-printable
for the whole article, because in my experience, only few newsreaders
will handle quoted-printable, while most can cope even with undeclared
8-bit characters.  That's a sad thing, but it's the reality.

> Please, implement the 4th option:
> 
>     * Message bodies: Tell Gnus to never encode Latin 1 chars in de.ALL.
>       Header lines: Encode non-ASCII characters.

Okay, it's in the CVS.  Now, Gnus should never use unencoded 8-bit
characters in message headers.  If this is a problem, please complain.
Maybe we can add a customize option or something like that if it's
necessary.

Re: de .* and gnus-group-posting-charset-alist

[Bjørn Mork]

Florian Weimer <fw@s.netic.de> writes:
> Karl EICHWALDER <ke@gnu.franken.de> writes:
> 
> > Please, implement the 4th option:
> > 
> >     * Message bodies: Tell Gnus to never encode Latin 1 chars in de.ALL.
> >       Header lines: Encode non-ASCII characters.
> 
> Okay, it's in the CVS.  Now, Gnus should never use unencoded 8-bit
> characters in message headers.  If this is a problem, please complain.

It's a problem if this is used everywhere. The no.* hierarchy still
requires unencoded 8bit latin1 in message headers.

> Maybe we can add a customize option or something like that if it's
> necessary.

I think it should be customizable based on hierarchy.


Bjørn

Re: de .* and gnus-group-posting-charset-alist

[Christophe Cuq]

>> "BM" == Bjørn Mork <bmork@dod.no> writes:

  BM> It's a problem if this is used everywhere. The no.* hierarchy still
  BM> requires unencoded 8bit latin1 in message headers.

Same for fr.*.

No encoding should be done, neither in the headers nor in
the body. Only 8bit characters.

  BM> I think it should be customizable based on hierarchy.

It must.

-- 
CHC

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Karl EICHWALDER <ke@gnu.franken.de> writes:

> Yes, of course.  gnus-msg.el and/or message.el has to take care about
> it.  AFAIK, encoding header lines is independend from the encoding of
> the body.

It should be indenpendent but AFAIK, it isn't.  Hence my `nil' entry
for de.ALL -- it's just a workaround, not a solution.  The solution
you desire (making encoding in headers independent from encoding in
the body) is the Right Thing, of course.  But what should we do until
it is done?

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

|   It should be indenpendent but AFAIK, it isn't.

Oops, I didn't know this fact.

|   But what should we do until it is done?

I vote to obey the standard and to restore the old settings (this means
Gnus should also encode the body of the message).

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I discussed with the guys in de.comm.software.newsreader, and they
> explained to me that Latin 1 chars in headers must be escaped 

Well, using QP in headers _used to_ cause all sorts of problems, and
getting you seriously flamed in most European hierarchies.  However,
maybe the consensus has changed.  I believe Free Agent is the last
significant newsreader without QP decoding, and the latest statistics
for the dk hierarchy put it behind Agent in popularity.  Last time I
saw someone complain about QP in headers, he was told it was his own
problem for using Free Agent instead of buying Agent ($29).

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > I discussed with the guys in de.comm.software.newsreader, and they
> > explained to me that Latin 1 chars in headers must be escaped 
> 
> Well, using QP in headers _used to_ cause all sorts of problems, and
> getting you seriously flamed in most European hierarchies. 

No one has flamed me for using QP yet.  However, people did complain
when I posted in UTF-8, though. ;)

> However, maybe the consensus has changed.  I believe Free Agent is
> the last significant newsreader without QP decoding,

Most of the wannabe newsreaders don't handle QP either.  IMHO, undeclared
8-bit characters are still less problematic.  But generating them is
an RFC infringement (and as such, quite unpopular in de.*).

> and the latest statistics for the dk hierarchy put it behind Agent
> in popularity.  Last time I saw someone complain about QP in
> headers, he was told it was his own problem for using Free Agent
> instead of buying Agent ($29).

Free Agent doesn't seem to be a problem any more, because you can
buy Agent again.  (I think the developers have discontinued from
discontinuing it).

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Florian Weimer <fw@s.netic.de> writes:

> Most of the wannabe newsreaders don't handle QP either.  IMHO, undeclared
> 8-bit characters are still less problematic.  But generating them is
> an RFC infringement (and as such, quite unpopular in de.*).

Well, in dk.*, given the choice between a solution that works and a
solution that follows a standard, we generally prefer the first. 

Re: de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

|   Well, in dk.*, given the choice between a solution that works and a
|   solution that follows a standard, we generally prefer the first. 

One of the problems is, that it seems not to work for Gnus.  Please,
have a look at

    Message-ID: <un1rfhryo.fsf@dnv.com>

This article lacks MIME information but contains 8bit character in the
>From line.  Gnus (under GNU Emacs 20.5) adds \201.

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Karl EICHWALDER <ke@gnu.franken.de> writes:

> This article lacks MIME information but contains 8bit character in the
> From line.  Gnus (under GNU Emacs 20.5) adds \201.

That's a Gnus/Emacs bug.

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Well, in dk.*, given the choice between a solution that works and a
> solution that follows a standard, we generally prefer the first. 

Isn't it wonderful?  Germans require standards compliance in all
cases.  I used to get flamed for the following From line because it
didn't contain a real name part:

    From: Kai.Grossjohann@CS.Uni-Dortmund.DE

Sometimes, I cannot understand my own nation :-/

They suggested that I use `ss' rather than `ß' in my name to make
header encoding unnecessary.  But my name is Großjohann.  Some Germans
are pronouncing my name wrongly because they think it is Grossjohann
:-(  And I ask you what would be the sense in having the following
>From line?

    From: Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE>

I'd really like to find a way to use ß in my name in de.ALL.

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

>     From: Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE>
> 
> I'd really like to find a way to use ß in my name in de.ALL.

You have to use MIME and encoded-words.  There's no other way.

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Florian Weimer <fw@s.netic.de> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > I'd really like to find a way to use ß in my name in de.ALL.
> 
> You have to use MIME and encoded-words.  There's no other way.

Yes, but you also need the `encode the header even if the body is
ASCII' thing.  Which I think you added.

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Florian Weimer <fw@s.netic.de> writes:
> 
> > Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> > 
> > > I'd really like to find a way to use ß in my name in de.ALL.
> > 
> > You have to use MIME and encoded-words.  There's no other way.
> 
> Yes, but you also need the `encode the header even if the body is
> ASCII' thing.  Which I think you added.

Yes, this was my intention.

I've tuned the regular expression in `gnus-group-posting-charset-alist'
a bit.  Now, it should handle crosspostings correctly.  Could somebody
who cares about this stuff test it, please?  Is the set of hierarchies
which require 8-bit characters complete (dk.*, fr.*, no.*)?

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Germans require standards compliance in all cases.

The fun part is that there isn't any standard RFC 1036 has expired,
and USEFOR does not even (officially) has an draft yet.  

MIME is a mail standard, the only thing giving it meaning on news is
the USEFOR working paper.  However, the USEFOR wp _does_ allow 8-bit
headers, however not in Latin 1, only in UTF-8.  But try sending 8-bit
UTF-8 headers, I suspect the people demanding QP will complain even
more.

In reality, Usenet has for many years lived on a set of unwritten
conventions, drawing from bit and parts of many standards, but mostly
just common usage.

To be fair, moving towards QP in headers is the right thing to do, it
will allow us to phase the hierarchy specific 8-bit interpretation out
and (much later) UTF-8 in.

Re: de .* and gnus-group-posting-charset-alist

[Russ Allbery]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> The fun part is that there isn't any standard RFC 1036 has expired, and
> USEFOR does not even (officially) has an draft yet.

RFC 1036 hasn't expired to my knowledge; I don't think informational RFCs
*can* expire.  It's certainly outdated enough to warrant expiration, but
so far as I know it's still an active standard.

> MIME is a mail standard, the only thing giving it meaning on news is the
> USEFOR working paper.

Well, and the RFC 1036 strongly implied statement that the mail standards
are canonical for news articles except for the exceptions given in that
RFC.

-- 
Russ Allbery (rra@stanford.edu)         <URL:http://www.eyrie.org/~eagle/>

Re: de .* and gnus-group-posting-charset-alist

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Russ Allbery <rra@stanford.edu>  on Tue, 21 Dec 1999
| RFC 1036 hasn't expired to my knowledge; I don't think informational RFCs
| *can* expire.  It's certainly outdated enough to warrant expiration, but
| so far as I know it's still an active standard.

RFCs do not expire so much as are superceded by newer RFCs describing newer 
and hopefully superior incarnations of the specification.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE4YCC9gl+vIlSVSNkRAkX0AKDkl9toR0iSWFA+eMgv4K+xCEgG7wCfYFvn
gcvcSFti+d82puB3KdqF3+Q=
=YlF2
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 

Re: de .* and gnus-group-posting-charset-alist

[smarkacz]

>>>>> "FW" == Florian Weimer <fw@s.netic.de> writes:

    FW> Per Abrahamsen <abraham@dina.kvl.dk> writes:
    >> However, maybe the consensus has changed.  I believe Free Agent is
    >> the last significant newsreader without QP decoding,

    FW> Most of the wannabe newsreaders don't handle QP either.  IMHO,
    FW> undeclared 8-bit characters are still less problematic.

How do you know whether these undeclared characters are Latin1, Latin2,
Windows1250 (yuck) or any other coding system?
-- 
[Jacek P. Szymański]##[smarkacz@anathema.eu.org]##[IRC: smarkacz]
[Network Admin]#####[ZmieńNa_Fanklub: http://anathema.eu.org/znf]
###[Monday is an awful way to spend one seventh of your life.]###

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

smarkacz@anathema.eu.org (smarkacz) writes:

>     FW> Most of the wannabe newsreaders don't handle QP either.  IMHO,
>     FW> undeclared 8-bit characters are still less problematic.
> 
> How do you know whether these undeclared characters are Latin1, Latin2,
> Windows1250 (yuck) or any other coding system?

That's no problem as long as both ends agree which charset to use.
Summarizing the discussions, such an agreement does exist for dk.*,
fr.*, and no.*, but not for de.* (after all, we are Germans, and it's
more important to follow the wording of the RFCs than to have a working
solution ;).

Of course, in general, you are right.  There isn't any agreement,
and that's why you should use MIME.

Re: de .* and gnus-group-posting-charset-alist

[Toby Speight]

Florian> Florian Weimer <URL:mailto:fw@s.netic.de>

0> In article <87so15xmug.fsf@deneb.cygnus.argh.org>, Florian wrote:

Florian> smarkacz@anathema.eu.org (smarkacz) writes:

>> How do you know whether these undeclared characters are Latin1,
>> Latin2, Windows1250 (yuck) or any other coding system?

Florian> That's no problem as long as both ends agree which charset to
Florian> use.  Summarizing the discussions, such an agreement does
Florian> exist for dk.*, fr.*, and no.*, but not for de.* (after all,
Florian> we are Germans, and it's more important to follow the wording
Florian> of the RFCs than to have a working solution ;).

Crossposting brings up a big problem with this approach.  Thinking of
this made me wonder: what does Gnus do if you post an article in groups
with different defaults?  Use the first-mentioned one?  (likely) Or
label it explicitly with MIME?  (better)

And for reading, does it use the default for the group you're reading
it in, or some other value?

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

> Crossposting brings up a big problem with this approach.  Thinking of
> this made me wonder: what does Gnus do if you post an article in groups
> with different defaults?  Use the first-mentioned one?  (likely) Or
> label it explicitly with MIME?  (better)

These things seem to be rather complicated. *sigh*

Perhaps we shouldn't change the current behaviour (always qp-encode
the header, as mandated by the RFCs).

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Florian Weimer <fw@s.netic.de> writes:

> Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:
> 
> > Crossposting brings up a big problem with this approach.  Thinking of
> > this made me wonder: what does Gnus do if you post an article in groups
> > with different defaults?  Use the first-mentioned one?  (likely) Or
> > label it explicitly with MIME?  (better)
> 
> These things seem to be rather complicated. *sigh*

But not different from the hierarchy specific rules for selecting
charset or encoding body-parts. 

> Perhaps we shouldn't change the current behaviour (always qp-encode
> the header, as mandated by the RFCs).

Which means Gnus will be broken for some hierarchies where it
currently works.

Re: de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

|   Which means Gnus will be broken for some hierarchies where it
|   currently works.

Some people might think so.  But it's that difficult to solve the
problem:

    Follow the written standards (RFCs) as close as possible.

    If the standards let room for interpretation, don't change Gnus if
    the feature is already implemented (at least wait until the "next"
    release of Gnus).

    Make it easy for the user to customize the tool (NEWS, README,
    *.texi).

(PS. I do read mailinglists where people are used to post in non
     standard encodings; this works for them quite well.  Because these
     people aren't able to follow good internet standards, I
     (resp. Gnus) was forced to work around the problem.  As it stands,
     Gnus is broken for some mailinglist -- to follow your
     argumentation.)

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

Karl EICHWALDER <ke@gnu.franken.de> writes:

> Per Abrahamsen <abraham@dina.kvl.dk> writes:
> 
> |   Which means Gnus will be broken for some hierarchies where it
> |   currently works.
> 
> Some people might think so.  But it's that difficult to solve the
> problem:

The current (5.8.2) solution, to make the behaviour depend on
hierarchy, is close to perfect.  Rare occurences such as
cross-hierachy crossposting should not affect this decision.  Maybe
the default is wrong for de.*, but that should not mean that the new
German preferences should be forced into all other hierarchies.

> (PS. I do read mailinglists where people are used to post in non
>      standard encodings; this works for them quite well.  Because these
>      people aren't able to follow good internet standards, I
>      (resp. Gnus) was forced to work around the problem.  As it stands,
>      Gnus is broken for some mailinglist -- to follow your
>      argumentation.)

Of course, mailing list conventions are even more subject to the
owners wishes than local hierarchies.  The owner usually prefer a
solution that works best for the users over a solution that follows
the words of some arbitrary standard.  Which is how it should be.

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> The current (5.8.2) solution, to make the behaviour depend on
> hierarchy, is close to perfect.

No, it isn't.  Currently, you've got two choices: Either you encode the
whole article with quoted-printable (which causes pain with clueless
newsreaders), or you use 8-bit in the article *and* the header, which
is against RFC 822, RFC 1036, RFC 2047 (and probably more...)

I'm currently testing the following extension (of course, a bit of
additional code is required):

(defcustom gnus-group-posting-charset-alist
  '(("^\\(no\\|fr\\|dk\\)\\.[^,]*" iso-8859-1 (iso-8859-1))
    (message-this-is-mail nil nil)
    (message-this-is-news nil t))
  "Alist of regexps and permitted unencoded charsets for posting.
Each element of the alist has the form (TEST HEADER BODY-LIST), where
TEST is either a regular expression matching the newsgroup header or a
variable to query,
HEADER is the charset which may be left unencoded in the header (nil
means encode all charsets),
BODY-LIST is a list of charsets which may be encoded using 8bit
content-transfer encoding in the body, or one of the special values
nil (always encode using quoted-printable) or t (always use 8bit).

Note that any value other tha nil for HEADER infringes some RFCs, so
use this option with care."
  :type '(repeat (list 
		  (choice :tag "Where"
		   (regexp :tag "Group")
		   (const :tag "Mail message" :value message-this-is-mail)
		   (const :tag "News article" :value message-this-is-news))
		  (choice :tag "Header"
		   (const :tag "Always encoded" nil)
		   (symbol :tag "Permitted charset"))
		  (choice :tag "Body"
			  (const :tag "Any charset" :value t)
			  (const :tag "Always encoded" :value nil)
			  (repeat :tag "Permitted charsets"
				  (symbol :tag "Charset")))))
  :group 'gnus-charset)

The idea is that you can chose the header charset and the charsets
which will not be encoded in the body in a completely independent manner.

It seems that my current implementation is nearly complete and mostly
correct, but I don't have Internet access at the moment and it's about
time to go to bed anyway ;), so I haven't committed the changes yet.

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Okay, here it is:

1999-12-21  Florian Weimer  <fw@s.netic.de>

	* gnus-msg.el (gnus-group-posting-charset-alist): New layout.
	(gnus-setup-message): No longer make `message-posting-charset'
	buffer-local.
	(gnus-setup-posting-charset): Reflect the new layout of
	`gnus-group-posting-charset-alist' and `message-posting-charset'.

	* message.el (message-send-mail): Bind `message-this-is-mail' and
	`message-posting-charset'.
	(message-send-news): Dito, and honour new layout of 
	`message-posting-charset'.
	(message-encode-message-body): Ignore `message-posting-charset'.
	
	* mm-bodies.el (mm-body-encoding): Consider
	`message-posting-charset' when deciding whether to use 8bit.

	* rfc2047.el (rfc2047-encode-message-header): Back out change.
	(rfc2047-encodable-p): Now solely for headers; use
	`message-posting-charset'.
	
`message-posting-charset' is now a list of the form
(HEADER-CHARSET BODY-CHARSETS...).  It's still taken from
`gnus-group-posting-charset-alist'.  If HEADER-CHARSET is nil, 8-bit
characters are always encoded in the header.  Otherwise, HEADER-CHARSET
is a symbol which denotes the charset which is to be left unencoded
(blechhh!).  BODY-CHARSETS is either t (which means: always use 8-bit
encoding) or a list of symbols naming charsets which can be encoded
using 8-bit (other charsets will be encoded as quoted-printable).
The special value nil indicates that quoted-printable should be used
for all charsets.

I sincerely hope that this will find its way into 5.8.3.  Some stuff like
this is certainly required to solve this nasty problem once and for all.

Re: de .* and gnus-group-posting-charset-alist

[Karl EICHWALDER]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

|   The current (5.8.2) solution, to make the behaviour depend on
|   hierarchy, is close to perfect.  Rare occurences such as
|   cross-hierachy crossposting should not affect this decision.

Thinking about it again, I've to admit that you are right -- sorry for
making noise ;)

|   Maybe the default is wrong for de.*, but that should not mean that
|   the new German preferences should be forced into all other
|   hierarchies.

Yes, difficult question -- I don't know what we want.  My wish is to
follow the standard as close as possible (= no 8bit header lines).  At
least, we've to make sure that courtesy copies are properly encoded.
There are quite a lot "firewall" configurations out there which reject
mails with 8bit subject lines.

|   Of course, mailing list conventions are even more subject to the
|   owners wishes than local hierarchies.  The owner usually prefer a
|   solution that works best for the users over a solution that follows
|   the words of some arbitrary standard.  Which is how it should be.

Nice theory ;)  Once I complained other participants signaled that they
didn't like 8bit postings.  It's a strange, strange world.

-- 
work    : ke@suse.de                          |
        : http://www.suse.de/~ke/             |       ------    ,__o
personal: ke@gnu.franken.de                   |      ------   _-\_<,
        : http://www.franken.de/users/gnu/ke/ |     ------   (*)/'(*)

Re: de .* and gnus-group-posting-charset-alist

[Per Abrahamsen]

smarkacz@anathema.eu.org (smarkacz) writes:

> How do you know whether these undeclared characters are Latin1, Latin2,
> Windows1250 (yuck) or any other coding system?

Easy, look at the newsgroups header.  In dk.* the default character
set is Latin-1.  In relcom.* it is koi-8.  Separate hierarchies have
separate rules.

Obviously this causes problems for software less smart than Gnus, but
up until recently (and maybe still?) QP caused even more problems.

Re: de .* and gnus-group-posting-charset-alist

[Hrvoje Niksic]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> smarkacz@anathema.eu.org (smarkacz) writes:
> 
> > How do you know whether these undeclared characters are Latin1, Latin2,
> > Windows1250 (yuck) or any other coding system?
> 
> Easy, look at the newsgroups header.

That doesn't really work.

Now that I use Gnus with Mule, I have real problems with Croatian
users' broken new agents, which mis-declare Latin 2 articles as Latin
1.  Poor Gnus thinks the message is Latin 1, and renders it that way.
:-(  Unix Slrn/Tin users are especially guilty of this, because many
of them don't bother to setup their newsreaders correctly.  After all,
*their* newsreaders just dump 8bit data to the terminal, which uses
Latin 2 fonts, so *they* see it right.

Crock like Windows-1250 can be handled, but not generally, i.e. such
handling is but a local hack, based on the fact that only two
Croatian-specific characters differ in positions between windows-1250
and Latin 2.  So you can "convert" between the two without loss of
information by swapping the two characters' codes.

It all just sucks.

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Now that I use Gnus with Mule, I have real problems with Croatian
> users' broken new agents, which mis-declare Latin 2 articles as Latin
> 1.  Poor Gnus thinks the message is Latin 1, and renders it that way.

I think it would be good to have a command in Gnus which says: pretend
that this MIME part (or maybe the whole message) has charset=foo even
though it says charset=bar in the header.

Then you could make a key binding and then you only need to hit F5 on
all such screwed messages and all will be well.

Any takers?  Or does this command exist already?

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Florian Weimer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I think it would be good to have a command in Gnus which says: pretend
> that this MIME part (or maybe the whole message) has charset=foo even
> though it says charset=bar in the header.

Well, there's `gnus-group-ignored-charsets-alist', and you can set the
default charset for a group in the group customization buffer.

Re: de .* and gnus-group-posting-charset-alist

[Kai Großjohann]

Florian Weimer <fw@s.netic.de> writes:

> Well, there's `gnus-group-ignored-charsets-alist', and you can set the
> default charset for a group in the group customization buffer.

This is nice, but I think it would be useful in its own right to have
a user-level command.  It would be useful for dealing with the
occasional strange article in groups where it isn't expected.  And
maybe some messages that Hrvoje is seeing are correctly labeled Latin-1?

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: de .* and gnus-group-posting-charset-alist

[Hrvoje Niksic]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Hrvoje Niksic <hniksic@iskon.hr> writes:
> 
> > Now that I use Gnus with Mule, I have real problems with Croatian
> > users' broken new agents, which mis-declare Latin 2 articles as Latin
> > 1.  Poor Gnus thinks the message is Latin 1, and renders it that way.
> 
> I think it would be good to have a command in Gnus which says:
> pretend that this MIME part (or maybe the whole message) has
> charset=foo even though it says charset=bar in the header.

Yes, that would be nice.  It could be a problem with multi-parts, but
that command could simply signal an error when encountering a
multipart.

A variable wouldn't cut it, because even in those groups legal Latin 1
messages.  *can* be found.