7bit and iso-8859-1

7bit and iso-8859-1

[Jon Kvebaek]

In some magical fashion, my current 20.3 w/PG 0.13 manages to rewrite
my mailheaders so the original Content-Transfer-Encoding: 8bit line
changes to 7bit instead. Where can I change this?

In addition, it has started to add the same 7bit mime headers to news
posts - is this a new default, or have i done something utterly
strange?
-- 
Jon Kvebaek <jkv@sys.sol.no>

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Jon Kvebaek <jkv@a.sol.no> writes:

> In some magical fashion, my current 20.3 w/PG 0.13 manages to rewrite
> my mailheaders so the original Content-Transfer-Encoding: 8bit line
> changes to 7bit instead. Where can I change this?

Even if the text is 8bit?  Thïs message is in Latin-1, and should go
out with an 8bit-encoding if everything is ok...  If there is only
ASCII chars in the messages, you should get a charset=us-ascii and a
CTE of 7bit.

> In addition, it has started to add the same 7bit mime headers to news
> posts - is this a new default, or have i done something utterly
> strange?

Messag is now an RFC2047-compliant mailer/poster, which means adding
stuff like that.  There should probably be a way to switch it off,
though, but I want it to be tested.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Jon Kvebaek]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
|
| Even if the text is 8bit?  Thïs message is in Latin-1, and should go
| out with an 8bit-encoding if everything is ok...  If there is only
| ASCII chars in the messages, you should get a charset=us-ascii and a
| CTE of 7bit.

Let's see. Thät mëans that this one should be as well. Maybe just
using æ, ø and å isn't good enough...

-- 
Jon Kvebaek <jkv@sys.sol.no>

Re: 7bit and iso-8859-1

[Lars Balker Rasmussen]

Jon Kvebaek <jkv@a.sol.no> writes:
> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> | Even if the text is 8bit?  Thïs message is in Latin-1, and should go
> | out with an 8bit-encoding if everything is ok...  If there is only
> | ASCII chars in the messages, you should get a charset=us-ascii and a
> | CTE of 7bit.
> 
> Let's see. Thät mëans that this one should be as well. Maybe just
> using æ, ø and å isn't good enough...

I've seen this behaviour as well with Emacs 20.2 and Gnus 5.4-5.6.

Apparantly the 
	(re-search-forward "[^\000-\177]" nil t)
in gnus-inews-insert-mime-headers (gnus-msg.el) isn't too terribly
thrilled with multibyte buffers, but I haven't been able to pinpoint
why.  Possibly all this is caused by a wrongly configured Emacs?

I do 
	(setq mime-editor/transfer-level 8)
	(setup-latin1-environment)
	(setq sendmail-coding-system 'iso-latin-1)
	(load-library "iso-insert")
	(require 'latin-1)
in my .emacs.  Anything wrong with that?

(This is sent from an MBSK'ed 20.2, which gets the re-search-forward right.)
-- 
Lars Balker Rasmussen, Software Engineer, Mjolner Informatics ApS
lbr@mjolner.dk

Re: 7bit and iso-8859-1

[Jon Kvebaek]

Lars Balker Rasmussen <lbr@mjolner.dk> writes:
| 
| Apparantly the 
| 	(re-search-forward "[^\000-\177]" nil t)
| in gnus-inews-insert-mime-headers (gnus-msg.el) isn't too terribly
| thrilled with multibyte buffers, but I haven't been able to pinpoint
| why.  Possibly all this is caused by a wrongly configured Emacs?

Maybe. I can't really see what I'm doing/not doing right
though. There's not much at all concerning charsets in my .emacs:

(standard-display-european t)
 
| I do 
| 	(setq mime-editor/transfer-level 8)
| 	(setup-latin1-environment)
| 	(setq sendmail-coding-system 'iso-latin-1)
| 	(load-library "iso-insert")
| 	(require 'latin-1)

I added these, but isn't mime-editor something out of TM? Anyway, nae
luck with that. Should i submit a bug report?

| in my .emacs.  Anything wrong with that?

Same "problem" - as I bet you can see.

-- 
Jon Kvebaek <jkv@sys.sol.no>
Wrinkled earlobes are a sign of impending heart attacks.

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Jon Kvebaek <jkv@a.sol.no> writes:

> | in my .emacs.  Anything wrong with that?
> 
> Same "problem" - as I bet you can see.

Your headers were right in this mail -- charset=us-ascii and CTE 7bit.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Lars Balker Rasmussen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Jon Kvebaek <jkv@a.sol.no> writes:
> > | in my .emacs.  Anything wrong with that?
> > 
> > Same "problem" - as I bet you can see.
> 
> Your headers were right in this mail -- charset=us-ascii and CTE 7bit.

Yes well, we knew they'd do that :-)  It's the other way around that's a
problem  (MULE and gnus-inews-insert-mime-headers will not detect 8bit.)
-- 
Lars Balker Rasmussen, Software Engineer, Mjolner Informatics ApS
lbr@mjolner.dk

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Lars Balker Rasmussen <lbr@mjolner.dk> writes:

> I've seen this behaviour as well with Emacs 20.2 and Gnus 5.4-5.6.
> 
> Apparantly the 
> 	(re-search-forward "[^\000-\177]" nil t)
> in gnus-inews-insert-mime-headers (gnus-msg.el) isn't too terribly
> thrilled with multibyte buffers, but I haven't been able to pinpoint
> why.

That function is not supposed to be used in pgnus, so I've removed it
now. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Jon Kvebaek <jkv@a.sol.no> writes:

> Let's see. Thät mëans that this one should be as well. Maybe just
> using æ, ø and å isn't good enough...

No, you got a "7bit" CTE, but the charset was right.

Try the following:

Put "Thïs" into an empty buffer, and then eval
`(progn (mm-encode-body) (mm-body-encoding))'.  You should get
`7bit'.  If not, go to the "ï" and eval
`(char-charset (following-char))'.  You should get `latin-iso8859-1'.
If you get `ascii', then, uhm.  Uhm?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Karl Kleinpaste]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

Jon Kvebaek <jkv@a.sol.no> writes:
>> Let's see. Thät mëans that this one should be as well. Maybe just
>> using æ, ø and å isn't good enough...

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> No, you got a "7bit" CTE, but the charset was right.

For the ?humor? value, I thought you should be aware that your choices
are at times being overridden by MTAs.  Both of your messages
contained these headers, as received here:

 MIME-Version: 1.0
 Content-Type: text/plain; charset="iso-8859-1"
 Content-Transfer-Encoding: 8bit
 X-MIME-Autoconverted: from quoted-printable to 8bit by pocari-sweat.jprc.com id KAA15510

Scary.

Standard sendmail; a bland RH5.1 installation with sendmail 8.8.7.

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Karl Kleinpaste <karl@jprc.com> writes:

> For the ?humor? value, I thought you should be aware that your choices
> are at times being overridden by MTAs.  Both of your messages
> contained these headers, as received here:
> 
>  MIME-Version: 1.0
>  Content-Type: text/plain; charset="iso-8859-1"
>  Content-Transfer-Encoding: 8bit
>  X-MIME-Autoconverted: from quoted-printable to 8bit by pocari-sweat.jprc.com id KAA15510
> 
> Scary.

What makes it even more scary is that it wasn't q-p'd when I sent it
out, so something must have q-p'd it before your MTA un-q-p'd it
again.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> What makes it even more scary is that it wasn't q-p'd when I sent it
> out, so something must have q-p'd it before your MTA un-q-p'd it
> again.  :-)

AFAIK sendmail does this as default today, if the MTA it's talking to
doesn't speak ESMTP and says that it understands 8BITMIME.


- Steinar

Re: 7bit and iso-8859-1

[Jon Kvebaek]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
|
| No, you got a "7bit" CTE, but the charset was right.

Yup.
 
| Try the following:
| 
| Put "Thïs" into an empty buffer, and then eval
| `(progn (mm-encode-body) (mm-body-encoding))'.  You should get
| `7bit'.  

Indeed I do.

|If not, go to the "ï" and eval

I'll do it anyway

| `(char-charset (following-char))'.  You should get `latin-iso8859-1'.
| If you get `ascii', then, uhm.  Uhm?

I do get ascii. On every fscking letter in "Thïs".

Another thing I just realised: If I hit C-\, it says:
 "Can't activate input method `latin-1-prefix'"

Now is that a Bad Thïng? Uhm?
-- 
Jon Kvebaek <jkv@sys.sol.no>
Wrinkled earlobes are a sign of impending heart attacks.

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="us-ascii", Size: 691 bytes --]

Jon Kvebaek <jkv@a.sol.no> writes:

> | Put "Thïs" into an empty buffer, and then eval
> | `(progn (mm-encode-body) (mm-body-encoding))'.  You should get
> | `7bit'.  
> 
> Indeed I do.

Sorry; you should get `8bit'.  :-)

> I do get ascii. On every fscking letter in "Thïs".

Eek.

> Another thing I just realised: If I hit C-\, it says:
>  "Can't activate input method `latin-1-prefix'"

I get the same.

default-input-method's value is 
"latin-1-prefix"

Eh.  What does this mean?  Hm.  I see.  But why is it that (by
default?) if Emacs can't activate it?  Is it a bug, or is it, uh.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Kai Grossjohann]

>>>>> Jon Kvebaek <jkv@a.sol.no> writes:

  Jon> Another thing I just realised: If I hit C-\, it says:
  Jon> "Can't activate input method `latin-1-prefix'"

>>>>> On 02 Sep 1998, Lars Magne Ingebrigtsen said:

  Lars> I get the same.

  Lars> default-input-method's value is 
  Lars> "latin-1-prefix"

  Lars> Eh.  What does this mean?  Hm.  I see.  But why is it that (by
  Lars> default?) if Emacs can't activate it?  Is it a bug, or is it, uh.

Well, things begin to work when you install LEIM along with Emacs.
But still, this is a bug, I'd suppose.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: 7bit and iso-8859-1

[Istvan Marko]

Jon Kvebaek <jkv@a.sol.no> writes:

> Another thing I just realised: If I hit C-\, it says:
>  "Can't activate input method `latin-1-prefix'"
> 
> Now is that a Bad Thong? Uhm?

This just means that you didn't include LEIM (Libraries of Emacs Input
Methods) in your Emacs installation. LEIM would allow you to use
various input methods to enter all those funky characters. Great stuff
if you want to write Tibetan text and your keyboard doesn't generate
all the Tibetan characters.

It's distributed separately in the file leim-20.3.tar.gz

See the Emacs INSTALL file for installation instructions


-- 
	Istvan

Re: 7bit and iso-8859-1

[Lars Balker Rasmussen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Try the following:
> 
> Put "Thïs" into an empty buffer, and then eval
> `(progn (mm-encode-body) (mm-body-encoding))'.  You should get
> `7bit'.  If not, go to the "ï" and eval
> `(char-charset (following-char))'.  You should get `latin-iso8859-1'.
> If you get `ascii', then, uhm.  Uhm?

I get the same problems as Jon (pgnus 0.13, Emacs-20.3), and I think I
know why:

(Seems that iso-insert is now default in 20.3, but it's not my build so
I can't say what they have put into it.)

Running `emacs-20.3 -q', `C-x 8 a' inserts what looks like an å, except
(char-charset (following-char)) on it says ascii.  

C-h k C-x 8 a
	C-x 8 a runs the command insert-a-ring

/usr/local/share/emacs/20.3/lisp/international/iso-insert.el:
	(defun insert-a-ring ()
	   (interactive "*")
	   (insert 229)
	)

Also, writing a simple test-message with just a couple of å's will have
Gnus complain about posting an empty message (?).

If I cut'n'paste an å from an xterm, I get the expected latin-iso8859-1
from (char-charset (following-char)).

Yes, I have an American keyboard, and will have to jump through hoops to
get accented chars.  So I'm wondering; what's the REAL way of getting
iso8859-1 chars under Mule, when iso-insert obviously is broken?
-- 
Lars Balker Rasmussen, Software Engineer, Mjolner Informatics ApS
lbr@mjolner.dk

Re: 7bit and iso-8859-1

[Lars Magne Ingebrigtsen]

Lars Balker Rasmussen <lbr@mjolner.dk> writes:

> Running `emacs-20.3 -q', `C-x 8 a' inserts what looks like an å, except
> (char-charset (following-char)) on it says ascii.  

Surely this must be a bug.  Report it with `M-x report-emacs-bug'.

> Also, writing a simple test-message with just a couple of å's will have
> Gnus complain about posting an empty message (?).

So it does.  I've now sent a bug report on both these things.

> If I cut'n'paste an å from an xterm, I get the expected latin-iso8859-1
> from (char-charset (following-char)).

So do I.  

> Yes, I have an American keyboard, and will have to jump through hoops to
> get accented chars.  So I'm wondering; what's the REAL way of getting
> iso8859-1 chars under Mule, when iso-insert obviously is broken?

I also have an American keyboard -- a Windows one, even.  I have all
the interesting chars on the Right-Alt keymap.  And who can live
without Hyper and Super modifiers?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: 7bit and iso-8859-1

[Kai Grossjohann]

>>>>> On 02 Sep 1998, Lars Balker Rasmussen said:

  Lars> Yes, I have an American keyboard, and will have to jump
  Lars> through hoops to get accented chars.  So I'm wondering; what's
  Lars> the REAL way of getting iso8859-1 chars under Mule, when
  Lars> iso-insert obviously is broken?

I installed Leim and I use the input method german-prefix.  So I type
" a and get ä.  Similar with " s and ß.  There is also a
latin-1-prefix input method.  Dunno about Danish, though.  Wait a
minute -- yes, there's a couple of Danish input methods, too.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: 7bit and iso-8859-1

[Steinar Bang]

>>>>> Jon Kvebaek <jkv@a.sol.no>:

> In some magical fashion, my current 20.3 w/PG 0.13 manages to rewrite
> my mailheaders so the original Content-Transfer-Encoding: 8bit line
> changes to 7bit instead. 

Ouch! Quite a few MUAs seems to label articles with 8bit content this
way.  It causes a bitstrip somewhere by either fetchmail or the
sendmail it delivers to (the sendmail and configuration, that came
standard with RH5.0).