decoding characters

decoding characters

[Laura Conrad]

I got an email which contained a table of TeX codings for characters,
and also the characters.  Most of them looked like I expected, but
there were 3 that displayed wrong in my gnus:

œ      \oe (displayed as \234)
Π     \OE (displayed as \214)
Ÿ      \"Y (displayed as \237)

I'm using:
xemacs*font: -etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1

in my Xdefaults file.

I also have (x-symbol-initialize) in my .emacs.

In the X-Symbol Grid, all the symbols display fine.  

When I just 'more' the mail file in and xterm, \oe and \OE are blank,
but \"y displays fine, using the same font as I have defined for
xemacs.

The relevant headers from the mail are:

X-Mailer: Mozilla 4.5 (Macintosh; I; PPC)
X-Accept-Language: en,pdf
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="------------4721D1F8D2DD403F6A49197F"
Content-Transfer-Encoding: 8bit

I'm using a hot-off-the-press cvs gnus, updated this afternoon.

Incidentally, this list of characters would make a good test file, if
there's somewhere it could be put for that purpose.

-- 
Laura (mailto:lconrad@world.std.com , http://www.world.std.com/~lconrad/ )
(617) 661-8097	fax: (801) 365-6574 
233 Broadway, Cambridge, MA 02139

Re: decoding characters

[Roland Mas]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 909 bytes --]

Laura Conrad (2000-05-11 15:26:07 -0400) :

> I got an email which contained a table of TeX codings for characters,
> and also the characters.  Most of them looked like I expected, but
> there were 3 that displayed wrong in my gnus:
> 
> œ      \oe (displayed as \234)
> Π     \OE (displayed as \214)
> Ÿ      \"Y (displayed as \237)
> 
> I'm using:
> xemacs*font: -etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1

The point is: these three characters are not in the iso-8859-1
character set (for somewhat stupid reasons, but anyway).  They are in
iso-8859-15, which is an update to the former and includes these three
characters as well as the Euro currency unit sign.

  Unfortunately, iso-8859-15 is not supported by Emacs (as far as I
can tell).  M-x report-emacs-bug?

Roland.
-- 
Roland Mas

It would be hard to be deader without special training.
  -- in Theatre of Cruelty (Terry Pratchett)

Re: decoding characters

[Toby Speight]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

Laura> Laura Conrad <URL:mailto:lconrad@world.std.com>

0> In article <kh7og6crir4.fsf@acme47.nineco.com>, Laura wrote:

Laura> I got an email which contained a table of TeX codings for
Laura> characters, and also the characters.  Most of them looked like
Laura> I expected, but there were 3 that displayed wrong in my gnus:
Laura>
Laura> œ      \oe (displayed as \234)
Laura> Π     \OE (displayed as \214)
Laura> Ÿ      \"Y (displayed as \237)
Laura>
Laura> I'm using: xemacs*font:
Laura> -etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1

Is your Emacs unibyte or multibyte?

IS8859 charsets don't assign characters in the range \200 - \237, so
if you're Unibyte, you shouldn't expect anything there.


Laura> I also have (x-symbol-initialize) in my .emacs.

This doesn't mean anything to me, so forgive me if I misunderstand
something later.


Laura> When I just 'more' the mail file in and xterm, \oe and \OE are
Laura> blank, but \"y displays fine, using the same font as I have
Laura> defined for xemacs.

I think the ETL fonts include \"Y at \237 - but you haven't told this
to Emacs.  (aset standard-display-table ?\237 [?\237])


Laura> The relevant headers from the mail are:
Laura>
Laura> Content-Type: multipart/mixed;
Laura>  boundary="------------4721D1F8D2DD403F6A49197F"

Actually, that's not the most relevant header.  What would be more
useful would be the Content-Type line of the particular part you're
having problems with.

If, as I suspect, it has charset=windows-1252, you might be interested
in the following display table I wrote for Emacs 19 (no warranty for
other emacsen):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Viewing Windows and Mac charsets in articles --]
[-- Type: text/x-emacs-lisp, Size: 4535 bytes --]

(defvar pc-quotes-display-table nil
  "Display table for viewing text containing CP1252 characters.")
(defun pc-quotes-article-display-pc-quotes ()
  "Change the display table of the *Article* buffer so it displays PC quotes."
  (interactive)
  (save-window-excursion
    (set-buffer gnus-article-buffer)
    (setq buffer-display-table pc-quotes-display-table)))

(defvar mac-quotes-display-table nil
  "Display table for viewing text containing Macintosh-specific characters.")
(defun pc-quotes-article-display-mac-quotes ()
  "Change the display table of the *Article* buffer so it displays Mac quotes."
  (interactive)
  (save-window-excursion
    (set-buffer gnus-article-buffer)
    (setq buffer-display-table mac-quotes-display-table)))

(defun pc-quotes-article-display-standard-quotes ()
  "Change the display table of the *Article* buffer to be the standard one."
  (interactive)
  (save-window-excursion
    (set-buffer gnus-article-buffer)
    (setq buffer-display-table standard-display-table)))

(or (boundp 'gnus-summary-mode-map) (require 'gnus-sum))

;;; Display tables
(setq pc-quotes-display-table
      (if standard-display-table
          (copy-sequence standard-display-table)
        (make-display-table)))
(mapcar
 (function
  (lambda (x)
    (aset pc-quotes-display-table (car x)
          (if (member window-system '(w32 win32)) (vector (car x)) (cdr x))
          ;;(cdr x)
          )))
 (list
  '(?\200 . [ 163 ?e ])               ; euro symbol

  '(?\202 . [ ?, ])
  '(?\203 . [ ?f ])
  '(?\204 . [ ?, ?, ])
  '(?\205 . [ ?. ?. ?. ])
  '(?\206 . [ ?+ ])
  '(?\207 . [ ?# ])
  '(?\210 . [ ?^ ])
  '(?\211 . [ ?% ?. ])
  '(?\212 . [ ?\\ ?^ ?S ])
  '(?\213 . [ ?< ])
  '(?\214 . [ ?O ?E ])

  '(?\216 . [ ?\\ ?^ ?Z ])


  '(?\221 . [ ?` ])
  '(?\222 . [ ?' ])
  '(?\223 . [ ?` ?` ])
  '(?\224 . [ ?' ?' ])
  '(?\225 . [ 183 ])                    ; middot (ISO 8859.1)
  '(?\226 . [ ?- ?- ])
  '(?\227 . [ ?- ?- ?- ])
  '(?\230 . [ ?~ ])
  '(?\231 . [ ?[ ?T ?M ?] ])            ; or (vector ?T ?M)
  '(?\232 . [ ?\\ ?^ ?s ])
  '(?\233 . [ ?> ])
  '(?\234 . [ ?o ?e ])

  '(?\236 . [ ?\\ ?^ ?z ])
  '(?\237 . [ ?\\ ?\" ?Y ])))

(setq mac-quotes-display-table
      (if standard-display-table
          (copy-sequence standard-display-table)
        (make-display-table)))
(mapcar
 (function
  (lambda (x)
    (aset mac-quotes-display-table (car x) (cdr x))))
 (list
  '(?\200 . [ ?\304 ])
  '(?\201 . [ ?\305 ])
  '(?\202 . [ ?\307 ])
  '(?\203 . [ ?\311 ])
  '(?\204 . [ ?\321 ])
  '(?\205 . [ ?\326 ])
  '(?\206 . [ ?\334 ])
  '(?\207 . [ ?\341 ])
  '(?\210 . [ ?\340 ])
  '(?\211 . [ ?\342 ])
  '(?\212 . [ ?\344 ])
  '(?\213 . [ ?\343 ])
  '(?\214 . [ ?\345 ])
  '(?\215 . [ ?\347 ])
  '(?\216 . [ ?\351 ])
  '(?\217 . [ ?\350 ])
  '(?\220 . [ ?\352 ])
  '(?\221 . [ ?\353 ])
  '(?\222 . [ ?\355 ])
  '(?\223 . [ ?\354 ])
  '(?\224 . [ ?\356 ])
  '(?\225 . [ ?\357 ])
  '(?\226 . [ ?\361 ])
  '(?\227 . [ ?\363 ])
  '(?\230 . [ ?\362 ])
  '(?\231 . [ ?\364 ])
  '(?\232 . [ ?\366 ])
  '(?\233 . [ ?\365 ])
  '(?\234 . [ ?\372 ])
  '(?\235 . [ ?\371 ])
  '(?\236 . [ ?\373 ])
  '(?\237 . [ ?\374 ])
  '(?\241 . [ ?\260 ])
  '(?\244 . [ ?\247 ])
  '(?\246 . [ ?\266 ])
  '(?\247 . [ ?\337 ])
  '(?\250 . [ ?\256 ])
  '(?\253 . [ ?\264 ])
  '(?\254 . [ ?\250 ])
  '(?\256 . [ ?\306 ])
  '(?\257 . [ ?\330 ])
  '(?\262 . [ ?= ?< ])
  '(?\263 . [ ?> ?= ])
  '(?\264 . [ ?\245 ])
  '(?\273 . [ ?\252 ])
  '(?\274 . [ ?\272 ])
  '(?\276 . [ ?\346 ])
  '(?\277 . [ ?\370 ])
  '(?\300 . [ ?\277 ])
  '(?\301 . [ ?\241 ])
  '(?\302 . [ ?\254 ])
  '(?\307 . [ ?\253 ])
  '(?\310 . [ ?\273 ])
  '(?\311 . [ ?. ?. ?. ])
  '(?\312 . [ ?\240 ])
  '(?\313 . [ ?\300 ])
  '(?\314 . [ ?\303 ])
  '(?\315 . [ ?\325 ])
  '(?\316 . [ ?O ?E ])
  '(?\317 . [ ?o ?e ])
  '(?\320 . [ ?- ?- ])
  '(?\321 . [ ?- ])
  '(?\322 . [ ?` ?` ])
  '(?\323 . [ ?' ?' ])
  '(?\324 . [ ?` ])
  '(?\325 . [ ?' ])
  '(?\326 . [ ?\367 ])
  '(?\327 . [ ?* ])
  '(?\330 . [ ?\377 ])
  '(?\331 . [ ?\377 ])
  '(?\332 . [ ?/ ])
  '(?\333 . [ ?\244 ])
  '(?\334 . [ ?< ])
  '(?\335 . [ ?> ])
  '(?\336 . [ ?f ?i ])
  '(?\337 . [ ?f ?l ])
  '(?\341 . [ ?\267 ])
  '(?\345 . [ ?\302 ])
  '(?\346 . [ ?\312 ])
  '(?\347 . [ ?\301 ])
  '(?\350 . [ ?\313 ])
  '(?\352 . [ ?\315 ])
  '(?\353 . [ ?\316 ])
  '(?\354 . [ ?\317 ])
  '(?\355 . [ ?\314 ])
  '(?\356 . [ ?\323 ])
  '(?\357 . [ ?\324 ])
  '(?\361 . [ ?\322 ])
  '(?\362 . [ ?\332 ])
  '(?\363 . [ ?\333 ])
  '(?\364 . [ ?\331 ])
  '(?\365 . [ ?\151 ])
  '(?\370 . [ ?\257 ])
  '(?\374 . [ ?\270 ])))

Re: decoding characters

[Laura Conrad]

>>>>> "Toby" == Toby Speight <streapadair@gmx.net> writes:

    Laura> Laura Conrad <URL:mailto:lconrad@world.std.com>
    0> In article <kh7og6crir4.fsf@acme47.nineco.com>, Laura wrote:

    Laura> I got an email which contained a table of TeX codings for
    Laura> characters, and also the characters.  Most of them looked like
    Laura> I expected, but there were 3 that displayed wrong in my gnus:
    Laura> 
    Laura> œ      \oe (displayed as \234)
    Laura> Π     \OE (displayed as \214)
    Laura> Ÿ      \"Y (displayed as \237)
    Laura> 
    Laura> I'm using: xemacs*font:
    Laura> -etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1

    Toby> Is your Emacs unibyte or multibyte?

How do I tell?  It's whatever came with RedHat.


    Laura> When I just 'more' the mail file in and xterm, \oe and \OE are
    Laura> blank, but \"y displays fine, using the same font as I have
    Laura> defined for xemacs.

    Toby> I think the ETL fonts include \"Y at \237 - but you haven't told this
    Toby> to Emacs.  (aset standard-display-table ?\237 [?\237])

This gives me 

        Wrong type argument: arrayp, nil


    Toby> Actually, that's not the most relevant header.  What would be more
    Toby> useful would be the Content-Type line of the particular part you're
    Toby> having problems with.

Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit


-- 
Laura (mailto:lconrad@world.std.com , http://www.world.std.com/~lconrad/ )
(617) 661-8097	fax: (801) 365-6574 
233 Broadway, Cambridge, MA 02139

Re: decoding characters

[Toby Speight]

Toby> Toby Speight <URL:mailto:streapadair@gmx.net>
Laura> Laura Conrad <URL:mailto:lconrad@world.std.com>

0> In article <kh7og6crir4.fsf@acme47.nineco.com>, Laura wrote:

Laura> I got an email which contained a table of TeX codings for
Laura> characters, and also the characters.  Most of them looked like
Laura> I expected, but there were 3 that displayed wrong in my gnus:
Laura> 
Laura>       \oe (displayed as \234)
Laura>       \OE (displayed as \214)
Laura>       \"Y (displayed as \237)
Laura> 
Laura> I'm using: xemacs*font:
Laura> -etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1


0> In article <ubt2c3vll.fsf@lanber.cam.eu.citrix.com>, Toby wrote:

Toby> Is your Emacs unibyte or multibyte?


0> In article <kh766sjrfyi.fsf@acme47.nineco.com>, Laura wrote:

Laura> How do I tell?  It's whatever came with RedHat.

Umm, pass.  I'd normally say to ask whoever built it, but that's
probably not convenient for you.  Anyone?


Toby> I think the ETL fonts include \"Y at \237 - but you haven't told
Toby> this to Emacs.  (aset standard-display-table ?\237 [?\237])

Laura> This gives me "Wrong type argument: arrayp, nil"

Hmm.  Works for me on GNU Emacs 20.4.1 (i386-*-nt4.0.1381).  It
probably makes the code I included useless to you.


Toby> Actually, that's not the most relevant header.  What would be more
Toby> useful would be the Content-Type line of the particular part you're
Toby> having problems with.

Laura> Content-Type: text/plain; charset=iso-8859-1
Laura> Content-Transfer-Encoding: 8bit

Well, you at least have the satisfaction of knowing that the message is
broken.  There are no printable characters at those codepoints in any
of the IS8859 encodings (for good reason - they are high-page control
characters).  Tell your correspondent to label messages correctly!

Re: decoding characters

[Toby Speight]

Toby> Toby Speight <URL:mailto:streapadair@gmx.net>
Laura> Laura Conrad <URL:mailto:lconrad@world.std.com>


0> In article <ubt2c3vll.fsf@lanber.cam.eu.citrix.com>, Toby wrote:

Toby> Is your Emacs unibyte or multibyte?


0> In article <kh766sjrfyi.fsf@acme47.nineco.com>, Laura wrote:

Laura> How do I tell?  It's whatever came with RedHat.


0> In article <uzopv3i9v.fsf@lanber.cam.eu.citrix.com>, Toby wrote:

Toby> Umm, pass.  I'd normally say to ask whoever built it, but that's
Toby> probably not convenient for you.  Anyone?

You could try viewing the HELLO file (C-h h); if that doesn't work,
it's either unibyte or doesn't have fonts for any of the scripts.

Re: decoding characters

[Mike Fabian]

Laura Conrad <lconrad@world.std.com> writes:

> >>>>> "Toby" == Toby Speight <streapadair@gmx.net> writes:
> 
>     Toby> Is your Emacs unibyte or multibyte?
> 
> How do I tell?  It's whatever came with RedHat.

I think the `Mule' item in the menubar of your XEmacs is only there if
you have a multibyte XEmacs.

I `--with-mule' was not used as an argument for `configure' you don't
get this menu.

-- 
Mike Fabian   <mike.fabian@gmx.de>

Re: decoding characters

[Laura Conrad]

>>>>> "Mike" == Mike Fabian <mike.fabian@gmx.de> writes:

    Mike> Laura Conrad <lconrad@world.std.com> writes:
    >> >>>>> "Toby" == Toby Speight <streapadair@gmx.net> writes:
    >> 
    Toby> Is your Emacs unibyte or multibyte?
    >> 
    >> How do I tell?  It's whatever came with RedHat.

    Mike> I think the `Mule' item in the menubar of your XEmacs is
    Mike> only there if you have a multibyte XEmacs.

Oh.  If "unibyte or mutibyte" is the same as "without or with Mule",
I'm unibyte.  That is, I don't have Mule.

-- 
Laura (mailto:lconrad@world.std.com , http://www.world.std.com/~lconrad/ )
(617) 661-8097	fax: (801) 365-6574 
233 Broadway, Cambridge, MA 02139

Re: decoding characters

[Kai Großjohann]

Laura Conrad <lconrad@world.std.com> writes:

> Oh.  If "unibyte or mutibyte" is the same as "without or with Mule",
> I'm unibyte.  That is, I don't have Mule.

No MULE means unibyte, but with MULE, you can be either unibyte or
multibyte.  Emacs has a command line option `--unibyte', and it is
possible to select multibyteness for each buffer (without that
option).  Dunno how XEmacs with MULE does this.

kai
-- 
Beware of flying birch trees.