non-ascii chars in the Lisp source files

non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

Hi,

Can you decode the following morse code using the `W m' key?

-·-· --·-  -·-· --·-  -·-· --·-  -·· ·  ·--- ·-· ·---- -·-· ·-· -··

I couldn't.  It is caused by the broken gnus-sum.elc file.
Mis-decoding of the string "·" which is contained in the
gnus-summary-morse-message function was occurred when compiling
it by XEmacs 21.4 in my system.  In that case,
buffer-file-coding-system is set to shift_jis when the
gnus-sum.el file is read.

It isn't a real problem on me since I seldom use that command
and changing the LANG environment variable from ja_JP.eucJP to C
when compiling (or putting the "-*-coding: iso-8859-1;-*-"
coding cookie in the file) helps it.  However, it is suggested
that it may cause a serious problem in the future, isn't it?
The following files also contain non-ascii chars:

deuglify.el gnus-delay.el gnus-spec.el mml1991.el nnultimate.el
utf7.el (the last one already contains the coding cookie.)
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: non-ascii chars in the Lisp source files

[Daniel Pittman]

On Wed, 03 Sep 2003, Katsumi Yamaoka wrote:
> Can you decode the following morse code using the `W m' key?
> 
> -·-· --·-  -·-· --·-  -·-· --·-  -·· ·  ·--- ·-· ·---- -·-· ·-· -··
> 
> I couldn't.  

I could, with XEmacs and a fairly current CVS Gnus.

I also found that it did hideous things to...

> It is caused by the broken gnus-sum.elc file. Mis-decoding of the
> string "·" which is contained in the gnus-summary-morse-message
> function was occurred when compiling it by XEmacs 21.4 in my system.

...the lines above, because it "decoded" the '-' in
gnus-summary-morse-message.

Maybe it should be enhanced to ignore a '.' or '-' between normal word
constituent characters?

            Daniel


> In that case, buffer-file-coding-system is set to shift_jis when the
> gnus-sum.el file is read.
> 
> It isn't a real problem on me since I seldom use that command
> and changing the LANG environment variable from ja_JP.eucJP to C
> when compiling (or putting the "-*-coding: iso-8859-1;-*-"
> coding cookie in the file) helps it.  However, it is suggested
> that it may cause a serious problem in the future, isn't it?
> The following files also contain non-ascii chars:
> 
> deuglify.el gnus-delay.el gnus-spec.el mml1991.el nnultimate.el
> utf7.el (the last one already contains the coding cookie.)

-- 
The fundamental delusion of humanity is to suppose
that I am here and you are there.
        -- Yasutani Roshi

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <86y8x6fdgz.fsf@i3d.home>
>>>>>	Robert Epprecht <epprecht@solnet.ch> wrote:

> And this one get's 'decoded' too:
>> gnus-summary-morse-message
>   gnustsummarytmorsetmessage   ;-)

There's also another problem that the `q' morse code is
represented as `n' by the default value of gnus-treat-emphasize.
:-p

Re: non-ascii chars in the Lisp source files

[Robert Epprecht]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Can you decode the following morse code using the `W m' key?
> -·-· --·-  -·-· --·-  -·-· --·-  -·· ·  ·--- ·-· ·---- -·-· ·-· -··

Yes, in GNU Emacs it works here.

And this one get's 'decoded' too:
> gnus-summary-morse-message
  gnustsummarytmorsetmessage   ;-)

Robert

Re: non-ascii chars in the Lisp source files

[Reiner Steib]

On Wed, Sep 03 2003, Katsumi Yamaoka wrote:

> The following files also contain non-ascii chars:
>
> deuglify.el gnus-delay.el gnus-spec.el mml1991.el nnultimate.el
> utf7.el (the last one already contains the coding cookie.)

All mentioned files already contain a cookie:

$ for i in deuglify.el gnus-delay.el gnus-spec.el mml1991.el \
         nnultimate.el utf7.el gnus-sum.el; do
  grep -ne coding: $i /dev/null
done
deuglify.el:468:;; coding: iso-8859-1
gnus-delay.el:192:;; coding: iso-8859-1
gnus-spec.el:754:;; coding: iso-8859-1
mml1991.el:296:;; coding: iso-8859-1
nnultimate.el:476:;; coding: iso-8859-1
utf7.el:1:;;; utf7.el --- UTF-7 encoding/decoding for Emacs   -*-coding: iso-8859-1;-*-
gnus-sum.el:11634:;; coding: iso-8859-1

Doesn't XEmacs allow the cookie in the local variables section?

,----
| ;; Local Variables:
| ;; coding: iso-8859-1
| ;; End:
| 
| ;;; gnus-sum.el ends here
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <v97k4qywwu.fsf@marauder.physik.uni-ulm.de>
>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:

> All mentioned files already contain a cookie:

Oops!  It just didn't occur to me to look the bottom of the
files.

> Doesn't XEmacs allow the cookie in the local variables section?

Unfortunately, it probably be so. :(
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <b9yk78q9kzm.fsf@jpl.org> Katsumi Yamaoka wrote:

>>>>>> In <v97k4qywwu.fsf@marauder.physik.uni-ulm.de>
>>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:

>> Doesn't XEmacs allow the cookie in the local variables section?

> Unfortunately, it probably be so. :(

XEmacs only does the following literally for the coding cookie
in the local variables section:

(make-local-variable 'coding)
(set 'coding 'iso-8859-1)

Using `buffer-file-coding-system' instead of `coding' there is
meaningless.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: non-ascii chars in the Lisp source files

[Reiner Steib]

On Wed, Sep 03 2003, Katsumi Yamaoka wrote:

>>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:
>
>> All mentioned files already contain a cookie:
[...]
>> Doesn't XEmacs allow the cookie in the local variables section?
>
> Unfortunately, it probably be so. :(

This seems to be a bug (see also [1]) in XEmacs 21.4 then, as (info
"(XEmacs)Recognize Coding") in XEmacs 21.4.12 mentions local
variables:

,----[ (info "(XEmacs)Recognize Coding") ]
|    You can specify the coding system for a particular file using the
| `-*-...-*-' construct at the beginning of a file, or a local variables
| list at the end (*note File Variables::).  You do this by defining a
| value for the "variable" named `coding'.  XEmacs does not really have a
| variable `coding'; instead of setting a variable, it uses the specified
| coding system for the file.
`----

Bye, Reiner.

[1]
<URL:http://news.gmane.org/onethread.php?group=gmane.emacs.xemacs.beta
&root=%3C20020506193719.30420.qmail%40ypy.tky.hut.fi%3E>
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <v9wucpj4br.fsf@marauder.physik.uni-ulm.de>
>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:

>>> Doesn't XEmacs allow the cookie in the local variables section?

>> Unfortunately, it probably be so. :(

> This seems to be a bug (see also [1]) in XEmacs 21.4 then, as (info
> "(XEmacs)Recognize Coding") in XEmacs 21.4.12 mentions local
> variables:

> ,----[ (info "(XEmacs)Recognize Coding") ]
>|    You can specify the coding system for a particular file using the
>| `-*-...-*-' construct at the beginning of a file, or a local variables
>| list at the end (*note File Variables::).

[...]

> [1]
> <URL:http://news.gmane.org/onethread.php?group=gmane.emacs.xemacs.beta
&root=%3C20020506193719.30420.qmail%40ypy.tky.hut.fi%3E>

Thanks.  I've confirmed both the latest XEmacs 21.4 and 21.5
don't handle the coding cookie at the end of a file yet.  So,
all files needing the coding cookie should contain the
"-*-...-*-" form at the beginning.  Although it's easy to modify
deuglify.el, gnus-delay.el, ..., gnus.texi so that they contain
the cookie at the beginning, there's no room to have it within
80-column.  In addition, we cannot do so in the ChangeLog files.
It's a distress.  I was increasingly disillusioned with XEmacs.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: non-ascii chars in the Lisp source files

[Juha Autero]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Thanks.  I've confirmed both the latest XEmacs 21.4 and 21.5
> don't handle the coding cookie at the end of a file yet.

What is the value of enable-local-variables? Local variables are sort
of security hole so maybe XEmacs nowadays disable them by default.

-- 
Juha Autero
http://www.iki.fi/jautero/
Eschew obscurity!

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <87k78oyi89.fsf@jautero.no-ip.org>
>>>>>	Juha Autero <Juha.Autero@iki.fi> wrote:

> Katsumi Yamaoka <yamaoka@jpl.org> writes:

>> Thanks.  I've confirmed both the latest XEmacs 21.4 and 21.5
>> don't handle the coding cookie at the end of a file yet.

> What is the value of enable-local-variables?

t.  Other local-eval related variables, enable-local-eval,
ignored-local-variables, and hack-local-variables-hook are
unchanged from the default.

> Local variables are sort of security hole so maybe XEmacs nowadays
> disable them by default.

Eh?  It is defined in files.el with the default value t.  Both
the latest XEmacs 21.4 and 21.5 do the same.

When I traced it using edebug, processing the `Local Variables:'
block was performed by the hack-one-local-variable function by
way of the hack-local-variables-last-page function as follows:

>>>>> In <b9y65ka9h7w.fsf@jpl.org> Katsumi Yamaoka wrote:

> (make-local-variable 'coding)
> (set 'coding 'iso-8859-1)
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: non-ascii chars in the Lisp source files

[Katsumi Yamaoka]

>>>>> In <b9yk78pl68f.fsf@jpl.org> Katsumi Yamaoka wrote:

> I've confirmed both the latest XEmacs 21.4 and 21.5 don't handle the
> coding cookie at the end of a file yet.

Here's a quick hack, not tested broadly:

(if (featurep 'xemacs)
    (defadvice hack-one-local-variable (around handle-coding-cookie (var val)
					       activate)
      "Handle a coding cookie at the Local Variables section."
      (if (eq var 'coding)
	  (if (and buffer-file-name
		   (symbolp val)
		   (find-coding-system val))
	      (let ((mod (buffer-modified-p))
		    (coding-system-for-read val)
		    buffer-read-only)
		(insert-file-contents buffer-file-name nil nil nil t)
		(setq buffer-file-coding-system val)
		(set-buffer-modified-p mod)))
	ad-do-it)))

It can also be used in dgnushack.el in order to build correct
elc files even in Japanese language environment.  I don't want
to put it into dgnushack.el actually, though.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>