RE: FAQ, German Umlauts (patch)

RE: FAQ, German Umlauts (patch)

[Amol Deshpande]

Peter,
Should this also be applicable to 3.0.x ? Also, would printing eight-bit
characters  be affected by inittyptab ? 
thanks,
-amol

> *** Doc/Zsh/options.yo.p8	Wed Apr 30 09:18:38 1997
> --- Doc/Zsh/options.yo	Thu Jun 26 18:33:51 1997
> ***************
> *** 622,627 ****
> --- 622,634 ----
>   tt(trap) and
>   tt(unset).
>   )
> + pindex(PRINT_EIGHT_BIT)
> + cindex(exit status, printing)
> + item(tt(PRINT_EIGHT_BIT) (tt(-1)))(
> + Print eight bit characters literally in completion lists, etc.
> + This option is not necessary if your system correctly returns the
> + printability of eight bit characters (see manref(ctype)(3)).
> + )
>   pindex(PRINT_EXIT_VALUE)
>   cindex(exit status, printing)
>   item(tt(PRINT_EXIT_VALUE) (tt(-1)))(
> *** Src/options.c.p8	Thu Jun 26 18:17:28 1997
> --- Src/options.c	Thu Jun 26 18:14:26 1997
> ***************
> *** 144,149 ****
> --- 144,150 ----
>   {NULL, "overstrike",	      0,
> OVERSTRIKE},
>   {NULL, "pathdirs",	      0,			 PATHDIRS},
>   {NULL, "posixbuiltins",	      OPT_EMULATE|OPT_BOURNE,
> POSIXBUILTINS},
> + {NULL, "printeightbit",       0,
> PRINTEIGHTBIT},
>   {NULL, "printexitvalue",      0,
> PRINTEXITVALUE},
>   {NULL, "privileged",	      OPT_SPECIAL,
> PRIVILEGED},
>   {NULL, "promptcr",	      OPT_ALL,			 PROMPTCR},
> *** Src/utils.c.p8	Thu Jun 26 18:17:18 1997
> --- Src/utils.c	Thu Jun 26 18:39:11 1997
> ***************
> *** 175,180 ****
> --- 175,182 ----
>       if (isprint(c))
>   	goto done;
>       if (c & 0x80) {
> + 	if (isset(PRINTEIGHTBIT))
> + 	    goto done;
>   	*s++ = '\\';
>   	*s++ = 'M';
>   	*s++ = '-';
> *** Src/zsh.h.p8	Tue Jun 24 09:09:11 1997
> --- Src/zsh.h	Fri Jun 27 09:35:25 1997
> ***************
> *** 1035,1040 ****
> --- 1035,1041 ----
>       OVERSTRIKE,
>       PATHDIRS,
>       POSIXBUILTINS,
> +     PRINTEIGHTBIT,
>       PRINTEXITVALUE,
>       PRIVILEGED,
>       PROMPTCR,
> 
> 
> -- 
> Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
> WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
> Deutsches Elektronen-Synchrotron --- Institut fuer Hochenergiephysik
> Zeuthen
> DESY-IfH, 15735 Zeuthen, Germany.
> 

Re: FAQ, German Umlauts (patch)

[Peter Stephenson]

Amol Deshpande wrote:
> Peter,
> Should this also be applicable to 3.0.x ? Also, would printing eight-bit
> characters  be affected by inittyptab ? 
> thanks,

Here's a patch for zsh 3.0.x.  The internal zsh table (set up in
ininittyptab()) isn't used in this case; isprint() is used in order to
give the system a chance of deciding what's printable.

*** Doc/zshoptions.man.p8	Mon Oct 21 21:01:15 1996
--- Doc/zshoptions.man	Mon Jun 30 09:36:36 1997
***************
*** 416,421 ****
--- 416,426 ----
  .BR trap " and"
  .BR unset .
  .TP
+ \fBPRINT_EIGHT_BIT\fP
+ Print eight bit characters literally in completion lists, etc.
+ This option is not necessary if your system correctly returns the
+ printability of eight bit characters (see ctype(3)).
+ .TP
  \fBPRINT_EXIT_VALUE\fP (\-\fB1\fP)
  Print the exit value of programs with non-zero exit status.
  .TP
*** Src/globals.h.p8	Tue Jun  3 07:11:26 1997
--- Src/globals.h	Mon Jun 30 10:04:01 1997
***************
*** 762,767 ****
--- 762,768 ----
      {"overstrike", 		0,    0,    0},
      {"pathdirs", 		'Q',  0,    0},
      {"posixbuiltins",		0,    0,    OPT_EMULATE|OPT_BOURNE},
+     {"printeightbit", 		0,    0,    0},
      {"printexitvalue", 		'1',  0,    0},
      {"privileged", 		'p',  'p',  OPT_SPECIAL},
      {"promptcr", 		x'V', 0,    OPT_ALL},
*** Src/utils.c.p8	Tue Jun  3 07:11:28 1997
--- Src/utils.c	Mon Jun 30 09:39:48 1997
***************
*** 174,179 ****
--- 174,181 ----
      if (isprint(c))
  	goto done;
      if (c & 0x80) {
+ 	if (isset(PRINTEIGHTBIT))
+ 	    goto done;
  	*s++ = '\\';
  	*s++ = 'M';
  	*s++ = '-';
*** Src/zsh.h.p8	Tue Jun  3 07:11:28 1997
--- Src/zsh.h	Mon Jun 30 09:37:01 1997
***************
*** 1124,1129 ****
--- 1124,1130 ----
      OVERSTRIKE,
      PATHDIRS,
      POSIXBUILTINS,
+     PRINTEIGHTBIT,
      PRINTEXITVALUE,
      PRIVILEGED,
      PROMPTCR,

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
Deutsches Elektronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: FAQ, German Umlauts (patch)

[Peter Stephenson]

Uli Zappe wrote:
> > Zsh doesn't even decide; the system does and it seems to be
> > getting it wrong.
> 
> But zsh is the *only* program with this problem on NEXTSTEP; so it  
> can't be as simple as "zsh is right, but NEXTSTEP is wrong"... :-(  
> perl, for instance, works perfectly well.

That's because they're simply not deciding.  Zsh, having a fairly
sophisticated editor, is trying to spare the user unreadable
characters on the terminal.  Perl doesn't know about characters, it
simply deals with byte streams.  You can probably see what I mean if
you have `less' and try to view something with non-ASCII characters.

> > The old zsh, so far as I remember, didn't even have the locale
> > mechanism, so is unlikely to be able to do non-ASCII characters.
> > Even a reasonable attempt at 8-bit characters is fairly new.
> 
> zsh 2.51 *can* deal perfectly with lower case Umlauts, as I said before.

That must be because it simply passed all 8-bit characters through.
This is unfortunately not a sensible default:  it'll screw up your old
VT100 something rotten, for example.

> > To clarify, zsh doesn't need the locale system
> > either, it just supports it if it is available.
> 
> Does this locale system consist of only the file /usr/lib/locale  
> (so that I simply could install such a file), or is there a  
> corresponding program that's necessary?

No, there's a whole mess of stuff in the operating system; either it
comes with it or it doesn't.

> > The alternative is Wolfgang Hukriede's suggestion of passing
> > through all characters.  That's dangerous in general, so it would
> > have to be an option.
> 
> Maybe it's not necessary to really pass thru *all* characters; I  
> can't judge, though.
> 
> Anyway, what do I have to do to make this option being implemented  
> in future versions of zsh?

It's very simple and I've just done it.  I've assumed that only eight
bit characters are a problem.  The option is called PRINT_EIGHT_BIT
(or printeightbit etc.); see the manual entry.  (Would PASS_EIGHT_BIT
be better? It sounded a little opaque to me.)

*** Doc/Zsh/options.yo.p8	Wed Apr 30 09:18:38 1997
--- Doc/Zsh/options.yo	Thu Jun 26 18:33:51 1997
***************
*** 622,627 ****
--- 622,634 ----
  tt(trap) and
  tt(unset).
  )
+ pindex(PRINT_EIGHT_BIT)
+ cindex(exit status, printing)
+ item(tt(PRINT_EIGHT_BIT) (tt(-1)))(
+ Print eight bit characters literally in completion lists, etc.
+ This option is not necessary if your system correctly returns the
+ printability of eight bit characters (see manref(ctype)(3)).
+ )
  pindex(PRINT_EXIT_VALUE)
  cindex(exit status, printing)
  item(tt(PRINT_EXIT_VALUE) (tt(-1)))(
*** Src/options.c.p8	Thu Jun 26 18:17:28 1997
--- Src/options.c	Thu Jun 26 18:14:26 1997
***************
*** 144,149 ****
--- 144,150 ----
  {NULL, "overstrike",	      0,			 OVERSTRIKE},
  {NULL, "pathdirs",	      0,			 PATHDIRS},
  {NULL, "posixbuiltins",	      OPT_EMULATE|OPT_BOURNE,	 POSIXBUILTINS},
+ {NULL, "printeightbit",       0,                         PRINTEIGHTBIT},
  {NULL, "printexitvalue",      0,			 PRINTEXITVALUE},
  {NULL, "privileged",	      OPT_SPECIAL,		 PRIVILEGED},
  {NULL, "promptcr",	      OPT_ALL,			 PROMPTCR},
*** Src/utils.c.p8	Thu Jun 26 18:17:18 1997
--- Src/utils.c	Thu Jun 26 18:39:11 1997
***************
*** 175,180 ****
--- 175,182 ----
      if (isprint(c))
  	goto done;
      if (c & 0x80) {
+ 	if (isset(PRINTEIGHTBIT))
+ 	    goto done;
  	*s++ = '\\';
  	*s++ = 'M';
  	*s++ = '-';
*** Src/zsh.h.p8	Tue Jun 24 09:09:11 1997
--- Src/zsh.h	Thu Jun 26 18:13:27 1997
***************
*** 1062,1068 ****
      VERBOSE,
      XTRACE,
      USEZLE,
!     OPT_SIZE
  };
  
  #undef isset
--- 1062,1069 ----
      VERBOSE,
      XTRACE,
      USEZLE,
!     OPT_SIZE,
!     PRINTEIGHTBIT
  };
  
  #undef isset

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
Deutsches Elektronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: FAQ, German Umlauts (patch)

[Zefram]

Peter Stephenson wrote:
>+ item(tt(PRINT_EIGHT_BIT) (tt(-1)))(
>+ Print eight bit characters literally in completion lists, etc.
>+ This option is not necessary if your system correctly returns the
>+ printability of eight bit characters (see manref(ctype)(3)).

If it's only needed if the system has a bug, then a run-time option is the
wrong way to do it.  Make it a configure-time option (--enable-pass8),
and conditionally compile in such a special version of isprint(), using
it instead of the system's version.

>      VERBOSE,
>      XTRACE,
>      USEZLE,
>!     OPT_SIZE,
>!     PRINTEIGHTBIT
>  };

Also this is wrong.  OPT_SIZE must be the last entry in the list, and
the options in the list are in alphabetical order.

-zefram

Re: FAQ, German Umlauts (patch)

[Uli Zappe]

Zefram wrote:

> If it's only needed if the system has a bug, then a run-time
> option is the wrong way to do it.  Make it a configure-time option
> (--enable-pass8), and conditionally compile in such a special
> version of isprint(), using it instead of the system's version.

This sounds very reasonable.

Is there a source for isprint() lying around somewhere? I could  
modify and/or at least test its correct behaviour for NEXTSTEP.


                Bye
                        Uli

_____________________________________________________________________

Uli Zappe               E-Mail: uli@tallowcross.uni-frankfurt.de
                                (NeXTMail,Mime,ASCII) PGP on request
Lorscher Strasse 5      WWW:    -
D-60489 Frankfurt       Fon:    +49 (69) 9784 0007
Germany                 Fax:    +49 (69) 9784 0042

staff member of NEXTTOYOU - the German NEXTSTEP/OPENSTEP magazine
_____________________________________________________________________

Re: FAQ, German Umlauts (patch)

[Uli Zappe]

I wrote:

> Is there a source for isprint() lying around somewhere? I could
> modify and/or at least test its correct behaviour for NEXTSTEP.

Aehem, to add to my own remark:

I've had the time now to look in the docs and found there is a  
NEXTSTEP equivalent to isprint, called NXIsPrint.

So obviously all that needs to be done is to make zsh use an  
alternative isprint and then use NXIsPrint for it (or "write" an  
alternative isprint that wraps NXIsPrint).

How could this be integrated in the source?


                Bye
                        Uli

_____________________________________________________________________

Uli Zappe               E-Mail: uli@tallowcross.uni-frankfurt.de
                                (NeXTMail,Mime,ASCII) PGP on request
Lorscher Strasse 5      WWW:    -
D-60489 Frankfurt       Fon:    +49 (69) 9784 0007
Germany                 Fax:    +49 (69) 9784 0042

staff member of NEXTTOYOU - the German NEXTSTEP/OPENSTEP magazine
_____________________________________________________________________

Re: FAQ, German Umlauts (patch)

[Peter Stephenson]

Zefram wrote:
> Peter Stephenson wrote:
> >+ item(tt(PRINT_EIGHT_BIT) (tt(-1)))(
> >+ Print eight bit characters literally in completion lists, etc.
> >+ This option is not necessary if your system correctly returns the
> >+ printability of eight bit characters (see manref(ctype)(3)).
> 
> If it's only needed if the system has a bug, then a run-time option is the
> wrong way to do it.  Make it a configure-time option (--enable-pass8),
> and conditionally compile in such a special version of isprint(), using
> it instead of the system's version.

I agree with Wolfgang on this:  you simply can't tell from the system
set-up whether it supports the hardware, i.e. whether the terminal
Does the Right Thing with eight bit characters or not. If you're using
a dumb character terminal as well as a smart terminal on the same system
you definitely need the run-time option.  In any case, there's no
way of deciding automatically and (as Uli has been saying) these NEXTSTEP
people like the thing just to work.  Configuring at compile-time whether
to dump raw characters at run-time to a terminal which could be anything
is just too brutal.  (What you possibly could configure is whether or not
to compile in the *option*, but I think it's obvious that that has less
than welcome side effects for code.  More plausibly you could decide at
compile time system by system whether the option should be in effect by
default.)

> >      VERBOSE,
> >      XTRACE,
> >      USEZLE,
> >!     OPT_SIZE,
> >!     PRINTEIGHTBIT
> >  };
> 
> Also this is wrong.  OPT_SIZE must be the last entry in the list, and
> the options in the list are in alphabetical order.

that was stupid... I simply noticed the last two were out of order and
thought therefore I could dump the new one on the end.  Here's a revised
version.

*** Doc/Zsh/options.yo.p8	Wed Apr 30 09:18:38 1997
--- Doc/Zsh/options.yo	Thu Jun 26 18:33:51 1997
***************
*** 622,627 ****
--- 622,634 ----
  tt(trap) and
  tt(unset).
  )
+ pindex(PRINT_EIGHT_BIT)
+ cindex(exit status, printing)
+ item(tt(PRINT_EIGHT_BIT) (tt(-1)))(
+ Print eight bit characters literally in completion lists, etc.
+ This option is not necessary if your system correctly returns the
+ printability of eight bit characters (see manref(ctype)(3)).
+ )
  pindex(PRINT_EXIT_VALUE)
  cindex(exit status, printing)
  item(tt(PRINT_EXIT_VALUE) (tt(-1)))(
*** Src/options.c.p8	Thu Jun 26 18:17:28 1997
--- Src/options.c	Thu Jun 26 18:14:26 1997
***************
*** 144,149 ****
--- 144,150 ----
  {NULL, "overstrike",	      0,			 OVERSTRIKE},
  {NULL, "pathdirs",	      0,			 PATHDIRS},
  {NULL, "posixbuiltins",	      OPT_EMULATE|OPT_BOURNE,	 POSIXBUILTINS},
+ {NULL, "printeightbit",       0,                         PRINTEIGHTBIT},
  {NULL, "printexitvalue",      0,			 PRINTEXITVALUE},
  {NULL, "privileged",	      OPT_SPECIAL,		 PRIVILEGED},
  {NULL, "promptcr",	      OPT_ALL,			 PROMPTCR},
*** Src/utils.c.p8	Thu Jun 26 18:17:18 1997
--- Src/utils.c	Thu Jun 26 18:39:11 1997
***************
*** 175,180 ****
--- 175,182 ----
      if (isprint(c))
  	goto done;
      if (c & 0x80) {
+ 	if (isset(PRINTEIGHTBIT))
+ 	    goto done;
  	*s++ = '\\';
  	*s++ = 'M';
  	*s++ = '-';
*** Src/zsh.h.p8	Tue Jun 24 09:09:11 1997
--- Src/zsh.h	Fri Jun 27 09:35:25 1997
***************
*** 1035,1040 ****
--- 1035,1041 ----
      OVERSTRIKE,
      PATHDIRS,
      POSIXBUILTINS,
+     PRINTEIGHTBIT,
      PRINTEXITVALUE,
      PRIVILEGED,
      PROMPTCR,


-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
Deutsches Elektronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.