Re: printf %s in UTF-8 is not POSIX-compliant

[Peter Stephenson <pws@csr.com>]

On Thu, 06 Mar 2008 09:09:01 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> I think "emulate sh" should emulate the POSIX shell to the greatest
> extent possible.  If that means turning off MULTIBYTE, turn it off.

That seems basically sensible.

> However, if "emulate bash" is going to mean something other than a
> synonym for "sh", then some effort should be put into being a bit
> closer to bash than it's currently possible to be.  For example,
> at least set the various BASH_* options, the way "emulate csh" sets
> the smattering of CSH_* options.

I'm not sure the first sentence agrees with the second.  Are you suggesting
new options?

> A final thought on MULTIBYTE:  Is it perhaps reasonable to split this
> into two options, one that affects line editor operations and one that
> affects internals?  If someone does "emulate sh; setopt zle" it seems
> there might be some expectation that ZLE can adapt to a terminal that
> displays multibyte even if the input is all treated as raw bytes once
> accept-line hands it off.  That might mean that e.g. _main_complete
> needs to look at the state of ZLE_MULTIBYTE (or whatever) and setopt
> MULTIBYTE locally to correspond.  Other widgets could also be affected,
> so the emphasis here is on "reasonable."

I think it can be done, and is reasonable if done properly, but is likely
to be bug-prone in the case where one option is on and the other off.  The
library code (mostly in utils.c) will need the correct option passing down
to it, widgets (including basic zle widgets) will need to be careful, and
the combination isn't likely to get well-tested anyway.

Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.56
diff -u -r1.56 options.yo
--- Doc/Zsh/options.yo	1 Feb 2008 19:59:48 -0000	1.56
+++ Doc/Zsh/options.yo	6 Mar 2008 17:36:57 -0000
@@ -427,10 +427,10 @@
 Append a trailing `tt(/)' to all directory
 names resulting from filename generation (globbing).
 )
-pindex(MULTIBYTE <D>)
+pindex(MULTIBYTE)
 cindex(characters, multibyte, in expansion and globbing)
 cindex(multibyte characters, in expansion and globbing)
-item(tt(MULTIBYTE))(
+item(tt(MULTIBYTE) <C> <K> <Z>)(
 Respect multibyte characters when found in strings.
 When this option is set, strings are examined using the
 system library to determine how many bytes form a character, depending
@@ -438,8 +438,10 @@
 pattern matching, parameter values and various delimiters.
 
 The option is on by default if the shell was compiled with
-tt(MULTIBYTE_SUPPORT); otherwise it is off by default and has no effect if
-turned on.
+tt(MULTIBYTE_SUPPORT) except in tt(sh) emulation; otherwise it is off by
+default and has no effect if turned on.  The mode is off in tt(sh)
+emulation for compatibility but for interative use may need to be
+turned on if the terminal interprets multibyte characters.
 
 If the option is off a single byte is always treated as a single
 character.  This setting is designed purely for examining strings
Index: Src/options.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/options.c,v
retrieving revision 1.38
diff -u -r1.38 options.c
--- Src/options.c	19 Dec 2007 21:49:35 -0000	1.38
+++ Src/options.c	6 Mar 2008 17:36:57 -0000
@@ -173,7 +173,7 @@
 {{NULL, "monitor",	      OPT_SPECIAL},		 MONITOR},
 {{NULL, "multibyte",
 #ifdef MULTIBYTE_SUPPORT
-			      OPT_ALL
+			      OPT_EMULATE|OPT_ZSH|OPT_CSH|OPT_KSH
 #else
 			      0
 #endif

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070