zsh-workers
 help / color / mirror / code / Atom feed
From: Peter Stephenson <pws@csr.com>
To: zsh workers <zsh-workers@zsh.org>
Subject: Re: vared/zle silently discards non-utf8 bytes
Date: Wed, 6 Jan 2010 11:37:23 +0000	[thread overview]
Message-ID: <20100106113723.4ecd8568@news01> (raw)
In-Reply-To: <237967ef0912230244i2ea13dfav734535262871db7e@mail.gmail.com>

On Wed, 23 Dec 2009 11:44:51 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> Ufortunately it seems vared discards
> anything after an invalid byte. To reproduce, just do
> 
> % a=hi$'\374'nothing
> % vared a

This is currently the designed behaviour if multibyte support is compiled
in.  In this case the editing line is a set of wide characters.  If it
can't convert the input into wide characters it's stuck.

Internally, there are two options

(i) I could simply make it ignore invalid characters, which gets you some
of the line, but is probably even more dangerous

(ii) you could have a go at rewriting the way characters are stored for
editing to use a marker that a character isn't a valid wide character but
is being stored to represent an octet.  This is a big job to get consistent
all the way through (display including width, character tests, conversion
back and forth).

Note that a simpler wrapper

varedquote() {
  # ignoring vared options for now....
  local var=${argv[-1]}
  local val=${(q)${(P)var}}
  # hmmm... if the user stripped some quoting the following is
  # a bit fraught...
  vared val && eval ${var}=${val}
}

should work because the (q) flag is already smart about unprintable
characters (except it does rely on the user not removing backslashes in the
variable).  This could be made a vared option.  It's a little bit hairy
making it default behaviour because it changes the meaning of special
characters in the string you're editing---it's no longer "raw" in other
ways than just $'...' quoting.

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom


  reply	other threads:[~2010-01-06 11:38 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-12-23 10:44 Mikael Magnusson
2010-01-06 11:37 ` Peter Stephenson [this message]
2010-01-06 11:42   ` Peter Stephenson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100106113723.4ecd8568@news01 \
    --to=pws@csr.com \
    --cc=zsh-workers@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).