From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-workers-return-39724-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 24710 invoked by alias); 25 Oct 2016 16:16:00 -0000
Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Workers List <zsh-workers.zsh.org>
List-Post: <mailto:zsh-workers@zsh.org>
List-Help: <mailto:zsh-workers-help@zsh.org>
X-Seq: 39724
Received: (qmail 22862 invoked from network); 25 Oct 2016 16:16:00 -0000
X-Qmail-Scanner-Diagnostics: from mailout4.w1.samsung.com by f.primenet.com.au (envelope-from <p.stephenson@samsung.com>, uid 7791) with qmail-scanner-2.11 
 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1.  
 Clear:RC:0(210.118.77.14):SA:0(-1.3/5.0):. 
 Processed in 0.453232 secs); 25 Oct 2016 16:16:00 -0000
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-1.3 required=5.0 tests=RP_MATCHES_RCVD
	autolearn=unavailable autolearn_force=no version=3.4.1
X-Envelope-From: p.stephenson@samsung.com
X-Qmail-Scanner-Mime-Attachments: |
X-Qmail-Scanner-Zip-Files: |
Received-SPF: none (ns1.primenet.com.au: domain at samsung.com does not designate permitted sender hosts)
X-AuditID: cbfec7f2-f79556d000002c42-5b-580f82da1af6
Date: Tue, 25 Oct 2016 17:05:44 +0100
From: Peter Stephenson <p.stephenson@samsung.com>
To: Zsh hackers list <zsh-workers@zsh.org>
Subject: Re: zsh/complist colours improperly handle multibyte characters
Message-id: <20161025170544.5862b0a8@pwslap01u.europe.root.pri>
In-reply-to: <161025085920.ZM7192@torch.brasslantern.com>
Organization: Samsung Cambridge Solution Centre
X-Mailer: Claws Mail 3.7.9 (GTK+ 2.22.0; i386-redhat-linux-gnu)
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
X-Brightmail-Tracker:
 H4sIAAAAAAAAA+NgFnrJIsWRmVeSWpSXmKPExsWy7djP87q3m/gjDLrns1ocbH7I5MDoserg
	B6YAxigum5TUnMyy1CJ9uwSujEk3UwrusFTM3/WPuYHxPHMXIweHhICJxJuWii5GTiBTTOLC
	vfVsXYxcHEICSxklJl1exQLh9DJJvL14ixmiykSi7+YmqMQyRolz159DOdOYJG5dvgnlnGGU
	2HDxOhOEc5ZRYv7WRnaQfhYBVYkJD9YwgthsAoYSUzfNBrNFBLQkdpw8yQRylLCAh0TjaSsQ
	k1fAXmLh9yKQCk4BS4mJv9ewgNj8AvoSV/9+YoK4yF5i5pUzYFN4BQQlfky+B1bDLKAjsW3b
	Y3YIW15i85q3zCDnSAj8ZpPYsnkRE8T/shKbDkB95iKx89stVghbWOLV8S3sELaMRGfHQahd
	/YwST7p9IebMYJQ4fWYHG0TCWqLv9kVGiGV8EpO2TYeGL69ER5sQRImHxJlZIJ+DhB0lFlyV
	mMCoOAvJ1bOQXD0LydULGJlXMYqklhbnpqcWG+sVJ+YWl+al6yXn525iBCaA0/+Of9rB+PWE
	1SFGAQ5GJR7eHen8EUKsiWXFlbmHGCU4mJVEeFc1AoV4UxIrq1KL8uOLSnNSiw8xSnOwKInz
	7llwJVxIID2xJDU7NbUgtQgmy8TBKdXA2L7D+EHkeudPOux7z7hNnqJ7NzfN89rp0yHc3YUP
	28z/3CrTuTT5H9+iNXLuZtbT5u+YUGrwdfr0AxYNq/6vOXpssqLwRfZ436mJbxQLere4KQRx
	v2qLfhl75eu6oInp8gXXrDa/Wcz4aPtF7xbBy90Z/BkWskVNrg4XVye/m9cwhf3+V4H9+kos
	xRmJhlrMRcWJAKI2s5T8AgAA
X-Brightmail-Tracker:
 H4sIAAAAAAAAA+NgFlrEIsWRmVeSWpSXmKPExsVy+t/xK7rnmvgjDP5f0bc42PyQyYHRY9XB
	D0wBjFFuNhmpiSmpRQqpecn5KZl56bZKoSFuuhZKCnmJuam2ShG6viFBSgpliTmlQJ6RARpw
	cA5wD1bSt0twy5h0M6XgDkvF/F3/mBsYzzN3MXJySAiYSPTd3MQCYYtJXLi3ng3EFhJYwiix
	c1Z6FyMXkD2DSaLp/UsWiMQ5RomV90UhEmcZJdbPOMIEkmARUJWY8GANI4jNJmAoMXXTbDBb
	REBLYsfJk0A1HBzCAh4SjaetQExeAXuJhd+LQCo4BSwlJv5ewwIx8jazxIqGl+wgCX4BfYmr
	fz8xQRxnLzHzyhmwkbwCghI/Jt8Du4cZaPzmbU2sELa8xOY1b5kh7lSXuHF3N/sERuFZSFpm
	IWmZhaRlASPzKkaR1NLi3PTcYkO94sTc4tK8dL3k/NxNjMD42Xbs5+YdjJc2Bh9iFOBgVOLh
	3ZHOHyHEmlhWXJl7iFGCg1lJhHdVI1CINyWxsiq1KD++qDQntfgQoykwXCYyS4km5wNjO68k
	3tDE0NzS0MjYwsLcyEhJnLfkw5VwIYH0xJLU7NTUgtQimD4mDk6pBsbZoUazRJ22TlPgcp4S
	fvJY5q4nUoU7vTMK3nXYOqwTbfTltBDTORA06ZTQJFmTP7sNz5jxh8VxdJy8F7pTVnza6bLQ
	6Ukqs2dMZNd/HuSRecKHS+VxkU37Mu/n+zS1jj8LTPxmJs26p0evPUDGL+92e3qYnHeXqMX7
	I3NzFDf/X2ag6Hv3qBJLcUaioRZzUXEiAI7p1py1AgAA
X-MTR: 20000000000000000@CPGS
X-CMS-MailID: 20161025160546eucas1p1975bf8ac57608afa20e8db6ab68a4e73
X-Msg-Generator: CA
X-Sender-IP: 182.198.249.179
X-Local-Sender: =?UTF-8?B?UGV0ZXIgU3RlcGhlbnNvbhtTQ1NDLURhdGEgUGxhbmUb?=
	=?UTF-8?B?7IK87ISx7KCE7J6QG1ByaW5jaXBhbCBFbmdpbmVlciwgU29mdHdhcmU=?=
X-Global-Sender: =?UTF-8?B?UGV0ZXIgU3RlcGhlbnNvbhtTQ1NDLURhdGEgUGxhbmUbU2Ft?=
	=?UTF-8?B?c3VuZyBFbGVjdHJvbmljcxtQcmluY2lwYWwgRW5naW5lZXIsIFNvZnR3YXJl?=
X-Sender-Code: =?UTF-8?B?QzEwG0VIURtDMTBDRDA1Q0QwNTAwNTg=?=
CMS-TYPE: 201P
X-HopCount: 7
X-CMS-RootMailID: 20161023195746epcas2p4cdccb2259bcdc15d35f26c8b5963148f
X-RootMTR: 20161023195746epcas2p4cdccb2259bcdc15d35f26c8b5963148f
References: <etPan.58096283.690aeb81.31f@dani-mac.newcastle.edu.au>
 <161020210735.ZM6446@torch.brasslantern.com>
 <20161023184641.4549e10a@ntlworld.com>
 <161023105652.ZM3309@torch.brasslantern.com>
 <20161023195908.40edc8cf@ntlworld.com>
 <CGME20161023195746epcas2p4cdccb2259bcdc15d35f26c8b5963148f@epcas2p4.samsung.com>
 <161023123416.ZM3982@torch.brasslantern.com>
 <20161025114425.64574aff@pwslap01u.europe.root.pri>
 <161025085920.ZM7192@torch.brasslantern.com>

On Tue, 25 Oct 2016 08:59:20 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Oct 25, 11:44am, Peter Stephenson wrote:
> }
> } To fix this safely, we need first to metafy the input string,
> } 
> } * E2 83 B4 82 *
> } 
> } then tokenise it with the change I previously posted to skip Meta,
> 
> This seems to have done it.  I tried the metafy before tokenize route,
> but without the change to zshtokenize that wasn't sufficient.
> 
> Do we also need this here?  And isn't it slightly better to pass -1
> to metafy() rather than call strlen()?

It looks highly likely.

pws