From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 29744 invoked by alias); 12 Sep 2015 16:46:09 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 36510 Received: (qmail 20647 invoked from network); 12 Sep 2015 16:46:08 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=de1wgAUkAnMEV9Xhp1AAS1nHHCkqDAeKmgRJi/T5nzE=; b=cspf+ZNcx42IkqW/sbIVrUd+OoTu19lmxh2OttSfN52jrkAbTT2GSerX4kH4jdZBpc JYlndaju99u7JOu+S44LY0ACtPbf57Z+ItWc10dZ4WEHrcwmdpTpqnyOZe5RHbfk3HVO DRUAmlD7Cm/kfd1BdasP5y8zePVwypyHMMvswbhPslCSF45AvUuebC7bN88FF04EBBiR qfGqhFcjBjbw3EPCDq5+t8prrBFNLrbhjdohCgdGst8f8lq8Wbs+NqO7K1UCTIVpzpfS MvXPagAie0Wr9ld0+j9o0L659jlz54iH0dhnyc7VAeoF7f75Vd57mAR+WloCXvVjHWNy YGCw== X-Gm-Message-State: ALoCoQk8Y+T8rsXioKubw4mS86R+FxzJ9BR7rVAZfg92OgFz0dOmIgPDidt8LkaOQXX3KKFwc93X MIME-Version: 1.0 X-Received: by 10.112.147.10 with SMTP id tg10mr3485424lbb.58.1442076365520; Sat, 12 Sep 2015 09:46:05 -0700 (PDT) In-Reply-To: <375b5c20-868c-41f6-af3d-297026ddbda5@email.android.com> References: <150911154207.ZM8630@torch.brasslantern.com> <375b5c20-868c-41f6-af3d-297026ddbda5@email.android.com> Date: Sat, 12 Sep 2015 09:46:05 -0700 Message-ID: Subject: Re: PATCH: read full multibyte string a bit more sooner From: Bart Schaefer To: Zsh hackers list Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sat, Sep 12, 2015 at 2:57 AM, Peter Stephenson wrote: > > On 11 Sep 2015 23:42, Bart Schaefer wrote: >> >> This breaks for me with bracketed-paste-magic when pasting the multibyte >> strings from Test/D07multibyte, specifically "More metafied characters >> in prompt expansion" test that has several different languages. I just reverted to the zsh-5.1.1 tag and tried again, and it breaks there, too, so this is probably not specific to the patch in 36483. > I won't have the source or anything more than > phones or tablets for a week, but it might be > meta aggro again. Unfortunately I don't know what that refers to. > I've a vague memory 'a grave' has one, if you > want an easy check. I threw in a { zle -M -- "$PASTED"; zle -R } in the read-command loop and got the following output (hope it comes through OK with the mutibyte in email). Here is the test string I'm pasting: =C4=84 =D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0= =D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5=A6=E7=94= =B1=E8=A8=98 And the result (5.1.1 without 36483): burner% =C4=83 =D0=83=D1=83=D1=82=D1=80 =D0=83=D0=BB=D1=83=D0=B8=D1=83 =D0= =A7=D0=B0=D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5= =A6=E7=83=B4=E8=A8=83 =C4=84 =D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0= =D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5=A6=E7=94= =B1=E8=A8=98 The first line is what got composed by mbchar+=3D$KEYS and the second line is what is actually in $PASTED. As you can see they match for some but not all characters. I then switched back to zsh-5.1.1-dev-0 and tried to repeat this. Here's where things get really interesting. The very first time I pasted the test string, I got this: burner% =C4=84 =D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0= =A7=D0=B0=D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5= =A6=E7=94=B1=E8=A8=98~ As you can see this is ALMOST correct, except for that unexpected trailing tilde, which must be part of the terminal escape for ending paste-mode? Sadly the next time I try pasting, I get this: burner% =C4=83 =D0=83=D1=83=D1=82=D1=80 =D0=83=D0=BB=D1=83=D0=B8=D1=83 =D0=A7=D0=B0= =D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5=A6=E7=83=B4=E8=A8=83 =C4=84 =D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0= =D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 =E6=A2=B6=E6=B5=A6=E7=94= =B1=E8=A8=98 (where all those are highlighted). So either there's some memory corruption, or the internal multibyte parsing state is messed up, or both. Is there someone who works in a multibyte character set all the time who can help with figuring out where this is going wrong? (Insight into what happened in the first [5.1.1] case would also be interesting.)