From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 16656 invoked by alias); 12 Dec 2017 13:44:00 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 42116 Received: (qmail 11947 invoked by uid 1010); 12 Dec 2017 13:44:00 -0000 X-Qmail-Scanner-Diagnostics: from rcpt-mqugw.biglobe.ne.jp by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(133.208.100.1):SA:0(-2.6/5.0):. Processed in 6.241357 secs); 12 Dec 2017 13:44:00 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_PASS,T_RP_MATCHES_RCVD autolearn=ham autolearn_force=no version=3.4.1 X-Envelope-From: takimoto-j@kba.biglobe.ne.jp X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | X-Biglobe-Sender: From: Jun T Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Zle widgets for vi-mode word motion: patch and a question Message-Id: <21671458-6017-48A0-B90E-D33CF207F81B@kba.biglobe.ne.jp> Date: Tue, 12 Dec 2017 22:01:53 +0900 To: zsh-workers@zsh.org X-Mailer: Apple Mail (2.3273) X-Biglobe-Spnum: 64553 The attached patch will fix a multibyte-related bug of a few Zle widgets (see [1] below), but it will also introduce an incompatibility with the current behavior of vi-backword-word-end (see [2]). I don't understand why currently vi-backword-word-end behaves this way, and the patch actually will increase the compatibility with the vim editor (/usr/bin/vim). Is it OK to change the behavior of vi-backword-word-end? [1] A problem of Zle widgets related with vi-mode word motion (vi-forward-word, vi-backward-word, etc.) For example, in the following command line (assume 'bindkey -v'): zsh% ls /ab/cd/ef if the cursor is on the 1st '/', hitting 'w' (vi-forward-word) repeatedly will move the cursor to 'a', '/', 'c', '/', 'e', 'f'; this is the expected behavior. But in the following case: zsh% ls /=E3=81=82=E3=81=84/=E3=81=86=E3=81=88/=E3=81=8A=E3=81=8B if the cursor is on the 1st '/', hitting a single 'w' will move the cursor to the last '=E3=81=8B'. This is because characters for which Z_vialnum(X) returns false are all considered to be in the same class, so that '/=E3=81=82=E3=81=84/=E3=81=86=E3=81=88/=E3=81=8A=E3=81=8B'= is just a single word. In the attached patch, I moved a function wordclass() from textobjects.c to zle_word.c and modified it so that it recognizes four character classes: blank(0), vialnum(1), punctuation(2) and all the other(3). [2] current (strange?) behavior of vi-backword-word-end Suppose we have the following 5-line command line (no multibyte characters but two blank lines): ------------- zsh% echo abc =3D+/ def ------------- (you need to type 'echo abcoo=3D+/oodef') If the cursor is on the 'f' of the last line, hitting 'ge' (bound to vi-backward-word-end) will move the cursor to the beginning of the previous blank line, and another 'ge' will move it to the '/' of the third line; these are as expected. But one more 'ge' will move the cursor to the 'c' on the first line, skipping a blank line. This seems to happen when the cursor is on a word made up only of non-alpha_numeric characters (such as '/', '=3D', etc.). The code for this widget is vibackwordwordend() in zle_word.c, but it is very hard to understand for me. In the vim editor, 'ge' never skips blank lines. vim's help document (type ':help ge' in vim) explicitly says "An empty line is also considered to be a WORD". I first thought this is just a bug of the current zsh, but X02zlevi.ztst explicitly tests this behavior (line 466 and below), so this may be intentional?? Is it OK to change this? diff --git a/Src/Zle/textobjects.c b/Src/Zle/textobjects.c index bf83906f2..c93777b65 100644 --- a/Src/Zle/textobjects.c +++ b/Src/Zle/textobjects.c @@ -30,13 +30,6 @@ #include "zle.mdh" #include "textobjects.pro" =20 -/* class of character: 0 is whitespace, 1 is word character, 2 is other = */ -static int -wordclass(ZLE_CHAR_T x) -{ - return (ZC_iblank(x) ? 0 : ((ZC_ialnum(x) || (ZWC('_') =3D=3D x)) ? = 1 : 2)); -} - static int blankwordclass(ZLE_CHAR_T x) { diff --git a/Src/Zle/zle.h b/Src/Zle/zle.h index 07b310180..8261da92b 100644 --- a/Src/Zle/zle.h +++ b/Src/Zle/zle.h @@ -67,6 +67,7 @@ typedef wint_t ZLE_INT_T; #define ZC_inblank iswspace #define ZC_iupper iswupper #define ZC_iword(x) wcsitype((x), IWORD) +#define ZC_ipunct iswpunct =20 #define ZC_tolower towlower #define ZC_toupper towupper @@ -153,6 +154,7 @@ static inline int ZS_strncmp(ZLE_STRING_T s1, = ZLE_STRING_T s2, size_t l) #define ZC_inblank inblank #define ZC_iupper isupper #define ZC_iword iword +#define ZC_ipunct ispunct =20 #define ZC_tolower tulower #define ZC_toupper tuupper diff --git a/Src/Zle/zle_word.c b/Src/Zle/zle_word.c index e4a878eab..4910d765b 100644 --- a/Src/Zle/zle_word.c +++ b/Src/Zle/zle_word.c @@ -64,7 +64,18 @@ forwardword(char **args) return 0; } =20 -#define Z_vialnum(X) (ZC_ialnum(X) || (ZWC('_') =3D=3D X)) +/* + * class of character (for vi-mode word motion) + * 0: blank, 1: alnum or _, 2: punctuation, 3: the others + */ + +/**/ +int +wordclass(ZLE_CHAR_T x) +{ + return (ZC_iblank(x) ? 0 : ((ZC_ialnum(x) || (ZWC('_') =3D=3D x)) ? = 1 : + ZC_ipunct(x) ? 2 : 3)); +} =20 /**/ int @@ -81,13 +92,10 @@ viforwardword(char **args) } while (n--) { int nl; - if (Z_vialnum(zleline[zlecs])) - while (zlecs !=3D zlell && Z_vialnum(zleline[zlecs])) - INCCS(); - else - while (zlecs !=3D zlell && !Z_vialnum(zleline[zlecs]) && - !ZC_inblank(zleline[zlecs])) - INCCS(); + int cc =3D wordclass(zleline[zlecs]); + while (zlecs !=3D zlell && wordclass(zleline[zlecs]) =3D=3D cc) = { + INCCS(); + } if (wordflag && !n) return 0; nl =3D (zleline[zlecs] =3D=3D ZWC('\n')); @@ -208,26 +216,17 @@ viforwardwordend(char **args) zlecs =3D pos; } if (zlecs !=3D zlell) { + int cc; pos =3D zlecs; INCPOS(pos); - if (Z_vialnum(zleline[pos])) { - for (;;) { - zlecs =3D pos; - if (zlecs =3D=3D zlell) + cc =3D wordclass(zleline[pos]); + for (;;) { + zlecs =3D pos; + if (zlecs =3D=3D zlell) + break; + INCPOS(pos); + if (wordclass(zleline[pos]) !=3D cc) break; - INCPOS(pos); - if (!Z_vialnum(zleline[pos])) - break; - } - } else { - for (;;) { - zlecs =3D pos; - if (zlecs =3D=3D zlell) - break; - INCPOS(pos); - if (Z_vialnum(zleline[pos]) || = ZC_inblank(zleline[pos])) - break; - } } } } @@ -295,24 +294,14 @@ vibackwardword(char **args) } if (zlecs) { int pos =3D zlecs; - if (Z_vialnum(zleline[pos])) { - for (;;) { - zlecs =3D pos; - if (zlecs =3D=3D 0) - break; - DECPOS(pos); - if (!Z_vialnum(zleline[pos])) - break; - } - } else { - for (;;) { - zlecs =3D pos; - if (zlecs =3D=3D 0) - break; - DECPOS(pos); - if (Z_vialnum(zleline[pos]) || = ZC_inblank(zleline[pos])) - break; - } + int cc =3D wordclass(zleline[pos]); + for (;;) { + zlecs =3D pos; + if (zlecs =3D=3D 0) + break; + DECPOS(pos); + if (wordclass(zleline[pos]) !=3D cc || = ZC_inblank(zleline[pos])) + break; } } } @@ -368,17 +357,10 @@ vibackwardwordend(char **args) return ret; } while (n-- && zlecs > 1) { - int start =3D 0; - if (Z_vialnum(zleline[zlecs])) - start =3D 1; - else if (!ZC_inblank(zleline[zlecs])) - start =3D 2; + int cc =3D wordclass(zleline[zlecs]); DECCS(); while (zlecs) { - int same =3D (start !=3D 1) && ZC_iblank(zleline[zlecs]); - if (start) - same |=3D Z_vialnum(zleline[zlecs]); - if (same =3D=3D (start =3D=3D 2)) + if (wordclass(zleline[zlecs]) !=3D cc || = ZC_iblank(zleline[zlecs])) break; DECCS(); } @@ -494,26 +476,17 @@ vibackwardkillword(UNUSED(char **args)) x =3D pos; } if (x > lim) { + int cc; int pos =3D x; DECPOS(pos); - if (Z_vialnum(zleline[pos])) { - for (;;) { - x =3D pos; - if (x <=3D lim) - break; - DECPOS(pos); - if (!Z_vialnum(zleline[pos])) - break; - } - } else { - for (;;) { - x =3D pos; - if (x <=3D lim) - break; - DECPOS(pos); - if (Z_vialnum(zleline[pos]) || = ZC_iblank(zleline[pos])) - break; - } + cc =3D wordclass(zleline[pos]); + for (;;) { + x =3D pos; + if (x < lim) + break; + DECPOS(pos); + if (wordclass(zleline[pos]) !=3D cc) + break; } } } diff --git a/Test/X02zlevi.ztst b/Test/X02zlevi.ztst index d3b533490..4e7966e12 100644 --- a/Test/X02zlevi.ztst +++ b/Test/X02zlevi.ztst @@ -1,6 +1,16 @@ # Tests of the vi mode of ZLE =20 %prep + unset -m LC_\* + ZSH_TEST_LANG=3D + langs=3D(en_{US,GB}.{UTF-,utf}8 en.UTF-8 + $(locale -a 2>/dev/null | egrep 'utf8|UTF-8')) + for LANG in $langs; do + if [[ =C3=A9 =3D ? ]]; then + ZSH_TEST_LANG=3D$LANG=20 + break; + fi + done if [[ $OSTYPE =3D cygwin ]]; then ZTST_unimplemented=3D"the zsh/zpty module does not work on Cygwin" elif ( zmodload zsh/zpty 2>/dev/null ); then @@ -463,12 +473,12 @@ > aww >CURSOR: 0 =20 - zletest $' --ww ww--\eo\eoww\eo\eo--\eo\eo ww\e' = gei{a,=3D,b,c,=3D,d,e,=3D,f}$'\e' + zletest $' --ww ww--\eo\eoww\eo\eo--\eo\eo ww\e' = gei{a,=3D,b,c,d,=3D,e,f,=3D,g}$'\e' 0:backward word end ->BUFFER: f -=3D-wew wdw-=3D- ->c ->wbw -> +>BUFFER: g -=3D-wfw wew-=3D- +>d +>wcw +>b >-=3D- >a > ww @@ -529,6 +539,15 @@ > wwe >CURSOR: 29 =20 + if [[ -z $ZSH_TEST_LANG ]]; then + ZTST_skip=3D"no UTF-8 locale for Zle vi-mode test" + else + zletest $'/=E3=81=82=E3=81=84=E3=81=86/=E3=81=88=E3=81=8A/=E3=81=8B=E3= =81=8D\ebxgegex0wxex' + fi +0:word motion with multibyte characters +>BUFFER: /=E3=81=84/=E3=81=88/=E3=81=8D +>CURSOR: 2 + zletest $' ----word ---- word word---- = ----\e42|daw30|daw22|daw14|daw2|daw' 0:delete all word on blanks >BUFFER: word