* Re: completion bug in UTF-8 locale [not found] <20060820071833.GA3850@localdomain> @ 2006-08-20 17:01 ` Peter Stephenson 2006-08-20 20:08 ` Roman Cheplyaka 0 siblings, 1 reply; 7+ messages in thread From: Peter Stephenson @ 2006-08-20 17:01 UTC (permalink / raw) To: Roman Cheplyaka; +Cc: zsh-workers On Sun, 20 Aug 2006 10:18:33 +0300 Roman Cheplyaka <roman.cheplyaka@gmail.com> wrote: > I have the following issue in UTF-8 locale: > zsh completion works ok in current folder, no matter which files are > here. > But suppose I have some directory named 'тест' (cyrillic letters). > Then > % ls те<Tab> > completes it to 'тест', but > % ls тест/<Tab> > doesn't show anything, even if there are no cyrillic-named items in that > directory. I can't get this, even with zsh -f; it seems to work fine both with my own completion set up and the plain old-fashioned defaul. I'm using the locale en_GB.UTF-8, but I tried it with ru_RU.UTF-8 as well. What actually are the contents of the directory (ls -a)? What locale are you using? What version of the shell? (UTF-8 support is new in 3.2, so if it's older it won't work well.) -- Peter Stephenson <p.w.stephenson@ntlworld.com> Web page now at http://homepage.ntlworld.com/p.w.stephenson/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-20 17:01 ` completion bug in UTF-8 locale Peter Stephenson @ 2006-08-20 20:08 ` Roman Cheplyaka 2006-08-20 22:18 ` Peter Stephenson 0 siblings, 1 reply; 7+ messages in thread From: Roman Cheplyaka @ 2006-08-20 20:08 UTC (permalink / raw) To: Peter Stephenson; +Cc: zsh-workers On Sun, Aug 20, 2006 at 06:01:51PM +0100, Peter Stephenson wrote: > On Sun, 20 Aug 2006 10:18:33 +0300 > Roman Cheplyaka <roman.cheplyaka@gmail.com> wrote: > > I have the following issue in UTF-8 locale: > > zsh completion works ok in current folder, no matter which files are > > here. > > But suppose I have some directory named 'тест' (cyrillic letters). > > Then > > % ls те<Tab> > > completes it to 'тест', but > > % ls тест/<Tab> > > doesn't show anything, even if there are no cyrillic-named items in that > > directory. > I can't get this, even with zsh -f; it seems to work fine both with my > own completion set up and the plain old-fashioned defaul. I've made some more tests. It seems to break at certain letters.. Try for example directory named "ц" (w/o quotes). > I'm using the > locale en_GB.UTF-8, but I tried it with ru_RU.UTF-8 as well. What > actually are the contents of the directory (ls -a)? It's irrelevant, I think. Ok, here it is: % ls -a ц ./ ../ aaaa > What locale are you > using? % locale LANG=POSIX LC_CTYPE=ru_UA.UTF-8 LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE=ru_UA.UTF-8 LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_PAPER="POSIX" LC_NAME="POSIX" LC_ADDRESS="POSIX" LC_TELEPHONE="POSIX" LC_MEASUREMENT="POSIX" LC_IDENTIFICATION="POSIX" LC_ALL= > What version of the shell? (UTF-8 support is new in 3.2, so if > it's older it won't work well.) % zsh --version zsh 4.3.2 (i686-pc-linux-gnu) % dpkg-query -W zsh zsh 4.3.2-14 -- Roman I. Cheplyaka ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-20 20:08 ` Roman Cheplyaka @ 2006-08-20 22:18 ` Peter Stephenson 2006-08-26 18:24 ` David Gómez 0 siblings, 1 reply; 7+ messages in thread From: Peter Stephenson @ 2006-08-20 22:18 UTC (permalink / raw) To: zsh-workers, Roman Cheplyaka On Sun, 20 Aug 2006 23:08:08 +0300 Roman Cheplyaka <roman.cheplyaka@gmail.com> wrote: > I've made some more tests. It seems to break at certain letters.. Try for > example directory named "ц" (w/o quotes). Aha. That needs one of the bytes to be encoded as a "Meta" byte (it overlaps with zsh's range of internal tokens). That's still broken in the patches I made for listing multibyte characters quite recently; the following should fix it. This means it's not surprising it's buggy in 4.3.2. Would you be able to try it with the code currently in CVS? Otherwise I'll probably make a pre-4.3.3 test version shortly. In the following patch I've assumed that any null-terminated string must be metafied. That's the usual rule, but nothing would surprise me about the completion code. Index: Src/Zle/complist.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v retrieving revision 1.94 diff -u -r1.94 complist.c --- Src/Zle/complist.c 17 Aug 2006 09:34:12 -0000 1.94 +++ Src/Zle/complist.c 20 Aug 2006 22:17:13 -0000 @@ -621,7 +621,11 @@ if (ml == mlend - 1 && (cc % columns) == columns - 1) return 0; - putc(*p, shout); + if (*p == Meta) { + p++; + putc(*p ^ 32, shout); + } else + putc(*p, shout); if ((beg = !(cc % columns))) ml++; if (mscroll && !(cc % columns) && @@ -1137,8 +1141,14 @@ dopr = 0; continue; } - while (len--) - putc(*p++, shout); + while (len--) { + if (*p == Meta) { + len--; + p++; + putc(*p++ ^ 32, shout); + } else + putc(*p++, shout); + } if ((beg = !(cc % columns)) && !stat) { ml++; fputs(" \010", shout); Index: Src/Zle/zle_tricky.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v retrieving revision 1.74 diff -u -r1.74 zle_tricky.c --- Src/Zle/zle_tricky.c 20 Aug 2006 18:07:49 -0000 1.74 +++ Src/Zle/zle_tricky.c 20 Aug 2006 22:17:15 -0000 @@ -2123,9 +2123,15 @@ tcout(TCUNDERLINEEND); break; case '{': - for (p++; *p && (*p != '%' || p[1] != '}'); p++) - if (dopr) + for (p++; *p && (*p != '%' || p[1] != '}'); p++) { + if (*p == Meta) { + p++; + if (dopr) + putc(*p ^ 32, shout); + } + else if (dopr) putc(*p, shout); + } if (*p) p++; else @@ -2164,8 +2170,14 @@ convchar_t cchar; int clen = MB_METACHARLENCONV(p, &cchar); if (dopr) { - while (clen--) - putc(*p++, shout); + while (clen--) { + if (*p == Meta) { + p++; + clen--; + putc(*p++ ^ 32, shout); + } else + putc(*p++, shout); + } } else p += clen; cc += WCWIDTH(cchar); -- Peter Stephenson <p.w.stephenson@ntlworld.com> Web page now at http://homepage.ntlworld.com/p.w.stephenson/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-20 22:18 ` Peter Stephenson @ 2006-08-26 18:24 ` David Gómez 2006-08-29 18:49 ` Peter Stephenson 0 siblings, 1 reply; 7+ messages in thread From: David Gómez @ 2006-08-26 18:24 UTC (permalink / raw) To: Peter Stephenson; +Cc: zsh-workers, Roman Cheplyaka Hi Peter, On Aug 20 at 11:18:38, Peter Stephenson wrote: > Roman Cheplyaka <roman.cheplyaka@gmail.com> wrote: > > I've made some more tests. It seems to break at certain letters.. Try for > > example directory named "ц" (w/o quotes). > > Aha. That needs one of the bytes to be encoded as a "Meta" byte (it > overlaps with zsh's range of internal tokens). > > That's still broken in the patches I made for listing multibyte > characters quite recently; the following should fix it. This means it's > not surprising it's buggy in 4.3.2. I'm testing latest CVS and the completion bug is still present. My locale is different from Roman's one (es_ES.UTF-8) and my tests are with japanese characters: [~] % ls 花 aaaa bbbb And [~] % cd 花/<Tab> doesn't complete. cheers, -- David Gómez Jabber ID: davidge@jabber.org ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-26 18:24 ` David Gómez @ 2006-08-29 18:49 ` Peter Stephenson 2006-08-30 15:54 ` Peter Stephenson 0 siblings, 1 reply; 7+ messages in thread From: Peter Stephenson @ 2006-08-29 18:49 UTC (permalink / raw) To: zsh-workers On Sat, 26 Aug 2006 20:24:33 +0200 David Gómez <david@pleyades.net> wrote: > Hi Peter, > I'm testing latest CVS and the completion bug is still present. My locale > is different from Roman's one (es_ES.UTF-8) and my tests are with japanese > characters: > > [~] % ls 花 > aaaa bbbb > > And > > [~] % cd 花/<Tab> > > doesn't complete. Is this after plain "zsh -f"? I can see the problem there, although I may not have a chance to look at it for a while since I'm in the middle of some holiday (it's probably a fairly trivial missing piece of Meta handling again). This isn't too surprising since I haven't been paying much attention to old-style completion. Or is this after compinit, possibly with some styles set? I can't see any problem there at the moment (and unless aaaa and bbbb are directories they wouldn't complete at that point). Generally speaking, the exact context is crucial in looking at completion bugs. -- Peter Stephenson <p.w.stephenson@ntlworld.com> Web page now at http://homepage.ntlworld.com/p.w.stephenson/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-29 18:49 ` Peter Stephenson @ 2006-08-30 15:54 ` Peter Stephenson 2006-08-30 17:13 ` David Gómez 0 siblings, 1 reply; 7+ messages in thread From: Peter Stephenson @ 2006-08-30 15:54 UTC (permalink / raw) To: zsh-workers Peter Stephenson <p.w.stephenson@ntlworld.com> wrote: > On Sat, 26 Aug 2006 20:24:33 +0200 > David Gómez <david@pleyades.net> wrote: > > Hi Peter, > > I'm testing latest CVS and the completion bug is still present. My locale > > is different from Roman's one (es_ES.UTF-8) and my tests are with japanese > > characters: > > > > [~] % ls 花 > > aaaa bbbb > > > > And > > > > [~] % cd 花/<Tab> > > > > doesn't complete. > > Is this after plain "zsh -f"? I can see the problem there, although I > may not have a chance to look at it for a while since I'm in the middle > of some holiday (it's probably a fairly trivial missing piece of Meta > handling again). This fixes the problem in this case, although I still don't know if this is the only problem. This isn't new with MULTIBYTE_SUPPORT, although in the case of this character you'd be seeing other strange effects if you didn't have that turned on. Index: Src/Zle/compctl.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/Zle/compctl.c,v retrieving revision 1.27 diff -u -r1.27 compctl.c --- Src/Zle/compctl.c 30 May 2006 22:35:04 -0000 1.27 +++ Src/Zle/compctl.c 30 Aug 2006 15:50:33 -0000 @@ -2127,9 +2127,9 @@ { DIR *d; struct stat buf; - char *n, p[PATH_MAX], *q = NULL, *e; + char *n, p[PATH_MAX], *q = NULL, *e, *pathpref; LinkList l = NULL; - int ns = 0, ng = opts[NULLGLOB], test, aw = addwhat; + int ns = 0, ng = opts[NULLGLOB], test, aw = addwhat, pathpreflen; opts[NULLGLOB] = 1; @@ -2145,12 +2145,18 @@ all = execs = 0; } /* Open directory. */ - if ((d = opendir((prpre && *prpre) ? prpre : "."))) { + if (prpre && *prpre) { + pathpref = dupstring(prpre); + unmetafy(pathpref, &pathpreflen); + } else { + pathpref = NULL; + pathpreflen = 0; + } + if ((d = opendir(pathpref ? pathpref : "."))) { /* If we search only special files, prepare a path buffer for stat. */ - if (!all && prpre) { - strcpy(p, prpre); - q = p + strlen(prpre); - } + if (!all && pathpreflen) + memcpy(p, pathpref, pathpreflen); + q = p + pathpreflen; /* Fine, now read the directory. */ while ((n = zreaddir(d, 1)) && !errflag) { /* Ignore files beginning with `.' unless the thing we found on * -- Peter Stephenson <pws@csr.com> Software Engineer CSR PLC, Churchill House, Cambridge Business Park, Cowley Road Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070 To access the latest news from CSR copy this link into a web browser: http://www.csr.com/email_sig.php ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: completion bug in UTF-8 locale 2006-08-30 15:54 ` Peter Stephenson @ 2006-08-30 17:13 ` David Gómez 0 siblings, 0 replies; 7+ messages in thread From: David Gómez @ 2006-08-30 17:13 UTC (permalink / raw) To: Peter Stephenson; +Cc: zsh-workers Hi Peter, On Aug 30 at 04:54:15, Peter Stephenson wrote: > > Is this after plain "zsh -f"? I can see the problem there, although I > > may not have a chance to look at it for a while since I'm in the middle > > of some holiday (it's probably a fairly trivial missing piece of Meta > > handling again). > > This fixes the problem in this case, although I still don't know if this is > the only problem. This isn't new with MULTIBYTE_SUPPORT, although in the > case of this character you'd be seeing other strange effects if you didn't > have that turned on. Great! That patch did the trick. I've tested it with another directories with different characters and with subdirectories and completion works in all cases. And yes, i was using old style completion, with no compinit at all loaded. Thanks for fixing this in your holiday ;-) cheers, -- David Gómez Jabber ID: davidge@jabber.org ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2006-08-31 0:56 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20060820071833.GA3850@localdomain> 2006-08-20 17:01 ` completion bug in UTF-8 locale Peter Stephenson 2006-08-20 20:08 ` Roman Cheplyaka 2006-08-20 22:18 ` Peter Stephenson 2006-08-26 18:24 ` David Gómez 2006-08-29 18:49 ` Peter Stephenson 2006-08-30 15:54 ` Peter Stephenson 2006-08-30 17:13 ` David Gómez
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).