* PATCH: (large) initial support for combining characters in ZLE.
@ 2008-04-13 16:54 Peter Stephenson
2008-04-13 17:32 ` Bart Schaefer
` (3 more replies)
0 siblings, 4 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-04-13 16:54 UTC (permalink / raw)
To: Zsh Hackers' List
Here is my first broad brush attempt to get combining characters working
in the line editor. I think it's now working well enough to make it
worth testing, even if most of the testing is going to be making sure I
haven't broken anything that currently works. SINGLE_LINE_ZLE in
particular could do with a workout since I've changed that without a
great deal of testing (it's not fundamentally kaput, but it could easily
be slightly bruised).
My definition of a combining character is slightly more restrictive than
I originally proposed: the base character must be an alphanumeric (and
I'm not sure about the numeric, I need to find a better definition), and
the zero-width characters afterwards (I haven't imposed a limit on how
many there are) must be punctuation. This reduces the chance of
silliness such as assuming a grave is going to appear over a space.
The ultimate goal is that a composed character in this format is treated
wherever it makes sense as a single character for editing.
To get combining characters to do anything useful, you will need to
set the option COMBINING_CHARS. There's no "ZLE" there because it
already has a minimal effect on the main shell in the [[:WORD:]] test:
arguably zero-width punctuation characters should be considered part of
the word regardless of the option, i.e. even if displayed specially, and
I might change this.
You will also need a terminal that supports this; if you don't have one,
turning the option on will just break things. So far I've only found
konsole, which works well, but presumably there are others.
Known gaps at the moment:
- Word tests need improving (a whole smattering HEREs)
- Vi "r" needs some work (another HERE)
- Delete-to-character needs some work (another couple of HEREs).
- As always, completion is so complicated it could do pretty much
anything
- Although you have the tools to test for all the relevant properties in
user-defined widgets, it's not as simple as you'd probably want, and
I've made no effort to fix up the widgets currently supplied. I'm
still in multiple minds as to how to improve this.
Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.57
diff -u -r1.57 options.yo
--- Doc/Zsh/options.yo 7 Mar 2008 09:44:30 -0000 1.57
+++ Doc/Zsh/options.yo 13 Apr 2008 16:22:13 -0000
@@ -1444,6 +1444,24 @@
item(tt(BEEP) (tt(PLUS()B)) <D>)(
Beep on error in ZLE.
)
+pindex(COMBINING_CHARS)
+cindex(characters, (Unicode) combining)
+cindex(combining characters (Unicode))
+cindex(Unicode combining characters)
+item(tt(COMBINING_CHARS))(
+Assume that the terminal displays combining characters correctly.
+Specifically, if a base alphanumeric character is followed by one or more
+zero-width punctuation characters, assume that the zero-width charaters
+will be displayed as modifications to the base character within the
+same width. Not all terminals handle this. If this option is not
+set, zero-width characters are displayed separately with special
+mark-up.
+
+If this option is set, the pattern test tt([[:WORD:]]) matches a
+zero-width punctuation character on the assumption that it will be
+used as part of a word in combination with a word character.
+Otherwise the base shell does not handle combining characters specially.
+)
pindex(EMACS)
item(tt(EMACS))(
If ZLE is loaded, turning on this option has the equivalent effect
Index: Src/options.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/options.c,v
retrieving revision 1.39
diff -u -r1.39 options.c
--- Src/options.c 7 Mar 2008 09:44:30 -0000 1.39
+++ Src/options.c 13 Apr 2008 16:22:14 -0000
@@ -101,6 +101,7 @@
{{NULL, "chaselinks", OPT_EMULATE}, CHASELINKS},
{{NULL, "checkjobs", OPT_EMULATE|OPT_ZSH}, CHECKJOBS},
{{NULL, "clobber", OPT_EMULATE|OPT_ALL}, CLOBBER},
+{{NULL, "combiningchars", 0}, COMBININGCHARS},
{{NULL, "completealiases", 0}, COMPLETEALIASES},
{{NULL, "completeinword", 0}, COMPLETEINWORD},
{{NULL, "correct", 0}, CORRECT},
Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.183
diff -u -r1.183 utils.c
--- Src/utils.c 14 Mar 2008 12:14:06 -0000 1.183
+++ Src/utils.c 13 Apr 2008 16:22:15 -0000
@@ -3081,6 +3081,14 @@
case IWORD:
if (iswalnum(c))
return 1;
+ /*
+ * If we are handling combining characters, anything
+ * printable with zero width needs to be considered
+ * part of a word.
+ */
+ if (isset(COMBININGCHARS) &&
+ iswprint(c) && wcwidth(c) == 0)
+ return 1;
return !!wmemchr(wordchars_wide.chars, c, wordchars_wide.len);
case ISEP:
Index: Src/zsh.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v
retrieving revision 1.120
diff -u -r1.120 zsh.h
--- Src/zsh.h 3 Apr 2008 11:39:11 -0000 1.120
+++ Src/zsh.h 13 Apr 2008 16:22:17 -0000
@@ -1715,6 +1715,7 @@
CHASELINKS,
CHECKJOBS,
CLOBBER,
+ COMBININGCHARS,
COMPLETEALIASES,
COMPLETEINWORD,
CORRECT,
@@ -1936,6 +1937,10 @@
#define tccan(X) (tclen[X])
+/*
+ * Text attributes for displaying in ZLE
+ */
+
#define TXTBOLDFACE 0x01
#define TXTSTANDOUT 0x02
#define TXTUNDERLINE 0x04
@@ -1955,6 +1960,12 @@
/* Bits to shift off right to get on */
#define TXT_ATTR_OFF_ON_SHIFT (4)
+/*
+ * Indicates to zle_refresh.c that the character entry is an
+ * index into the list of multiword symbols.
+ */
+#define TXT_MULTIWORD_MASK 0x100
+
#define txtchangeisset(T,X) ((T) & (X))
#define txtchangeset(X, Y) (txtchange |= (X), txtchange &= ~(Y))
Index: Src/Zle/compcore.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/compcore.c,v
retrieving revision 1.92
diff -u -r1.92 compcore.c
--- Src/Zle/compcore.c 28 Nov 2007 21:14:14 -0000 1.92
+++ Src/Zle/compcore.c 13 Apr 2008 16:22:17 -0000
@@ -349,7 +349,7 @@
if (makecomplist(s, incmd, lst)) {
/* Error condition: feeeeeeeeeeeeep(). */
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
inststr(origline);
zlemetacs = origcs;
clearlist = 1;
@@ -381,7 +381,7 @@
} else if (!useline && uselist) {
/* All this and the guy only wants to see the list, sigh. */
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
inststr(origline);
zlemetacs = origcs;
showinglist = -2;
@@ -429,7 +429,7 @@
if (forcelist)
clearlist = 1;
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
inststr(origline);
zlemetacs = origcs;
}
@@ -519,7 +519,7 @@
if (ret >= 2) {
fixsuffix();
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
inststr(origline);
zlemetacs = origcs;
if (ret == 2) {
Index: Src/Zle/complist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v
retrieving revision 1.109
diff -u -r1.109 complist.c
--- Src/Zle/complist.c 28 Mar 2008 23:21:26 -0000 1.109
+++ Src/Zle/complist.c 13 Apr 2008 16:22:18 -0000
@@ -2052,7 +2052,7 @@
s[lastend - zlemetacs] = '\0';
}
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(sll);
memcpy(zlemetaline, sline, sll);
zlemetacs = scs;
@@ -2298,7 +2298,7 @@
*/
mode = MM_INTER;
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(l);
strncpy(zlemetaline, origline, l);
zlemetacs = origcs;
@@ -2501,7 +2501,7 @@
*/
mode = MM_INTER;
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(l);
strncpy(zlemetaline, origline, l);
zlemetacs = origcs;
@@ -2560,7 +2560,7 @@
* characters typed by the user.
*/
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(l);
strncpy(zlemetaline, origline, l);
zlemetacs = origcs;
@@ -2701,7 +2701,7 @@
handleundo();
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(l = strlen(u->line));
strncpy(zlemetaline, u->line, l);
zlemetacs = u->cs;
@@ -3090,7 +3090,7 @@
origcs = modecs;
origll = modell;
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(origll);
strncpy(zlemetaline, origline, origll);
zlemetacs = origcs;
Index: Src/Zle/compresult.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/compresult.c,v
retrieving revision 1.73
diff -u -r1.73 compresult.c
--- Src/Zle/compresult.c 14 Feb 2008 15:21:38 -0000 1.73
+++ Src/Zle/compresult.c 13 Apr 2008 16:22:19 -0000
@@ -469,7 +469,7 @@
memcpy(r, zlemetaline + ocs, i);
r[i] = '\0';
zlemetacs = ocs;
- foredel(i);
+ foredel(i, CUT_RAW);
if (csp)
*csp = ncs - ocs;
@@ -707,7 +707,7 @@
instmatch(m, NULL);
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(oll);
memcpy(zlemetaline, oline, oll);
zlemetacs = ocs;
@@ -778,7 +778,7 @@
tcs = zlemetacs;
zlemetacs = wb;
memcpy(old, zlemetaline + wb, we - wb);
- foredel(we - wb);
+ foredel(we - wb, CUT_RAW);
/* Now get the unambiguous string and insert it into the line. */
cline_str(ainfo->line, 1, NULL, NULL);
@@ -789,7 +789,7 @@
if (lastend < we && !lenchanged && !hasunmatched) {
zlemetacs = wb;
- foredel(lastend - wb);
+ foredel(lastend - wb, CUT_RAW);
inststrlen(old, 0, we - wb);
lastend = we;
zlemetacs = tcs;
@@ -978,7 +978,7 @@
minfo.insc = 0;
zlemetacs = minfo.pos;
- foredel(l);
+ foredel(l, CUT_RAW);
if (m->flags & CMF_ALL) {
do_allmatches(0);
@@ -1316,7 +1316,7 @@
l = zlemetacs;
zlemetacs = minfo.pos + minfo.len + minfo.insc - (*(minfo.cur))->qisl;
if (zlemetacs < l)
- foredel(l - zlemetacs);
+ foredel(l - zlemetacs, CUT_RAW);
else if (zlemetacs > zlemetall)
zlemetacs = zlemetall;
inststrlen(" ", 1, 1);
Index: Src/Zle/deltochar.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/deltochar.c,v
retrieving revision 1.8
diff -u -r1.8 deltochar.c
--- Src/Zle/deltochar.c 6 Jul 2007 21:52:40 -0000 1.8
+++ Src/Zle/deltochar.c 13 Apr 2008 16:22:19 -0000
@@ -46,10 +46,11 @@
while (dest != zlell && (ZLE_INT_T)zleline[dest] != c)
dest++;
if (dest != zlell) {
+ /* HERE adjust dest for trailing combining chars */
if (!zap || n > 0)
- dest++;
+ INCCS();
if (!n) {
- forekill(dest - zlecs, 0);
+ forekill(dest - zlecs, CUT_RAW);
ok++;
}
}
@@ -63,7 +64,8 @@
dest--;
if ((ZLE_INT_T)zleline[dest] == c) {
if (!n) {
- backkill(zlecs - dest - zap, 1);
+ /* HERE adjust zap for trailing combining chars */
+ backkill(zlecs - dest - zap, CUT_RAW|CUT_FRONT);
ok++;
}
if (dest)
Index: Src/Zle/zle.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle.h,v
retrieving revision 1.36
diff -u -r1.36 zle.h
--- Src/Zle/zle.h 3 Apr 2008 11:39:12 -0000 1.36
+++ Src/Zle/zle.h 13 Apr 2008 16:22:23 -0000
@@ -74,6 +74,16 @@
#define LASTFULLCHAR lastchar_wide
#define LASTFULLCHAR_T ZLE_INT_T
+/* We may need to handle combining character alignment */
+#define CCLEFT() alignmultiwordleft(1)
+#define CCRIGHT() alignmultiwordright(1)
+/*
+ * Increment or decrement the cursor position, skipping over
+ * combining characters.
+ */
+#define INCCS() inccs()
+#define DECCS() deccs()
+
#else /* Not MULTIBYTE_SUPPORT: old single-byte code */
typedef char ZLE_CHAR_T;
@@ -133,6 +143,15 @@
#define LASTFULLCHAR lastchar
#define LASTFULLCHAR_T int
+/* Combining character alignment: none in this mode */
+#define CCLEFT()
+#define CCRIGHT()
+/*
+ * Increment or decrement the cursor position: simple in this case.
+ */
+#define INCCS() ((void)(zlecs++))
+#define DECCS() ((void)(zlecs--))
+
#endif
@@ -202,6 +221,12 @@
#define zmult (zmod.mult)
+/* flags to cut() and cuttext() and other front-ends */
+
+#define CUT_FRONT (1<<0) /* Text goes in front of cut buffer */
+#define CUT_REPLACE (1<<1) /* Text replaces cut buffer */
+#define CUT_RAW (1<<2) /* Raw character counts (not used in cut itself) */
+
/* undo system */
struct change {
@@ -340,7 +365,11 @@
* Description of one screen cell in zle_refresh.c
*/
typedef struct {
- /* The (possibly wide) character */
+ /*
+ * The (possibly wide) character.
+ * If atr contains TXT_MULTIWORD_MASK, an index into the set of multiword
+ * symbols (only if MULTIBYTE_SUPPORT is present).
+ */
REFRESH_CHAR chr;
/*
* Its attributes. 'On' attributes (TXT_ATTR_ON_MASK) are
@@ -349,7 +378,7 @@
* need the effect; 'off' attributes are only present for the
* last character in the sequence.
*/
- REFRESH_CHAR atr;
+ int atr;
} REFRESH_ELEMENT;
/* A string of screen cells */
Index: Src/Zle/zle_hist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_hist.c,v
retrieving revision 1.38
diff -u -r1.38 zle_hist.c
--- Src/Zle/zle_hist.c 11 Apr 2008 11:38:10 -0000 1.38
+++ Src/Zle/zle_hist.c 13 Apr 2008 16:22:23 -0000
@@ -640,7 +640,7 @@
if (deleteword) {
int pos = zlemetacs;
zlemetacs = lastpos;
- foredel(pos - zlemetacs);
+ foredel(pos - zlemetacs, CUT_RAW);
/*
* Mark that this has been deleted.
* For consistency with history lines, we really ought to
@@ -701,7 +701,7 @@
if (deleteword > 0) {
int pos = zlemetacs;
zlemetacs = lastpos;
- foredel(pos - zlemetacs);
+ foredel(pos - zlemetacs, CUT_RAW);
}
if (lastinsert) {
zfree(lastinsert, lastlen);
Index: Src/Zle/zle_misc.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_misc.c,v
retrieving revision 1.44
diff -u -r1.44 zle_misc.c
--- Src/Zle/zle_misc.c 3 Apr 2008 15:20:30 -0000 1.44
+++ Src/Zle/zle_misc.c 13 Apr 2008 16:22:23 -0000
@@ -105,6 +105,7 @@
int
deletechar(char **args)
{
+ int n;
if (zmult < 0) {
int ret;
zmult = -zmult;
@@ -112,12 +113,15 @@
zmult = -zmult;
return ret;
}
- if (zlecs + zmult <= zlell) {
- zlecs += zmult;
- backdel(zmult);
- return 0;
+
+ n = zmult;
+ while (n--) {
+ if (zlecs == zlell)
+ return 1;
+ INCCS();
}
- return 1;
+ backdel(zmult, 0);
+ return 0;
}
/**/
@@ -131,7 +135,7 @@
zmult = -zmult;
return ret;
}
- backdel(zmult > zlecs ? zlecs : zmult);
+ backdel(zmult > zlecs ? zlecs : zmult, 0);
return 0;
}
@@ -149,7 +153,7 @@
while (zlecs && zleline[zlecs - 1] != '\n')
zlecs--;
for (i = zlecs; i != zlell && zleline[i] != '\n'; i++);
- forekill(i - zlecs + (i != zlell), fg);
+ forekill(i - zlecs + (i != zlell), fg ? (CUT_FRONT|CUT_RAW) : CUT_RAW);
}
clearlist = 1;
return 0;
@@ -160,7 +164,7 @@
killbuffer(UNUSED(char **args))
{
zlecs = 0;
- forekill(zlell, 0);
+ forekill(zlell, CUT_RAW);
clearlist = 1;
return 0;
}
@@ -185,7 +189,7 @@
while (zlecs && zleline[zlecs - 1] != '\n')
zlecs--, i++;
}
- forekill(i, 1);
+ forekill(i, CUT_FRONT|CUT_RAW);
clearlist = 1;
return 0;
}
@@ -267,13 +271,13 @@
zlecs = findeol();
}
} else {
- foredel(1);
+ foredel(1, 0);
zlecs = findeol();
while(zlecs != zlell) {
zlecs++;
vifirstnonblank(zlenoargs);
if(zleline[zlecs] == '#')
- foredel(1);
+ foredel(1, 0);
zlecs = findeol();
}
}
@@ -319,7 +323,7 @@
while (zlecs != zlell && zleline[zlecs] != ZWC('\n'))
zlecs++, i++;
}
- backkill(i, 0);
+ backkill(i, CUT_RAW);
clearlist = 1;
return 0;
}
@@ -331,9 +335,9 @@
if (mark > zlell)
mark = zlell;
if (mark > zlecs)
- forekill(mark - zlecs, 0);
+ forekill(mark - zlecs, CUT_RAW);
else
- backkill(zlecs - mark, 1);
+ backkill(zlecs - mark, CUT_FRONT|CUT_RAW);
return 0;
}
@@ -344,7 +348,7 @@
if (*args) {
int len;
ZLE_STRING_T line = stringaszleline(*args, 0, &len, NULL, NULL);
- cuttext(line, len, -1);
+ cuttext(line, len, CUT_REPLACE);
free(line);
} else {
if (mark > zlell)
@@ -352,7 +356,7 @@
if (mark > zlecs)
cut(zlecs, mark - zlecs, 0);
else
- cut(mark, zlecs - mark, 1);
+ cut(mark, zlecs - mark, CUT_FRONT);
}
return 0;
}
@@ -441,7 +445,7 @@
} while (!buf->buf || *buf->buf == ZWC('\0'));
zlecs = yankb;
- foredel(yanke - yankb);
+ foredel(yanke - yankb, CUT_RAW);
cc = buf->len;
spaceinline(cc);
ZS_memcpy(zleline + zlecs, buf->buf, cc);
@@ -766,7 +770,7 @@
}
str = (ZLE_STRING_T)hcalloc((len = mark - zlecs) * ZLE_CHAR_SIZE);
ZS_memcpy(str, zleline + zlecs, len);
- foredel(len);
+ foredel(len, CUT_RAW);
str = makequote(str, &len);
spaceinline(len);
ZS_memcpy(zleline + zlecs, str, len);
@@ -1321,10 +1325,10 @@
/* must be shifting wide character lengths */
if (zlemetaline != NULL) {
unmetafy_line();
- backdel(sl);
+ backdel(sl, CUT_RAW);
metafy_line();
} else
- backdel(sl);
+ backdel(sl, CUT_RAW);
if (!keep)
invalidatelist();
}
Index: Src/Zle/zle_move.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_move.c,v
retrieving revision 1.10
diff -u -r1.10 zle_move.c
--- Src/Zle/zle_move.c 3 Apr 2008 11:39:12 -0000 1.10
+++ Src/Zle/zle_move.c 13 Apr 2008 16:22:23 -0000
@@ -32,6 +32,112 @@
static int vimarkcs[27], vimarkline[27];
+#ifdef MULTIBYTE_SUPPORT
+/*
+ * Take account of combining characters when moving left. If
+ * we are on a zero-width printable wide character and are
+ * treating these as part of the base character for display purposes,
+ * move left until we reach a non-zero-width printable character
+ * (the base character). If we reach something else first, stay where we
+ * were.
+ *
+ * If setpos is non-zero, update zlecs on success.
+ * Return 1 if we were on a combining char and could move, else 0.
+ */
+/**/
+int
+alignmultiwordleft(int setpos)
+{
+ int loccs;
+
+ /* generic nothing to do test */
+ if (!isset(COMBININGCHARS) || zlecs == zlell || zlecs == 0)
+ return 0;
+
+ /* need to be on zero-width punctuation character */
+ if (!iswpunct(zleline[zlecs]) || wcwidth(zleline[zlecs]) != 0)
+ return 0;
+
+ /* yes, go left */
+ loccs = zlecs - 1;
+
+ for (;;) {
+ /* second test here is paranoia */
+ if (iswalnum(zleline[loccs]) && wcwidth(zleline[loccs]) > 0) {
+ /* found start position */
+ if (setpos)
+ zlecs = loccs;
+ return 1;
+ } else if (!iswpunct(zleline[loccs]) ||
+ wcwidth(zleline[loccs]) != 0) {
+ /* no go */
+ return 0;
+ }
+ /* combining char, keep going */
+ if (loccs-- == 0)
+ return 0;
+ }
+}
+
+
+/*
+ * Same principle when moving right. We need to check if
+ * alignmultiwordleft() would be successful in order to decide
+ * if we're on a combining character, and if so we move right to
+ * anything that isn't one.
+ */
+/**/
+int
+alignmultiwordright(int setpos)
+{
+ int loccs;
+
+ /*
+ * Are we on a suitable character?
+ */
+ if (!alignmultiwordleft(0))
+ return 0;
+
+ /* yes, go right */
+ loccs = zlecs + 1;
+
+ while (loccs < zlell) {
+ /* Anything other than a combining char will do here */
+ if (!iswpunct(zleline[loccs]) || wcwidth(zleline[loccs]) != 0) {
+ if (setpos)
+ zlecs = loccs;
+ return 1;
+ }
+ loccs++;
+ }
+
+ zlecs = zlell;
+ return 1;
+}
+
+
+/* Move cursor right, checking for combining characters */
+
+/**/
+mod_export void
+inccs(void)
+{
+ zlecs++;
+ alignmultiwordright(1);
+}
+
+
+/* Move cursor left, checking for combining characters */
+
+/**/
+mod_export void
+deccs(void)
+{
+ zlecs--;
+ alignmultiwordleft(1);
+}
+#endif
+
/**/
int
beginningofline(char **args)
@@ -157,25 +263,43 @@
/**/
int
-forwardchar(UNUSED(char **args))
+forwardchar(char **args)
{
- zlecs += zmult;
- if (zlecs > zlell)
- zlecs = zlell;
- if (zlecs < 0)
- zlecs = 0;
+ int n = zmult;
+
+ if (n < 0) {
+ int ret;
+ zmult = -n;
+ ret = backwardchar(args);
+ zmult = n;
+ return ret;
+ }
+
+ /*
+ * If handling combining characters with the base character,
+ * we skip over the whole set in one go, so need to check.
+ */
+ while (zlecs < zlell && n--)
+ INCCS();
return 0;
}
/**/
int
-backwardchar(UNUSED(char **args))
+backwardchar(char **args)
{
- zlecs -= zmult;
- if (zlecs > zlell)
- zlecs = zlell;
- if (zlecs < 0)
- zlecs = 0;
+ int n = zmult;
+
+ if (n < 0) {
+ int ret;
+ zmult = -n;
+ ret = forwardchar(args);
+ zmult = n;
+ return ret;
+ }
+
+ while (zlecs > 0 && n--)
+ DECCS();
return 0;
}
@@ -216,17 +340,21 @@
int
vigotocolumn(UNUSED(char **args))
{
- int x, y;
+ int x, y, n = zmult;
findline(&x, &y);
- if (zmult >= 0)
- zlecs = x + zmult - (zmult > 0);
- else
- zlecs = y + zmult;
- if (zlecs > y)
- zlecs = y;
- if (zlecs < x)
+ if (n >= 0) {
+ if (n)
+ n--;
zlecs = x;
+ while (zlecs < y && n--)
+ INCCS();
+ } else {
+ zlecs = y;
+ n = -n;
+ while (zlecs > x && n--)
+ DECCS();
+ }
return 0;
}
@@ -286,7 +414,7 @@
zlecs = ocs;
return 1;
} else if(dir > 0 && virangeflag)
- zlecs++;
+ INCCS();
return 0;
}
@@ -307,7 +435,7 @@
if (zlecs >= lim)
return 1;
while (n-- && zlecs < lim)
- zlecs++;
+ INCCS();
return 0;
}
@@ -326,9 +454,9 @@
}
if (zlecs == findbol())
return 1;
- while (n--) {
- zlecs--;
- if (zlecs < 0 || zleline[zlecs] == '\n') {
+ while (n-- && zlecs > 0) {
+ DECCS();
+ if (zleline[zlecs] == '\n') {
zlecs++;
break;
}
@@ -432,7 +560,10 @@
}
while (n--) {
do {
- zlecs += vfinddir;
+ if (vfinddir > 0)
+ INCCS();
+ else
+ DECCS();
} while (zlecs >= 0 && zlecs < zlell
&& (ZLE_INT_T)zleline[zlecs] != vfindchar
&& zleline[zlecs] != ZWC('\n'));
@@ -441,9 +572,12 @@
return 1;
}
}
- zlecs += tailadd;
+ if (tailadd > 0)
+ INCCS();
+ else if (tailadd < 0)
+ DECCS();
if (vfinddir == 1 && virangeflag)
- zlecs++;
+ INCCS();
return 0;
}
@@ -471,7 +605,7 @@
{
zlecs = findbol();
while (zlecs != zlell && ZC_iblank(zleline[zlecs]))
- zlecs++;
+ INCCS();
return 0;
}
Index: Src/Zle/zle_refresh.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_refresh.c,v
retrieving revision 1.56
diff -u -r1.56 zle_refresh.c
--- Src/Zle/zle_refresh.c 4 Apr 2008 16:32:16 -0000 1.56
+++ Src/Zle/zle_refresh.c 13 Apr 2008 16:22:23 -0000
@@ -29,7 +29,58 @@
#include "zle.mdh"
+#ifdef MULTIBYTE_SUPPORT
+/*
+ * Handling for glyphs that contain more than one wide character,
+ * if ZLE_COMBINING_CHARS is set. Each glyph is one character with
+ * non-zero width followed by an arbitrary (but typically small)
+ * number of characters that have zero width (combining characters).
+ *
+ * The allocated size for each array is given by ?mw_size; nmw_ind
+ * is the next free element, i.e. nmwbuf[nmw_ind] will be the next
+ * element to be written (we never insert into omwbuf). We initialise
+ * nmw_ind to 1 to avoid the index stored in the character looking like a
+ * NULL. This wastees a word but it's safer than messing with pointers.
+ *
+ * The layout of the buffer is as a string of entries that consist of multiple
+ * elements of the allocated array with no boundary (the code keeps track of
+ * where each entry starts). Note distinction between (logical) entries and
+ * (array) elements. Each entry consists of an element giving the total
+ * number of wide characters for the entry (there are N+1 wide characters,
+ * where N >= 1 is the number of trailing zero width characters), followed by
+ * those characters.
+ */
+static REFRESH_CHAR
+ *omwbuf = NULL, /* old multiword glyph buffer */
+ *nmwbuf = NULL; /* new multiword glyph buffer */
+#endif
+
+/*
+ * Compare if two characters are equal.
+ */
+#ifdef MULTIBYTE_SUPPORT
+/*
+ * We may need to compare values in multiword arrays. As the arrays are
+ * different for the old and new video arrays, it is vital that the comparison
+ * always be done in the correct order: an element of the old video array,
+ * followed by an element of the new one. In this case, having ascertained
+ * that both elements are multiword (because they have the some attributes),
+ * we do the character comparison in two stages: first we check that the
+ * lengths are the same, then we check that the characters stored are the
+ * same. This ensures we can't read past the end of either array. If either
+ * character is a constant, then TXT_MULTIWORD_MASK is guaranteed not to be
+ * set and this doesn't matter.
+ */
+#define ZR_equal(oldzr, newzr) \
+ ((oldzr).atr == (newzr).atr && \
+ (((oldzr).atr & TXT_MULTIWORD_MASK) ? \
+ (omwbuf[(oldzr).chr] == nmwbuf[(newzr).chr] && \
+ !memcmp(omwbuf + (oldzr).chr + 1, nmwbuf + (newzr).chr + 1, \
+ omwbuf[(oldzr).chr] * sizeof(*omwbuf))) : \
+ (oldzr).chr == (newzr).chr))
+#else
#define ZR_equal(zr1, zr2) ((zr1).chr == (zr2).chr && (zr1).atr == (zr2).atr)
+#endif
static void
ZR_memset(REFRESH_ELEMENT *dst, REFRESH_ELEMENT rc, int len)
@@ -61,17 +112,22 @@
/*
* Simplified strcmp: we don't need the sign, just whether
* the strings and their attributes are equal.
+ *
+ * In the multibyte case, the two elements must be in the order
+ * element from old video array, element from new video array.
*/
static int
-ZR_strncmp(const REFRESH_ELEMENT *wstr1, const REFRESH_ELEMENT *wstr2, int len)
+ZR_strncmp(const REFRESH_ELEMENT *oldwstr, const REFRESH_ELEMENT *newwstr,
+ int len)
{
while (len--) {
- if (!wstr1->chr || !wstr2->chr)
- return !ZR_equal(*wstr1, *wstr2);
- if (!ZR_equal(*wstr1, *wstr2))
+ if ((!(oldwstr->atr & TXT_MULTIWORD_MASK) && !oldwstr->chr) ||
+ (!(newwstr->atr & TXT_MULTIWORD_MASK) && !newwstr->chr))
+ return !ZR_equal(*oldwstr, *newwstr);
+ if (!ZR_equal(*oldwstr, *newwstr))
return 1;
- wstr1++;
- wstr2++;
+ oldwstr++;
+ newwstr++;
}
return 0;
@@ -502,9 +558,19 @@
}
+/*
+ * Output the character. This must come from the new video
+ * buffer, nbuf, since we access the multiword buffer nmwbuf
+ * directly.
+ *
+ * curatrp may be NULL, otherwise points to an integer specifying
+ * what attributes were turned on for a character output immediately
+ * before, in order to optimise output of attribute changes.
+ */
+
/**/
void
-zwcputc(const REFRESH_ELEMENT *c, REFRESH_CHAR *curatrp)
+zwcputc(const REFRESH_ELEMENT *c, int *curatrp)
{
/*
* Safety: turn attributes off if last heard of turned on.
@@ -536,7 +602,17 @@
}
#ifdef MULTIBYTE_SUPPORT
- if (c->chr != WEOF) {
+ if (c->atr & TXT_MULTIWORD_MASK) {
+ /* Multiword glyph stored in nmwbuf */
+ int nchars = nmwbuf[c->chr];
+ REFRESH_CHAR *wcptr = nmwbuf + c->chr + 1;
+
+ memset(&mbstate, 0, sizeof(mbstate_t));
+ while (nchars--) {
+ if ((i = wcrtomb(mbtmp, (wchar_t)*wcptr++, &mbstate)) > 0)
+ fwrite(mbtmp, i, 1, shout);
+ }
+ } else if (c->chr != WEOF) {
memset(&mbstate, 0, sizeof(mbstate_t));
if ((i = wcrtomb(mbtmp, (wchar_t)c->chr, &mbstate)) > 0)
fwrite(mbtmp, i, 1, shout);
@@ -545,6 +621,10 @@
fputc(c->chr, shout);
#endif
+ /*
+ * Always output "off" attributes since we only turn off at
+ * the end of a chunk of highlighted text.
+ */
if (c->atr & TXT_ATTR_OFF_MASK) {
settextattributes(c->atr & TXT_ATTR_OFF_MASK);
lastatr &= ~((c->atr & TXT_ATTR_OFF_MASK) >> TXT_ATTR_OFF_ON_SHIFT);
@@ -563,7 +643,7 @@
zwcwrite(const REFRESH_STRING s, size_t i)
{
size_t j;
- REFRESH_CHAR curatr = 0;
+ int curatr = 0;
for (j = 0; j < i; j++)
zwcputc(s + j, &curatr);
@@ -593,6 +673,17 @@
winprompt, /* singlelinezle: part of lprompt showing */
winw_alloc = -1, /* allocated window width */
winh_alloc = -1; /* allocates window height */
+#ifdef MULTIBYTE_SUPPORT
+static int
+ omw_size, /* allocated size of omwbuf */
+ nmw_size, /* allocated size of nmwbuf */
+ nmw_ind; /* next insert point in nmw_ind */
+#endif
+
+/*
+ * Number of words to allocate in one go for the multiword buffers.
+ */
+#define DEF_MWBUF_ALLOC (32)
static void
freevideo(void)
@@ -605,6 +696,12 @@
}
free(nbuf);
free(obuf);
+#ifdef MULTIBYTE_SUPPORT
+ zfree(nmwbuf, nmw_size * sizeof(*nmwbuf));
+ zfree(omwbuf, omw_size * sizeof(*omwbuf));
+ omw_size = nmw_size = 0;
+ nmw_ind = 1;
+#endif
nbuf = NULL;
obuf = NULL;
}
@@ -631,6 +728,15 @@
nbuf[0] = (REFRESH_STRING)zalloc((winw + 2) * sizeof(**nbuf));
obuf[0] = (REFRESH_STRING)zalloc((winw + 2) * sizeof(**obuf));
+#ifdef MULTIBYTE_SUPPORT
+ nmw_size = DEF_MWBUF_ALLOC;
+ nmw_ind = 1;
+ nmwbuf = (REFRESH_CHAR *)zalloc(nmw_size * sizeof(*nmwbuf));
+
+ omw_size = DEF_MWBUF_ALLOC;
+ omwbuf = (REFRESH_CHAR *)zalloc(omw_size * sizeof(*omwbuf));
+#endif
+
winw_alloc = winw;
winh_alloc = winh;
}
@@ -803,6 +909,72 @@
tsetcap(TCUNDERLINEBEG, 0);
}
+#ifdef MULTIBYTE_SUPPORT
+/*
+ * Add a multiword glyph at the screen location base.
+ * tptr points to the source and there are ichars characters.
+ */
+static void
+addmultiword(REFRESH_ELEMENT *base, ZLE_STRING_T tptr, int ichars)
+{
+ /* Number of characters needed in buffer incl. count */
+ int iadd = ichars + 1, icnt;
+ REFRESH_CHAR *nmwptr;
+ base->atr |= TXT_MULTIWORD_MASK;
+ /* check allocation */
+ if (nmw_ind + iadd > nmw_size) {
+ /* need more space in buffer */
+ int mw_more = (iadd > DEF_MWBUF_ALLOC) ? iadd :
+ DEF_MWBUF_ALLOC;
+ nmwbuf = (REFRESH_CHAR *)
+ zrealloc(nmwbuf, (nmw_size += mw_more) *
+ sizeof(*nmwbuf));
+ }
+ /* make buffer entry: count, then characters */
+ nmwptr = nmwbuf + nmw_ind;
+ *nmwptr++ = ichars;
+ for (icnt = 0; icnt < ichars; icnt++)
+ *nmwptr++ = tptr[icnt];
+ /* save index and update */
+ base->chr = (wint_t)nmw_ind;
+ nmw_ind += iadd;
+}
+#endif
+
+
+/*
+ * Swap the old and new video buffers, plus any associated multiword
+ * buffers. The new buffer becomes the old one; the new new buffer
+ * will be filled with the command line next time.
+ */
+static void
+bufswap(void)
+{
+ REFRESH_STRING *qbuf;
+#ifdef MULTIBYTE_SUPPORT
+ REFRESH_CHAR *qmwbuf;
+ int itmp;
+#endif
+
+ qbuf = nbuf;
+ nbuf = obuf;
+ obuf = qbuf;
+
+#ifdef MULTIBYTE_SUPPORT
+/* likewise multiword buffers */
+ qmwbuf = nmwbuf;
+ nmwbuf = omwbuf;
+ omwbuf = qmwbuf;
+
+ itmp = nmw_size;
+ nmw_size = omw_size;
+ omw_size = itmp;
+
+ nmw_ind = 1;
+#endif
+}
+
+
/**/
mod_export void
zrefresh(void)
@@ -814,7 +986,6 @@
t, /* pointer into the real buffer */
scs, /* pointer to cursor position in real buffer */
u; /* pointer for status line stuff */
- REFRESH_STRING *qbuf; /* tmp */
int tmpcs, tmpll; /* ditto cursor position and line length */
int tmppos; /* t - tmpline */
int tmpalloced; /* flag to free tmpline when finished */
@@ -867,6 +1038,7 @@
/* this will create region_highlights if it's still NULL */
zle_set_highlight();
+ /* check for region between point ($CURSOR) and mark ($MARK) */
if (region_active) {
if (zlecs <= mark) {
region_highlights->start = zlecs;
@@ -1009,9 +1181,6 @@
struct region_highlight *rhp;
/*
* Calculate attribute based on region.
- * HERE: we may need to be smarter about turning
- * attributes off if bailing out before the end of the
- * region.
*/
for (ireg = 0, rhp = region_highlights;
ireg < n_region_highlights;
@@ -1054,13 +1223,13 @@
}
#ifdef MULTIBYTE_SUPPORT
else if (iswprint(*t) && (width = wcwidth(*t)) > 0) {
+ int ichars;
if (width > rpms.sen - rpms.s) {
int started = 0;
/*
* Too wide to fit. Insert spaces to end of current line.
*/
do {
- /* HERE highlight */
rpms.s->chr = ZWC(' ');
if (!started)
started = 1;
@@ -1076,12 +1245,21 @@
rpms.nvcs = rpms.s - nbuf[rpms.nvln = rpms.ln];
}
}
+ if (isset(COMBININGCHARS) && iswalnum(*t)) {
+ /*
+ * Look for combining characters: trailing punctuation
+ * characters with printing width zero.
+ */
+ for (ichars = 1; tmppos + ichars < tmpll; ichars++) {
+ if (!iswpunct(t[ichars]) || wcwidth(t[ichars]) != 0)
+ break;
+ }
+ } else
+ ichars = 1;
if (width > rpms.sen - rpms.s || width == 0) {
/*
* The screen width is too small to fit even one
* occurrence.
- *
- * HERE highlight
*/
rpms.s->chr = ZWC('?');
rpms.s->atr = special_atr_on | special_atr_off |
@@ -1089,12 +1267,22 @@
rpms.s++;
} else {
/* We can fit it without reaching the end of the line. */
- rpms.s->chr = *t;
/*
* As we don't actually output the WEOF, we attach
* any off attributes to the character itself.
*/
rpms.s->atr = base_atr_on | base_atr_off;
+ if (ichars > 1) {
+ /*
+ * Glyph includes combining characters.
+ * Write these into the multiword buffer and put
+ * the index into the value at the screen location.
+ */
+ addmultiword(rpms.s, t, ichars);
+ } else {
+ /* Single wide character */
+ rpms.s->chr = *t;
+ }
rpms.s++;
while (--width > 0) {
rpms.s->chr = WEOF;
@@ -1103,6 +1291,11 @@
rpms.s++;
}
}
+ if (ichars > 1) {
+ /* allow for normal increment */
+ tmppos += ichars - 1;
+ t += ichars - 1;
+ }
}
#endif
else if (ZC_icntrl(*t)
@@ -1110,7 +1303,6 @@
&& (unsigned)*t <= 0xffU
#endif
) { /* other control character */
- /* HERE highlight */
rpms.s->chr = ZWC('^');
rpms.s->atr = special_atr_on | base_atr_on;
rpms.s++;
@@ -1131,8 +1323,6 @@
/*
* Not printable or zero width.
* Resort to hackery.
- *
- * HERE: highlight
*/
char dispchars[11];
char *dispptr = dispchars;
@@ -1214,7 +1404,6 @@
snextline(&rpms);
}
if (width > rpms.sen - rpms.s) {
- /* HERE: highlight */
rpms.s->chr = ZWC('?');
rpms.s->atr = special_atr_on | special_atr_off;
rpms.s++;
@@ -1232,7 +1421,6 @@
else
#endif
if (ZC_icntrl(*u)) { /* simplified processing in the status line */
- /* HERE: highlight */
rpms.s->chr = ZWC('^');
rpms.s->atr = special_atr_on;
rpms.s++;
@@ -1375,10 +1563,10 @@
if (!clearf && iln > 0 && iln < olnct - 1 &&
!(hasam && vcs == winw) &&
nbuf[iln] && obuf[iln] &&
- ZR_strncmp(nbuf[iln], obuf[iln], 16)) {
+ ZR_strncmp(obuf[iln], nbuf[iln], 16)) {
if (tccan(TCDELLINE) && obuf[iln + 1] &&
obuf[iln + 1][0].chr && nbuf[iln] &&
- !ZR_strncmp(nbuf[iln], obuf[iln + 1], 16)) {
+ !ZR_strncmp(obuf[iln + 1], nbuf[iln], 16)) {
moveto(iln, 0);
tcout(TCDELLINE);
zfree(obuf[iln], (winw + 2) * sizeof(**obuf));
@@ -1391,8 +1579,7 @@
go off the end of the screen. */
else if (tccan(TCINSLINE) && olnct < vmaxln && nbuf[iln + 1] &&
- obuf[iln] && !ZR_strncmp(nbuf[iln + 1],
- obuf[iln], 16)) {
+ obuf[iln] && !ZR_strncmp(obuf[iln], nbuf[iln + 1], 16)) {
moveto(iln, 0);
tcout(TCINSLINE);
for (t0 = olnct; t0 != iln; t0--)
@@ -1454,9 +1641,8 @@
moveto(rpms.nvln, rpms.nvcs);
/* swap old and new buffers - better than freeing/allocating every time */
- qbuf = nbuf;
- nbuf = obuf;
- obuf = qbuf;
+ bufswap();
+
/* store current values so we can use them next time */
ovln = rpms.nvln;
olnct = nlnct;
@@ -1496,13 +1682,17 @@
#define tc_upcurs(X) (void) tcmultout(TCUP, TCMULTUP, (X))
#define tc_leftcurs(X) (void) tcmultout(TCLEFT, TCMULTLEFT, (X))
+/*
+ * Once again, in the multibyte case the arguments must be in the
+ * order: element of old video array, element of new video array.
+ */
static int
-wpfxlen(const REFRESH_ELEMENT *s, const REFRESH_ELEMENT *t)
+wpfxlen(const REFRESH_ELEMENT *olds, const REFRESH_ELEMENT *news)
{
int i = 0;
- while (s->chr && ZR_equal(*s, *t))
- s++, t++, i++;
+ while (olds->chr && ZR_equal(*olds, *news))
+ olds++, news++, i++;
return i;
}
@@ -1579,13 +1769,12 @@
else {
col_cleareol = -1;
if (tccan(TCCLEAREOL) && (nllen == winw || put_rpmpt != oput_rpmpt)) {
- /* HERE: watch for change of attributes */
- for (i = nllen; i && ZR_equal(nl[i - 1], zr_sp); i--)
+ for (i = nllen; i && ZR_equal(zr_sp, nl[i - 1]); i--)
;
for (j = ollen; j && ZR_equal(ol[j - 1], zr_sp); j--)
;
if ((j > i + tclen[TCCLEAREOL]) /* new buf has enough spaces */
- || (nllen == winw && ZR_equal(nl[winw - 1], zr_sp)))
+ || (nllen == winw && ZR_equal(zr_sp, nl[winw - 1])))
col_cleareol = i;
}
}
@@ -1641,13 +1830,13 @@
#ifdef MULTIBYTE_SUPPORT
if ((!nl->chr || nl->chr != WEOF) && (!ol->chr || ol->chr != WEOF)) {
#endif
- if (nl->chr && ol->chr && ZR_equal(nl[1], ol[1])) {
+ if (nl->chr && ol->chr && ZR_equal(ol[1], nl[1])) {
/* skip only if second chars match */
#ifdef MULTIBYTE_SUPPORT
int ccs_was = ccs;
#endif
/* skip past all matching characters */
- for (; nl->chr && ZR_equal(*nl, *ol); nl++, ol++, ccs++)
+ for (; nl->chr && ZR_equal(*ol, *nl); nl++, ol++, ccs++)
;
#ifdef MULTIBYTE_SUPPORT
/* Make sure ol and nl are pointing to real characters */
@@ -1723,7 +1912,7 @@
#ifdef MULTIBYTE_SUPPORT
&& ol->chr != WEOF && nl->chr != WEOF
#endif
- && nl[1].chr && ol[1].chr && !ZR_equal(nl[1], ol[1])) {
+ && nl[1].chr && ol[1].chr && !ZR_equal(ol[1], nl[1])) {
/* deleting characters - see if we can find a match series that
makes it cheaper to delete intermediate characters
@@ -1762,7 +1951,7 @@
if (tccan(TCINS) && (vln != lines - 1)) { /* not on last line */
for (i = 1; nl[i].chr; i++)
- if (tcinscost(i) < wpfxlen(nl + i, ol)) {
+ if (tcinscost(i) < wpfxlen(ol, nl + i)) {
tc_inschars(i);
zwrite(nl, i);
nl += i;
@@ -2055,25 +2244,45 @@
singlerefresh(ZLE_STRING_T tmpline, int tmpll, int tmpcs)
{
REFRESH_STRING vbuf, vp, /* video buffer and pointer */
- *qbuf, /* tmp */
refreshop; /* pointer to old video buffer */
int t0, /* tmp */
vsiz, /* size of new video buffer */
nvcs = 0, /* new video cursor column */
owinpos = winpos, /* previous window position */
owinprompt = winprompt; /* previous winprompt */
+#ifdef MULTIBYTE_SUPPORT
+ int width; /* width of multibyte character */
+#endif
nlnct = 1;
/* generate the new line buffer completely */
- for (vsiz = 1 + lpromptw, t0 = 0; t0 != tmpll; t0++, vsiz++)
+ for (vsiz = 1 + lpromptw, t0 = 0; t0 != tmpll; t0++) {
if (tmpline[t0] == ZWC('\t'))
- vsiz = (vsiz | 7) + 1;
+ vsiz = (vsiz | 7) + 2;
+#ifdef MULTIBYTE_SUPPORT
+ else if (iswprint(tmpline[t0]) && (width = wcwidth(tmpline[t0]) > 0)) {
+ vsiz += width;
+ if (isset(COMBININGCHARS) && iswalnum(tmpline[t0])) {
+ while (t0 < tmpll-1 && iswpunct(tmpline[t0+1]) &&
+ wcwidth(tmpline[t0+1]) == 0)
+ t0++;
+ }
+ }
+#endif
+ else if (ZC_icntrl(tmpline[t0])
#ifdef MULTIBYTE_SUPPORT
- else if (iswprint(tmpline[t0]))
- vsiz += wcwidth(tmpline[t0]);
+ && (unsigned)tmpline[t0] <= 0xffU
#endif
- else if (ZC_icntrl(tmpline[t0]))
+ )
+ vsiz += 2;
+#ifdef MULTIBYTE_SUPPORT
+ else
+ vsiz += 10;
+#else
+ else
vsiz++;
+#endif
+ }
vbuf = (REFRESH_STRING)zalloc(vsiz * sizeof(*vbuf));
if (tmpcs < 0) {
@@ -2094,9 +2303,6 @@
struct region_highlight *rhp;
/*
* Calculate attribute based on region.
- * HERE: we may need to be smarter about turning
- * attributes off if bailing out before the end of the
- * region.
*/
for (ireg = 0, rhp = region_highlights;
ireg < n_region_highlights;
@@ -2122,7 +2328,6 @@
*vp++ = zr_sp;
vp[-1].atr |= base_atr_off;
} else if (tmpline[t0] == ZWC('\n')) {
- /* HERE highlight */
vp->chr = ZWC('\\');
vp->atr = special_atr_on | base_atr_on;
vp++;
@@ -2131,21 +2336,39 @@
base_atr_on | base_atr_off;
vp++;
#ifdef MULTIBYTE_SUPPORT
- } else if (iswprint(tmpline[t0])) {
- int width;
- vp->chr = tmpline[t0];
- vp->atr = base_atr_on;
+ } else if (iswprint(tmpline[t0]) &&
+ (width = wcwidth(tmpline[t0])) > 0) {
+ int ichars;
+ if (isset(COMBININGCHARS) && iswalnum(tmpline[t0])) {
+ /*
+ * Look for combining characters: trailing printable
+ * characters with printing width zero.
+ */
+ for (ichars = 1; t0 + ichars < tmpll; ichars++) {
+ if (!iswpunct(tmpline[t0+ichars]) ||
+ wcwidth(tmpline[t0+ichars]) != 0)
+ break;
+ }
+ } else
+ ichars = 1;
+ vp->atr = base_atr_on | base_atr_off;
+ if (ichars > 1)
+ addmultiword(vp, tmpline+t0, ichars);
+ else
+ vp->chr = tmpline[t0];
vp++;
- width = wcwidth(tmpline[t0]);
while (--width > 0) {
vp->chr = WEOF;
- vp->atr = base_atr_on;
+ vp->atr = base_atr_on | base_atr_off;
vp++;
}
- vp[-1].atr |= base_atr_off;
+ t0 += ichars - 1;
#endif
- } else if (ZC_icntrl(tmpline[t0])) {
- /* HERE: highlight */
+ } else if (ZC_icntrl(tmpline[t0])
+#ifdef MULTIBYTE_SUPPORT
+ && (unsigned)tmpline[t0] <= 0xffU
+#endif
+ ) {
ZLE_INT_T t = tmpline[++t0];
vp->chr = ZWC('^');
@@ -2156,11 +2379,39 @@
vp->atr = special_atr_on | special_atr_off | base_atr_on |
base_atr_off;
vp++;
- } else {
+ }
+#ifdef MULTIBYTE_SUPPORT
+ else {
+ char dispchars[11];
+ char *dispptr = dispchars;
+ wchar_t wc;
+ int started = 0;
+
+ if ((unsigned)tmpline[t0] > 0xffffU) {
+ sprintf(dispchars, "<%.08x>", (unsigned)tmpline[t0]);
+ } else {
+ sprintf(dispchars, "<%.04x>", (unsigned)tmpline[t0]);
+ }
+ while (*dispptr) {
+ if (mbtowc(&wc, dispptr, 1) == 1 /* paranoia */) {
+ vp->chr = wc;
+ if (!started)
+ started = 1;
+ vp->atr = special_atr_on | base_atr_on;
+ vp++;
+ }
+ dispptr++;
+ }
+ if (started)
+ vp[-1].atr |= special_atr_off | base_atr_off;
+ }
+#else
+ else {
vp->chr = tmpline[t0];
vp->atr = base_atr_on | base_atr_off;
vp++;
}
+#endif
if (t0 == tmpcs)
nvcs = vp - vbuf - 1;
}
@@ -2252,7 +2503,7 @@
* nastiness may be around.
*/
if (vp - *nbuf >= owinprompt)
- for (; vp->chr && ZR_equal(*vp, *refreshop);
+ for (; vp->chr && ZR_equal(*refreshop, *vp);
t0++, vp++, refreshop++)
;
@@ -2282,9 +2533,7 @@
/* move to the new cursor position */
singmoveto(nvcs);
- qbuf = nbuf;
- nbuf = obuf;
- obuf = qbuf;
+ bufswap();
}
/**/
Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.90
diff -u -r1.90 zle_tricky.c
--- Src/Zle/zle_tricky.c 9 Feb 2008 17:32:45 -0000 1.90
+++ Src/Zle/zle_tricky.c 13 Apr 2008 16:22:23 -0000
@@ -685,7 +685,7 @@
}
ocs = zlemetacs;
zlemetacs = 0;
- foredel(chl);
+ foredel(chl, CUT_RAW);
zlemetacs = ocs;
}
freeheap();
@@ -796,7 +796,7 @@
if (inull(*q))
*q = Nularg;
zlemetacs = wb;
- foredel(we - wb);
+ foredel(we - wb, CUT_RAW);
untokenize(x = ox = dupstring(w));
if (*w == Tilde || *w == Equals || *w == String)
@@ -848,7 +848,7 @@
* parts of the code re-install them, but for expansion
* we have to do it here. */
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(origll);
memcpy(zlemetaline, origline, origll);
zlemetacs = origcs;
@@ -1669,7 +1669,7 @@
if (i < ocs)
offs -= skipchars;
/* Move the tail of the line up */
- foredel(skipchars);
+ foredel(skipchars, CUT_RAW);
/*
* Update the offset into the command line to the
* cursor position if that's after the current position.
@@ -1724,7 +1724,7 @@
} else {
ocs = zlemetacs;
zlemetacs = i;
- foredel(skipchars);
+ foredel(skipchars, CUT_RAW);
if ((zlemetacs = ocs) > (i -= skipchars))
zlemetacs -= skipchars;
we -= skipchars;
@@ -1732,7 +1732,7 @@
} else {
ocs = zlemetacs;
zlemetacs = we;
- backdel(skipchars);
+ backdel(skipchars, CUT_RAW);
if (ocs == we)
zlemetacs = we - skipchars;
else
@@ -2085,7 +2085,7 @@
/* Only the list of expansions was requested. Restore the
* command line. */
zlemetacs = 0;
- foredel(zlemetall);
+ foredel(zlemetall, CUT_RAW);
spaceinline(origll);
memcpy(zlemetaline, origline, origll);
zlemetacs = origcs;
@@ -2095,7 +2095,7 @@
}
/* Remove the current word and put the expansions there. */
zlemetacs = wb;
- foredel(we - wb);
+ foredel(we - wb, CUT_RAW);
while ((ss = (char *)ugetnode(vl))) {
ret = 0;
ss = quotename(ss, NULL);
@@ -2811,7 +2811,7 @@
if (!str)
return 1;
zlecs = cmdwb;
- foredel(cmdwe - cmdwb);
+ foredel(cmdwe - cmdwb, CUT_RAW);
zlestr = stringaszleline(str, 0, &strll, NULL, NULL);
spaceinline(strll);
ZS_strncpy(zleline + zlecs, zlestr, strll);
Index: Src/Zle/zle_utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_utils.c,v
retrieving revision 1.46
diff -u -r1.46 zle_utils.c
--- Src/Zle/zle_utils.c 11 Apr 2008 11:38:10 -0000 1.46
+++ Src/Zle/zle_utils.c 13 Apr 2008 16:22:23 -0000
@@ -357,6 +357,17 @@
}
+/*
+ * Basic utility functions for adding to line or removing from line.
+ * At this level the counts supplied are raw character counts, so
+ * the calling code must be aware of combining characters where
+ * necessary, e.g. if we want to delete a + combing grave forward
+ * from the cursor, then shiftchars() gets the count 2 (not 1).
+ *
+ * This is necessary because these utility functions don't know about
+ * zlecs, and we need to count combined characters from there.
+ */
+
/* insert space for ct chars at cursor position */
/**/
@@ -412,27 +423,6 @@
region_active = 0;
}
-/**/
-mod_export void
-backkill(int ct, int dir)
-{
- int i = (zlecs -= ct);
-
- cut(i, ct, dir);
- shiftchars(i, ct);
-}
-
-/**/
-mod_export void
-forekill(int ct, int dir)
-{
- int i = zlecs;
-
- cut(i, ct, dir);
- shiftchars(i, ct);
-}
-
-
/*
* Put the ct characters starting at zleline + i into the
* cutbuffer, circling the kill ring if necessary (it's
@@ -448,9 +438,9 @@
/**/
void
-cut(int i, int ct, int dir)
+cut(int i, int ct, int flags)
{
- cuttext(zleline + i, ct, dir);
+ cuttext(zleline + i, ct, flags);
}
/*
@@ -459,7 +449,7 @@
/**/
void
-cuttext(ZLE_STRING_T line, int ct, int dir)
+cuttext(ZLE_STRING_T line, int ct, int flags)
{
if (!ct)
return;
@@ -504,7 +494,7 @@
cutbuf.buf = (ZLE_STRING_T)zalloc(ZLE_CHAR_SIZE);
cutbuf.buf[0] = ZWC('\0');
cutbuf.len = cutbuf.flags = 0;
- } else if (!(lastcmd & ZLE_KILL) || dir < 0) {
+ } else if (!(lastcmd & ZLE_KILL) || (flags & CUT_RAW)) {
Cutbuffer kptr;
if (!kring) {
kringsize = KRINGCTDEF;
@@ -519,7 +509,7 @@
cutbuf.buf[0] = ZWC('\0');
cutbuf.len = cutbuf.flags = 0;
}
- if (dir) {
+ if (flags & (CUT_FRONT|CUT_REPLACE)) {
ZLE_STRING_T s = (ZLE_STRING_T)zalloc((cutbuf.len + ct)*ZLE_CHAR_SIZE);
ZS_memcpy(s, line, ct);
@@ -539,24 +529,87 @@
cutbuf.flags &= ~CUTBUFFER_LINE;
}
+/*
+ * Now we're back in the world of zlecs where we need to keep
+ * track of whether we're on a combining character.
+ */
+
/**/
mod_export void
-backdel(int ct)
+backkill(int ct, int flags)
{
- if (zlemetaline != NULL)
- shiftchars(zlemetacs -= ct, ct);
- else
- shiftchars(zlecs -= ct, ct);
+ int i;
+
+ if (flags & CUT_RAW) {
+ i = (zlecs -= ct);
+ } else {
+ int n = ct;
+ while (n--)
+ DECCS();
+ i = zlecs;
+ }
+
+ cut(i, ct, flags);
+ shiftchars(i, ct);
}
/**/
mod_export void
-foredel(int ct)
+forekill(int ct, int flags)
{
- if (zlemetaline != NULL)
- shiftchars(zlemetacs, ct);
- else
+ int i = zlecs;
+
+ if (!(flags & CUT_RAW)) {
+ int n = ct;
+ while (n--)
+ INCCS();
+ ct = zlecs - i;
+ zlecs = i;
+ }
+
+ cut(i, ct, flags);
+ shiftchars(i, ct);
+}
+
+/**/
+mod_export void
+backdel(int ct, int flags)
+{
+ if (flags & CUT_RAW) {
+ if (zlemetaline != NULL) {
+ shiftchars(zlemetacs -= ct, ct);
+ } else {
+ shiftchars(zlecs -= ct, ct);
+ }
+ } else {
+ int n = ct, origcs = zlecs;
+ DPUTS(zlemetaline != NULL, "backdel needs CUT_RAW when metafied");
+ while (n--)
+ DECCS();
+ shiftchars(zlecs, origcs - zlecs);
+ }
+}
+
+/**/
+mod_export void
+foredel(int ct, int flags)
+{
+ if (flags & CUT_RAW) {
+ if (zlemetaline != NULL) {
+ shiftchars(zlemetacs, ct);
+ } else if (flags & CUT_RAW) {
+ shiftchars(zlecs, ct);
+ }
+ } else {
+ int origcs = zlecs;
+ int n = ct;
+ DPUTS(zlemetaline != NULL, "backdel needs CUT_RAW when metafied");
+ while (n--)
+ INCCS();
+ ct = zlecs - origcs;
+ zlecs = origcs;
shiftchars(zlecs, ct);
+ }
}
/**/
@@ -578,7 +631,7 @@
zleline = stringaszleline(scp, 0, &zlell, &linesz, NULL);
if ((flags & ZSL_TOEND) && (zlecs = zlell) && invicmdmode())
- zlecs--;
+ DECCS();
else if (zlecs > zlell)
zlecs = zlell;
@@ -1035,7 +1088,7 @@
}
zlecs = ch->off;
if(ch->ins)
- foredel(ch->insl);
+ foredel(ch->insl, CUT_RAW);
if(ch->del) {
spaceinline(ch->dell);
ZS_memcpy(zleline + zlecs, ch->del, ch->dell);
@@ -1075,7 +1128,7 @@
}
zlecs = ch->off;
if(ch->del)
- foredel(ch->dell);
+ foredel(ch->dell, CUT_RAW);
if(ch->ins) {
spaceinline(ch->insl);
ZS_memcpy(zleline + zlecs, ch->ins, ch->insl);
Index: Src/Zle/zle_vi.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_vi.c,v
retrieving revision 1.15
diff -u -r1.15 zle_vi.c
--- Src/Zle/zle_vi.c 3 Apr 2008 15:20:31 -0000 1.15
+++ Src/Zle/zle_vi.c 13 Apr 2008 16:22:23 -0000
@@ -210,7 +210,13 @@
* moving to the opening bracket, meaning that we need to *
* change the *starting* position. */
if(virangeflag == -1)
- pos++;
+ {
+ int origcs = zlecs;
+ zlecs = pos;
+ INCCS();
+ pos = zlecs;
+ zlecs = origcs;
+ }
/* Get the range the right way round. zlecs is placed at the *
* start of the range, and pos (the return value of this *
@@ -314,12 +320,12 @@
startvichange(1);
if ((c2 = getvirange(0)) != -1) {
- forekill(c2 - zlecs, 0);
+ forekill(c2 - zlecs, CUT_RAW);
ret = 0;
if (vilinerange && zlell) {
if (zlecs == zlell)
- zlecs--;
- foredel(1);
+ DECCS();
+ foredel(1, 0);
vifirstnonblank(zlenoargs);
}
}
@@ -350,7 +356,7 @@
if (n > findeol() - zlecs)
n = findeol() - zlecs;
/* do the deletion */
- forekill(n, 0);
+ forekill(n, CUT_RAW);
return 0;
}
@@ -363,7 +369,7 @@
startvichange(1);
if ((c2 = getvirange(1)) != -1) {
ret = 0;
- forekill(c2 - zlecs, 0);
+ forekill(c2 - zlecs, CUT_RAW);
selectkeymap("main", 1);
viinsbegin = zlecs;
undoing = 0;
@@ -388,7 +394,7 @@
if (n > findeol() - zlecs)
n = findeol() - zlecs;
/* do the substitution */
- forekill(n, 0);
+ forekill(n, CUT_RAW);
startvitext(1);
return 0;
}
@@ -397,7 +403,7 @@
int
vichangeeol(UNUSED(char **args))
{
- forekill(findeol() - zlecs, 0);
+ forekill(findeol() - zlecs, CUT_RAW);
startvitext(1);
return 0;
}
@@ -488,11 +494,22 @@
vireplacechars(UNUSED(char **args))
{
ZLE_INT_T ch;
- int n = zmult;
+ int n = zmult, origcs = zlecs, fail = 0;
+ if (n > 0) {
+ while (n > 0) {
+ if (zlecs == zlell || zleline[zlell] == ZWC('\n')) {
+ fail = 1;
+ break;
+ }
+ INCCS();
+ }
+ n = zlecs - origcs;
+ zlecs = origcs;
+ }
startvichange(1);
/* check argument range */
- if (n < 1 || n + zlecs > findeol()) {
+ if (n < 1 || fail) {
if(vichgrepeat)
vigetkey();
if(vichgflag) {
@@ -511,9 +528,10 @@
if (ch == ZWC('\r') || ch == ZWC('\n')) {
/* <return> handled specially */
zlecs += n - 1;
- backkill(n - 1, 0);
+ backkill(n - 1, CUT_RAW);
zleline[zlecs++] = '\n';
} else {
+ /* HERE: we shouldn't replace combining chars, we should delete them */
while (n--)
zleline[zlecs++] = ch;
zlecs--;
@@ -531,7 +549,7 @@
undoing = 1;
vichgflag = 0;
if (zlecs != findbol())
- zlecs--;
+ DECCS();
return 0;
}
@@ -575,7 +593,7 @@
zleline[zlecs] = ZC_toupper(zleline[zlecs]);
else if (ZC_iupper(zleline[zlecs]))
zleline[zlecs] = ZC_tolower(zleline[zlecs]);
- zlecs++;
+ INCCS();
}
/* go back to the first line of the range */
zlecs = oldcs;
@@ -664,7 +682,7 @@
/* remove a tab from the beginning of each line within range */
while (zlecs < c2) {
if (zleline[zlecs] == '\t')
- foredel(1);
+ foredel(1, 0);
zlecs = findeol() + 1;
}
/* go back to the first line of the range */
@@ -699,7 +717,7 @@
if (n > zlecs - findbol())
n = zlecs - findbol();
/* do the deletion */
- backkill(n, 1);
+ backkill(n, CUT_FRONT|CUT_RAW);
return 0;
}
@@ -709,7 +727,7 @@
{
if (viinsbegin > zlecs)
return 1;
- backdel(zlecs - viinsbegin);
+ backdel(zlecs - viinsbegin, CUT_RAW);
return 0;
}
@@ -790,7 +808,7 @@
return 1;
zlecs = x + 1;
for (x = 1; zlecs != zlell && ZC_iblank(zleline[zlecs]); zlecs++, x++);
- backdel(x);
+ backdel(x, CUT_RAW);
if (zlecs && ZC_iblank(zleline[zlecs-1]))
zlecs--;
else {
@@ -893,7 +911,7 @@
viinsbegin++;
zlecs = oldcs + (zlecs <= oldcs);
} else {
- foredel(1);
+ foredel(1, 0);
if (zlecs < viinsbegin)
viinsbegin--;
zlecs = oldcs - (zlecs < oldcs);
@@ -921,7 +939,7 @@
#ifndef HAS_TIO
zsetterm();
#endif
- foredel(1);
+ foredel(1, 0);
if(LASTFULLCHAR == ZLEEOF)
return 1;
else
Index: Src/Zle/zle_word.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_word.c,v
retrieving revision 1.9
diff -u -r1.9 zle_word.c
--- Src/Zle/zle_word.c 20 Mar 2006 14:40:43 -0000 1.9
+++ Src/Zle/zle_word.c 13 Apr 2008 16:22:23 -0000
@@ -30,6 +30,12 @@
#include "zle.mdh"
#include "zle_word.pro"
+/*
+ * HERE: our handling of combining characters may be wrong. We
+ * should make sure we only consider a combining character part of
+ * a word if the base character is.
+ */
+
/**/
int
forwardword(char **args)
@@ -45,11 +51,11 @@
}
while (n--) {
while (zlecs != zlell && ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
if (wordflag && !n)
return 0;
while (zlecs != zlell && !ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
}
return 0;
}
@@ -72,14 +78,14 @@
while (n--) {
if (Z_vialnum(zleline[zlecs]))
while (zlecs != zlell && Z_vialnum(zleline[zlecs]))
- zlecs++;
+ INCCS();
else
while (zlecs != zlell && !Z_vialnum(zleline[zlecs]) && !ZC_iblank(zleline[zlecs]))
- zlecs++;
+ INCCS();
if (wordflag && !n)
return 0;
while (zlecs != zlell && ZC_inblank(zleline[zlecs]))
- zlecs++;
+ INCCS();
}
return 0;
}
@@ -99,11 +105,11 @@
}
while (n--) {
while (zlecs != zlell && !ZC_iblank(zleline[zlecs]))
- zlecs++;
+ INCCS();
if (wordflag && !n)
return 0;
while (zlecs != zlell && ZC_iblank(zleline[zlecs]))
- zlecs++;
+ INCCS();
}
return 0;
}
@@ -123,11 +129,11 @@
}
while (n--) {
while (zlecs != zlell && !ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
if (wordflag && !n)
return 0;
while (zlecs != zlell && ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
}
return 0;
}
@@ -141,13 +147,14 @@
if (n < 0)
return 1;
while (n--) {
+ /* HERE: the zlecs + 1 here is suspect */
while (zlecs != zlell && ZC_iblank(zleline[zlecs + 1]))
- zlecs++;
+ INCCS();
while (zlecs != zlell && !ZC_iblank(zleline[zlecs + 1]))
- zlecs++;
+ INCCS();
}
if (zlecs != zlell && virangeflag)
- zlecs++;
+ INCCS();
return 0;
}
@@ -165,18 +172,19 @@
return ret;
}
while (n--) {
+ /* HERE: the zlecs + 1 here is suspect */
if (ZC_iblank(zleline[zlecs + 1]))
while (zlecs != zlell && ZC_iblank(zleline[zlecs + 1]))
- zlecs++;
+ INCCS();
if (Z_vialnum(zleline[zlecs + 1]))
while (zlecs != zlell && Z_vialnum(zleline[zlecs + 1]))
- zlecs++;
+ INCCS();
else
while (zlecs != zlell && !Z_vialnum(zleline[zlecs + 1]) && !ZC_iblank(zleline[zlecs + 1]))
- zlecs++;
+ INCCS();
}
if (zlecs != zlell && virangeflag)
- zlecs++;
+ INCCS();
return 0;
}
@@ -194,10 +202,11 @@
return ret;
}
while (n--) {
+ /* HERE: the zlecs - 1 here is suspect */
while (zlecs && !ZC_iword(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
while (zlecs && ZC_iword(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
}
return 0;
}
@@ -216,14 +225,15 @@
return ret;
}
while (n--) {
+ /* HERE: the zlecs - 1 here is suspect */
while (zlecs && ZC_iblank(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
if (Z_vialnum(zleline[zlecs - 1]))
while (zlecs && Z_vialnum(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
else
while (zlecs && !Z_vialnum(zleline[zlecs - 1]) && !ZC_iblank(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
}
return 0;
}
@@ -243,9 +253,9 @@
}
while (n--) {
while (zlecs && ZC_iblank(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
while (zlecs && !ZC_iblank(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
}
return 0;
}
@@ -265,9 +275,9 @@
}
while (n--) {
while (zlecs && !ZC_iword(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
while (zlecs && ZC_iword(zleline[zlecs - 1]))
- zlecs--;
+ DECCS();
}
return 0;
}
@@ -286,12 +296,16 @@
return ret;
}
while (n--) {
+ /*
+ * HERE: the zlecs - 1 here is suspect, and we should
+ * do the DECCS() thing.
+ */
while (x && !ZC_iword(zleline[x - 1]))
x--;
while (x && ZC_iword(zleline[x - 1]))
x--;
}
- backdel(zlecs - x);
+ backdel(zlecs - x, CUT_RAW);
return 0;
}
@@ -306,6 +320,10 @@
return 1;
/* this taken from "vibackwardword" */
while (n--) {
+ /*
+ * HERE: the zlecs - 1 here is suspect, and we should
+ * do the DECCS() thing.
+ */
while ((x > lim) && ZC_iblank(zleline[x - 1]))
x--;
if (Z_vialnum(zleline[x - 1]))
@@ -315,7 +333,7 @@
while ((x > lim) && !Z_vialnum(zleline[x - 1]) && !ZC_iblank(zleline[x - 1]))
x--;
}
- backkill(zlecs - x, 1);
+ backkill(zlecs - x, CUT_FRONT);
return 0;
}
@@ -334,12 +352,16 @@
return ret;
}
while (n--) {
+ /*
+ * HERE: the zlecs - 1 here is suspect, and we should
+ * do the DECCS() thing.
+ */
while (x && !ZC_iword(zleline[x - 1]))
x--;
while (x && ZC_iword(zleline[x - 1]))
x--;
}
- backkill(zlecs - x, 1);
+ backkill(zlecs - x, CUT_FRONT);
return 0;
}
@@ -354,10 +376,10 @@
n = -n;
while (n--) {
while (zlecs != zlell && !ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
while (zlecs != zlell && ZC_iword(zleline[zlecs])) {
zleline[zlecs] = ZC_toupper(zleline[zlecs]);
- zlecs++;
+ INCCS();
}
}
if (neg)
@@ -376,10 +398,10 @@
n = -n;
while (n--) {
while (zlecs != zlell && !ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
while (zlecs != zlell && ZC_iword(zleline[zlecs])) {
zleline[zlecs] = ZC_tolower(zleline[zlecs]);
- zlecs++;
+ INCCS();
}
}
if (neg)
@@ -399,14 +421,14 @@
while (n--) {
first = 1;
while (zlecs != zlell && !ZC_iword(zleline[zlecs]))
- zlecs++;
+ INCCS();
while (zlecs != zlell && ZC_iword(zleline[zlecs]) && !ZC_ialpha(zleline[zlecs]))
- zlecs++;
+ INCCS();
while (zlecs != zlell && ZC_iword(zleline[zlecs])) {
zleline[zlecs] = (first) ? ZC_toupper(zleline[zlecs]) :
ZC_tolower(zleline[zlecs]);
first = 0;
- zlecs++;
+ INCCS();
}
}
if (neg)
@@ -429,12 +451,13 @@
return ret;
}
while (n--) {
+ /* HERE: we should do the INCCS() thing */
while (x != zlell && !ZC_iword(zleline[x]))
x++;
while (x != zlell && ZC_iword(zleline[x]))
x++;
}
- foredel(x - zlecs);
+ foredel(x - zlecs, CUT_RAW);
return 0;
}
@@ -453,12 +476,13 @@
return ret;
}
while (n--) {
+ /* HERE: we should do the INCCS() thing */
while (x != zlell && !ZC_iword(zleline[x]))
x++;
while (x != zlell && ZC_iword(zleline[x]))
x++;
}
- forekill(x - zlecs, 0);
+ forekill(x - zlecs, CUT_RAW);
return 0;
}
@@ -474,6 +498,10 @@
if (neg)
n = -n;
while (n--) {
+ /*
+ * HERE: we should do the INCCS() thing.
+ * A great deal of the following needs rewriting.
+ */
while (x != zlell && zleline[x] != ZWC('\n') && !ZC_iword(zleline[x]))
x++;
if (x == zlell || zleline[x] == ZWC('\n')) {
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-13 16:54 PATCH: (large) initial support for combining characters in ZLE Peter Stephenson
@ 2008-04-13 17:32 ` Bart Schaefer
2008-04-14 9:02 ` Peter Stephenson
2008-04-14 12:00 ` Peter Stephenson
` (2 subsequent siblings)
3 siblings, 1 reply; 102+ messages in thread
From: Bart Schaefer @ 2008-04-13 17:32 UTC (permalink / raw)
To: Zsh Hackers' List
On Apr 13, 5:54pm, Peter Stephenson wrote:
}
} To get combining characters to do anything useful, you will need to
} set the option COMBINING_CHARS. There's no "ZLE" there because it
} already has a minimal effect on the main shell in the [[:WORD:]]
} test: arguably zero-width punctuation characters should be considered
} part of the word regardless of the option, i.e. even if displayed
} specially, and I might change this.
Pardon if I'm just not thinking this through far enough, but what if
any affect does this have on completion of strings that contain zero-
width characters?
Mainly I'm wondering whether NO_combiningchars needs to be added to
_comp_options, or something of that ilk.
} - As always, completion is so complicated it could do pretty much
} anything
Well, yeah. So perhaps I should refine my question: What effect is
this eventually intended to have on completion when all of the details
have been worked out? (Something which may, conceivably, never come
entirely to pass, I admit).
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-13 17:32 ` Bart Schaefer
@ 2008-04-14 9:02 ` Peter Stephenson
0 siblings, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-04-14 9:02 UTC (permalink / raw)
To: Zsh Hackers' List
Bart Schaefer wrote:
> On Apr 13, 5:54pm, Peter Stephenson wrote:
> } To get combining characters to do anything useful, you will need to
> } set the option COMBINING_CHARS. There's no "ZLE" there because it
> } already has a minimal effect on the main shell in the [[:WORD:]]
> } test: arguably zero-width punctuation characters should be considered
> } part of the word regardless of the option, i.e. even if displayed
> } specially, and I might change this.
>
> Pardon if I'm just not thinking this through far enough, but what if
> any affect does this have on completion of strings that contain zero-
> width characters?
>
> Mainly I'm wondering whether NO_combiningchars needs to be added to
> _comp_options, or something of that ilk.
It shouldn't have a major effect. The completion system inserts
multibyte strings that get converted to wide characters later on; when
this is redisplayed it goes into the same logic in zrefresh() that is
already basically working.
The only issue I can see is if the cursor ends up on a combining
character at that point; for example, when completing characters that
only differ by the subsequent combining characters. Actually, I can't
think of a case where that can happen. Either we insert the unique part
to the left of the cursor, or we insert one of the choices completely.
As long as we don't put anything under the cursor, things will work.
There's the slightly odd effect that characters differing by combining
characters will appear treated as if the base character matches: that's
a bug, but no worse than the current situation.
Hence the only effects I'm expecting are minor edge effects which can
probably be fixed up with a suitably placed alignmultiwordright().
I wonder, in fact, if it might be worth putting that in
unmetafy_line(), as below...
In any case I'd like to keep the option turned on and see what happens,
since I'm not expecting anything catastrophic and I need to know if
there is.
> } - As always, completion is so complicated it could do pretty much
> } anything
>
> Well, yeah. So perhaps I should refine my question: What effect is
> this eventually intended to have on completion when all of the details
> have been worked out? (Something which may, conceivably, never come
> entirely to pass, I admit).
I don't think it should have any, for the reasons given above. Anything
that looks different, other than having the characters displayed
more correctly, is probably a bug.
For example, I've made no attempt to do anything clever when displaying
completion lists. They need to be fixed up along the same lines as the
command line.
Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.91
diff -u -r1.91 zle_tricky.c
--- Src/Zle/zle_tricky.c 13 Apr 2008 16:58:43 -0000 1.91
+++ Src/Zle/zle_tricky.c 14 Apr 2008 08:59:11 -0000
@@ -978,6 +978,14 @@
zlemetaline[zlemetall] = '\0';
zleline = stringaszleline(zlemetaline, zlemetacs, &zlell, &linesz, &zlecs);
+#ifdef MULTIBYTE_SUPPORT
+ /*
+ * If we inserted combining characters under the cursor we
+ * won't have tested the effect yet. So fix it up now.
+ */
+ alignmultiwordright(1);
+#endif
+
free(zlemetaline);
zlemetaline = NULL;
}
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-13 16:54 PATCH: (large) initial support for combining characters in ZLE Peter Stephenson
2008-04-13 17:32 ` Bart Schaefer
@ 2008-04-14 12:00 ` Peter Stephenson
2008-04-14 13:34 ` Mikael Magnusson
2008-04-17 18:33 ` Jun T.
3 siblings, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-04-14 12:00 UTC (permalink / raw)
To: Zsh Hackers' List
Documention changes, some of them somewhat overdue.
I wonder if it's time to merge the FAQ into the main documentation as
zshfaq.1?
Index: Doc/Zsh/roadmap.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/roadmap.yo,v
retrieving revision 1.10
diff -u -r1.10 roadmap.yo
--- Doc/Zsh/roadmap.yo 1 Feb 2008 19:59:48 -0000 1.10
+++ Doc/Zsh/roadmap.yo 14 Apr 2008 11:50:10 -0000
@@ -44,6 +44,13 @@
tt(HISTSIZE) and tt(SAVEHIST) in ifzman(zmanref(zshparam))\
ifnzman(noderef(Parameters Used By The Shell)).
+The shell now supports the UTF-8 character set (and also others if
+supported by the operating system). This is (mostly) handled transparently
+by the shell, but the degree of support in terminal emulators is variable.
+There is some discussion of this in the shell FAQ,
+http://zsh.dotsrc.org/FAQ/ . Note in particular that for combining
+characters to be handled the option tt(COMBINING_CHARS) needs to be set.
+
subsect(Completion)
Completion is a feature present in many shells. It allows the user to
Index: Etc/FAQ.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Etc/FAQ.yo,v
retrieving revision 1.37
diff -u -r1.37 FAQ.yo
--- Etc/FAQ.yo 31 Mar 2008 15:03:11 -0000 1.37
+++ Etc/FAQ.yo 14 Apr 2008 11:50:14 -0000
@@ -126,11 +126,11 @@
4.5. How do I get started with programmable completion?
4.6. Suppose I want to complete all files during a special completion?
-Chapter 5: Multibyte input
+Chapter 5: Multibyte input and output
5.1. What is multibyte input?
-5.2. How does zsh handle multibyte input?
-5.3. How do I ensure multibyte input works on my system?
+5.2. How does zsh handle multibyte input and output?
+5.3. How do I ensure multibyte input and output work on my system?
5.4. How can I input characters that aren't on my keyboard?
Chapter 6: The future of zsh
@@ -1961,7 +1961,7 @@
such as expansion or approximate completion.
-chapter(Multibyte input)
+chapter(Multibyte input and output)
label(c5)
sect(What is multibyte input?)
@@ -2012,7 +2012,7 @@
in those formats.)
-sect(How does zsh handle multibyte input?)
+sect(How does zsh handle multibyte input and output?)
Until version 4.3, zsh didn't handle multibyte input properly at all.
Each octet in a multibyte character would look to the shell like a
@@ -2021,50 +2021,44 @@
cause all sorts of odd effects. (It was possible to edit in zsh using
single-byte extensions of ASCII such as the ISO 8859 family, however.)
- From version 4.3, multibyte input is handled in the line editor if zsh
- has been compiled with the appropriate definitions. This will happen
- automatically if the compiler defines __STDC_ISO_10646__, which is true
- for many recent GNU-based systems. On other systems you must configure
- zsh with the argument --enable-multibyte to configure. Explicit use of
- --enable-multibyte should work on many other recent UNIX systems; if it
- works on yours, and that's not mentioned in the shell documentation,
- please report this to zsh-workers@sunsite.dk, and if it doesn't but you
- can work out why not we'd also be interested in hearing.
-
- (The reason for the test for __STDC_ISO_10646__ is that its presence
- happens to indicate that the required library support is likely to be
- present, short-circuiting a large number of configuration tests. This
- isn't strictly guaranteed, since the definition indicates the rather more
- limited fact that the wide character representation used internally by
- the shell is Unicode. However, in practice such systems provide the
- right level of support for zsh to use. It would be better to test
- individually for the library features the shell needs; unfortunately
- there are a lot of them.)
-
- You can test if multibyte handling is compiled into your version of the
- shell by running:
- verb(
- (bindkey -m)
- )
- which should output a warning:
- verb(
- bindkey: warning: `bindkey -m' disables multibyte support
- )
- If it doesn't, you don't have multibyte support in your shell. The
- parentheses are there to run the command in a subshell, which protects
- your interactive shell from the effects being warned about.
-
- Multibyte strings are not yet handled anywhere else in the shell. This
- means, for example, patterns treat multibyte characters as a set of single
- octets and the ${#var} syntax counts octets, not characters. There will
- probably be new syntax to ensure that zsh can work both in its traditional
- way as well as when interpreting multibyte characters.
+ From version 4.3.4, multibyte input is handled in the line editor if zsh
+ has been compiled with the appropriate definitions, and is automatically
+ activated. This is indicated by the option tt(MULTIBYTE), which is
+ set by default on shells that support multibyte mode. Hence you
+ can test this with a standard option test: `tt([[ -o multibyte ]])'.
+
+ The tt(MULTIBYTE) option affects the entire shell: parameter expansion,
+ pattern matching, etc. count valid multibyte character strings as a
+ single character. You can unset the option locally in a function to
+ revert to single-byte operation.
+
+ Note that if the shell is emulating a Bourne shell the tt(MULTIBYTE)
+ option is unset by default. This allows various POSIX modes to
+ work normally (POSIX does not deal with multibyte characters). If
+ you use a "sh" or "ksh" emulation interactively you shouldprobably
+ set the tt(MULTIBYTE) option.
+
+ The other option that affects multibyte support is tt(COMBINING_CHARS),
+ new in version 4.3.7. When this is set, any zero-length punctuation
+ characters that follow an alphanumeric character (the base character) are
+ assumed to be modifications (accents etc.) to the base character and to
+ be displayed within the same screen area as the base character. As not
+ all terminals handle this, even if they correctly display the base
+ multibyte character, this option is not on by default. The KDE terminal
+ emulator tt(konsole) is known to handle combining characters.
+
+ The tt(COMBINING_CHARS) option only affects output; combining characters
+ may always be input, but when the option is off will be displayed
+ specially. By default this is as a code point (the index of the
+ character in the character set) between angle brackets, usually
+ in inverse video. Highlighting of such special characters can
+ be modified using the new array parameter tt(zle_highlight).
-sect(How do I ensure multibyte input works on my system?)
+sect(How do I ensure multibyte input and output work on my system?)
Once you have a version of zsh with multibyte support, you need to
- ensure the envivronment is correct. We'll assume you're using UTF-8.
+ ensure the environment is correct. We'll assume you're using UTF-8.
Many modern systems may come set up correctly already. Try one of
the editing widgets described in the next section to see.
@@ -2163,6 +2157,9 @@
however, using UTF-8 massively extends the number of valid characters
that can be produced.
+ See also url(http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)
+ for general information on entering Unicode characters from a keyboard.
+
chapter(The future of zsh)
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-13 16:54 PATCH: (large) initial support for combining characters in ZLE Peter Stephenson
2008-04-13 17:32 ` Bart Schaefer
2008-04-14 12:00 ` Peter Stephenson
@ 2008-04-14 13:34 ` Mikael Magnusson
2008-04-14 13:54 ` Peter Stephenson
2008-04-17 18:33 ` Jun T.
3 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-04-14 13:34 UTC (permalink / raw)
To: Zsh Hackers' List
On 13/04/2008, Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> Here is my first broad brush attempt to get combining characters working
> in the line editor. I think it's now working well enough to make it
> worth testing, even if most of the testing is going to be making sure I
> haven't broken anything that currently works. SINGLE_LINE_ZLE in
> particular could do with a workout since I've changed that without a
> great deal of testing (it's not fundamentally kaput, but it could easily
> be slightly bruised).
Hi, I'm not sure if you want bug reports at this point, but one thing
I noticed and can reproduce in zsh -f is that the cursor sometimes
gets misplaced in the terminal with combiningchars on.
% zsh -f
% setopt combiningchars
% print -z $'\u0342'
the buffer should now contain an inverted <0342>, press left or ctrl-a
or whatever to go to the start of the line and press 'a'. Here, in
both urxvt and xterm (though in xterm no ~ appears over the a) the
cursor goes to sit on the start of the previous line in the terminal.
Funnily when you backspace it goes back to the proper place in the
current buffer again.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-14 13:34 ` Mikael Magnusson
@ 2008-04-14 13:54 ` Peter Stephenson
2008-04-15 13:58 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-04-14 13:54 UTC (permalink / raw)
To: Zsh Hackers' List
On Mon, 14 Apr 2008 15:34:19 +0200
"Mikael Magnusson" <mikachu@gmail.com> wrote:
> Hi, I'm not sure if you want bug reports at this point, but one thing
> I noticed and can reproduce in zsh -f is that the cursor sometimes
> gets misplaced in the terminal with combiningchars on.
> % zsh -f
> % setopt combiningchars
> % print -z $'\u0342'
> the buffer should now contain an inverted <0342>, press left or ctrl-a
> or whatever to go to the start of the line and press 'a'. Here, in
> both urxvt and xterm (though in xterm no ~ appears over the a) the
> cursor goes to sit on the start of the previous line in the terminal.
> Funnily when you backspace it goes back to the proper place in the
> current buffer again.
Here are a few trivial fixes for some simple problems like that---it's
simply a case of moving right until we get off the combining characters.
I should find a general fix for the jumping cursor problem, but for now
it's quite convenient (for me, anyway) to see the problems so clearly.
Index: Src/Zle/zle_hist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_hist.c,v
retrieving revision 1.39
diff -u -r1.39 zle_hist.c
--- Src/Zle/zle_hist.c 13 Apr 2008 16:58:42 -0000 1.39
+++ Src/Zle/zle_hist.c 14 Apr 2008 13:52:18 -0000
@@ -756,10 +756,12 @@
ZS_memcpy(zleline, he->zle_text, zlell);
if ((zlecs = zlell) && invicmdmode())
- zlecs--;
+ DECCS();
} else {
setline(he->node.nam, ZSL_COPY|ZSL_TOEND);
}
+ /* Move right if we're on a zero-width combining character */
+ CCRIGHT();
setlastline();
clearlist = 1;
}
@@ -1548,6 +1550,7 @@
zletextfree(&zt);
zle_setline(he);
zlecs = cpos;
+ CCRIGHT();
return 0;
}
}
@@ -1588,6 +1591,7 @@
zletextfree(&zt);
zle_setline(he);
zlecs = cpos;
+ CCRIGHT();
return 0;
}
}
Index: Src/Zle/zle_misc.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_misc.c,v
retrieving revision 1.45
diff -u -r1.45 zle_misc.c
--- Src/Zle/zle_misc.c 13 Apr 2008 16:58:42 -0000 1.45
+++ Src/Zle/zle_misc.c 14 Apr 2008 13:52:33 -0000
@@ -56,6 +56,8 @@
zleline[zlecs++] = *s;
if(neg)
zlecs += zmult * len;
+ /* if we ended up on a combining character, skip over it */
+ CCRIGHT();
}
/**/
Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.92
diff -u -r1.92 zle_tricky.c
--- Src/Zle/zle_tricky.c 14 Apr 2008 09:13:18 -0000 1.92
+++ Src/Zle/zle_tricky.c 14 Apr 2008 13:52:45 -0000
@@ -980,14 +980,11 @@
free(zlemetaline);
zlemetaline = NULL;
-
-#ifdef MULTIBYTE_SUPPORT
/*
* If we inserted combining characters under the cursor we
* won't have tested the effect yet. So fix it up now.
*/
- alignmultiwordright(1);
-#endif
+ CCRIGHT();
}
/* Free a brinfo list. */
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-14 13:54 ` Peter Stephenson
@ 2008-04-15 13:58 ` Mikael Magnusson
2008-04-15 16:46 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-04-15 13:58 UTC (permalink / raw)
To: Zsh Hackers' List
On 14/04/2008, Peter Stephenson <pws@csr.com> wrote:
> On Mon, 14 Apr 2008 15:34:19 +0200
> "Mikael Magnusson" <mikachu@gmail.com> wrote:
> > Hi, I'm not sure if you want bug reports at this point, but one thing
> > I noticed and can reproduce in zsh -f is that the cursor sometimes
> > gets misplaced in the terminal with combiningchars on.
> > % zsh -f
> > % setopt combiningchars
> > % print -z $'\u0342'
> > the buffer should now contain an inverted <0342>, press left or ctrl-a
> > or whatever to go to the start of the line and press 'a'. Here, in
> > both urxvt and xterm (though in xterm no ~ appears over the a) the
> > cursor goes to sit on the start of the previous line in the terminal.
> > Funnily when you backspace it goes back to the proper place in the
> > current buffer again.
>
>
> Here are a few trivial fixes for some simple problems like that---it's
> simply a case of moving right until we get off the combining characters.
>
> I should find a general fix for the jumping cursor problem, but for now
> it's quite convenient (for me, anyway) to see the problems so clearly.
This is better, but I can still put the cursor between the a and the ~.
> > % zsh -f
> > % setopt combiningchars
> > % print -z $'\u0342'
> > the buffer should now contain an inverted <0342>, press left or ctrl-a
> > or whatever to go to the start of the line and press 'a'.
But instead of pressing a, type a '!' and go left again and press 'a',
then press ctrl-d. (or ctrl-t)
Another thing I noticed is that ctrl-t moves just the combining char,
not the whole base+combiningchar, but that might already be on your
todo as a subset of "fix widgets" :).
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-15 13:58 ` Mikael Magnusson
@ 2008-04-15 16:46 ` Peter Stephenson
2008-04-16 1:28 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-04-15 16:46 UTC (permalink / raw)
To: Zsh Hackers' List
On Tue, 15 Apr 2008 15:58:54 +0200
"Mikael Magnusson" <mikachu@gmail.com> wrote:
> This is better, but I can still put the cursor between the a and the ~.
> > > % zsh -f
> > > % setopt combiningchars
> > > % print -z $'\u0342'
> > > the buffer should now contain an inverted <0342>, press left or ctrl-a
> > > or whatever to go to the start of the line and press 'a'.
> But instead of pressing a, type a '!' and go left again and press 'a',
> then press ctrl-d. (or ctrl-t)
OK, so we need to fix things up right at the end after an operation in case
it generated some combinations a sneaky way. Hence some more CCRIGHT()s.
> Another thing I noticed is that ctrl-t moves just the combining char,
> not the whole base+combiningchar, but that might already be on your
> todo as a subset of "fix widgets" :).
This should help with fixing the word stuff, when I get round to it...
I'm fed up with the names zle_misc.c and zle_utils.c. I can never find
anything.
Index: Src/Zle/zle.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle.h,v
retrieving revision 1.37
diff -u -r1.37 zle.h
--- Src/Zle/zle.h 13 Apr 2008 16:58:42 -0000 1.37
+++ Src/Zle/zle.h 15 Apr 2008 16:42:04 -0000
@@ -75,14 +75,19 @@
#define LASTFULLCHAR_T ZLE_INT_T
/* We may need to handle combining character alignment */
-#define CCLEFT() alignmultiwordleft(1)
-#define CCRIGHT() alignmultiwordright(1)
+#define CCLEFT() alignmultiwordleft(&zlecs, 1)
+#define CCRIGHT() alignmultiwordright(&zlecs, 1)
/*
* Increment or decrement the cursor position, skipping over
* combining characters.
*/
#define INCCS() inccs()
#define DECCS() deccs()
+/*
+ * Same for any other position.
+ */
+#define INCPOS(pos) incpos(&pos)
+#define DECPOS(pos) decpos(&pos)
#else /* Not MULTIBYTE_SUPPORT: old single-byte code */
@@ -151,6 +156,11 @@
*/
#define INCCS() ((void)(zlecs++))
#define DECCS() ((void)(zlecs--))
+/*
+ * Same for any other position.
+ */
+#define INCPOS(pos) ((void)(pos++))
+#define DECPOS(pos) ((void)(pos--))
#endif
Index: Src/Zle/zle_misc.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_misc.c,v
retrieving revision 1.46
diff -u -r1.46 zle_misc.c
--- Src/Zle/zle_misc.c 14 Apr 2008 14:57:54 -0000 1.46
+++ Src/Zle/zle_misc.c 15 Apr 2008 16:42:07 -0000
@@ -196,23 +196,66 @@
return 0;
}
+#ifdef MULTIBYTE_SUPPORT
+/*
+ * Transpose the chunk of the line from start to middle with
+ * that from middle to end.
+ */
+
+static void
+transpose_swap(int start, int middle, int end)
+{
+ int len1, len2;
+ ZLE_STRING_T first;
+
+ len1 = middle - start;
+ len2 = end - middle;
+
+ first = (ZLE_STRING_T)zalloc(len1 * ZLE_CHAR_SIZE);
+ ZS_memcpy(first, zleline + start, len1);
+ /* Move may be overlapping... */
+ ZS_memmove(zleline + start, zleline + middle, len2);
+ ZS_memcpy(zleline + start + len2, first, len1);
+ zfree(first, len1 * ZLE_CHAR_SIZE);
+}
+#endif
+
/**/
int
gosmacstransposechars(UNUSED(char **args))
{
- int cc;
-
if (zlecs < 2 || zleline[zlecs - 1] == '\n' || zleline[zlecs - 2] == '\n') {
- if (zlecs == zlell || zleline[zlecs] == '\n' ||
- ((zlecs + 1 == zlell || zleline[zlecs + 1] == '\n') &&
- (!zlecs || zleline[zlecs - 1] == '\n'))) {
+ int twice = (zlecs == 0 || zleline[zlecs - 1] == '\n');
+
+ if (zlecs == zlell || zleline[zlecs] == '\n')
return 1;
+
+ INCCS();
+ if (twice) {
+ if (zlecs == zlell || zleline[zlecs] == '\n')
+ return 1;
+ INCCS();
}
- zlecs += (zlecs == 0 || zleline[zlecs - 1] == '\n') ? 2 : 1;
}
- cc = zleline[zlecs - 2];
- zleline[zlecs - 2] = zleline[zlecs - 1];
- zleline[zlecs - 1] = cc;
+#ifdef MULTIBYTE_SUPPORT
+ {
+ int start, middle;
+
+ middle = zlecs;
+ DECPOS(middle);
+
+ start = middle;
+ DECPOS(start);
+
+ transpose_swap(start, middle, zlecs);
+ }
+#else
+ {
+ ZLE_CHAR_T cc = zleline[zlecs - 2];
+ zleline[zlecs - 2] = zleline[zlecs - 1];
+ zleline[zlecs - 1] = cc;
+ }
+#endif
return 0;
}
@@ -220,7 +263,7 @@
int
transposechars(UNUSED(char **args))
{
- int cc, ct;
+ int ct;
int n = zmult;
int neg = n < 0;
@@ -231,26 +274,43 @@
if (zlell == zlecs || zleline[zlecs] == '\n')
return 1;
if (!neg)
- zlecs++;
- ct++;
+ INCCS();
+ INCPOS(ct);
}
if (neg) {
if (zlecs && zleline[zlecs - 1] != '\n') {
- zlecs--;
- if (ct > 1 && zleline[ct - 2] != '\n')
- ct--;
+ DECCS();
+ if (ct > 1 && zleline[ct - 2] != '\n') {
+ DECPOS(ct);
+ }
}
} else {
if (zlecs != zlell && zleline[zlecs] != '\n')
- zlecs++;
+ INCCS();
+ }
+ if (ct == zlell || zleline[ct] == '\n') {
+ DECPOS(ct);
}
- if (ct == zlell || zleline[ct] == '\n')
- ct--;
if (ct < 1 || zleline[ct - 1] == '\n')
return 1;
- cc = zleline[ct - 1];
- zleline[ct - 1] = zleline[ct];
- zleline[ct] = cc;
+#ifdef MULTIBYTE_SUPPORT
+ {
+ /*
+ * We should keep any accents etc. on their original characters.
+ */
+ int start = ct, end = ct;
+ DECPOS(start);
+ INCPOS(end);
+
+ transpose_swap(start, ct, end);
+ }
+#else
+ {
+ ZLE_CHAR_T cc = zleline[ct - 1];
+ zleline[ct - 1] = zleline[ct];
+ zleline[ct] = cc;
+ }
+#endif
}
return 0;
}
Index: Src/Zle/zle_move.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_move.c,v
retrieving revision 1.11
diff -u -r1.11 zle_move.c
--- Src/Zle/zle_move.c 13 Apr 2008 16:58:42 -0000 1.11
+++ Src/Zle/zle_move.c 15 Apr 2008 16:42:08 -0000
@@ -46,27 +46,27 @@
*/
/**/
int
-alignmultiwordleft(int setpos)
+alignmultiwordleft(int *pos, int setpos)
{
- int loccs;
+ int loccs = *pos;
/* generic nothing to do test */
- if (!isset(COMBININGCHARS) || zlecs == zlell || zlecs == 0)
+ if (!isset(COMBININGCHARS) || loccs == zlell || loccs == 0)
return 0;
/* need to be on zero-width punctuation character */
- if (!iswpunct(zleline[zlecs]) || wcwidth(zleline[zlecs]) != 0)
+ if (!iswpunct(zleline[loccs]) || wcwidth(zleline[loccs]) != 0)
return 0;
/* yes, go left */
- loccs = zlecs - 1;
+ loccs--;
for (;;) {
/* second test here is paranoia */
if (iswalnum(zleline[loccs]) && wcwidth(zleline[loccs]) > 0) {
/* found start position */
if (setpos)
- zlecs = loccs;
+ *pos = loccs;
return 1;
} else if (!iswpunct(zleline[loccs]) ||
wcwidth(zleline[loccs]) != 0) {
@@ -88,30 +88,31 @@
*/
/**/
int
-alignmultiwordright(int setpos)
+alignmultiwordright(int *pos, int setpos)
{
int loccs;
/*
* Are we on a suitable character?
*/
- if (!alignmultiwordleft(0))
+ if (!alignmultiwordleft(pos, 0))
return 0;
/* yes, go right */
- loccs = zlecs + 1;
+ loccs = *pos + 1;
while (loccs < zlell) {
/* Anything other than a combining char will do here */
if (!iswpunct(zleline[loccs]) || wcwidth(zleline[loccs]) != 0) {
if (setpos)
- zlecs = loccs;
+ *pos = loccs;
return 1;
}
loccs++;
}
- zlecs = zlell;
+ if (setpos)
+ *pos = loccs;
return 1;
}
@@ -123,7 +124,7 @@
inccs(void)
{
zlecs++;
- alignmultiwordright(1);
+ alignmultiwordright(&zlecs, 1);
}
@@ -134,7 +135,26 @@
deccs(void)
{
zlecs--;
- alignmultiwordleft(1);
+ alignmultiwordleft(&zlecs, 1);
+}
+
+/* Same utilities for general position */
+
+/**/
+mod_export void
+incpos(int *pos)
+{
+ (*pos)++;
+ alignmultiwordright(pos, 1);
+}
+
+
+/**/
+mod_export void
+decpos(int *pos)
+{
+ (*pos)--;
+ alignmultiwordleft(pos, 1);
}
#endif
Index: Src/Zle/zle_utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_utils.c,v
retrieving revision 1.47
diff -u -r1.47 zle_utils.c
--- Src/Zle/zle_utils.c 13 Apr 2008 16:58:44 -0000 1.47
+++ Src/Zle/zle_utils.c 15 Apr 2008 16:42:12 -0000
@@ -551,6 +551,7 @@
cut(i, ct, flags);
shiftchars(i, ct);
+ CCRIGHT();
}
/**/
@@ -569,6 +570,7 @@
cut(i, ct, flags);
shiftchars(i, ct);
+ CCRIGHT();
}
/**/
@@ -588,6 +590,7 @@
DECCS();
shiftchars(zlecs, origcs - zlecs);
}
+ CCRIGHT();
}
/**/
@@ -603,13 +606,14 @@
} else {
int origcs = zlecs;
int n = ct;
- DPUTS(zlemetaline != NULL, "backdel needs CUT_RAW when metafied");
+ DPUTS(zlemetaline != NULL, "foredel needs CUT_RAW when metafied");
while (n--)
INCCS();
ct = zlecs - origcs;
zlecs = origcs;
shiftchars(zlecs, ct);
}
+ CCRIGHT();
}
/**/
@@ -634,6 +638,7 @@
DECCS();
else if (zlecs > zlell)
zlecs = zlell;
+ CCRIGHT();
if (flags & ZSL_COPY)
free(scp);
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-15 16:46 ` Peter Stephenson
@ 2008-04-16 1:28 ` Mikael Magnusson
2008-04-16 8:47 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-04-16 1:28 UTC (permalink / raw)
To: Zsh Hackers' List
On 15/04/2008, Peter Stephenson <pws@csr.com> wrote:
> On Tue, 15 Apr 2008 15:58:54 +0200
>
> "Mikael Magnusson" <mikachu@gmail.com> wrote:
>
> > This is better, but I can still put the cursor between the a and the ~.
> > > > % zsh -f
> > > > % setopt combiningchars
> > > > % print -z $'\u0342'
> > > > the buffer should now contain an inverted <0342>, press left or ctrl-a
> > > > or whatever to go to the start of the line and press 'a'.
> > But instead of pressing a, type a '!' and go left again and press 'a',
> > then press ctrl-d. (or ctrl-t)
>
>
> OK, so we need to fix things up right at the end after an operation in case
> it generated some combinations a sneaky way. Hence some more CCRIGHT()s.
>
>
> > Another thing I noticed is that ctrl-t moves just the combining char,
> > not the whole base+combiningchar, but that might already be on your
> > todo as a subset of "fix widgets" :).
>
>
> This should help with fixing the word stuff, when I get round to it...
>
> I'm fed up with the names zle_misc.c and zle_utils.c. I can never find
> anything.
I wonder if I'm just being evil now...
zsh -f
setopt interactivecomments
function toggleopt() {
#is there some easier way to do this?
[[ $options[$1] = on ]] && setopt no$1 || setopt $1
}
compdef _options toggleopt=setopt
function _togglecombining() {
toggleopt combiningchars
}
zle -N _togglecombining
bindkey ^K _togglecombining
print -z $'a\u0342'
<press ctrl-k and then left, and ctrl-k again>
...
sorry :)
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-16 1:28 ` Mikael Magnusson
@ 2008-04-16 8:47 ` Peter Stephenson
2008-04-17 9:28 ` Stephane Chazelas
2008-09-22 18:16 ` Mikael Magnusson
0 siblings, 2 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-04-16 8:47 UTC (permalink / raw)
To: Zsh Hackers' List
"Mikael Magnusson" wrote:
> zsh -f
> setopt interactivecomments
> function toggleopt() {
> #is there some easier way to do this?
> [[ $options[$1] = on ]] && setopt no$1 || setopt $1
> }
> compdef _options toggleopt=setopt
> function _togglecombining() {
> toggleopt combiningchars
> }
> zle -N _togglecombining
> bindkey ^K _togglecombining
> print -z $'a\u0342'
> <press ctrl-k and then left, and ctrl-k again>
I haven't looked at the behaviour of user-defined widgets at all, which
will need much more work, but on exit from a widget is a good place to
fix up the cursor position.
Some more for the FAQ, too.
Index: Etc/FAQ.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Etc/FAQ.yo,v
retrieving revision 1.38
diff -u -r1.38 FAQ.yo
--- Etc/FAQ.yo 14 Apr 2008 12:53:37 -0000 1.38
+++ Etc/FAQ.yo 16 Apr 2008 08:45:51 -0000
@@ -2045,7 +2045,8 @@
be displayed within the same screen area as the base character. As not
all terminals handle this, even if they correctly display the base
multibyte character, this option is not on by default. The KDE terminal
- emulator tt(konsole) is known to handle combining characters.
+ emulator tt(konsole), tt(rxvt-unicode), and the Unicode version of
+ xterm, tt(uxterm), are known to handle combining characters.
The tt(COMBINING_CHARS) option only affects output; combining characters
may always be input, but when the option is off will be displayed
@@ -2157,7 +2158,7 @@
however, using UTF-8 massively extends the number of valid characters
that can be produced.
- See also url(http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)
+ See also url(http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)(http://www.cl.cam.ac.uk/~mgk25/unicode.html#input)
for general information on entering Unicode characters from a keyboard.
Index: Src/Zle/zle_main.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_main.c,v
retrieving revision 1.107
diff -u -r1.107 zle_main.c
--- Src/Zle/zle_main.c 3 Apr 2008 15:30:44 -0000 1.107
+++ Src/Zle/zle_main.c 16 Apr 2008 08:45:52 -0000
@@ -1344,6 +1344,12 @@
}
if (set_bindk)
bindk = save_bindk;
+ /*
+ * Goodness knows where the user's left us; make sure
+ * it's not on a combining character that won't be displayed
+ * directly.
+ */
+ CCRIGHT();
return ret;
}
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-16 8:47 ` Peter Stephenson
@ 2008-04-17 9:28 ` Stephane Chazelas
2008-09-22 18:16 ` Mikael Magnusson
1 sibling, 0 replies; 102+ messages in thread
From: Stephane Chazelas @ 2008-04-17 9:28 UTC (permalink / raw)
To: Peter Stephenson; +Cc: Zsh Hackers' List
On Wed, Apr 16, 2008 at 09:47:54AM +0100, Peter Stephenson wrote:
[...]
> + emulator tt(konsole), tt(rxvt-unicode), and the Unicode version of
> + xterm, tt(uxterm), are known to handle combining characters.
[...]
Hi Peter,
Note that uxterm is generally a script that calls xterm with the
-u8 function, so you could say "recent versions of xterm"
instead. From http://invisible-island.net/xterm/xterm.log.html,
it seems that support for combining characters was added in
version 141 (2000/8/14).
Cheers,
Stéphane
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-16 8:47 ` Peter Stephenson
2008-04-17 9:28 ` Stephane Chazelas
@ 2008-09-22 18:16 ` Mikael Magnusson
2008-09-22 18:36 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-09-22 18:16 UTC (permalink / raw)
To: Peter Stephenson; +Cc: Zsh Hackers' List
2008/4/16 Peter Stephenson <pws@csr.com>:
> "Mikael Magnusson" wrote:
>> zsh -f
>> setopt interactivecomments
>> function toggleopt() {
>> #is there some easier way to do this?
>> [[ $options[$1] = on ]] && setopt no$1 || setopt $1
>> }
>> compdef _options toggleopt=setopt
>> function _togglecombining() {
>> toggleopt combiningchars
>> }
>> zle -N _togglecombining
>> bindkey ^K _togglecombining
>> print -z $'a\u0342'
>> <press ctrl-k and then left, and ctrl-k again>
>
> I haven't looked at the behaviour of user-defined widgets at all, which
> will need much more work, but on exit from a widget is a good place to
> fix up the cursor position.
>
> Some more for the FAQ, too.
I suddenly realized while doing something completely different that this
might still work (where work is defined as not work), and it does.
zsh -f
print -z $'a\u0342'
<alt-x>push-input
setopt combiningchars
boom
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-09-22 18:16 ` Mikael Magnusson
@ 2008-09-22 18:36 ` Peter Stephenson
2008-09-22 18:39 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-09-22 18:36 UTC (permalink / raw)
To: Zsh Hackers' List
"Mikael Magnusson" wrote:
> zsh -f
> print -z $'a\u0342'
> <alt-x>push-input
> setopt combiningchars
> boom
I can get something untoward to happen if I move the cursor on top of
the combining tilde first, if that's what you mean.
Index: Src/Zle/zle_main.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_main.c,v
retrieving revision 1.116
diff -u -r1.116 zle_main.c
--- Src/Zle/zle_main.c 16 Sep 2008 15:12:17 -0000 1.116
+++ Src/Zle/zle_main.c 22 Sep 2008 18:33:21 -0000
@@ -1173,6 +1173,7 @@
stackcs = -1;
if (zlecs > zlell)
zlecs = zlell;
+ CCLEFT();
}
if (stackhist != -1) {
histline = stackhist;
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-09-22 18:36 ` Peter Stephenson
@ 2008-09-22 18:39 ` Mikael Magnusson
0 siblings, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-09-22 18:39 UTC (permalink / raw)
To: Peter Stephenson; +Cc: Zsh Hackers' List
2008/9/22 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> "Mikael Magnusson" wrote:
>> zsh -f
>> print -z $'a\u0342'
>> <alt-x>push-input
>> setopt combiningchars
>> boom
>
> I can get something untoward to happen if I move the cursor on top of
> the combining tilde first, if that's what you mean.
Ah yes, I forgot that bit. Thanks :).
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-13 16:54 PATCH: (large) initial support for combining characters in ZLE Peter Stephenson
` (2 preceding siblings ...)
2008-04-14 13:34 ` Mikael Magnusson
@ 2008-04-17 18:33 ` Jun T.
2008-04-18 9:40 ` Peter Stephenson
3 siblings, 1 reply; 102+ messages in thread
From: Jun T. @ 2008-04-17 18:33 UTC (permalink / raw)
To: zsh-workers
Thank you for starting the combining character support!
At 17:54 +0100 08.4.13, Peter Stephenson wrote:
>the base character must be an alphanumeric (and
>I'm not sure about the numeric, I need to find a better definition),
and
I think this is too restrictive, because in some Asian languages
(Japanese, Korean, Thai, etc.) the base character can be non-alphaget.
For example, in Japanese, Hiragana/Katakana can be combined with
U+3099 (VOICED SOUND MARK) or U+309A (SEMI-VOICED SOUND MARK).
Example: U+3057 U+3099 = "じ"
the base character U+3057 = "し" is not an alphanumeric.
>the zero-width characters afterwards (I haven't imposed a limit on how
>many there are) must be punctuation.
I guess this is also too restrictive. I have run the code like the
following
on Fedora7:
wchar_t w;
setlocale(LC_ALL,"");
for(w=1; w<0x2ffff; ++w) {
if(wcwidth(w)==0 && iswpunct(w)==0) {
printf("%05x: %lc\n",w,w);
}
}
It listed 166 characters, all of which seem to be combining chars in
Thai or Korean (U+0e4e and U+1160 may not be combining, I'm not sure).
I think strictly defining combined char is virtually impossible,
because there are so many "nonsensical" combinations like
"Hiragana + umlaut". Even within alphabet, a combination like
"x + U+0318" is almost as strange as "space + grave".
How about accepting any combination?
If terminal emulator displays garbage, the user can turn off the
option COMBINING_CHARS to see the hex code.
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-17 18:33 ` Jun T.
@ 2008-04-18 9:40 ` Peter Stephenson
2008-04-18 15:48 ` Jun T.
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-04-18 9:40 UTC (permalink / raw)
To: zsh-workers
On Fri, 18 Apr 2008 03:33:36 +0900
"Jun T." <takimoto-j@kba.biglobe.ne.jp> wrote:
> At 17:54 +0100 08.4.13, Peter Stephenson wrote:
> >the base character must be an alphanumeric (and
> >I'm not sure about the numeric, I need to find a better definition),
> and
>
> I think this is too restrictive, because in some Asian languages
> (Japanese, Korean, Thai, etc.) the base character can be non-alphaget.
> For example, in Japanese, Hiragana/Katakana can be combined with
> U+3099 (VOICED SOUND MARK) or U+309A (SEMI-VOICED SOUND MARK).
> Example: U+3057 U+3099 = "じ"
> the base character U+3057 = "し" is not an alphanumeric.
It's treated as alphanumeric here, but what you say doesn't surprise me. I
think we can widen it without problems to anything that isn't a whitespace
or a control character, at least, so iswgraph() might be the thing. We
definitely need to avoid special whitespace characters (tabs, feeds,
newline, carriage return) since we don't know what's going to happen.
Also, as far as I can see marking a character as "control" is a signal not
to print it directly.
> >the zero-width characters afterwards (I haven't imposed a limit on how
> >many there are) must be punctuation.
>
> I guess this is also too restrictive. I have run the code like the
> following
> on Fedora7:
>
> wchar_t w;
> setlocale(LC_ALL,"");
> for(w=1; w<0x2ffff; ++w) {
> if(wcwidth(w)==0 && iswpunct(w)==0) {
> printf("%05x: %lc\n",w,w);
> }
> }
>
> It listed 166 characters, all of which seem to be combining chars in
> Thai or Korean (U+0e4e and U+1160 may not be combining, I'm not sure).
Probably looking for a graphic zero width character is good enough.
There may not be control character with zero width, anyway.
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-18 9:40 ` Peter Stephenson
@ 2008-04-18 15:48 ` Jun T.
2008-04-18 16:05 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Jun T. @ 2008-04-18 15:48 UTC (permalink / raw)
To: zsh-workers
On 2008/04/18, at 18:40, Peter Stephenson wrote:
> so iswgraph() might be the thing.
There are about 15 characters for which "wcwidth() > 0 && ! iswgraph()"
is true, all of them are a kind of white space (no tab or such).
I personaly think "space + combining-char" is OK and
just "wcwidth()>0" is enough for defining the base character.
(wcwidth() is -1 for any control chars including tabs.)
But if you think white space should not be a base character,
then "wcwidth() > 0 && iswgraph()" would be the best definition
of the base character.
> Probably looking for a graphic zero width character is good enough.
> There may not be control character with zero width, anyway.
Yes, I believe so. "wcwidth()==0" would be just enough for
defining the combining characters. wcwidth() is -1 for control
characters.
Jun
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: (large) initial support for combining characters in ZLE.
2008-04-18 15:48 ` Jun T.
@ 2008-04-18 16:05 ` Peter Stephenson
2008-04-19 15:04 ` Jun T.
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-04-18 16:05 UTC (permalink / raw)
To: zsh-workers
"Jun T." wrote:
> On 2008/04/18, at 18:40, Peter Stephenson wrote:
> > so iswgraph() might be the thing.
>
> There are about 15 characters for which "wcwidth() > 0 && ! iswgraph()"
> is true, all of them are a kind of white space (no tab or such).
> I personaly think "space + combining-char" is OK and
> just "wcwidth()>0" is enough for defining the base character.
> (wcwidth() is -1 for any control chars including tabs.)
It may well be OK from a standards point of view, but it's more
complicated in the shell, even if we restrict it to a space character
(which would still require an explicit test). Space is a word delimiter
and without major surgery in the main shell any attached combining
character will be stripped off in lexical analysis; either that, or we
would need to ignore any spaces with combining characters. This
fundamentally breaks the model that the main shell deals with an ASCII
byte stream with multibyte characters that aren't lexically significant.
Hence I'd prefer to make this clear from the manner of display.
A lexically non-significant (i.e. quoted) space in principle doesn't
need this special handling, but that would mean the line editor needs
lexical information, which I want to avoid: this is one of the things
that makes the completion code so hard to handle.
(There's no general problem with elements of $IFS, by the way: the
initial command line is special and the delimiters have to be whitespace,
presumably to avoid just this sort of problem.)
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread
* Identify "active" region?
@ 2008-05-04 0:52 Bart Schaefer
2008-05-04 7:16 ` Mikael Magnusson
2008-05-04 12:21 ` Peter Stephenson
0 siblings, 2 replies; 102+ messages in thread
From: Bart Schaefer @ 2008-05-04 0:52 UTC (permalink / raw)
To: zsh-workers
I'd like to write a widget that does
if [[ there is an active region ]]
then zle kill-region
else zle backward-kill-word
fi
(In fact, I'm tempted to write it in C and make it the default binding
for ^W, but that's somewhat beside the point of this message.)
There doesn't seem to be any way to determine that there's an active
region except to override set-mark-command and exchange-point-and-mark
to stash something in a global variable. Have I missed something?
(I thought perhaps I could peek into $region_highlight in the latest
from CVS, but that doesn't update on the fly.)
Opinions on usefulness going forward of detecting the active region?
A distinguished value of MARK appears too problematic, so it'd have to
be another zle variable, or ...?
For the time being I'm using the less-than-satisfactory
(( MARK > 0 && MARK != CURSOR ))
which isn't *that* bad since if I meant to kill all the way to the
beginning of the line I'd probably use kill-whole-line, and at worst
I just have to do exchange-point-and-mark first.
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Identify "active" region?
2008-05-04 0:52 Identify "active" region? Bart Schaefer
@ 2008-05-04 7:16 ` Mikael Magnusson
2008-05-04 12:21 ` Peter Stephenson
1 sibling, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-05-04 7:16 UTC (permalink / raw)
To: zsh-workers
2008/5/4 Bart Schaefer <schaefer@brasslantern.com>:
> I'd like to write a widget that does
>
> if [[ there is an active region ]]
> then zle kill-region
> else zle backward-kill-word
> fi
>
> (In fact, I'm tempted to write it in C and make it the default binding
> for ^W, but that's somewhat beside the point of this message.)
>
> There doesn't seem to be any way to determine that there's an active
> region except to override set-mark-command and exchange-point-and-mark
> to stash something in a global variable. Have I missed something?
>
> (I thought perhaps I could peek into $region_highlight in the latest
> from CVS, but that doesn't update on the fly.)
>
> Opinions on usefulness going forward of detecting the active region?
> A distinguished value of MARK appears too problematic, so it'd have to
> be another zle variable, or ...?
>
> For the time being I'm using the less-than-satisfactory
> (( MARK > 0 && MARK != CURSOR ))
> which isn't *that* bad since if I meant to kill all the way to the
> beginning of the line I'd probably use kill-whole-line, and at worst
> I just have to do exchange-point-and-mark first.
fwiw, i'd like to second this, i've been meaning to ask it for a while
but didn't get around to it. My widget looks like this:
autoload -U narrow-to-region
function _narrow_to_region_marked()
{
local right
local left
if ((MARK == 0)); then
zle set-mark-command
fi
if ((MARK < CURSOR)); then
left="$LBUFFER[0,$((MARK-CURSOR-1))]"
right="$RBUFFER"
else
left="$LBUFFER"
right="$BUFFER[$((MARK+1)),-1]"
fi
narrow-to-region -p "$left>>|" -P "|<<$right"
}
I'd like to replace the "MARK == 0" with "is there an active region?".
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Identify "active" region?
2008-05-04 0:52 Identify "active" region? Bart Schaefer
2008-05-04 7:16 ` Mikael Magnusson
@ 2008-05-04 12:21 ` Peter Stephenson
2008-05-04 12:33 ` Mikael Magnusson
1 sibling, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-05-04 12:21 UTC (permalink / raw)
To: zsh-workers
On Sat, 03 May 2008 17:52:19 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> There doesn't seem to be any way to determine that there's an active
> region except to override set-mark-command and exchange-point-and-mark
> to stash something in a global variable. Have I missed something?
No, that's worth adding. I'll do it if no one gets to it first. It
should be trivial: a new ZLE variable tied into the integer controlling
this. There's no reason it shouldn't be writable, either.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Identify "active" region?
2008-05-04 12:21 ` Peter Stephenson
@ 2008-05-04 12:33 ` Mikael Magnusson
2008-05-04 12:35 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-05-04 12:33 UTC (permalink / raw)
To: zsh-workers
2008/5/4 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> On Sat, 03 May 2008 17:52:19 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > There doesn't seem to be any way to determine that there's an active
> > region except to override set-mark-command and exchange-point-and-mark
> > to stash something in a global variable. Have I missed something?
>
> No, that's worth adding. I'll do it if no one gets to it first. It
> should be trivial: a new ZLE variable tied into the integer controlling
> this. There's no reason it shouldn't be writable, either.
This seems to do the trick:
diff --git a/Src/Zle/zle_params.c b/Src/Zle/zle_params.c
index a04ee72..5b8cc30 100644
--- a/Src/Zle/zle_params.c
+++ b/Src/Zle/zle_params.c
@@ -89,6 +89,8 @@ static const struct gsu_integer numeric_gsu =
{ get_numeric, set_numeric, unset_numeric };
static const struct gsu_integer pending_gsu =
{ get_pending, NULL, zleunsetfn };
+static const struct gsu_integer region_gsu =
+{ get_region_active, set_region_active, zleunsetfn };
static const struct gsu_array killring_gsu =
{ get_killring, set_killring, unset_killring };
@@ -125,7 +127,8 @@ static struct zleparam {
{ "POSTDISPLAY", PM_SCALAR, GSU(postdisplay_gsu), NULL },
{ "PREBUFFER", PM_SCALAR | PM_READONLY, GSU(prebuffer_gsu), NULL },
{ "PREDISPLAY", PM_SCALAR, GSU(predisplay_gsu), NULL },
- { "RBUFFER", PM_SCALAR, GSU(rbuffer_gsu), NULL },
+ { "RBUFFER", PM_SCALAR, GSU(rbuffer_gsu), NULL },
+ { "REGION", PM_INTEGER, GSU(region_gsu), NULL},
{ "region_highlight", PM_ARRAY, GSU(region_highlight_gsu), NULL },
{ "WIDGET", PM_SCALAR | PM_READONLY, GSU(widget_gsu), NULL },
{ "WIDGETFUNC", PM_SCALAR | PM_READONLY, GSU(widgetfunc_gsu), NULL },
@@ -252,6 +255,20 @@ get_mark(UNUSED(Param pm))
/**/
static void
+set_region_active(UNUSED(Param pm), zlong x)
+{
+ region_active = !!x;
+}
+
+/**/
+static zlong
+get_region_active(UNUSED(Param pm))
+{
+ return region_active;
+}
+
+/**/
+static void
set_lbuffer(UNUSED(Param pm), char *x)
{
ZLE_STRING_T y;
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Identify "active" region?
2008-05-04 12:33 ` Mikael Magnusson
@ 2008-05-04 12:35 ` Mikael Magnusson
2008-05-04 13:28 ` Mikael Magnusson
2008-05-04 16:38 ` Bart Schaefer
0 siblings, 2 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-05-04 12:35 UTC (permalink / raw)
To: zsh-workers
2008/5/4 Mikael Magnusson <mikachu@gmail.com>:
> 2008/5/4 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> > No, that's worth adding. I'll do it if no one gets to it first. It
> > should be trivial: a new ZLE variable tied into the integer controlling
> > this. There's no reason it shouldn't be writable, either.
>
> This seems to do the trick:
> + { "REGION", PM_INTEGER, GSU(region_gsu), NULL},
I realize two things just after hitting send, 1) the variable should
probably be called something like REGIONACTIVE, and 2) I didn't write
any documentation for it.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Identify "active" region?
2008-05-04 12:35 ` Mikael Magnusson
@ 2008-05-04 13:28 ` Mikael Magnusson
2008-05-04 18:05 ` Peter Stephenson
2008-05-04 16:38 ` Bart Schaefer
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-05-04 13:28 UTC (permalink / raw)
To: zsh-workers
2008/5/4 Mikael Magnusson <mikachu@gmail.com>:
> 2008/5/4 Mikael Magnusson <mikachu@gmail.com>:
>
> > 2008/5/4 Peter Stephenson <p.w.stephenson@ntlworld.com>:
>
> > > No, that's worth adding. I'll do it if no one gets to it first. It
> > > should be trivial: a new ZLE variable tied into the integer controlling
> > > this. There's no reason it shouldn't be writable, either.
> >
> > This seems to do the trick:
> >
> > + { "REGION", PM_INTEGER, GSU(region_gsu), NULL},
>
> I realize two things just after hitting send, 1) the variable should
> probably be called something like REGIONACTIVE, and 2) I didn't write
> any documentation for it.
diff --git a/Doc/Zsh/zle.yo b/Doc/Zsh/zle.yo
index f8435c1..e07c955 100644
--- a/Doc/Zsh/zle.yo
+++ b/Doc/Zsh/zle.yo
@@ -774,6 +774,11 @@ The part of the buffer that lies to the right of
the cursor position.
If it is assigned to, only that part of the buffer is replaced, and the
cursor remains between the old tt($LBUFFER) and the new tt($RBUFFER).
)
+vindex(REGIONACTIVE)
+item(tt(REGIONACTIVE) (integer))(
+Indicates if the region is currently active. It can be assigned 0 or 1
+to deactivate and deactivate the region respectively.
+)
vindex(region_highlight)
item(tt(region_highlight) (array))(
Each element of this array may be set to a string that describes
diff --git a/Src/Zle/zle_params.c b/Src/Zle/zle_params.c
index a04ee72..a114596 100644
--- a/Src/Zle/zle_params.c
+++ b/Src/Zle/zle_params.c
@@ -89,6 +89,8 @@ static const struct gsu_integer numeric_gsu =
{ get_numeric, set_numeric, unset_numeric };
static const struct gsu_integer pending_gsu =
{ get_pending, NULL, zleunsetfn };
+static const struct gsu_integer region_gsu =
+{ get_region_active, set_region_active, zleunsetfn };
static const struct gsu_array killring_gsu =
{ get_killring, set_killring, unset_killring };
@@ -125,7 +127,8 @@ static struct zleparam {
{ "POSTDISPLAY", PM_SCALAR, GSU(postdisplay_gsu), NULL },
{ "PREBUFFER", PM_SCALAR | PM_READONLY, GSU(prebuffer_gsu), NULL },
{ "PREDISPLAY", PM_SCALAR, GSU(predisplay_gsu), NULL },
- { "RBUFFER", PM_SCALAR, GSU(rbuffer_gsu), NULL },
+ { "RBUFFER", PM_SCALAR, GSU(rbuffer_gsu), NULL },
+ { "REGIONACTIVE", PM_INTEGER, GSU(region_gsu), NULL},
{ "region_highlight", PM_ARRAY, GSU(region_highlight_gsu), NULL },
{ "WIDGET", PM_SCALAR | PM_READONLY, GSU(widget_gsu), NULL },
{ "WIDGETFUNC", PM_SCALAR | PM_READONLY, GSU(widgetfunc_gsu), NULL },
@@ -252,6 +255,20 @@ get_mark(UNUSED(Param pm))
/**/
static void
+set_region_active(UNUSED(Param pm), zlong x)
+{
+ region_active = !!x;
+}
+
+/**/
+static zlong
+get_region_active(UNUSED(Param pm))
+{
+ return region_active;
+}
+
+/**/
+static void
set_lbuffer(UNUSED(Param pm), char *x)
{
ZLE_STRING_T y;
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Identify "active" region?
2008-05-04 12:35 ` Mikael Magnusson
2008-05-04 13:28 ` Mikael Magnusson
@ 2008-05-04 16:38 ` Bart Schaefer
2008-05-04 17:52 ` Mikael Magnusson
1 sibling, 1 reply; 102+ messages in thread
From: Bart Schaefer @ 2008-05-04 16:38 UTC (permalink / raw)
To: zsh-workers
On May 4, 2:35pm, Mikael Magnusson wrote:
}
} I realize two things just after hitting send, 1) the variable should
} probably be called something like REGIONACTIVE, and 2) I didn't write
} any documentation for it.
Or it could be slightly more complicated and be an associative array
with fields active, start, and end. Then one could write
(( $REGION[active] )) && BUFFER[$REGION[start],$REGION[end]]=...
instead of
if (( $REGIONACTIVE ))
then
integer start=$MARK end=$CURSOR
if (( MARK > CURSOR ))
start=$CURSOR
end=$MARK
fi
BUFFER[$start,$end]=...
fi
However, $REGIONACTIVE is much better than nothing; thanks.
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Identify "active" region?
2008-05-04 16:38 ` Bart Schaefer
@ 2008-05-04 17:52 ` Mikael Magnusson
0 siblings, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-05-04 17:52 UTC (permalink / raw)
To: zsh-workers
2008/5/4 Bart Schaefer <schaefer@brasslantern.com>:
> On May 4, 2:35pm, Mikael Magnusson wrote:
> }
> } I realize two things just after hitting send, 1) the variable should
> } probably be called something like REGIONACTIVE, and 2) I didn't write
> } any documentation for it.
>
> Or it could be slightly more complicated and be an associative array
> with fields active, start, and end. Then one could write
>
> (( $REGION[active] )) && BUFFER[$REGION[start],$REGION[end]]=...
>
> instead of
>
> if (( $REGIONACTIVE ))
> then
> integer start=$MARK end=$CURSOR
> if (( MARK > CURSOR ))
> start=$CURSOR
> end=$MARK
> fi
> BUFFER[$start,$end]=...
> fi
>
> However, $REGIONACTIVE is much better than nothing; thanks.
Hm, that might be nicer. But I see there are no zle variables that are
associative arrays, so I don't really know how to do that. (Actually
the two that are normal arrays look a bit scary too :).
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
[parent not found: <mikachu@gmail.com>]
* Completion lockup
@ 2008-02-06 15:02 ` Mikael Magnusson
2008-02-06 15:07 ` Ismail Dönmez
2008-02-09 17:21 ` Peter Stephenson
0 siblings, 2 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-02-06 15:02 UTC (permalink / raw)
To: zsh-workers
zsh -f
setopt rcquotes
a'''<tab>
Tried on two machines with 4.3.4-dev-7, both started spinning 100% cpu
longer than i felt like waiting.
Some random gdb backtraces (not crashed, just pressed ctrl-c and bt,
then c and repeated a few times).
Program received signal SIGINT, Interrupt.
0xa7968e2a in docomplete () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
(gdb) bt
#0 0xa7968e2a in docomplete () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#1 0xa79651b0 in completecall () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#2 0xa7958318 in execzlefunc () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#3 0xa795850a in zlecore () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#4 0xa7958b7f in zleread () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#5 0x080829ca in inputline ()
#6 0x080830b9 in ingetc ()
#7 0x0807d537 in ihgetc ()
#8 0x0808cb4b in yylex ()
#9 0x080abc8a in parse_event ()
#10 0x08081404 in loop ()
#11 0x0808247e in zsh_main ()
#12 0x08054f82 in main ()
(gdb) Quit
(gdb) c
Continuing.
Program received signal SIGINT, Interrupt.
0x080c60e7 in mb_metacharlenconv ()
(gdb) bt
#0 0x080c60e7 in mb_metacharlenconv ()
#1 0xa7968e10 in docomplete () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#2 0xa79651b0 in completecall () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#3 0xa7958318 in execzlefunc () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#4 0xa795850a in zlecore () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#5 0xa7958b7f in zleread () from /usr/local/lib/zsh/4.3.4-dev-7/zsh/zle.so
#6 0x080829ca in inputline ()
#7 0x080830b9 in ingetc ()
#8 0x0807d537 in ihgetc ()
#9 0x0808cb4b in yylex ()
#10 0x080abc8a in parse_event ()
#11 0x08081404 in loop ()
#12 0x0808247e in zsh_main ()
#13 0x08054f82 in main ()
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Completion lockup
2008-02-06 15:02 ` Completion lockup Mikael Magnusson
@ 2008-02-06 15:07 ` Ismail Dönmez
2008-02-09 17:21 ` Peter Stephenson
1 sibling, 0 replies; 102+ messages in thread
From: Ismail Dönmez @ 2008-02-06 15:07 UTC (permalink / raw)
To: Mikael Magnusson; +Cc: zsh-workers
At Wednesday 06 February 2008 around 17:02:49 Mikael Magnusson wrote:
> zsh -f
> setopt rcquotes
> a'''<tab>
Can confirm with latest CVS.
Regards,
ismail
--
Never learn by your mistakes, if you do you may never dare to try again.
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Completion lockup
2008-02-06 15:02 ` Completion lockup Mikael Magnusson
2008-02-06 15:07 ` Ismail Dönmez
@ 2008-02-09 17:21 ` Peter Stephenson
2008-02-09 18:04 ` Mikael Magnusson
1 sibling, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-02-09 17:21 UTC (permalink / raw)
To: zsh-workers
"Mikael Magnusson" wrote:
> zsh -f
> setopt rcquotes
> a'''<tab>
> Tried on two machines with 4.3.4-dev-7, both started spinning 100% cpu
> longer than i felt like waiting.
get_comp_string() is the second most horrible function ever written.
The following not only stopped the crash it apparently worked, i.e.
it complete a file a\'foobar I created in the path! I didn't like
to look any further. I'm sure you'll let me know if anything funny
is going on.
Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.89
diff -u -r1.89 zle_tricky.c
--- Src/Zle/zle_tricky.c 3 Oct 2007 16:18:38 -0000 1.89
+++ Src/Zle/zle_tricky.c 9 Feb 2008 17:18:16 -0000
@@ -1230,11 +1230,16 @@
/* This is done when the lexer reached the word the cursor is on. */
tt = tokstr ? dupstring(tokstr) : NULL;
- if (isset(RCQUOTES) && *tt == Snull) {
- char *p, *e = tt + zlemetacs - wb;
- for (p = tt; *p && p < e; p++)
- if (*p == '\'')
- qsub++;
+ if (isset(RCQUOTES)) {
+ char *tt1, *e = tt + zlemetacs - wb;
+ for (tt1 = tt; *tt1; tt1++) {
+ if (*tt1 == Snull) {
+ char *p;
+ for (p = tt1; *p && p < e; p++)
+ if (*p == '\'')
+ qsub++;
+ }
+ }
}
/* If we added a `x', remove it. */
if (addedx && tt)
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Completion lockup
2008-02-09 17:21 ` Peter Stephenson
@ 2008-02-09 18:04 ` Mikael Magnusson
2008-02-09 19:13 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-02-09 18:04 UTC (permalink / raw)
To: zsh-workers
On 09/02/2008, Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> "Mikael Magnusson" wrote:
> > zsh -f
> > setopt rcquotes
> > a'''<tab>
> > Tried on two machines with 4.3.4-dev-7, both started spinning 100% cpu
> > longer than i felt like waiting.
>
> get_comp_string() is the second most horrible function ever written.
> The following not only stopped the crash it apparently worked, i.e.
> it complete a file a\'foobar I created in the path! I didn't like
> to look any further. I'm sure you'll let me know if anything funny
> is going on.
This seems to have done the trick, it's always nice when fixing something
makes it work too. The only funny thing I've noticed in the past 5 minutes
is that if i type a''''<tab> i get the file as a\'foobar, but if i
leave out the a and
just tab the four quotes, i get it in the command line as 'a''foobar' which
seems weird, but maybe the _correct (i think?) completer does things
differently. This is not something that bothers me in the least, just thought
i should mention it.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Completion lockup
2008-02-09 18:04 ` Mikael Magnusson
@ 2008-02-09 19:13 ` Peter Stephenson
0 siblings, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-02-09 19:13 UTC (permalink / raw)
To: zsh-workers
"Mikael Magnusson" wrote:
> The only funny thing I've noticed in the past 5 minutes
> is that if i type a''''<tab> i get the file as a\'foobar, but if i
> leave out the a and
> just tab the four quotes, i get it in the command line as 'a''foobar' which
> seems weird, but maybe the _correct (i think?) completer does things
> differently. This is not something that bothers me in the least, just thought
> i should mention it.
The code that handles quotes in completion is gruesome (and includes, in
case you hadn't guessed, the most horrible function ever written,
too---that's the one that deals with nested quotes when something inside
quotes is to be treated as a command line in its own right). One of its
tricks is that it makes some kind of guess early on how to quote the
line. If there's a ' at the start it will use ' quoting and if there
isn't it will use backslash quoting. It doesn't make much attempt to
keep the form of quotes it started off with. Typically this is a pretty
reasonable guess.
You'd see this if the test file was called \'foobar and you typed
'''f<TAB>.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Who is sorting my completion results?
@ 2008-08-30 12:02 ` Mikael Magnusson
2008-08-30 12:16 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-08-30 12:02 UTC (permalink / raw)
To: zsh-workers
I'm trying to add reflog completion for git (nevermind what that is if you don't
know :), they look like foo@{1}, foo@{2}, etc. git reflog gives an
already sorted
list of reflog entries, but when I try my completion in the shell, they are re-
sorted as foo@{1}, foo@{10}, foo@{100}, foo@{2}, etc. Who and why is this list
resorted by? (+grammar).
(( $+functions[__git_heads] )) ||
__git_heads () {
local expl
declare -a branch_names_with_desc branch_names ref_with_desc ref
if [[ $PREFIX = *@* ]]; then
#XXX this gets sorted by sha1 at least from git-checkout which is
obviously bad...
compset -P '*@'
ref_with_desc=(${(f)"$(_call_program reflogentries git reflog show
${IPREFIX%@})"})
__git_command_successful || return
ref=(${ref_with_desc//(#b)*. (*\}):*/$match[1]})
unset PREFIX IPREFIX
_wanted reflog expl reflog-entry compadd $* -ld ref_with_desc - $ref
else
branch_names_with_desc=(${${(f)"$(_call_program headrefs git
for-each-ref --format='"%(refname) [%(subject)]"' refs/heads
refs/remotes 2>/dev/null)"}#refs/(heads|remotes)/})
__git_command_successful || return
branch_names=(${branch_names_with_desc// *})
_wanted heads expl branch-name compadd $* -ld
branch_names_with_desc - $branch_names HEAD
fi
}
--
Mikael Magnusson
PS I lied a bit in the description wrt this code, but it gets true if you
remove -ld ref_with_desc, the results are even worse with it since it gets
sorted by the sha1 hash of the commit. DS
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Who is sorting my completion results?
2008-08-30 12:02 ` Who is sorting my completion results? Mikael Magnusson
@ 2008-08-30 12:16 ` Peter Stephenson
2008-08-30 12:39 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2008-08-30 12:16 UTC (permalink / raw)
To: zsh-workers
"Mikael Magnusson" wrote:
> I'm trying to add reflog completion for git (nevermind what that is if
> you don't know :), they look like foo@{1}, foo@{2}, etc. git reflog
> gives an already sorted list of reflog entries, but when I try my
> completion in the shell, they are re-sorted as foo@{1}, foo@{10},
> foo@{100}, foo@{2}, etc. Who and why is this list resorted by?
> (+grammar).
Sorting or not sorting is done by whether you pass the option "-J group"
or "-V group" to compadd. In your case it looks like the right thing to
do is to pass the option "-V" (only) to _wanted, which will construct an
appropriate argument list for compadd.
Completion/Zsh/Type/_directory_stack was the simplest example I noticed
of this happening.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Who is sorting my completion results?
2008-08-30 12:16 ` Peter Stephenson
@ 2008-08-30 12:39 ` Mikael Magnusson
0 siblings, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-08-30 12:39 UTC (permalink / raw)
To: zsh-workers
2008/8/30 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> "Mikael Magnusson" wrote:
>> I'm trying to add reflog completion for git (nevermind what that is if
>> you don't know :), they look like foo@{1}, foo@{2}, etc. git reflog
>> gives an already sorted list of reflog entries, but when I try my
>> completion in the shell, they are re-sorted as foo@{1}, foo@{10},
>> foo@{100}, foo@{2}, etc. Who and why is this list resorted by?
>> (+grammar).
>
> Sorting or not sorting is done by whether you pass the option "-J group"
> or "-V group" to compadd. In your case it looks like the right thing to
> do is to pass the option "-V" (only) to _wanted, which will construct an
> appropriate argument list for compadd.
> Completion/Zsh/Type/_directory_stack was the simplest example I noticed
> of this happening.
Thanks, with some additional trickery, that did it. With the given
information, I could almost reverse engineer the location in the manpage
where this is written :). _wanted says it has some options in common
with _requested, which in turn hints you might find some information
in the _description text, which does explain what -V does. I am still
somewhat confused about what the difference is between -V, -1V and
-2V though.
The following diff is not submitted in any way, but only to illustrate
the needed changes.
--- a/Completion/Unix/Command/_git
+++ b/Completion/Unix/Command/_git
@@ -1563,6 +1563,7 @@ _git-checkout () {
else
_arguments -C -S \
'-q[suppress feedback messages]' \
+ '::branch or tree-ish:__git_tree_ishs' \
- switch-branch \
'-f[force a complete re-read]' \
'-b[create a new branch based at given branch]:
:__git_guard_branch-name' \
@@ -1571,9 +1572,7 @@ _git-checkout () {
'-l[create the branch'\''s reflog]' \
$new_branch_reflog_arg \
'-m[3way merge current branch, working tree and new branch]' \
- '::branch:__git_revisions' \
- update-files \
- '::tree-ish:__git_tree_ishs' \
'*::file:->files' && ret=0
fi
@@ -2870,9 +2869,8 @@ __git_tree_files () {
# if both exists, they need to be completed to heads/x and tags/x.
(( $+functions[__git_commits] )) ||
__git_commits () {
- _alternative \
- 'heads::__git_heads' \
- 'tags::__git_tags'
+ __git_heads $*
+ __git_tags $*
}
(( $+functions[__git_committishs] )) ||
@@ -3038,7 +3036,7 @@ __git_heads () {
unset PREFIX IPREFIX
- _wanted reflog expl reflog-entry compadd $* -ld ref_with_desc - $ref
+ _wanted -V reflog expl reflog-entry compadd $* -ld ref_with_desc - $ref
else
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
[parent not found: <237967ef0902140622s7389d2c8h5a0c786dcf207422@mail.gmail.com>]
* Infinite loop, can't reproduce with zsh -f
@ 2011-01-06 19:22 ` Mikael Magnusson
2011-01-06 20:03 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2011-01-06 19:22 UTC (permalink / raw)
To: zsh workers
Hi, I was going to test something in relation to the =~ bug in the
other thread and forgot what -regex-match was called, so i tried
set -x; [[ a =~ b ]]; set +x
but it just starts spinning forever. It doesn't in zsh -f though.
Any ideas what to look for here? Note that running set -x and +x on
separate lines before and after works fine... In fact, having set -x
on the same line works too, the problematic case is
[[ a =~ b ]]; set +x
for some reason.
^C
Program received signal SIGINT, Interrupt.
gettext2 (state=0x7fffffffd640) at text.c:350
350 if (!(stack = (WC_SUBLIST_TYPE(code) == WC_SUBLIST_END))) {
(gdb) bt
#0 gettext2 (state=0x7fffffffd640) at text.c:350
#1 0x0000000000489bca in getjobtext (prog=0x7ffff7fefc00,
c=0x7ffff7fefc38) at text.c:223
#2 0x00000000004427e8 in loop (toplevel=1, justonce=0) at init.c:174
#3 0x0000000000445771 in zsh_main (argc=1, argv=0x7fffffffd828) at init.c:1508
#4 0x0000000000410674 in main (argc=1, argv=0x7fffffffd828) at ./main.c:93
(gdb) n
361 if (stack < 1 && (WC_SUBLIST_FLAGS(s->code) & WC_SUBLIST_SIMPLE))
(gdb) [i'll leave out the following empty (gdb) lines]
363 break;
845 }
302 if (stack) {
303 if (!(s = tstack))
305 if (s->pop) {
310 code = s->code;
311 stack = 0;
316 switch (wc_code(code)) {
340 if (!s) {
350 if (!(stack = (WC_SUBLIST_TYPE(code) == WC_SUBLIST_END))) {
361 if (stack < 1 && (WC_SUBLIST_FLAGS(s->code) & WC_SUBLIST_SIMPLE))
363 break;
(gdb) print *s
$7 = {prev = 0x0, code = 4098, pop = 0, u = {_redir = {list =
0xf000d000c000b}, _funcdef = {
strs = 0xf000d000c000b <Address 0xf000d000c000b out of bounds>,
end = 0x14001200110010}, _case = {end = 0xf000d000c000b}, _if =
{cond = 786443,
end = 0x14001200110010}, _cond = {par = 786443}, _subsh = {end =
0xf000d000c000b}}}
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* PATCH: Fix leaks of desthost in ztcp
@ 2014-11-23 21:07 ` Mikael Magnusson
2014-11-23 21:53 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2014-11-23 21:07 UTC (permalink / raw)
To: zsh-workers
Found by coverity. Is that tcp_close right too? It's closed in another
error case nearby so I'm pretty sure it is.
---
Src/Modules/tcp.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/Src/Modules/tcp.c b/Src/Modules/tcp.c
index 3f92050..0d95220 100644
--- a/Src/Modules/tcp.c
+++ b/Src/Modules/tcp.c
@@ -623,6 +623,7 @@ bin_ztcp(char *nam, char **args, Options ops, UNUSED(int func))
zthost = zsh_getipnodebyname(desthost, AF_INET, 0, &herrno);
if (!zthost || errflag) {
zwarnnam(nam, "host resolution failure: %s", desthost);
+ zsfree(desthost);
return 1;
}
@@ -630,6 +631,7 @@ bin_ztcp(char *nam, char **args, Options ops, UNUSED(int func))
if (!sess) {
zwarnnam(nam, "unable to allocate a TCP session slot");
+ zsfree(desthost);
return 1;
}
@@ -665,6 +667,8 @@ bin_ztcp(char *nam, char **args, Options ops, UNUSED(int func))
sess->fd = redup(sess->fd, targetfd);
if (sess->fd < 0) {
zerrnam(nam, "could not duplicate socket fd to %d: %e", targetfd, errno);
+ zsfree(desthost);
+ tcp_close(sess);
return 1;
}
}
--
2.2.0-rc0
^ permalink raw reply [flat|nested] 102+ messages in thread
[parent not found: <okiddle@yahoo.co.uk>]
* another bug: zsh_directory_name
@ 2008-10-30 21:20 ` Oliver Kiddle
2008-10-30 21:26 ` Mikael Magnusson
2008-10-30 22:16 ` Peter Stephenson
0 siblings, 2 replies; 102+ messages in thread
From: Oliver Kiddle @ 2008-10-30 21:20 UTC (permalink / raw)
To: Zsh workers
Just thought I'd check that the case of calling this new ~[...] stuff
recursively had been handled. The following caused an instant seg fault:
zsh_directory_name() {
echo ~[]
}
Out of interest what are people using this for. It seems like a nice
idea, but I've yet to think of an actual use that can't be done with a
normal named directory. Transformations on the current $PWD seem to be
one logical option - ${PWD/src/include} for instance. Or an easier syntax
for (../)#
Oliver
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: another bug: zsh_directory_name
2008-10-30 21:20 ` another bug: zsh_directory_name Oliver Kiddle
@ 2008-10-30 21:26 ` Mikael Magnusson
2008-10-30 22:13 ` Peter Stephenson
2008-10-30 22:16 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-10-30 21:26 UTC (permalink / raw)
To: Oliver Kiddle; +Cc: Zsh workers
2008/10/30 Oliver Kiddle <okiddle@yahoo.co.uk>:
> Just thought I'd check that the case of calling this new ~[...] stuff
> recursively had been handled. The following caused an instant seg fault:
>
> zsh_directory_name() {
> echo ~[]
> }
I don't think this has anything to do with ~[],
% function a() { a }
% a
zsh: segmentation fault zsh
(i appear to not have debug symbols but maybe better than nothing),
(gdb) bt
#0 0x08069b4c in execcmd ()
#1 0x0806df95 in execpline2 ()
#2 0x0806e39d in execpline ()
#3 0x00000000 in ?? ()
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: another bug: zsh_directory_name
2008-10-30 21:26 ` Mikael Magnusson
@ 2008-10-30 22:13 ` Peter Stephenson
2008-10-30 23:44 ` Vincent Lefevre
` (2 more replies)
0 siblings, 3 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-10-30 22:13 UTC (permalink / raw)
To: Zsh workers, pws
"Mikael Magnusson" wrote:
> 2008/10/30 Oliver Kiddle <okiddle@yahoo.co.uk>:
> > Just thought I'd check that the case of calling this new ~[...] stuff
> > recursively had been handled. The following caused an instant seg fault:
> >
> > zsh_directory_name() {
> > echo ~[]
> > }
>
> I don't think this has anything to do with ~[],
> % function a() { a }
> % a
> zsh: segmentation fault zsh
Right. I get:
pws-pc% function a() { a }
pws-pc% a
a: maximum nested function level reached
and
pws-pc% zsh_directory_name() {
function> echo ~[]
function> }
pws-pc% echo ~[]
zsh_directory_name:1: job table full or recursion limit exceeded
so the problem is that the max function depth is too large on those
systems (or, equivalently, the resource limits are too small).
There's no easy way of handling this more gracefully in the shell, since
we could hit the buffers on any of a zillion zillion allocations with
untested return values.
The default max function depth is currently 1000. See
zsh-workers/22501 (http://www.zsh.org/mla/workers/2006/msg00388.html).
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: another bug: zsh_directory_name
2008-10-30 22:13 ` Peter Stephenson
@ 2008-10-30 23:44 ` Vincent Lefevre
2008-10-31 0:14 ` Mikael Magnusson
2008-10-31 9:44 ` Oliver Kiddle
2 siblings, 0 replies; 102+ messages in thread
From: Vincent Lefevre @ 2008-10-30 23:44 UTC (permalink / raw)
To: Zsh workers
[-- Attachment #1: Type: text/plain, Size: 627 bytes --]
On 2008-10-30 22:13:02 +0000, Peter Stephenson wrote:
> "Mikael Magnusson" wrote:
> > % function a() { a }
> > % a
> > zsh: segmentation fault zsh
>
> Right. I get:
>
> pws-pc% function a() { a }
> pws-pc% a
> a: maximum nested function level reached
With zsh 4.3.6 under Mac OS X 10.4.11, it crashes.
I've attached ktrace information. If I understand correctly, it reaches
the default 8-MB stack size.
--
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)
[-- Attachment #2: kdump.out.gz --]
[-- Type: application/x-gunzip, Size: 17890 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: another bug: zsh_directory_name
2008-10-30 22:13 ` Peter Stephenson
2008-10-30 23:44 ` Vincent Lefevre
@ 2008-10-31 0:14 ` Mikael Magnusson
2008-10-31 9:44 ` Oliver Kiddle
2 siblings, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-10-31 0:14 UTC (permalink / raw)
To: Peter Stephenson; +Cc: Zsh workers, pws
2008/10/30 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> "Mikael Magnusson" wrote:
>> 2008/10/30 Oliver Kiddle <okiddle@yahoo.co.uk>:
>> > Just thought I'd check that the case of calling this new ~[...] stuff
>> > recursively had been handled. The following caused an instant seg fault:
>> >
>> > zsh_directory_name() {
>> > echo ~[]
>> > }
>>
>> I don't think this has anything to do with ~[],
>> % function a() { a }
>> % a
>> zsh: segmentation fault zsh
>
> Right. I get:
>
> pws-pc% function a() { a }
> pws-pc% a
> a: maximum nested function level reached
I get this when compiling with -O0 -ggdb3, and the segfault when
compiling with -O3 and no -g. Didn't try other combinations.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: another bug: zsh_directory_name
2008-10-30 22:13 ` Peter Stephenson
2008-10-30 23:44 ` Vincent Lefevre
2008-10-31 0:14 ` Mikael Magnusson
@ 2008-10-31 9:44 ` Oliver Kiddle
2008-10-31 9:58 ` Peter Stephenson
2 siblings, 1 reply; 102+ messages in thread
From: Oliver Kiddle @ 2008-10-31 9:44 UTC (permalink / raw)
To: Zsh workers
Peter Stephenson wrote:
> so the problem is that the max function depth is too large on those
> systems (or, equivalently, the resource limits are too small).
Probably so but it should be noted that I get the seg fault on systems
with 4.3.9 where older versions of zsh print the "maximum nested function
level reached" message. Has some change meant that more stack space is
being used than before when calling functions recursively?
Oliver
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: another bug: zsh_directory_name
2008-10-31 9:44 ` Oliver Kiddle
@ 2008-10-31 9:58 ` Peter Stephenson
0 siblings, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-10-31 9:58 UTC (permalink / raw)
To: Zsh workers
Oliver Kiddle wrote:
> Peter Stephenson wrote:
> > so the problem is that the max function depth is too large on those
> > systems (or, equivalently, the resource limits are too small).
>
> Probably so but it should be noted that I get the seg fault on systems
> with 4.3.9 where older versions of zsh print the "maximum nested function
> level reached" message. Has some change meant that more stack space is
> being used than before when calling functions recursively?
The function stack debugging information is now larger.
It also runs a dupstring() at each level to save the name of the file
from which the function was loaded; although this doesn't affect the
stack, it is quite inefficent, but I think the "shfunc" structure can be
removed at any time so it's not easy to do better.
We can reduce the maximum function depth if it's getting flaky on too
many systems.
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: another bug: zsh_directory_name
2008-10-30 21:20 ` another bug: zsh_directory_name Oliver Kiddle
2008-10-30 21:26 ` Mikael Magnusson
@ 2008-10-30 22:16 ` Peter Stephenson
1 sibling, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-10-30 22:16 UTC (permalink / raw)
To: Zsh workers
Oliver Kiddle wrote:
> Out of interest what are people using this for. It seems like a nice
> idea, but I've yet to think of an actual use that can't be done with a
> normal named directory. Transformations on the current $PWD seem to be
> one logical option - ${PWD/src/include} for instance. Or an easier syntax
> for (../)#
I'm using it for simplified hierarchies. ~[p] means my normal Perforce
client, ~[p1] my alternative one, so ~[p:u] is
/home/pws/perforce/uwb/main/fw/src while ~[p1:u] is
/home/pws/perforcel1/uwb/main/fw/src, and so on. In other words, I can
replace multiple chunks in the directory path very briefly; I could only
do that with static names by naming every possible combination.
See my previous post for how I'm doing completion on these.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* PATCH: bug with hash builtin
@ 2008-10-31 11:10 ` Oliver Kiddle
2008-10-31 21:07 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Oliver Kiddle @ 2008-10-31 11:10 UTC (permalink / raw)
To: Zsh workers
This was found by coverity:
hash =
causes a seg fault.
A patch is below. I'm not sure about the error message because it isn't
really an assignment but I can't think of anything better. Does anyone
think it would be better to avoid the use of the continue statement here
at the cost of having to further indent quite a few lines.
Oliver
Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.215
diff -u -r1.215 builtin.c
--- Src/builtin.c 30 Oct 2008 19:52:44 -0000 1.215
+++ Src/builtin.c 31 Oct 2008 11:00:22 -0000
@@ -3262,7 +3262,7 @@
}
queue_signals();
- while (*argv) {
+ for (;*argv;++argv) {
void *hn;
if (OPT_ISSET(ops,'m')) {
/* with the -m option, treat the argument as a glob pattern
*/
@@ -3275,7 +3275,12 @@
zwarnnam(name, "bad pattern : %s", *argv);
returnval = 1;
}
- } else if ((asg = getasg(*argv)) && asg->value) {
+ continue;
+ }
+ if (!(asg = getasg(*argv))) {
+ zwarnnam(name, "bad assignment");
+ returnval = 1;
+ } else if (asg->value) {
if(isset(RESTRICTED)) {
zwarnnam(name, "restricted: %s", asg->value);
returnval = 1;
@@ -3323,7 +3328,6 @@
ht->printnode(hn, 0);
} else if(OPT_ISSET(ops,'v'))
ht->printnode(hn, 0);
- argv++;
}
unqueue_signals();
return returnval;
^ permalink raw reply [flat|nested] 102+ messages in thread
* PATCH: edit-command-line with spaces in EDITOR
@ 2008-12-16 15:38 Clint Adams
2008-12-16 17:07 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Clint Adams @ 2008-12-16 15:38 UTC (permalink / raw)
To: zsh-workers
Apparently people want to set EDITOR to things like
"connect-emacs editor -t".
Index: Functions/Zle/edit-command-line
===================================================================
RCS file: /cvsroot/zsh/zsh/Functions/Zle/edit-command-line,v
retrieving revision 1.6
diff -u -r1.6 edit-command-line
--- Functions/Zle/edit-command-line 11 Jul 2008 19:12:24 -0000 1.6
+++ Functions/Zle/edit-command-line 16 Dec 2008 15:37:21 -0000
@@ -10,7 +10,7 @@
print -R - "$PREBUFFER$BUFFER" >$tmpfile
exec </dev/tty
-${VISUAL:-${EDITOR:-vi}} $tmpfile
+${=${VISUAL:-${EDITOR:-vi}}} $tmpfile
print -Rz - "$(<$tmpfile)"
command rm -f $tmpfile
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 15:38 PATCH: edit-command-line with spaces in EDITOR Clint Adams
@ 2008-12-16 17:07 ` Mikael Magnusson
2008-12-16 19:22 ` Peter Stephenson
` (2 more replies)
0 siblings, 3 replies; 102+ messages in thread
From: Mikael Magnusson @ 2008-12-16 17:07 UTC (permalink / raw)
To: zsh-workers
2008/12/16 Clint Adams <clint@zsh.org>:
> Apparently people want to set EDITOR to things like
> "connect-emacs editor -t".
I'm not convinced this is a good idea. If the binary or path to it
contains a space, you just broke it. It's probably better to just tell
people to make a short wrapper script. Also, if some apps parse spaces
in EDITOR and others don't, then you effectively have to _both_ keep
the path+binary space-free and not give any arguments. That seems
suboptimal. FWIW git interprets spaces in EDITOR but... AFAIK it has
always been the name of a binary and nothing more.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 17:07 ` Mikael Magnusson
@ 2008-12-16 19:22 ` Peter Stephenson
2008-12-16 19:27 ` Mikael Magnusson
2008-12-16 21:45 ` Clint Adams
2 siblings, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2008-12-16 19:22 UTC (permalink / raw)
To: zsh-workers
"Mikael Magnusson" wrote:
> 2008/12/16 Clint Adams <clint@zsh.org>:
> > Apparently people want to set EDITOR to things like
> > "connect-emacs editor -t".
>
> I'm not convinced this is a good idea. If the binary or path to it
> contains a space, you just broke it.
A style might be useful, but I don't know what the best default is.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 17:07 ` Mikael Magnusson
2008-12-16 19:22 ` Peter Stephenson
@ 2008-12-16 19:27 ` Mikael Magnusson
2008-12-16 20:51 ` Richard Hartmann
2008-12-16 21:45 ` Clint Adams
2 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-12-16 19:27 UTC (permalink / raw)
To: zsh-workers
2008/12/16 Mikael Magnusson <mikachu@gmail.com>:
> 2008/12/16 Clint Adams <clint@zsh.org>:
>> Apparently people want to set EDITOR to things like
>> "connect-emacs editor -t".
>
> I'm not convinced this is a good idea. If the binary or path to it
> contains a space, you just broke it. It's probably better to just tell
> people to make a short wrapper script. Also, if some apps parse spaces
> in EDITOR and others don't, then you effectively have to _both_ keep
> the path+binary space-free and not give any arguments. That seems
> suboptimal. FWIW git interprets spaces in EDITOR but... AFAIK it has
> always been the name of a binary and nothing more.
I had a feeling I was forgetting something when I was typing that,
just realized what it was;
% EDITOR="connect-emacs editor -t"
% $EDITOR myfile
zsh: command not found: connect-emacs editor -t
I guess you can still type $=EDITOR though...
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 19:27 ` Mikael Magnusson
@ 2008-12-16 20:51 ` Richard Hartmann
0 siblings, 0 replies; 102+ messages in thread
From: Richard Hartmann @ 2008-12-16 20:51 UTC (permalink / raw)
To: Mikael Magnusson; +Cc: zsh-workers
On Tue, Dec 16, 2008 at 20:27, Mikael Magnusson <mikachu@gmail.com> wrote:
> % EDITOR="connect-emacs editor -t"
> % $EDITOR myfile
> zsh: command not found: connect-emacs editor -t
I do that not often, but sometimes. Of course, I don't use spaces in $EDITOR
either. While one can argue that it's the user's duty to ensure a non-borked
config, the tools should not make it too easy to create broken & hard to
debug effects.
Personally, I would vote for extending the docs on why spaces in $EDITOR
can not be supported and disallowing it.
An alternative would be to have an option, but that seems like overkill, to
me.
RIchard
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 17:07 ` Mikael Magnusson
2008-12-16 19:22 ` Peter Stephenson
2008-12-16 19:27 ` Mikael Magnusson
@ 2008-12-16 21:45 ` Clint Adams
2008-12-16 22:31 ` Mikael Magnusson
2008-12-17 4:04 ` Bart Schaefer
2 siblings, 2 replies; 102+ messages in thread
From: Clint Adams @ 2008-12-16 21:45 UTC (permalink / raw)
To: Mikael Magnusson; +Cc: zsh-workers
On Tue, Dec 16, 2008 at 06:07:50PM +0100, Mikael Magnusson wrote:
> I'm not convinced this is a good idea. If the binary or path to it
> contains a space, you just broke it. It's probably better to just tell
> people to make a short wrapper script. Also, if some apps parse spaces
> in EDITOR and others don't, then you effectively have to _both_ keep
> the path+binary space-free and not give any arguments. That seems
> suboptimal. FWIW git interprets spaces in EDITOR but... AFAIK it has
> always been the name of a binary and nothing more.
Are you aware of any applications which _do not_ split EDITOR on spaces
or is this all theoretical?
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 21:45 ` Clint Adams
@ 2008-12-16 22:31 ` Mikael Magnusson
2008-12-17 12:16 ` Romain Francoise
2008-12-17 4:04 ` Bart Schaefer
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2008-12-16 22:31 UTC (permalink / raw)
To: zsh-workers
2008/12/16 Clint Adams <clint@zsh.org>:
> On Tue, Dec 16, 2008 at 06:07:50PM +0100, Mikael Magnusson wrote:
>> I'm not convinced this is a good idea. If the binary or path to it
>> contains a space, you just broke it. It's probably better to just tell
>> people to make a short wrapper script. Also, if some apps parse spaces
>> in EDITOR and others don't, then you effectively have to _both_ keep
>> the path+binary space-free and not give any arguments. That seems
>> suboptimal. FWIW git interprets spaces in EDITOR but... AFAIK it has
>> always been the name of a binary and nothing more.
>
> Are you aware of any applications which _do not_ split EDITOR on spaces
> or is this all theoretical?
sudo, http://www.google.com/codesearch?hl=en&q=getenv.*EDITOR+show:PzdGoZtAqeI:G9wXYPpcdgc:AfyqUXSwU3U&sa=N&cd=9&ct=rc&cs_p=ftp://ftp.sunfreeware.com/pub/freeware/SOURCES/sudo-1.6.8p9.tar.gz&cs_f=sudo-1.6.8p9/visudo.c
tla, http://www.google.com/codesearch?hl=en&q=getenv.*EDITOR+show:6rdpA64V62I:9dG97f3kZF0:vHcDzWcDvPI&sa=N&cd=11&ct=rc&cs_p=http://freshmeat.net/redir/archrevctl/41124/url_tgz/tla-1.3.5.tar.gz&cs_f=tla-1.3.5/src/tla/libarch/editor.c
(at this point i would like to note there are a sad number of programs
that do sprintf(cmd, "%s '%s'", editor, file); system(cmd);)
ironically emacs,
% ALTERNATE_EDITOR="ls" emacsclient =emacs
emacsclient: can't find socket; have you started the server?
To start the server in Emacs, type "M-x server-start".
/usr/bin/emacs
% ALTERNATE_EDITOR="ls -l" emacsclient =emacs
emacsclient: can't find socket; have you started the server?
To start the server in Emacs, type "M-x server-start".
emacsclient: error executing alternate editor "ls -l"
So far, i have only found lots of strange code and programs that do
not split EDITOR, I have yet to find one that does (except for git
which i already knew about). To be fair, some of the "strange code"
does split it, but they also split filenames and run arbitrary code so
I don't think those count.
pam_unix in osx (i think?),
http://www.google.com/codesearch?hl=en&q=getenv.*EDITOR+show:uJDRNZUL70A:waEx2HuwGy4:ADkvXgRqXEk&sa=N&cd=107&ct=rc&cs_p=http://www.opensource.apple.com/darwinsource/tarballs/other/pam_modules-13.tar.gz&cs_f=pam_modules-13/pam_unix/pw_util.c
crontab in ditto,
http://www.google.com/codesearch?hl=en&q=getenv.*EDITOR+show:_00kONdzoCk:TrU12ZEvJTg:zznBZOnQrgA&sa=N&cd=111&ct=rc&cs_p=http://www.opensource.apple.com/darwinsource/tarballs/other/cron-12.2.tar.gz&cs_f=cron-12.2/crontab/crontab.c
here's one that purposely calls system() to split on spaces in EDITOR,
http://www.google.com/codesearch?hl=en&q=getenv.*EDITOR+show:X1tmsVlC7-g:7ffgu2ZLT7E:8ZS2TlXP2Xs&sa=N&cd=120&ct=rc&cs_p=ftp://153.19.33.250/software/shadow/shadow-4.0.18.1.tar.bz2&cs_f=shadow-4.0.18.1/src/vipw.c
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: PATCH: edit-command-line with spaces in EDITOR
2008-12-16 21:45 ` Clint Adams
2008-12-16 22:31 ` Mikael Magnusson
@ 2008-12-17 4:04 ` Bart Schaefer
1 sibling, 0 replies; 102+ messages in thread
From: Bart Schaefer @ 2008-12-17 4:04 UTC (permalink / raw)
To: zsh-workers
On Dec 16, 9:45pm, Clint Adams wrote:
}
} Are you aware of any applications which _do not_ split EDITOR on spaces
} or is this all theoretical?
I suspect that most applications do the equivalent of
eval "$EDITOR $1"
which accomplishes the splitting of $EDITOR at some security risk if the
environment is tainted.
Then the spaces-in-the-path problem is solvable by embedding quotes or
backslashes in the value of $EDITOR.
Zsh could meet the two extremes halfway by using ${(Q)${(z)EDITOR}}.
^ permalink raw reply [flat|nested] 102+ messages in thread
* zsh eats 100% CPU with completion in /
@ 2009-10-30 16:10 Frank Terbeck
2009-10-30 21:14 ` Mikael Magnusson
2009-10-30 21:29 ` Benjamin R. Haskell
0 siblings, 2 replies; 102+ messages in thread
From: Frank Terbeck @ 2009-10-30 16:10 UTC (permalink / raw)
To: zsh-workers
The subject is not entirely correct.
Zsh eats 100% CPU on a colleague's laptop if you do this:
% cd /
% cd ..
% ./<tab>
Happens with the most recent CVS HEAD, too.
That's on linux (debian stable, actually).
The problem is reproducible on his machine. And it doesn't happen if
you do this:
% cd /
% ./<tab>
The 'cd ..' is required.
I cannot reproduce the problem on my system with exactly the same
configuration and with exactly the same zsh version. And I am in fact
using the very same OS (debian stable), too.
As you can imagine, I am quite puzzled and I don't know how to tackle
this.
Any hints or suggestions are highly welcome.
Regards, Frank
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-30 16:10 zsh eats 100% CPU with completion in / Frank Terbeck
@ 2009-10-30 21:14 ` Mikael Magnusson
2009-10-30 21:25 ` Mikael Magnusson
2009-10-30 21:33 ` Mikael Magnusson
2009-10-30 21:29 ` Benjamin R. Haskell
1 sibling, 2 replies; 102+ messages in thread
From: Mikael Magnusson @ 2009-10-30 21:14 UTC (permalink / raw)
To: zsh-workers
2009/10/30 Frank Terbeck <ft@bewatermyfriend.org>:
> The subject is not entirely correct.
>
> Zsh eats 100% CPU on a colleague's laptop if you do this:
> % cd /
> % cd ..
> % ./<tab>
>
> Happens with the most recent CVS HEAD, too.
> That's on linux (debian stable, actually).
>
> The problem is reproducible on his machine. And it doesn't happen if
> you do this:
> % cd /
> % ./<tab>
>
> The 'cd ..' is required.
>
> I cannot reproduce the problem on my system with exactly the same
> configuration and with exactly the same zsh version. And I am in fact
> using the very same OS (debian stable), too.
>
> As you can imagine, I am quite puzzled and I don't know how to tackle
> this.
>
> Any hints or suggestions are highly welcome.
I can reproduce this, but not under gdb, and, curiously, not if I
first start another instance of zsh, ie:
%(1) zsh
%(2) cd /
%(2) cd ..
%(2) <tab> -> produces listing
%(2) exit
%(1) cd /
%(1) cd ..
%(1) <tab> -> hangs
Here's a backtrace without debug symbols while i rebuild:
#0 0x080d5cc0 in itype_end ()
#1 0x6fafd51f in docomplete () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#2 0x6faf91d0 in completecall () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#3 0x6fae935d in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#4 0x6fae9582 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#5 0x6fae9b89 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#6 0x6faebbf7 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#7 0x08083df0 in zleentry ()
#8 0x08087a53 in inputline ()
#9 0x08087c88 in ingetc ()
#10 0x08082958 in ihgetc ()
#11 0x08090fc6 in zshlex ()
#12 0x080b30aa in parse_event ()
#13 0x08085004 in loop ()
#14 0x08086f2e in zsh_main ()
#15 0x080554b2 in main ()
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-30 21:14 ` Mikael Magnusson
@ 2009-10-30 21:25 ` Mikael Magnusson
2009-10-30 21:33 ` Mikael Magnusson
1 sibling, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2009-10-30 21:25 UTC (permalink / raw)
To: zsh-workers
2009/10/30 Mikael Magnusson <mikachu@gmail.com>:
> 2009/10/30 Frank Terbeck <ft@bewatermyfriend.org>:
>> The subject is not entirely correct.
>>
>> Zsh eats 100% CPU on a colleague's laptop if you do this:
>> % cd /
>> % cd ..
>> % ./<tab>
>>
>> Happens with the most recent CVS HEAD, too.
>> That's on linux (debian stable, actually).
>>
>> The problem is reproducible on his machine. And it doesn't happen if
>> you do this:
>> % cd /
>> % ./<tab>
>>
>> The 'cd ..' is required.
>>
>> I cannot reproduce the problem on my system with exactly the same
>> configuration and with exactly the same zsh version. And I am in fact
>> using the very same OS (debian stable), too.
>>
>> As you can imagine, I am quite puzzled and I don't know how to tackle
>> this.
>>
>> Any hints or suggestions are highly welcome.
>
> I can reproduce this, but not under gdb.
I also cannot reproduce it in xterm, while it happens in rxvt-unicode.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-30 21:14 ` Mikael Magnusson
2009-10-30 21:25 ` Mikael Magnusson
@ 2009-10-30 21:33 ` Mikael Magnusson
2009-10-31 20:00 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-10-30 21:33 UTC (permalink / raw)
To: zsh-workers
2009/10/30 Mikael Magnusson <mikachu@gmail.com>:
> 2009/10/30 Frank Terbeck <ft@bewatermyfriend.org>:
>> The subject is not entirely correct.
>>
>> Zsh eats 100% CPU on a colleague's laptop if you do this:
>> % cd /
>> % cd ..
>> % ./<tab>
>>
>> Happens with the most recent CVS HEAD, too.
>> That's on linux (debian stable, actually).
>>
>> The problem is reproducible on his machine. And it doesn't happen if
>> you do this:
>> % cd /
>> % ./<tab>
>>
>> The 'cd ..' is required.
>>
>> I cannot reproduce the problem on my system with exactly the same
>> configuration and with exactly the same zsh version. And I am in fact
>> using the very same OS (debian stable), too.
>>
>> As you can imagine, I am quite puzzled and I don't know how to tackle
>> this.
>>
>> Any hints or suggestions are highly welcome.
>
> I can reproduce this, but not under gdb, and, curiously, not if I
> first start another instance of zsh, ie:
> %(1) zsh
> %(2) cd /
> %(2) cd ..
> %(2) <tab> -> produces listing
> %(2) exit
> %(1) cd /
> %(1) cd ..
> %(1) <tab> -> hangs
>
> Here's a backtrace without debug symbols while i rebuild:
And here's a real one, and stepping until it seems to repeat. The /*
Probably shouldn't happen */ comment looks particularly suspicious.
(gdb) bt
#0 mb_metacharinit () at utils.c:448
#1 0x080d378c in itype_end (ptr=0x830a100 "", itype=128, once=1) at
utils.c:3310
#2 0x6fa39a38 in get_comp_string () at zle_tricky.c:1450
#3 0x6fa372bc in docomplete (lst=4) at zle_tricky.c:659
#4 0x6fa3675f in expandorcomplete (args=0x6fa4cd40) at zle_tricky.c:315
#5 0x6fa36302 in completecall (args=0x6fa4cd40) at zle_tricky.c:208
#6 0x6fa25fcd in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#7 0x6fa261f2 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#8 0x6fa267f9 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#9 0x6fa28867 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
#10 0x08086cda in zleentry (cmd=1) at init.c:1304
#11 0x08087b93 in inputline ()
#12 0x08087dc8 in ingetc ()
#13 0x0807d595 in ihgetc () at hist.c:263
#14 0x08091c26 in zshlex ()
#15 0x080ab108 in parse_event () at parse.c:451
#16 0x08084209 in loop (toplevel=1, justonce=0) at init.c:132
#17 0x08087053 in zsh_main (argc=1, argv=0x77c043c4) at init.c:1454
#18 0x08055522 in main ()
(gdb) s
449 }
(gdb)
itype_end (ptr=0x830a100 "", itype=128, once=1) at utils.c:3311
3311 while (*ptr) {
(gdb)
3376 return (char *)ptr;
(gdb)
3377 }
(gdb)
get_comp_string () at zle_tricky.c:1451
1451 nnb = tt + nclen;
(gdb)
1452 tt += nclen;
(gdb)
1439 for (tt = s; tt < s + zlemetacs_qsub - wb;) {
(gdb)
1440 if (*tt == Inbrack) {
(gdb)
1445 } else if (i && *tt == Outbrack) {
(gdb)
1449 int nclen = MB_METACHARLEN(tt);
(gdb)
mb_metacharlenconv (s=0x830a100 "", wcp=0x0) at utils.c:4254
4254 if (!isset(MULTIBYTE)) {
(gdb)
4270 if (itok(*s)) {
(gdb)
4276 return mb_metacharlenconv_r(s, wcp, &mb_shiftstate);
(gdb)
mb_metacharlenconv_r (s=0x830a100 "", wcp=0x0, mbsp=0x80f0ccc) at utils.c:4205
4205 size_t ret = MB_INVALID;
(gdb)
4210 for (ptr = s; *ptr; ) {
(gdb)
4229 if (wcp)
(gdb)
4232 memset(mbsp, 0, sizeof(*mbsp));
(gdb)
4233 if (ptr > s) {
(gdb)
4236 return 0; /* Probably shouldn't happen */
(gdb)
4237 }
(gdb)
mb_metacharlenconv (s=0x830a100 "", wcp=0x0) at utils.c:4277
4277 }
(gdb)
get_comp_string () at zle_tricky.c:1450
1450 if (itype_end(tt, IIDENT, 1) == tt)
(gdb)
itype_end (ptr=0x830a100 "", itype=128, once=1) at utils.c:3308
3308 if (isset(MULTIBYTE) &&
(gdb)
3310 mb_metacharinit();
(gdb)
mb_metacharinit () at utils.c:448
448 memset(&mb_shiftstate, 0, sizeof(mb_shiftstate));
(gdb)
449 }
(gdb)
itype_end (ptr=0x830a100 "", itype=128, once=1) at utils.c:3311
3311 while (*ptr) {
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-10-30 21:33 ` Mikael Magnusson
@ 2009-10-31 20:00 ` Peter Stephenson
2009-10-31 22:43 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2009-10-31 20:00 UTC (permalink / raw)
To: zsh-workers
Mikael Magnusson wrote:
> 2009/10/30 Mikael Magnusson <mikachu@gmail.com>:
> > 2009/10/30 Frank Terbeck <ft@bewatermyfriend.org>:
> >> The subject is not entirely correct.
> >>
> >> Zsh eats 100% CPU on a colleague's laptop if you do this:
> >> % cd /
> >> % cd ..
> >> % ./<tab>
Thanks for the additional details.
So it appears something's going on in this loop in get_comp_string()
around line 1439 of zle_tricky.c:
for (tt = s; tt < s + zlemetacs_qsub - wb;) {
if (*tt == Inbrack) {
i++;
nb = nnb;
ne = tt;
tt++;
} else if (i && *tt == Outbrack) {
i--;
tt++;
} else {
int nclen = MB_METACHARLEN(tt);
if (itype_end(tt, IIDENT, 1) == tt)
nnb = tt + nclen;
tt += nclen;
}
}
It would therefore probably help to know what the various quantities are
in this loop: tt, s, zlemetacs_qsub and wb in particular, and also what
nclen is being returned. (You might need a debugging build to get these
out.) It seems plausible nclen is zero, which obviously causes a
problem. If eveything's in range (and unless we're going backwards the
loop test suggests it should be) this presumably means we encountered a
character we didn't like somewhere, so possibly the LC_CTYPE locale is
useful (I would guess it's UTF-8).
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-10-31 20:00 ` Peter Stephenson
@ 2009-10-31 22:43 ` Mikael Magnusson
2009-10-31 23:00 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-10-31 22:43 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/10/31 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> Mikael Magnusson wrote:
>> 2009/10/30 Mikael Magnusson <mikachu@gmail.com>:
>> > 2009/10/30 Frank Terbeck <ft@bewatermyfriend.org>:
>> >> The subject is not entirely correct.
>> >>
>> >> Zsh eats 100% CPU on a colleague's laptop if you do this:
>> >> % cd /
>> >> % cd ..
>> >> % ./<tab>
>
> Thanks for the additional details.
>
> So it appears something's going on in this loop in get_comp_string()
> around line 1439 of zle_tricky.c:
>
> for (tt = s; tt < s + zlemetacs_qsub - wb;) {
> if (*tt == Inbrack) {
> i++;
> nb = nnb;
> ne = tt;
> tt++;
> } else if (i && *tt == Outbrack) {
> i--;
> tt++;
> } else {
> int nclen = MB_METACHARLEN(tt);
> if (itype_end(tt, IIDENT, 1) == tt)
> nnb = tt + nclen;
> tt += nclen;
> }
> }
>
> It would therefore probably help to know what the various quantities are
> in this loop: tt, s, zlemetacs_qsub and wb in particular, and also what
> nclen is being returned. (You might need a debugging build to get these
> out.) It seems plausible nclen is zero, which obviously causes a
> problem. If eveything's in range (and unless we're going backwards the
> loop test suggests it should be) this presumably means we encountered a
> character we didn't like somewhere, so possibly the LC_CTYPE locale is
> useful (I would guess it's UTF-8).
Yes, my LC_CTYPE is en_US.UTF-8, here's a bt full
#0 0x6fae4a7b in get_comp_string () at zle_tricky.c:1439
nnb = 0x82d7248 ""
nb = 0x0
ne = 0x0
t0 = 34
tt0 = 34
i = 0
j = 0
k = 0
cp = 1
rd = 0
sl = 1
ocs = 0
ins = 0
oins = 0
ia = 0
parct = 0
varq = 0
ona = 0
qsub = 0
zlemetacs_qsub = 2
redirpos = 0
s = 0x82d7248 ""
tmp = 0x80ff770 "./"
p = 0x0
tt = 0x82d7248 ""
rdop = "\000\367\017\b\370\031\244wq\367\017\b\001\000\000\000h0\031\b"
linptr = 0x6ffed840 "./x"
u = 0x80ff772 ""
#1 0x6fae22bc in docomplete (lst=4) at zle_tricky.c:659
s = 0x41b6cedb "\201\303\031\301\v"
ol = 0x0
olst = 4
chl = 0
ne = 0
ocs = 2
ret = 0
dat = {1878464192, 2007243640}
active = 1
#2 0x6fae175f in expandorcomplete (args=0x6faf7d40) at zle_tricky.c:315
ret = 2007244248
#3 0x6fae1302 in completecall (args=0x6faf7d40) at zle_tricky.c:208
No locals.
and inside the last bit
(gdb) print zlemetacs_qsub
$3 = 2
get_comp_string () at zle_tricky.c:1450
1450 if (itype_end(tt, IIDENT, 1) == tt)
(gdb) print nclen
$7 = 0
(gdb) print tt
$8 = 0x82d7248 ""
As a curiousity, I couldn't reproduce the bug as my user now, but it
"worked" as root, or if I did the cd .. twice.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-10-31 22:43 ` Mikael Magnusson
@ 2009-10-31 23:00 ` Peter Stephenson
2009-11-01 1:50 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2009-10-31 23:00 UTC (permalink / raw)
To: zsh-workers
Mikael Magnusson wrote:
> zlemetacs_qsub = 2
> s = 0x82d7248 ""
> tt = 0x82d7248 ""
It looks like it's already gone wrong by that point; s is an empty
string while it thinks the cursor is 2 characters in (which appears to
be correct from the original completion; zlemetaline should show "./x",
which I think it must since that's what's in tmp at this point).
So somehow the command line string has been emptied incorrectly. I can
fix the symptom at this point but it doesn't look like the real bug. It
would be good to find out where s is being set in get_comp_string(). In
this simple case it should have come from line 1356 (the token type
should be STRING), in which case maybe the clwords array contains
something funny or clwpos is incorrect (it should be 0 from line 1306 or
1262, I'm not quite sure what the difference is).
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-31 23:00 ` Peter Stephenson
@ 2009-11-01 1:50 ` Mikael Magnusson
2009-11-01 18:31 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-01 1:50 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/1 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> Mikael Magnusson wrote:
>> zlemetacs_qsub = 2
>> s = 0x82d7248 ""
>> tt = 0x82d7248 ""
>
> It looks like it's already gone wrong by that point; s is an empty
> string while it thinks the cursor is 2 characters in (which appears to
> be correct from the original completion; zlemetaline should show "./x",
> which I think it must since that's what's in tmp at this point).
>
> So somehow the command line string has been emptied incorrectly. I can
> fix the symptom at this point but it doesn't look like the real bug. It
> would be good to find out where s is being set in get_comp_string(). In
> this simple case it should have come from line 1356 (the token type
> should be STRING), in which case maybe the clwords array contains
> something funny or clwpos is incorrect (it should be 0 from line 1306 or
> 1262, I'm not quite sure what the difference is).
>
> --
> Peter Stephenson <p.w.stephenson@ntlworld.com>
> Web page now at http://homepage.ntlworld.com/p.w.stephenson/
>
(gdb) break get_comp_string
Breakpoint 1 at 0x6fad5420: file zle_tricky.c, line 1067.
(gdb) c
Continuing.
Breakpoint 1, get_comp_string () at zle_tricky.c:1067
1067 int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;
(gdb) break 1346
Breakpoint 2 at 0x6fad64b5: file zle_tricky.c, line 1346.
(gdb) c
Continuing.
Breakpoint 2, get_comp_string () at zle_tricky.c:1346
1346 if (inwhat == IN_MATH)
(gdb) print inwhat
$1 = 0
(gdb) print t0
$3 = 34
(gdb) print ENDINPUT+0
$5 = 37
(gdb) print STRING+0
$7 = 34
(gdb) print clwpos
$8 = 0
(gdb) print clwords
$9 = (char **) 0x8144278
(gdb) print clwords[0]
$10 = 0x82fb758 ""
(gdb) s
1348 else if (!t0 || t0 == ENDINPUT) {
(gdb)
1354 } else if (t0 == STRING) {
(gdb)
1356 s = ztrdup(clwords[clwpos]);
(gdb)
1403 if (we > zlemetall)
(gdb) print s
$11 = 0x82fb7a0 ""
for fun i tried '(gdb) set zlemetacs_qsub=0' at the appropriate time
and broke out of the loop, got this
(eval):3: no such file or directory:
it is probably not helpful but you never know.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-01 1:50 ` Mikael Magnusson
@ 2009-11-01 18:31 ` Peter Stephenson
2009-11-01 19:33 ` Bart Schaefer
2009-11-01 21:12 ` Mikael Magnusson
0 siblings, 2 replies; 102+ messages in thread
From: Peter Stephenson @ 2009-11-01 18:31 UTC (permalink / raw)
To: zsh-workers
Mikael Magnusson wrote:
> (gdb) print clwpos
> $8 = 0
> (gdb) print clwords
> $9 = (char **) 0x8144278
> (gdb) print clwords[0]
> $10 = 0x82fb758 ""
Thanks---I think that means somehow it's got confused in the lexical
analyser, there's not much scope for more problems in get_comp_string().
That first word obviously ought to be "./x". Possibly something hasn't
been reset properly after the last operation; we've seen a couple of
things like that. We need to find out why the first
zshlex(), and hence gettok(), is setting tok to STRING but not tokstr to
"./x" (if it is, the problem is in get_comp_string() after all). It
might well be something quite simple.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-01 18:31 ` Peter Stephenson
@ 2009-11-01 19:33 ` Bart Schaefer
2009-11-01 21:12 ` Mikael Magnusson
1 sibling, 0 replies; 102+ messages in thread
From: Bart Schaefer @ 2009-11-01 19:33 UTC (permalink / raw)
To: zsh-workers
On Nov 1, 6:31pm, Peter Stephenson wrote:
}
} Thanks---I think that means somehow it's got confused in the lexical
} analyser, there's not much scope for more problems in get_comp_string().
} That first word obviously ought to be "./x".
Hmm. Somewhere in the completion system -- I forget now whether it's in
the C code (though I believe so) -- the string on the line is modified
to insert an "x" as a placeholder, which is then later deleted. I wonder
if something is going wrong there -- either the pointer isn't being
advanced properly when the "x" is inserted, or it's being decremented
incorrectly (or too often) when the "x" is removed?
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-01 18:31 ` Peter Stephenson
2009-11-01 19:33 ` Bart Schaefer
@ 2009-11-01 21:12 ` Mikael Magnusson
2009-11-01 22:20 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-01 21:12 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/1 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> Mikael Magnusson wrote:
>> (gdb) print clwpos
>> $8 = 0
>> (gdb) print clwords
>> $9 = (char **) 0x8144278
>> (gdb) print clwords[0]
>> $10 = 0x82fb758 ""
>
> Thanks---I think that means somehow it's got confused in the lexical
> analyser, there's not much scope for more problems in get_comp_string().
> That first word obviously ought to be "./x". Possibly something hasn't
> been reset properly after the last operation; we've seen a couple of
> things like that. We need to find out why the first
> zshlex(), and hence gettok(), is setting tok to STRING but not tokstr to
> "./x" (if it is, the problem is in get_comp_string() after all). It
> might well be something quite simple.
I didn't spend too much thinking about when to print variable values
here, but if you point to somewhere I can do a bt full there.
I did the break after typing ./ and before pressing tab.
(gdb) break zshlex
Breakpoint 1 at 0x8091bf6
(gdb) break get_comp_string
Breakpoint 2 at 0x6fa7b420: file zle_tricky.c, line 1067.
(gdb) c
Continuing.
Breakpoint 1, 0x08091bf6 in zshlex ()
(gdb) bt full
#0 0x08091bf6 in zshlex ()
No symbol table info available.
#1 0x080927ab in ctxtlex ()
No symbol table info available.
#2 0x6fa808a4 in doexpandhist () at zle_tricky.c:2662
ol = 0x6fb89740 "./"
oll = 2
ocs = 2
ne = 0
err = 1873283000
ona = 0
#3 0x6fa7a144 in docomplete (lst=4) at zle_tricky.c:624
s = 0x41b6cedb "\201\303\031\301\v"
ol = 0x6fa8bff4 "p\233\003"
olst = 4
chl = 0
ne = 0
ocs = 1873222770
ret = 0
dat = {1877915328, 2006971800}
active = 1
#4 0x6fa7975f in expandorcomplete (args=0x6fa8fd40) at zle_tricky.c:315
ret = 2006972408
#5 0x6fa79302 in completecall (args=0x6fa8fd40) at zle_tricky.c:208
No locals.
#6 0x6fa68fcd in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#7 0x6fa691f2 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#8 0x6fa697f9 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#9 0x6fa6b867 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#10 0x08086cda in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x779ff8f4 "\004\071\017\b\214\070\017\b\a"
#11 0x08087b93 in inputline ()
No symbol table info available.
#12 0x08087dc8 in ingetc ()
No symbol table info available.
#13 0x0807d595 in ihgetc () at hist.c:263
c = 0
#14 0x08091c26 in zshlex ()
No symbol table info available.
#15 0x080ab108 in parse_event () at parse.c:451
No locals.
#16 0x08084209 in loop (toplevel=1, justonce=0) at init.c:132
prog = 0x6fb89ab0
err = 50
non_empty = 1
#17 0x08087053 in zsh_main (argc=1, argv=0x779ffaf4) at init.c:1454
t = 0x779ffaf8
t0 = 158
#18 0x08055522 in main ()
No symbol table info available.
(gdb) s
Single stepping until exit from function zshlex,
which has no line number information.
ihgetc () at hist.c:263
263 int c = ingetc();
(gdb)
265 qbang = 0;
(gdb)
266 if (!stophist && !(inbufflags & INP_ALIAS)) {
(gdb)
275 if ((inbufflags & INP_HIST) && !stophist) {
(gdb)
286 } else if (stophist || (inbufflags & INP_ALIAS))
(gdb)
294 qbang = c == bangchar && (stophist < 2);
(gdb)
295 hwaddc(c);
(gdb)
ihwaddc (c=46) at hist.c:212
212 if (chline && !(errflag || lexstop)) {
(gdb)
214 if (c == bangchar && stophist < 2 && qbang)
(gdb)
221 *hptr++ = c;
(gdb)
224 if (hptr - chline >= hlinesz) {
(gdb)
231 }
(gdb)
ihgetc () at hist.c:296
296 addtoline(c);
(gdb)
iaddtoline (c=46) at hist.c:242
242 if (!expanding || lexstop)
(gdb)
244 if (qbang && c == bangchar && stophist < 2) {
(gdb)
248 if (excs > zlemetacs) {
(gdb)
249 excs += 1 + inbufct - exlast;
(gdb)
250 if (excs < zlemetacs)
(gdb)
255 exlast = inbufct;
(gdb)
256 zleentry(ZLE_CMD_ADD_TO_LINE, itok(c) ? ztokens[c - Pound] : c);
(gdb)
zleentry (cmd=2) at init.c:1273
1273 char *ret = NULL;
(gdb)
1277 VA_START(ap, cmd);
(gdb)
1282 switch (zle_load_state) {
(gdb)
1304 ret = zle_entry_ptr(cmd, ap);
(gdb)
1306 cmd = -1;
(gdb)
1315 switch (cmd) {
(gdb)
1353 return ret;
(gdb)
1354 }
(gdb)
iaddtoline (c=46) at hist.c:257
257 }
(gdb)
ihgetc () at hist.c:298
298 return c;
(gdb)
299 }
(gdb)
0x08091c26 in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
ihwbegin (offset=-1) at hist.c:1300
1300 if (stophist == 2)
(gdb)
1302 if (chwordpos%2)
(gdb)
1307 if ((inbufflags & INP_ALIAS) && !(inbufflags & INP_HIST))
(gdb)
1310 hwgetword = -1;
(gdb)
1311 chwords[chwordpos++] = hptr - chline + offset;
(gdb)
1312 }
(gdb)
0x08091dea in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
0x080927ab in ctxtlex ()
(gdb)
Single stepping until exit from function ctxtlex,
which has no line number information.
doexpandhist () at zle_tricky.c:2663
2663 } while (tok != ENDINPUT && tok != LEXERR);
(gdb)
2662 ctxtlex();
(gdb)
Breakpoint 1, 0x08091bf6 in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
ihgetc () at hist.c:263
263 int c = ingetc();
(gdb)
265 qbang = 0;
(gdb)
266 if (!stophist && !(inbufflags & INP_ALIAS)) {
(gdb)
275 if ((inbufflags & INP_HIST) && !stophist) {
(gdb)
286 } else if (stophist || (inbufflags & INP_ALIAS))
(gdb)
294 qbang = c == bangchar && (stophist < 2);
(gdb)
295 hwaddc(c);
(gdb)
ihwaddc (c=32) at hist.c:212
212 if (chline && !(errflag || lexstop)) {
(gdb)
231 }
(gdb)
ihgetc () at hist.c:296
296 addtoline(c);
(gdb)
iaddtoline (c=32) at hist.c:242
242 if (!expanding || lexstop)
(gdb)
257 }
(gdb)
ihgetc () at hist.c:298
298 return c;
(gdb)
299 }
(gdb)
0x08091c26 in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
0x080927ab in ctxtlex ()
(gdb)
Single stepping until exit from function ctxtlex,
which has no line number information.
doexpandhist () at zle_tricky.c:2663
2663 } while (tok != ENDINPUT && tok != LEXERR);
(gdb)
2664 while (!lexstop)
(gdb)
2669 err = errflag;
(gdb)
2670 noerrs = ne;
(gdb)
2671 noaliases = ona;
(gdb)
2672 strinend();
(gdb)
strinend () at hist.c:799
799 hend(NULL);
(gdb)
hend (prog=0x0) at hist.c:1144
1144 LinkList hookargs = newlinklist();
(gdb)
1145 int flag, save = 1, hookret, stack_pos = histsave_stack_pos;
(gdb)
1150 queue_signals();
(gdb)
1151 if (histdone & HISTFLAG_SETTY)
(gdb)
1153 if (!(histactive & HA_NOINC))
(gdb)
1155 if (histactive & HA_NOINC) {
(gdb)
1156 zfree(chline, hlinesz);
(gdb)
1157 zfree(chwords, chwordlen*sizeof(short));
(gdb)
1158 chline = NULL;
(gdb)
1159 chwords = NULL;
(gdb)
1160 histactive = 0;
(gdb)
1161 unqueue_signals();
(gdb)
1162 return 1;
(gdb)
1287 }
(gdb)
strinend () at hist.c:801
801 strin--;
(gdb)
802 isfirstch = 1;
(gdb)
803 histdone = 0;
(gdb)
804 }
(gdb)
doexpandhist () at zle_tricky.c:2673
2673 inpop();
(gdb)
2674 zleparse = 0;
(gdb)
2675 lexrestore();
(gdb)
2676 expanding = 0;
(gdb)
2678 if (!err) {
(gdb)
2679 zlemetacs = excs;
(gdb)
2680 if (strcmp(zlemetaline, ol)) {
(gdb)
2692 strcpy(zlemetaline, ol);
(gdb)
2693 zlemetall = oll;
(gdb)
2694 zlemetacs = ocs;
(gdb)
2695 unmetafy_line();
(gdb)
unmetafy_line () at zle_tricky.c:978
978 zlemetaline[zlemetall] = '\0';
(gdb)
979 zleline = stringaszleline(zlemetaline, zlemetacs, &zlell,
&linesz, &zlecs);
(gdb)
981 free(zlemetaline);
(gdb)
982 zlemetaline = NULL;
(gdb)
987 CCRIGHT();
(gdb)
988 }
(gdb)
doexpandhist () at zle_tricky.c:2697
2697 popheap();
(gdb)
2699 return 0;
(gdb)
2700 }
(gdb)
docomplete (lst=4) at zle_tricky.c:629
629 metafy_line();
(gdb)
metafy_line () at zle_tricky.c:960
960 zlemetaline = zlelineasstring(zleline, zlell, zlecs,
(gdb)
962 metalinesz = zlemetall;
(gdb)
967 free(zleline);
(gdb)
968 zleline = NULL;
(gdb)
969 }
(gdb) print zlemetaline
$1 = 0x8223058 "./"
(gdb) s
docomplete (lst=4) at zle_tricky.c:631
631 ocs = zlemetacs;
(gdb) print ocs
$2 = 1873222770
(gdb) print zlemetacs
$3 = 2
(gdb) s
632 origline = dupstring(zlemetaline);
(gdb)
633 origcs = zlemetacs;
(gdb) print origline
$4 = 0x6fb89740 "./"
(gdb) s
634 origll = zlemetall;
(gdb) print origcs
$5 = 2
(gdb) s
635 if (!isfirstln && chline != NULL) {
(gdb) print origll
$6 = 2
(gdb) s
648 ol = NULL;
(gdb)
649 inwhat = IN_NOTHING;
(gdb)
650 zsfree(qipre);
(gdb)
651 qipre = ztrdup("");
(gdb)
652 zsfree(qisuf);
(gdb)
653 qisuf = ztrdup("");
(gdb)
654 zsfree(autoq);
(gdb)
655 autoq = NULL;
(gdb)
658 noerrs = 1;
(gdb)
659 s = get_comp_string();
(gdb)
Breakpoint 2, get_comp_string () at zle_tricky.c:1067
1067 int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;
(gdb) frame 1
#1 0x6fa7a2bc in docomplete (lst=4) at zle_tricky.c:659
659 s = get_comp_string();
(gdb) print s
$7 = 0x41b6cedb "\201\303\031\301\v"
(gdb) frame 0
#0 get_comp_string () at zle_tricky.c:1067
1067 int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;
(gdb) bt full
#0 get_comp_string () at zle_tricky.c:1067
t0 = 0
tt0 = 137351280
i = 2006971508
j = 2006971636
k = 2006971576
cp = 136458330
rd = 1103269876
sl = 136458330
ocs = 0
ins = 1103269876
oins = 1102143328
ia = 136458329
parct = 2006971560
varq = 1102499901
ona = 135251752
qsub = 2006971504
zlemetacs_qsub = 2006971540
redirpos = 2006971576
s = 0x6fb89740 "./"
tmp = 0x82fc830 "MMAND"
p = 0x0
tt = 0x779ff498
"\333ζA\364\277\250oZ0\"\bt\r\266A\b\365\237w</\250oX0\"\b
\241\302A\030"
rdop = "_0\"\b\030\364\237wY0\"\b\001\000\000\000\060\310/\b"
linptr = 0x0
u = 0x1 <Address 0x1 out of bounds>
#1 0x6fa7a2bc in docomplete (lst=4) at zle_tricky.c:659
s = 0x41b6cedb "\201\303\031\301\v"
ol = 0x0
olst = 4
---Type <return> to continue, or q <return> to quit---
chl = 0
ne = 0
ocs = 2
ret = 0
dat = {1877915328, 2006971800}
active = 1
#2 0x6fa7975f in expandorcomplete (args=0x6fa8fd40) at zle_tricky.c:315
ret = 2006972408
#3 0x6fa79302 in completecall (args=0x6fa8fd40) at zle_tricky.c:208
No locals.
#4 0x6fa68fcd in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#5 0x6fa691f2 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#6 0x6fa697f9 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#7 0x6fa6b867 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#8 0x08086cda in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x779ff8f4 "\004\071\017\b\214\070\017\b\a"
#9 0x08087b93 in inputline ()
No symbol table info available.
#10 0x08087dc8 in ingetc ()
No symbol table info available.
#11 0x0807d595 in ihgetc () at hist.c:263
c = 0
#12 0x08091c26 in zshlex ()
No symbol table info available.
#13 0x080ab108 in parse_event () at parse.c:451
No locals.
---Type <return> to continue, or q <return> to quit---
#14 0x08084209 in loop (toplevel=1, justonce=0) at init.c:132
prog = 0x6fb89ab0
err = 50
non_empty = 1
#15 0x08087053 in zsh_main (argc=1, argv=0x779ffaf4) at init.c:1454
t = 0x779ffaf8
t0 = 158
#16 0x08055522 in main ()
No symbol table info available.
(gdb) s
1068 int ona = noaliases;
(gdb)
1079 int qsub, zlemetacs_qsub = 0;
(gdb)
1086 char *s = NULL, *tmp, *p, *tt = NULL, rdop[20];
(gdb)
1091 freebrinfo(brbeg);
(gdb)
Breakpoint 1, 0x08091bf6 in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
nohw (c=-1) at hist.c:812
812 }
(gdb)
0x08091dea in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
0x080927ab in ctxtlex ()
(gdb)
Single stepping until exit from function ctxtlex,
which has no line number information.
get_comp_string () at zle_tricky.c:1166
1166 if (tok == LEXERR) {
(gdb)
1179 } else if (tok == ENVSTRING)
(gdb)
1181 if (tok == ENVARRAY) {
(gdb)
1185 } else if (tok == INPAR)
(gdb)
1187 else if (tok == OUTPAR) {
(gdb)
1193 if (inredir && IS_REDIROP(tok)) {
(gdb)
1204 if (tok == DINPAR)
(gdb)
1208 if (tok == ENDINPUT)
(gdb)
1210 if ((ins && (tok == DOLOOP || tok == SEPER)) ||
(gdb)
1231 if (lincmd && (tok == STRING || tok == FOR || tok == FOREACH ||
(gdb)
1235 ins = (tok == REPEAT ? 2 : (tok != STRING));
(gdb)
1236 zsfree(cmdstr);
(gdb)
1237 cmdstr = ztrdup(tokstr);
(gdb)
1239 if (i != redirpos)
(gdb)
1242 if (!zleparse && !tt0) {
(gdb)
1244 tt = tokstr ? dupstring(tokstr) : NULL;
(gdb)
1246 if (isset(RCQUOTES) && tt) {
(gdb)
1247 char *tt1, *e = tt + zlemetacs - wb;
(gdb) print tokstr
$8 = 0x6fb89758 "."
(gdb) print tt
$9 = 0x6fb89778 "."
(gdb) s
1248 for (tt1 = tt; *tt1; tt1++) {
(gdb)
1249 if (*tt1 == Snull) {
(gdb)
1248 for (tt1 = tt; *tt1; tt1++) {
(gdb)
1258 if (addedx && tt)
(gdb)
1259 chuck(tt + zlemetacs - wb - qsub);
(gdb)
Breakpoint 1, 0x08091bf6 in zshlex ()
(gdb)
Single stepping until exit from function zshlex,
which has no line number information.
0x080927ab in ctxtlex ()
(gdb)
Single stepping until exit from function ctxtlex,
which has no line number information.
get_comp_string () at zle_tricky.c:1166
1166 if (tok == LEXERR) {
(gdb)
1179 } else if (tok == ENVSTRING)
(gdb)
1181 if (tok == ENVARRAY) {
(gdb)
1185 } else if (tok == INPAR)
(gdb)
1187 else if (tok == OUTPAR) {
(gdb)
1193 if (inredir && IS_REDIROP(tok)) {
(gdb)
1204 if (tok == DINPAR)
(gdb)
1208 if (tok == ENDINPUT)
(gdb)
1314 clwnum = (tt || !i) ? i : i - 1;
(gdb)
1315 zsfree(clwords[clwnum]);
(gdb) print clwnum
$10 = 1
(gdb) s
1316 clwords[clwnum] = NULL;
(gdb)
1317 t0 = tt0;
(gdb)
1318 if (ia) {
(gdb) print t0
$11 = 34
(gdb) s
1322 lincmd = cp;
(gdb)
1323 linredir = rd;
(gdb)
1325 strinend();
(gdb)
strinend () at hist.c:799
799 hend(NULL);
(gdb)
hend (prog=0x0) at hist.c:1144
1144 LinkList hookargs = newlinklist();
(gdb)
1145 int flag, save = 1, hookret, stack_pos = histsave_stack_pos;
(gdb)
1150 queue_signals();
(gdb)
1151 if (histdone & HISTFLAG_SETTY)
(gdb)
1153 if (!(histactive & HA_NOINC))
(gdb)
1155 if (histactive & HA_NOINC) {
(gdb)
1156 zfree(chline, hlinesz);
(gdb)
1157 zfree(chwords, chwordlen*sizeof(short));
(gdb) print chwords
$12 = (short int *) 0x0
(gdb) s
1158 chline = NULL;
(gdb)
1159 chwords = NULL;
(gdb)
1160 histactive = 0;
(gdb)
1161 unqueue_signals();
(gdb)
1162 return 1;
(gdb)
1287 }
(gdb)
strinend () at hist.c:801
801 strin--;
(gdb)
802 isfirstch = 1;
(gdb)
803 histdone = 0;
(gdb)
804 }
(gdb)
get_comp_string () at zle_tricky.c:1326
1326 inpop();
(gdb)
1327 errflag = zleparse = 0;
(gdb)
1328 if (parbegin != -1) {
(gdb)
1346 if (inwhat == IN_MATH)
(gdb)
1348 else if (!t0 || t0 == ENDINPUT) {
(gdb)
1354 } else if (t0 == STRING) {
(gdb) print s
$13 = 0x0
(gdb) print t0
$14 = 34
(gdb) bt full
#0 get_comp_string () at zle_tricky.c:1354
t0 = 34
tt0 = 34
i = 1
j = 0
k = 0
cp = 1
rd = 0
sl = 1
ocs = 0
ins = 0
oins = 0
ia = 0
parct = 0
varq = 0
ona = 0
qsub = 0
zlemetacs_qsub = 2
redirpos = 0
s = 0x0
tmp = 0x8223058 "./"
p = 0x0
tt = 0x6fb89778 "."
rdop = "\000\060\"\b\030\364\237wY0\"\b\001\000\000\000\060\310/\b"
linptr = 0x6fb89748 "./x"
u = 0x822305a ""
#1 0x6fa7a2bc in docomplete (lst=4) at zle_tricky.c:659
s = 0x41b6cedb "\201\303\031\301\v"
ol = 0x0
olst = 4
chl = 0
---Type <return> to continue, or q <return> to quit---
ne = 0
ocs = 2
ret = 0
dat = {1877915328, 2006971800}
active = 1
#2 0x6fa7975f in expandorcomplete (args=0x6fa8fd40) at zle_tricky.c:315
ret = 2006972408
#3 0x6fa79302 in completecall (args=0x6fa8fd40) at zle_tricky.c:208
No locals.
#4 0x6fa68fcd in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#5 0x6fa691f2 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#6 0x6fa697f9 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#7 0x6fa6b867 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#8 0x08086cda in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x779ff8f4 "\004\071\017\b\214\070\017\b\a"
#9 0x08087b93 in inputline ()
No symbol table info available.
#10 0x08087dc8 in ingetc ()
No symbol table info available.
#11 0x0807d595 in ihgetc () at hist.c:263
c = 0
#12 0x08091c26 in zshlex ()
No symbol table info available.
#13 0x080ab108 in parse_event () at parse.c:451
No locals.
#14 0x08084209 in loop (toplevel=1, justonce=0) at init.c:132
---Type <return> to continue, or q <return> to quit---
prog = 0x6fb89ab0
err = 50
non_empty = 1
#15 0x08087053 in zsh_main (argc=1, argv=0x779ffaf4) at init.c:1454
t = 0x779ffaf8
t0 = 158
#16 0x08055522 in main ()
No symbol table info available.
(gdb) s
1356 s = ztrdup(clwords[clwpos]);
(gdb) print clwpos
$15 = 0
(gdb) print clwords[0]
$16 = 0x82fb758 ""
(gdb) print clwords[1]
$17 = 0x0
(gdb) s
1403 if (we > zlemetall)
(gdb)
1405 tt = zlemetaline;
(gdb)
1406 if (tmp) {
(gdb) print tt
$18 = 0x6fb89748 "./x"
(gdb) print tmp
$19 = 0x8223058 "./"
(gdb) s
1407 zlemetaline = tmp;
(gdb)
1408 zlemetall = strlen(zlemetaline);
(gdb)
1410 if (t0 != STRING && inwhat != IN_MATH) {
(gdb) print zlemetall
$20 = 2
(gdb) print zlemetaline
$21 = 0x8223058 "./"
(gdb) s
1423 noaliases = ona;
(gdb)
1430 if (inwhat != IN_MATH) {
(gdb)
1431 char *nnb, *nb = NULL, *ne = NULL;
(gdb)
1433 i = 0;
(gdb)
1434 MB_METACHARINIT();
(gdb)
mb_metacharinit () at utils.c:448
448 memset(&mb_shiftstate, 0, sizeof(mb_shiftstate));
(gdb)
449 }
(gdb)
get_comp_string () at zle_tricky.c:1435
1435 if (itype_end(s, IIDENT, 1) == s)
(gdb)
itype_end (ptr=0x82fb7a0 "", itype=128, once=1) at utils.c:3308
3308 if (isset(MULTIBYTE) &&
(gdb)
3310 mb_metacharinit();
(gdb)
mb_metacharinit () at utils.c:448
448 memset(&mb_shiftstate, 0, sizeof(mb_shiftstate));
(gdb)
449 }
(gdb)
itype_end (ptr=0x82fb7a0 "", itype=128, once=1) at utils.c:3311
3311 while (*ptr) {
(gdb)
3376 return (char *)ptr;
(gdb)
3377 }
(gdb)
get_comp_string () at zle_tricky.c:1436
1436 nnb = s + MB_METACHARLEN(s);
(gdb)
mb_metacharlenconv (s=0x82fb7a0 "", wcp=0x0) at utils.c:4254
4254 if (!isset(MULTIBYTE)) {
(gdb)
4270 if (itok(*s)) {
(gdb)
4276 return mb_metacharlenconv_r(s, wcp, &mb_shiftstate);
(gdb)
mb_metacharlenconv_r (s=0x82fb7a0 "", wcp=0x0, mbsp=0x80f0ccc) at utils.c:4205
4205 size_t ret = MB_INVALID;
(gdb)
4210 for (ptr = s; *ptr; ) {
(gdb)
4229 if (wcp)
(gdb) print s
$22 = 0x82fb7a0 ""
(gdb) s
4232 memset(mbsp, 0, sizeof(*mbsp));
(gdb)
4233 if (ptr > s) {
(gdb) print ptr
$23 = 0x82fb7a0 ""
(gdb) s
4236 return 0; /* Probably shouldn't happen */
(gdb) bt full
#0 mb_metacharlenconv_r (s=0x82fb7a0 "", wcp=0x0, mbsp=0x80f0ccc) at
utils.c:4236
ret = 4294967295
inchar = 0 '\000'
ptr = 0x82fb7a0 ""
wc = 238448 L'p'
#1 0x080d4e67 in mb_metacharlenconv (s=0x82fb7a0 "", wcp=0x0) at utils.c:4276
No locals.
#2 0x6fa7c97c in get_comp_string () at zle_tricky.c:1436
nnb = 0x779ff448 "H\227\270oZ0\"\by\227\270oz\227\270o\001"
nb = 0x0
ne = 0x0
t0 = 34
tt0 = 34
i = 0
j = 0
k = 0
cp = 1
rd = 0
sl = 1
ocs = 0
ins = 0
oins = 0
ia = 0
parct = 0
varq = 0
ona = 0
qsub = 0
zlemetacs_qsub = 2
redirpos = 0
s = 0x82fb7a0 ""
tmp = 0x8223058 "./"
---Type <return> to continue, or q <return> to quit---
p = 0x0
tt = 0x6fb89748 "./x"
rdop = "\000\060\"\b\030\364\237wY0\"\b\001\000\000\000\060\310/\b"
linptr = 0x6fb89748 "./x"
u = 0x822305a ""
#3 0x6fa7a2bc in docomplete (lst=4) at zle_tricky.c:659
s = 0x41b6cedb "\201\303\031\301\v"
ol = 0x0
olst = 4
chl = 0
ne = 0
ocs = 2
ret = 0
dat = {1877915328, 2006971800}
active = 1
#4 0x6fa7975f in expandorcomplete (args=0x6fa8fd40) at zle_tricky.c:315
ret = 2006972408
#5 0x6fa79302 in completecall (args=0x6fa8fd40) at zle_tricky.c:208
No locals.
#6 0x6fa68fcd in execzlefunc () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#7 0x6fa691f2 in zlecore () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#8 0x6fa697f9 in zleread () from /usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#9 0x6fa6b867 in zle_main_entry () from
/usr/local/lib/zsh/4.3.10-dev-1/zsh/zle.so
No symbol table info available.
#10 0x08086cda in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x779ff8f4 "\004\071\017\b\214\070\017\b\a"
#11 0x08087b93 in inputline ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#12 0x08087dc8 in ingetc ()
No symbol table info available.
#13 0x0807d595 in ihgetc () at hist.c:263
c = 0
#14 0x08091c26 in zshlex ()
No symbol table info available.
#15 0x080ab108 in parse_event () at parse.c:451
No locals.
#16 0x08084209 in loop (toplevel=1, justonce=0) at init.c:132
prog = 0x6fb89ab0
err = 50
non_empty = 1
#17 0x08087053 in zsh_main (argc=1, argv=0x779ffaf4) at init.c:1454
t = 0x779ffaf8
t0 = 158
#18 0x08055522 in main ()
No symbol table info available.
(gdb) s
4237 }
(gdb)
mb_metacharlenconv (s=0x82fb7a0 "", wcp=0x0) at utils.c:4277
4277 }
(gdb)
get_comp_string () at zle_tricky.c:1439
1439 for (tt = s; tt < s + zlemetacs_qsub - wb;) {
(gdb) print tt
$24 = 0x6fb89748 "./x"
(gdb) print s
$25 = 0x82fb7a0 ""
(gdb) print zlemetacs_qsub
$26 = 2
(gdb) print wb
$27 = 0
(and at this point the infinite loop starts)
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-01 21:12 ` Mikael Magnusson
@ 2009-11-01 22:20 ` Peter Stephenson
2009-11-02 0:57 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2009-11-01 22:20 UTC (permalink / raw)
To: zsh-workers
Mikael Magnusson wrote:
> I did the break after typing ./ and before pressing tab.
Nothing leapt out of that. "tokstr" seems to be coming back as "." and
then the "." gets removed, as far as I can see, but looking at it out of
context I might be wrong.
Might be worth checking zlemetacs as well as zlemetaline and zlemetall
on entry to get_comp_string() in case they're already wrong, otherwise
it's probably down in the bowels of gettok() where it puts together the
string it's returning. I suspect a counting error---one of the myriad
substractions to correct for all the things that need fixing on the way
through the position calculations is wrong.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-01 22:20 ` Peter Stephenson
@ 2009-11-02 0:57 ` Mikael Magnusson
2009-11-02 1:26 ` Mikael Magnusson
2009-11-02 10:06 ` Peter Stephenson
0 siblings, 2 replies; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-02 0:57 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/1 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> Mikael Magnusson wrote:
>> I did the break after typing ./ and before pressing tab.
>
> Nothing leapt out of that. "tokstr" seems to be coming back as "." and
> then the "." gets removed, as far as I can see, but looking at it out of
> context I might be wrong.
>
> Might be worth checking zlemetacs as well as zlemetaline and zlemetall
> on entry to get_comp_string() in case they're already wrong, otherwise
> it's probably down in the bowels of gettok() where it puts together the
> string it's returning. I suspect a counting error---one of the myriad
> substractions to correct for all the things that need fixing on the way
> through the position calculations is wrong.
Before I actually have to start thinking and looking at the code, does
this tell you anything? In the working trace, I didn't do the cd ..
(this is from break zshlex)
I tried breaking on gettok but it appears my modules don't have debug
symbols, I might get back in a while if anything interesting shows up there.
--- /tmp/working 2009-11-02 01:43:42.060760548 +0100
+++ /tmp/broken 2009-11-02 01:46:26.331314015 +0100
@@ -357,14 +357,6 @@
(gdb)
1248 for (tt1 = tt; *tt1; tt1++) {
(gdb)
-1249 if (*tt1 == Snull) {
-(gdb)
-1248 for (tt1 = tt; *tt1; tt1++) {
-(gdb)
-1249 if (*tt1 == Snull) {
-(gdb)
-1248 for (tt1 = tt; *tt1; tt1++) {
-(gdb)
1258 if (addedx && tt)
(gdb)
1259 chuck(tt + zlemetacs - wb - qsub);
@@ -496,7 +488,7 @@
get_comp_string () at zle_tricky.c:1435
1435 if (itype_end(s, IIDENT, 1) == s)
(gdb)
-itype_end (ptr=0x8192ca8 "./", itype=128, once=1) at utils.c:3308
+itype_end (ptr=0x82fb8c0 "", itype=128, once=1) at utils.c:3308
3308 if (isset(MULTIBYTE) &&
(gdb)
3310 mb_metacharinit();
@@ -506,55 +498,9 @@
(gdb)
449 }
(gdb)
-itype_end (ptr=0x8192ca8 "./", itype=128, once=1) at utils.c:3311
+itype_end (ptr=0x82fb8c0 "", itype=128, once=1) at utils.c:3311
3311 while (*ptr) {
(gdb)
-3313 int len = mb_metacharlenconv(ptr, &wc);
-(gdb)
-mb_metacharlenconv (s=0x8192ca8 "./", wcp=0x77991ac8) at utils.c:4254
-4254 if (!isset(MULTIBYTE)) {
-(gdb)
-4270 if (itok(*s)) {
-(gdb)
-4276 return mb_metacharlenconv_r(s, wcp, &mb_shiftstate);
-(gdb)
-mb_metacharlenconv_r (s=0x8192ca8 "./", wcp=0x77991ac8,
mbsp=0x80f0ccc) at utils.c:4205
-4205 size_t ret = MB_INVALID;
-(gdb)
-4210 for (ptr = s; *ptr; ) {
-(gdb)
-4211 if (*ptr == Meta) {
-(gdb)
-4216 inchar = *ptr;
-(gdb)
-4217 ptr++;
-(gdb)
-4218 ret = mbrtowc(&wc, &inchar, 1, mbsp);
-(gdb)
-4220 if (ret == MB_INVALID)
-(gdb)
-4222 if (ret == MB_INCOMPLETE)
-(gdb)
-4224 if (wcp)
-(gdb)
-4225 *wcp = wc;
-(gdb)
-4226 return ptr - s;
-(gdb)
-4237 }
-(gdb)
-mb_metacharlenconv (s=0x8192ca8 "./", wcp=0x77991ac8) at utils.c:4277
-4277 }
-(gdb)
-itype_end (ptr=0x8192ca8 "./", itype=128, once=1) at utils.c:3315
-3315 if (!len)
-(gdb)
-3318 if (wc == WEOF) {
-(gdb)
-3324 } else if (len == 1 && isascii(*ptr)) {
-(gdb)
-3326 if (!zistype(*ptr,itype))
-(gdb)
3376 return (char *)ptr;
(gdb)
3377 }
@@ -562,150 +508,30 @@
get_comp_string () at zle_tricky.c:1436
1436 nnb = s + MB_METACHARLEN(s);
(gdb)
-mb_metacharlenconv (s=0x8192ca8 "./", wcp=0x0) at utils.c:4254
+mb_metacharlenconv (s=0x82fb8c0 "", wcp=0x0) at utils.c:4254
4254 if (!isset(MULTIBYTE)) {
(gdb)
4270 if (itok(*s)) {
(gdb)
4276 return mb_metacharlenconv_r(s, wcp, &mb_shiftstate);
(gdb)
-mb_metacharlenconv_r (s=0x8192ca8 "./", wcp=0x0, mbsp=0x80f0ccc) at
utils.c:4205
+mb_metacharlenconv_r (s=0x82fb8c0 "", wcp=0x0, mbsp=0x80f0ccc) at utils.c:4205
4205 size_t ret = MB_INVALID;
(gdb)
4210 for (ptr = s; *ptr; ) {
(gdb)
-4211 if (*ptr == Meta) {
-(gdb)
-4216 inchar = *ptr;
+4229 if (wcp)
(gdb)
-4217 ptr++;
+4232 memset(mbsp, 0, sizeof(*mbsp));
(gdb)
-4218 ret = mbrtowc(&wc, &inchar, 1, mbsp);
+4233 if (ptr > s) {
(gdb)
-4220 if (ret == MB_INVALID)
-(gdb)
-4222 if (ret == MB_INCOMPLETE)
-(gdb)
-4224 if (wcp)
-(gdb)
-4226 return ptr - s;
+4236 return 0; /* Probably shouldn't happen */
(gdb)
4237 }
(gdb)
-mb_metacharlenconv (s=0x8192ca8 "./", wcp=0x0) at utils.c:4277
+mb_metacharlenconv (s=0x82fb8c0 "", wcp=0x0) at utils.c:4277
4277 }
(gdb)
get_comp_string () at zle_tricky.c:1439
1439 for (tt = s; tt < s + zlemetacs_qsub - wb;) {
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-02 0:57 ` Mikael Magnusson
@ 2009-11-02 1:26 ` Mikael Magnusson
2009-11-02 16:38 ` Peter Stephenson
2009-11-02 10:06 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-02 1:26 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/2 Mikael Magnusson <mikachu@gmail.com>:
> 2009/11/1 Peter Stephenson <p.w.stephenson@ntlworld.com>:
>> Mikael Magnusson wrote:
>>> I did the break after typing ./ and before pressing tab.
>>
>> Nothing leapt out of that. "tokstr" seems to be coming back as "." and
>> then the "." gets removed, as far as I can see, but looking at it out of
>> context I might be wrong.
>>
>> Might be worth checking zlemetacs as well as zlemetaline and zlemetall
>> on entry to get_comp_string() in case they're already wrong, otherwise
>> it's probably down in the bowels of gettok() where it puts together the
>> string it's returning. I suspect a counting error---one of the myriad
>> substractions to correct for all the things that need fixing on the way
>> through the position calculations is wrong.
>
> Before I actually have to start thinking and looking at the code, does
> this tell you anything? In the working trace, I didn't do the cd ..
> (this is from break zshlex)
> I tried breaking on gettok but it appears my modules don't have debug
> symbols, I might get back in a while if anything interesting shows up there.
Well, I obviously have no idea what I'm doing now, first I just tried
breaking on
gettok, but it went on and on forever, then i started doing s 100 and
it still took
a good while until we came to the infinite loop. Then I tried the new reversible
debugging thing in gdb 7, breaked on zshlex, then continued backward
to gettok, and
stepped forward from there. I'm not exactly sure if I went far enough though.
(gdb) target record
(gdb) break zle_tricky.c:1248
Breakpoint 1 at 0x6faf8088: file zle_tricky.c, line 1248.
(gdb) c
Continuing.
Breakpoint 1, get_comp_string () at zle_tricky.c:1248
1248 for (tt1 = tt; *tt1; tt1++) {
(gdb) print tt1
$1 = 0x0
(gdb) print tt
$2 = 0x6fc05778 "."
(gdb) rs
1247 char *tt1, *e = tt + zlemetacs - wb;
(gdb)
1246 if (isset(RCQUOTES) && tt) {
(gdb)
dupstring (s=0x6fc05758 ".") at string.c:42
42 }
(gdb)
41 return t;
(gdb)
40 strcpy(t, s);
(gdb)
zhalloc (size=8) at mem.c:417
417 }
(gdb)
376 return ret;
(gdb)
375 unqueue_signals();
(gdb)
374 ret = arena(h) + n - size;
(gdb)
373 h->used = n;
(gdb)
370 if (ARENA_SIZEOF(h) >= (n = size + h->used)) {
(gdb)
368 ? fheap : heaps);
(gdb)
359 queue_signals();
(gdb)
357 size = (size + H_ISIZE - 1) & ~(H_ISIZE - 1);
(gdb)
dupstring (s=0x6fc05758 ".") at string.c:39
39 t = (char *) zhalloc(strlen((char *)s) + 1);
(gdb)
37 if (!s)
(gdb)
get_comp_string () at zle_tricky.c:1244
1244 tt = tokstr ? dupstring(tokstr) : NULL;
(gdb)
1242 if (!zleparse && !tt0) {
(gdb) print tokstr
$3 = 0x6fc05758 "."
(gdb) print tt0
$4 = 0
(gdb) rs
1239 if (i != redirpos)
(gdb)
ztrdup (s=0x6fc05758 ".") at string.c:55
55 }
(gdb)
54 return t;
(gdb)
53 strcpy(t, s);
(gdb)
zalloc (size=2) at mem.c:590
590 }
(gdb)
589 return ptr;
(gdb)
587 unqueue_signals();
(gdb)
583 if (!(ptr = (void *) malloc(size))) {
(gdb)
582 queue_signals();
(gdb)
580 if (!size)
(gdb)
ztrdup (s=0x6fc05758 ".") at string.c:52
52 t = (char *)zalloc(strlen((char *)s) + 1);
(gdb)
50 if (!s)
(gdb)
get_comp_string () at zle_tricky.c:1237
1237 cmdstr = ztrdup(tokstr);
(gdb) print cmdstr
$5 = 0x0
(gdb) print tokstr
$6 = 0x6fc05758 "."
(gdb) rs
zsfree (p=0x0) at mem.c:1509
1509 }
(gdb) s
get_comp_string () at zle_tricky.c:1237
1237 cmdstr = ztrdup(tokstr);
(gdb)
ztrdup (s=0x6fc05758 ".") at string.c:50
50 if (!s)
(gdb) print cmdstr
$7 = 0x0
(gdb) print tokstr
$8 = 0x6fc05758 "."
(gdb) rs
get_comp_string () at zle_tricky.c:1237
1237 cmdstr = ztrdup(tokstr);
(gdb)
zsfree (p=0x0) at mem.c:1509
1509 }
(gdb)
1507 if (p)
(gdb)
get_comp_string () at zle_tricky.c:1236
1236 zsfree(cmdstr);
(gdb)
1235 ins = (tok == REPEAT ? 2 : (tok != STRING));
(gdb) print cmdstr
$9 = 0x0
(gdb) rs
1231 if (lincmd && (tok == STRING || tok == FOR || tok == FOREACH ||
(gdb)
1210 if ((ins && (tok == DOLOOP || tok == SEPER)) ||
(gdb)
1208 if (tok == ENDINPUT)
(gdb)
1204 if (tok == DINPAR)
(gdb)
1193 if (inredir && IS_REDIROP(tok)) {
(gdb)
1187 else if (tok == OUTPAR) {
(gdb)
1185 } else if (tok == INPAR)
(gdb)
1181 if (tok == ENVARRAY) {
(gdb)
1179 } else if (tok == ENVSTRING)
(gdb)
1166 if (tok == LEXERR) {
(gdb)
ctxtlex () at lex.c:451
451 }
(gdb)
447 } else if (inredir) {
(gdb)
443 if (IS_REDIROP(tok) || tok == FOR || tok == FOREACH || tok == SELECT) {
(gdb)
442 infor = tok == FOR ? 2 : 0;
(gdb)
441 if (tok != DINPAR)
(gdb)
438 incmdpos = 0;
(gdb) help rs
Step program backward until it reaches the beginning of another source line.
Argument N means do this N times (or till program stops for another reason).
(gdb) help rc
Continue program being debugged but run it in reverse.
If proceeding from breakpoint, a number N may be used as an argument,
which means to set the ignore count of that breakpoint to N - 1 (so that
the breakpoint won't break until the Nth time it is reached).
(gdb) break gettok
Breakpoint 2 at 0x808f3fa: file lex.c, line 673.
(gdb) rc
Continuing.
Breakpoint 2, gettok () at lex.c:673
673 int peekfd = -1, peek;
(gdb) s
676 tokstr = NULL;
(gdb)
677 while (iblank(c = hgetc()) && !lexstop);
(gdb)
ingetc () at input.c:185
185 if (lexstop)
(gdb)
188 if (inbufleft) {
(gdb)
189 inbufleft--;
(gdb)
190 inbufct--;
(gdb)
191 if (itok(lastc = STOUC(*inbufptr++)))
(gdb) print inbufptr
$10 = 0x6fc05750 "./x "
(gdb) s
193 if (((inbufflags & INP_LINENO) || !strin) && lastc == '\n')
(gdb) print lastc
$11 = 46
(gdb) s
195 return lastc;
(gdb)
216 }
(gdb)
217 }
(gdb)
gettok () at lex.c:678
678 toklineno = lineno;
(gdb)
679 if (lexstop)
(gdb)
681 isfirstln = 0;
(gdb)
682 wordbeg = inbufct - (qbang && c == bangchar);
(gdb)
683 hwbegin(-1-(qbang && c == bangchar));
(gdb)
nohw (c=-1) at hist.c:812
812 }
(gdb)
gettok () at lex.c:685
685 if (dbparens) {
(gdb)
703 } else if (idigit(c)) { /* handle 1< foo */
(gdb)
727 if (c == hashchar && !nocomments &&
(gdb)
750 switch (lexact1[STOUC(c)]) {
(gdb)
929 return gettokstr(c, 0);
(gdb) print c
$12 = 46
(gdb) s
gettokstr (c=46, sub=0) at lex.c:949
949 int bct = 0, pct = 0, brct = 0, fdpar = 0;
(gdb)
950 int intpos = 1, in_brace_param = 0;
(gdb)
951 int peek, inquote, unmatched = 0;
(gdb)
956 peek = STRING;
(gdb)
957 if (!sub) {
(gdb)
958 len = 0;
(gdb)
959 bptr = tokstr = (char *) hcalloc(bsiz = 32);
(gdb)
hcalloc (size=32) at mem.c:567
567 ptr = zhalloc(size);
(gdb)
zhalloc (size=32) at mem.c:357
357 size = (size + H_ISIZE - 1) & ~(H_ISIZE - 1);
(gdb)
359 queue_signals();
(gdb)
368 ? fheap : heaps);
(gdb)
370 if (ARENA_SIZEOF(h) >= (n = size + h->used)) {
(gdb)
373 h->used = n;
(gdb)
374 ret = arena(h) + n - size;
(gdb)
375 unqueue_signals();
(gdb)
376 return ret;
(gdb)
417 }
(gdb)
hcalloc (size=32) at mem.c:568
568 memset(ptr, 0, size);
(gdb)
569 return ptr;
(gdb)
570 }
(gdb)
gettokstr (c=46, sub=0) at lex.c:964
964 int inbl = inblank(c);
(gdb)
966 if (fdpar && !inbl && c != ')')
(gdb)
969 if (inbl && !in_brace_param && !pct)
(gdb)
972 act = lexact2[STOUC(c)];
(gdb)
973 c = lextok2[STOUC(c)];
(gdb) bt full
#0 gettokstr (c=46, sub=0) at lex.c:973
act = 18
e = 1103274760
inbl = 0
bct = 0
pct = 0
brct = 0
fdpar = 0
intpos = 1
in_brace_param = 0
peek = 34
inquote = 0
unmatched = 0
#1 0x0808fd9e in gettok () at lex.c:929
c = 46
d = 5
peekfd = -1
peek = 2005139576
#2 0x0808ec98 in zshlex () at lex.c:364
No locals.
#3 0x0808ee5e in ctxtlex () at lex.c:408
oldpos = 0
#4 0x6faf7b84 in get_comp_string () at zle_tricky.c:1164
t0 = 1
tt0 = 0
i = 0
j = 0
k = 0
cp = 0
rd = 0
sl = 2005139972
---Type <return> to continue, or q <return> to quit---
ocs = 2005139960
ins = 0
oins = 0
ia = 0
parct = 0
varq = 0
ona = 0
qsub = 0
zlemetacs_qsub = 0
redirpos = 0
s = 0x0
tmp = 0x821f0a0 "./"
p = 0x80f8728 ""
tt = 0x0
rdop = "\000\000\000\000\004\000\000\000\000P\300o@W\300o\247\360!\b"
linptr = 0x6fc05748 "./x"
u = 0x821f0a2 ""
#5 0x6faf6574 in docomplete (lst=4) at zle_tricky.c:659
s = 0x778404b8 "\350\004\204wV\017\256o9\002\024\b"
ol = 0x0
olst = 4
chl = 0
ne = 0
ocs = 2
ret = 0
dat = {137335288, 1873838068}
active = 1
#6 0x6faf5a17 in expandorcomplete (args=0x6fb0bd40) at zle_tricky.c:315
ret = 2005140840
#7 0x6faf55ba in completecall (args=0x6fb0bd40) at zle_tricky.c:208
No locals.
---Type <return> to continue, or q <return> to quit---
#8 0x6fae5148 in execzlefunc (func=0x6fb09d28, args=0x6fb0bd40, set_bindk=0)
at zle_main.c:1312
atcurhist = 1
wflags = 134
r = 0
ret = 0
remetafy = 0
w = 0x81499f0
save_bindk = 0x6fb09d28
#9 0x6fae45b3 in zlecore () at zle_main.c:1050
No locals.
#10 0x6fae4ce5 in zleread (lp=0x80efb44, rp=0x80efacc, flags=7,
context=0) at zle_main.c:1213
s = 0x0
old_errno = 3
tmout = 0
initthingy = 0x0
#11 0x6fae6e33 in zle_main_entry (cmd=1, ap=0x778406c4 "") at zle_main.c:1866
lp = 0x80efb44
rp = 0x80efacc
flags = 7
context = 0
#12 0x08086faa in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x778406b4 "D\373\016\b\314\372\016\b\a"
#13 0x0808786d in inputline () at input.c:278
flags = 7
ingetcline = 0x0
ingetcpmptl = 0x80efb44
ingetcpmptr = 0x80efacc
context = 0
#14 0x080876dc in ingetc () at input.c:214
---Type <return> to continue, or q <return> to quit---
lastc = 1102191123
#15 0x0807d865 in ihgetc () at hist.c:263
c = 3
#16 0x0808f412 in gettok () at lex.c:677
c = 0
d = 0
peekfd = -1
peek = 0
#17 0x0808ec98 in zshlex () at lex.c:364
No locals.
#18 0x080aa9d8 in parse_event () at parse.c:451
No locals.
#19 0x080844d9 in loop (toplevel=1, justonce=0) at init.c:132
prog = 0x6fc05ab0
err = 50
non_empty = 1
#20 0x08087323 in zsh_main (argc=1, argv=0x77840934) at init.c:1454
t = 0x77840938
t0 = 158
#21 0x080553d6 in main (argc=Cannot access memory at address 0x0
) at ./main.c:93
No locals.
(gdb) s
975 switch (act) {
(gdb)
1353 add(c);
(gdb)
add (c=46) at lex.c:535
535 *bptr++ = c;
(gdb)
536 if (bsiz == ++len) {
(gdb)
555 }
(gdb)
gettokstr (c=46, sub=0) at lex.c:1354
1354 c = hgetc();
(gdb)
ingetc () at input.c:185
185 if (lexstop)
(gdb)
188 if (inbufleft) {
(gdb)
189 inbufleft--;
(gdb)
190 inbufct--;
(gdb)
191 if (itok(lastc = STOUC(*inbufptr++)))
(gdb)
193 if (((inbufflags & INP_LINENO) || !strin) && lastc == '\n')
(gdb)
195 return lastc;
(gdb)
216 }
(gdb)
217 }
(gdb)
gettokstr (c=47, sub=0) at lex.c:1355
1355 if (intpos)
(gdb)
1356 intpos--;
(gdb)
1357 if (lexstop)
(gdb)
964 int inbl = inblank(c);
(gdb)
966 if (fdpar && !inbl && c != ')')
(gdb)
969 if (inbl && !in_brace_param && !pct)
(gdb)
972 act = lexact2[STOUC(c)];
(gdb)
973 c = lextok2[STOUC(c)];
(gdb)
975 switch (act) {
(gdb)
1353 add(c);
(gdb)
add (c=0) at lex.c:535
535 *bptr++ = c;
(gdb)
536 if (bsiz == ++len) {
(gdb)
555 }
(gdb)
gettokstr (c=0, sub=0) at lex.c:1354
1354 c = hgetc();
(gdb)
ingetc () at input.c:185
185 if (lexstop)
(gdb)
188 if (inbufleft) {
(gdb)
189 inbufleft--;
(gdb)
190 inbufct--;
(gdb)
191 if (itok(lastc = STOUC(*inbufptr++)))
(gdb) print inbufptr
$13 = 0x6fc05752 "x "
(gdb) s
193 if (((inbufflags & INP_LINENO) || !strin) && lastc == '\n')
(gdb)
195 return lastc;
(gdb)
216 }
(gdb)
217 }
(gdb)
gettokstr (c=120, sub=0) at lex.c:1355
1355 if (intpos)
(gdb)
1357 if (lexstop)
(gdb)
964 int inbl = inblank(c);
(gdb)
966 if (fdpar && !inbl && c != ')')
(gdb)
969 if (inbl && !in_brace_param && !pct)
(gdb)
972 act = lexact2[STOUC(c)];
(gdb)
973 c = lextok2[STOUC(c)];
(gdb)
975 switch (act) {
(gdb)
1353 add(c);
(gdb)
add (c=120) at lex.c:535
535 *bptr++ = c;
(gdb)
536 if (bsiz == ++len) {
(gdb) print bptr
$14 = 0x6fc0575b ""
(gdb) s
555 }
(gdb)
gettokstr (c=120, sub=0) at lex.c:1354
1354 c = hgetc();
(gdb)
ingetc () at input.c:185
185 if (lexstop)
(gdb)
188 if (inbufleft) {
(gdb)
189 inbufleft--;
(gdb)
190 inbufct--;
(gdb)
191 if (itok(lastc = STOUC(*inbufptr++)))
(gdb)
193 if (((inbufflags & INP_LINENO) || !strin) && lastc == '\n')
(gdb) print inbufptr
$15 = 0x6fc05754 ""
(gdb) s
195 return lastc;
(gdb)
216 }
(gdb)
217 }
(gdb)
gettokstr (c=32, sub=0) at lex.c:1355
1355 if (intpos)
(gdb)
1357 if (lexstop)
(gdb)
964 int inbl = inblank(c);
(gdb)
966 if (fdpar && !inbl && c != ')')
(gdb)
969 if (inbl && !in_brace_param && !pct)
(gdb)
970 act = LX2_BREAK;
(gdb)
975 switch (act) {
(gdb)
977 if (!in_brace_param && !sub)
(gdb)
1361 hungetc(c);
(gdb)
inungetc (c=32) at input.c:358
358 if (!lexstop) {
(gdb)
359 if (inbufptr != inbuf) {
(gdb)
368 inbufptr--;
(gdb)
369 inbufct++;
(gdb)
370 inbufleft++;
(gdb)
371 if (((inbufflags & INP_LINENO) || !strin) && c == '\n')
(gdb)
395 if (inbufptr == inbufpush && inbufflags & INP_ALCONT) {
(gdb)
410 }
(gdb)
gettokstr (c=32, sub=0) at lex.c:1362
1362 if (unmatched)
(gdb)
1364 if (in_brace_param) {
(gdb)
1368 } else if (unset(IGNOREBRACES) && !sub && len > 1 &&
(gdb)
1376 *bptr = '\0';
(gdb)
1378 return peek;
(gdb) print peek
$16 = 34
(gdb) s
1379 }
(gdb)
gettok () at lex.c:930
930 }
(gdb)
zshlex () at lex.c:365
365 while (tok != ENDINPUT && exalias());
(gdb)
exalias () at lex.c:1716
1716 hwend();
(gdb)
nohwe () at hist.c:817
817 }
(gdb)
exalias () at lex.c:1717
1717 if (interact && isset(SHINSTDIN) && !strin && !incasepat &&
(gdb)
1722 if (!tokstr) {
(gdb)
1727 VARARR(char, copy, (strlen(tokstr) + 1));
(gdb)
1729 if (has_token(tokstr)) {
(gdb)
has_token (s=0x6fc05758 ".") at utils.c:1785
1785 if(itok(*s++))
(gdb)
1784 while(*s)
(gdb)
1787 return 0;
(gdb)
1788 }
(gdb)
exalias () at lex.c:1736
1736 zshlextext = tokstr;
(gdb)
1738 if (zleparse && !(inbufflags & INP_ALIAS)) {
(gdb)
1739 int zp = zleparse;
(gdb)
1741 gotword();
(gdb)
gotword () at lex.c:1700
1700 we = zlemetall + 1 - inbufct + (addedx == 2 ? 1 : 0);
(gdb)
1701 if (zlemetacs <= we) {
(gdb)
1702 wb = zlemetall - wordbeg + addedx;
(gdb)
1703 zleparse = 0;
(gdb)
1705 }
(gdb)
exalias () at lex.c:1742
1742 if (zp == 1 && !zleparse) {
(gdb)
1743 if (zshlextext == copy)
(gdb)
1745 return 0;
(gdb)
1799 }
(gdb)
zshlex () at lex.c:366
366 nocorrect &= 1;
(gdb)
367 if (tok == NEWLIN || tok == ENDINPUT) {
(gdb)
394 if (tok != NEWLIN)
(gdb)
395 isnewlin = 0;
(gdb)
398 if (tok == SEMI || tok == NEWLIN)
(gdb)
400 }
(gdb)
ctxtlex () at lex.c:409
409 switch (tok) {
(gdb)
438 incmdpos = 0;
(gdb)
441 if (tok != DINPAR)
(gdb)
442 infor = tok == FOR ? 2 : 0;
(gdb)
443 if (IS_REDIROP(tok) || tok == FOR || tok == FOREACH || tok == SELECT) {
(gdb)
447 } else if (inredir) {
(gdb)
451 }
(gdb)
get_comp_string () at zle_tricky.c:1166
1166 if (tok == LEXERR) {
(gdb)
1179 } else if (tok == ENVSTRING)
(gdb)
1181 if (tok == ENVARRAY) {
(gdb)
1185 } else if (tok == INPAR)
(gdb)
1187 else if (tok == OUTPAR) {
(gdb)
1193 if (inredir && IS_REDIROP(tok)) {
(gdb)
1204 if (tok == DINPAR)
(gdb)
1208 if (tok == ENDINPUT)
(gdb)
1210 if ((ins && (tok == DOLOOP || tok == SEPER)) ||
(gdb)
1231 if (lincmd && (tok == STRING || tok == FOR || tok == FOREACH ||
(gdb)
1235 ins = (tok == REPEAT ? 2 : (tok != STRING));
(gdb)
1236 zsfree(cmdstr);
(gdb)
zsfree (p=0x0) at mem.c:1507
1507 if (p)
(gdb)
1509 }
(gdb)
get_comp_string () at zle_tricky.c:1237
1237 cmdstr = ztrdup(tokstr);
(gdb)
ztrdup (s=0x6fc05758 ".") at string.c:50
50 if (!s)
(gdb)
52 t = (char *)zalloc(strlen((char *)s) + 1);
(gdb)
zalloc (size=2) at mem.c:580
580 if (!size)
(gdb)
582 queue_signals();
(gdb)
583 if (!(ptr = (void *) malloc(size))) {
(gdb)
587 unqueue_signals();
(gdb)
589 return ptr;
(gdb)
590 }
(gdb)
ztrdup (s=0x6fc05758 ".") at string.c:53
53 strcpy(t, s);
(gdb) bt full
#0 ztrdup (s=0x6fc05758 ".") at string.c:53
t = 0x818e950 "\270x/\bSEARCH"
#1 0x6faf7fbd in get_comp_string () at zle_tricky.c:1237
t0 = 1
tt0 = 0
i = 0
j = 0
k = 0
cp = 0
rd = 0
sl = 2005139972
ocs = 2005139960
ins = 0
oins = 0
ia = 0
parct = 0
varq = 0
ona = 0
qsub = 0
zlemetacs_qsub = 0
redirpos = 0
s = 0x0
tmp = 0x821f0a0 "./"
p = 0x80f8728 ""
tt = 0x0
rdop = "\000\000\000\000\004\000\000\000\000P\300o@W\300o\247\360!\b"
linptr = 0x6fc05748 "./x"
u = 0x821f0a2 ""
#2 0x6faf6574 in docomplete (lst=4) at zle_tricky.c:659
s = 0x778404b8 "\350\004\204wV\017\256o9\002\024\b"
ol = 0x0
---Type <return> to continue, or q <return> to quit---
olst = 4
chl = 0
ne = 0
ocs = 2
ret = 0
dat = {137335288, 1873838068}
active = 1
#3 0x6faf5a17 in expandorcomplete (args=0x6fb0bd40) at zle_tricky.c:315
ret = 2005140840
#4 0x6faf55ba in completecall (args=0x6fb0bd40) at zle_tricky.c:208
No locals.
#5 0x6fae5148 in execzlefunc (func=0x6fb09d28, args=0x6fb0bd40, set_bindk=0)
at zle_main.c:1312
atcurhist = 1
wflags = 134
r = 0
ret = 0
remetafy = 0
w = 0x81499f0
save_bindk = 0x6fb09d28
#6 0x6fae45b3 in zlecore () at zle_main.c:1050
No locals.
#7 0x6fae4ce5 in zleread (lp=0x80efb44, rp=0x80efacc, flags=7,
context=0) at zle_main.c:1213
s = 0x0
old_errno = 3
tmout = 0
initthingy = 0x0
#8 0x6fae6e33 in zle_main_entry (cmd=1, ap=0x778406c4 "") at zle_main.c:1866
lp = 0x80efb44
rp = 0x80efacc
flags = 7
---Type <return> to continue, or q <return> to quit---
context = 0
#9 0x08086faa in zleentry (cmd=1) at init.c:1304
ret = 0x0
ap = 0x778406b4 "D\373\016\b\314\372\016\b\a"
#10 0x0808786d in inputline () at input.c:278
flags = 7
ingetcline = 0x0
ingetcpmptl = 0x80efb44
ingetcpmptr = 0x80efacc
context = 0
#11 0x080876dc in ingetc () at input.c:214
lastc = 1102191123
#12 0x0807d865 in ihgetc () at hist.c:263
c = 3
#13 0x0808f412 in gettok () at lex.c:677
c = 0
d = 0
peekfd = -1
peek = 0
#14 0x0808ec98 in zshlex () at lex.c:364
No locals.
#15 0x080aa9d8 in parse_event () at parse.c:451
No locals.
#16 0x080844d9 in loop (toplevel=1, justonce=0) at init.c:132
prog = 0x6fc05ab0
err = 50
non_empty = 1
#17 0x08087323 in zsh_main (argc=1, argv=0x77840934) at init.c:1454
t = 0x77840938
t0 = 158
#18 0x080553d6 in main (argc=0, argv=0x0) at ./main.c:93
---Type <return> to continue, or q <return> to quit---
No locals.
(gdb) s
54 return t;
(gdb)
55 }
(gdb)
get_comp_string () at zle_tricky.c:1239
1239 if (i != redirpos)
(gdb)
1242 if (!zleparse && !tt0) {
(gdb)
1244 tt = tokstr ? dupstring(tokstr) : NULL;
(gdb)
dupstring (s=0x6fc05758 ".") at string.c:37
37 if (!s)
(gdb)
39 t = (char *) zhalloc(strlen((char *)s) + 1);
(gdb)
zhalloc (size=2) at mem.c:357
357 size = (size + H_ISIZE - 1) & ~(H_ISIZE - 1);
(gdb)
359 queue_signals();
(gdb)
368 ? fheap : heaps);
(gdb)
370 if (ARENA_SIZEOF(h) >= (n = size + h->used)) {
(gdb)
373 h->used = n;
(gdb)
374 ret = arena(h) + n - size;
(gdb)
375 unqueue_signals();
(gdb)
376 return ret;
(gdb)
417 }
(gdb)
dupstring (s=0x6fc05758 ".") at string.c:40
40 strcpy(t, s);
(gdb)
41 return t;
(gdb)
42 }
(gdb)
get_comp_string () at zle_tricky.c:1246
1246 if (isset(RCQUOTES) && tt) {
(gdb)
1247 char *tt1, *e = tt + zlemetacs - wb;
No more reverse-execution history.
get_comp_string () at zle_tricky.c:1248
1248 for (tt1 = tt; *tt1; tt1++) {
(gdb) print tt
$17 = 0x6fc05778 "."
(gdb) c
Continuing.
Breakpoint 2, gettok () at lex.c:673
At this point I got a bit tired of it though, I might try more tomorrow.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-02 1:26 ` Mikael Magnusson
@ 2009-11-02 16:38 ` Peter Stephenson
2009-11-02 20:58 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2009-11-02 16:38 UTC (permalink / raw)
To: zsh-workers
On Mon, 2 Nov 2009 02:26:54 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> Well, I obviously have no idea what I'm doing now, first I just tried
> breaking on
> gettok, but it went on and on forever, then i started doing s 100 and
> it still took
> a good while until we came to the infinite loop. Then I tried the new reversible
> debugging thing in gdb 7, breaked on zshlex, then continued backward
> to gettok, and
> stepped forward from there. I'm not exactly sure if I went far enough though.
Thanks again.
Somehow you've got tokstr equal to "." rather then either "" or "./x" in
this case. That might be consistent---at that point we haven't removed the
"x", so we might remove what we think is the "x" while it's actually the
".", ending up with the "" we see later. That would imply the end index is
two too short.
tmp is from zlemetaline: that's OK, "./", except the "x" has disappeared
already. That happens at zle_tricky.c:1259. I didn't think you'd got that
far yet, but I'm quite confused about what's actually happening, so it may
be this is a red herring (it doesn't seem to fit with the other evidence).
The key thing to find out (probably) is how gettokstr() puts together the
string it's returning, which should be "./x", and how zle fudges it. There
are no special characters in that, so we should bypass the switch at
lex.c:975 each time until we get to the end of the input. So we should be
calling add('.'), add('/'), add('x'). I see three calls to add in the
trace, followed by hitting LX2_BREAK, which would be OK for the end of the
string but I've no idea if it corresponds to that or not.
Those "*bptr++ = c"s ought to be a help: that's where we're appending to
tokstr. I see three of those following by a "*bptr = '\0'", which looks
right. It might be worth tracking the value of tokstr after that
point ("d tokstr" would probably do it).
I think the nasty fix-up-for-zle code in zle is the stuff in exalias()
around the call to gotwork(). This is grotesque and uncommented, so I'm
not sure what it's doing, but if it's working it should leave tokstr as
"./x". We're returning from inside that code at line 1745, which I presume
is correct, but I don't know, it's just too opaque. gotword() should be
setting "we" (the word end index) to 2 (or maybe 3? This is full of opaque
adjustments, too), and "wb" (the word beginning) to 0.
(Hmm... even if tokstr were "./x" an incorrect "we" might screw it up, but
from other evidence it looks likes it's getting to be "." somehow.)
Anyway, I wouldn't be surprised if the difference from the working case was
down here, somewhere, with the end index getting fudged wrongly, so if you
can work out how to break on the right gettok() or gettokstr() you should
"just" be able to post traces for both cases.
Maybe if we work out what's happening we can even stick some comments in
the code...
--
Peter Stephenson <pws@csr.com> Software Engineer
Tel: +44 (0)1223 692070 Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-02 16:38 ` Peter Stephenson
@ 2009-11-02 20:58 ` Mikael Magnusson
2009-11-02 21:06 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-02 20:58 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/2 Peter Stephenson <pws@csr.com>:
> On Mon, 2 Nov 2009 02:26:54 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
>> Well, I obviously have no idea what I'm doing now, first I just tried
>> breaking on
>> gettok, but it went on and on forever, then i started doing s 100 and
>> it still took
>> a good while until we came to the infinite loop. Then I tried the new reversible
>> debugging thing in gdb 7, breaked on zshlex, then continued backward
>> to gettok, and
>> stepped forward from there. I'm not exactly sure if I went far enough though.
>
> Thanks again.
>
> Somehow you've got tokstr equal to "." rather then either "" or "./x" in
> this case. That might be consistent---at that point we haven't removed the
> "x", so we might remove what we think is the "x" while it's actually the
> ".", ending up with the "" we see later. That would imply the end index is
> two too short.
>
> tmp is from zlemetaline: that's OK, "./", except the "x" has disappeared
> already. That happens at zle_tricky.c:1259. I didn't think you'd got that
> far yet, but I'm quite confused about what's actually happening, so it may
> be this is a red herring (it doesn't seem to fit with the other evidence).
>
> The key thing to find out (probably) is how gettokstr() puts together the
> string it's returning, which should be "./x", and how zle fudges it. There
> are no special characters in that, so we should bypass the switch at
> lex.c:975 each time until we get to the end of the input. So we should be
> calling add('.'), add('/'), add('x'). I see three calls to add in the
> trace, followed by hitting LX2_BREAK, which would be OK for the end of the
> string but I've no idea if it corresponds to that or not.
>
> Those "*bptr++ = c"s ought to be a help: that's where we're appending to
> tokstr. I see three of those following by a "*bptr = '\0'", which looks
> right. It might be worth tracking the value of tokstr after that
> point ("d tokstr" would probably do it).
>
> I think the nasty fix-up-for-zle code in zle is the stuff in exalias()
> around the call to gotwork(). This is grotesque and uncommented, so I'm
> not sure what it's doing, but if it's working it should leave tokstr as
> "./x". We're returning from inside that code at line 1745, which I presume
> is correct, but I don't know, it's just too opaque. gotword() should be
> setting "we" (the word end index) to 2 (or maybe 3? This is full of opaque
> adjustments, too), and "wb" (the word beginning) to 0.
>
> (Hmm... even if tokstr were "./x" an incorrect "we" might screw it up, but
> from other evidence it looks likes it's getting to be "." somehow.)
>
> Anyway, I wouldn't be surprised if the difference from the working case was
> down here, somewhere, with the end index getting fudged wrongly, so if you
> can work out how to break on the right gettok() or gettokstr() you should
> "just" be able to post traces for both cases.
>
> Maybe if we work out what's happening we can even stick some comments in
> the code...
I found something now, but I'm not sure if it's the gdb record log being weird.
It looks like lextok2['/'] is 0, which ends up making tokstr not "."
as we thought, but ".\000x" (maybe). I did try the whole thing without
any recording and this is what I see:
Breakpoint 1, get_comp_string () at zle_tricky.c:1067
1067 int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;
then a lot of stepping to
1164 ctxtlex();
then
(gdb) print tokstr
$1 = 0x6fb71758 "."
(gdb) print tokstr+1
$2 = 0x6fb71759 ""
(gdb) print tokstr+2
$3 = 0x6fb7175a "x"
(gdb) print tokstr+3
$4 = 0x6fb7175b ""
Here are some other variables
(gdb) print bptr
$10 = 0x6fb7175b ""
(gdb) print bptr-1
$11 = 0x6fb7175a "x"
(gdb) print bptr-2
$12 = 0x6fb71759 ""
(gdb) print bptr-3
$13 = 0x6fb71758 "."
(gdb) print bptr-4
$14 = 0x6fb71757 ""
(gdb) print inbuf
$15 = 0x6fb71750 "./x "
(gdb) print inbufptr
$16 = 0x6fb71753 " "
(gdb) print inbufct
$17 = 1
(gdb) print inbufleft
$18 = 1
Tried that again and stepped into ctxtlex a bit, and
Breakpoint 2, ctxtlex () at lex.c:408
408 zshlex();
(gdb) print tokstr
$2 = 0x0
(gdb) s
zshlex () at lex.c:361
361 if (tok == LEXERR)
(gdb) n
364 tok = gettok();
(gdb)
365 while (tok != ENDINPUT && exalias());
(gdb) print tokstr
$3 = 0x6fb23758 "."
(gdb) print tokstr+1
$4 = 0x6fb23759 ""
(gdb) print tokstr+2
$5 = 0x6fb2375a "x"
(gdb) print tokstr+3
$6 = 0x6fb2375b ""
So I don't think it's exalias() that's breaking it? So far I have no
idea how all this works though, has all this already been called one
(or more) times by the time we call it from get_comp_string()?
Just confirmed ['/'] is set to 0 by doing cd .. in /
Here we are,
(gdb) watch lextok2['/']
Hardware watchpoint 9: lextok2['/']
Continuing.
Hardware watchpoint 9: lextok2['/']
Old value = 47 '/'
New value = 0 '\000'
0x080c962f in xsymlinks (s=0x818e8f1 "..") at utils.c:696
696 *p = '\0';
(start over and add breakpoints (i'm not in reversible now just to be safe))
692 if (!strcmp(xbuf, "/"))
(gdb) print xbuf
$22 = '\000' <repeats 8191 times>
(gdb) n
694 p = xbuf + strlen(xbuf);
(gdb)
695 while (*--p != '/');
(gdb) n
at this point i pressed ctrl-c because i was curious what was taking
so long. Apparently this loops over all memory until it finds a slash,
which takes a while under gdb. Also apparently, the first / it finds
is in lextok2. Amusingly, ctrl-c and breaking on the next line is
instant, compared to waiting 30 seconds for 'n' to finish.
Breakpoint 11, xsymlinks (s=0x818e8f1 "..") at utils.c:696
696 *p = '\0';
(gdb) print p
$25 = 0x80e9b6f
"/0123456789:;<=>\225@ABCDEFGHIJKLMNOPQRSTUVWXYZ\217\\]\206_`abcdefghijklmnopqrstuvwxyz\215|}\226\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365",
<incomplete sequence \366>...
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: zsh eats 100% CPU with completion in /
2009-11-02 20:58 ` Mikael Magnusson
@ 2009-11-02 21:06 ` Mikael Magnusson
2009-11-02 21:30 ` Mikael Magnusson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-02 21:06 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/2 Mikael Magnusson <mikachu@gmail.com>:
> Here we are,
>
> (gdb) watch lextok2['/']
> Hardware watchpoint 9: lextok2['/']
> Continuing.
> Hardware watchpoint 9: lextok2['/']
>
> Old value = 47 '/'
> New value = 0 '\000'
> 0x080c962f in xsymlinks (s=0x818e8f1 "..") at utils.c:696
> 696 *p = '\0';
>
> (start over and add breakpoints (i'm not in reversible now just to be safe))
>
> 692 if (!strcmp(xbuf, "/"))
> (gdb) print xbuf
> $22 = '\000' <repeats 8191 times>
> (gdb) n
> 694 p = xbuf + strlen(xbuf);
> (gdb)
> 695 while (*--p != '/');
> (gdb) n
> at this point i pressed ctrl-c because i was curious what was taking
> so long. Apparently this loops over all memory until it finds a slash,
> which takes a while under gdb. Also apparently, the first / it finds
> is in lextok2.
And this seems to fix it. Not sure if checking that inside the loop is the best
place, but there seems to be some freeing of *pp going on that I
didn't feel like
investigating. Maybe checking xbuf before slashsplit would work?
diff --git a/Src/utils.c b/Src/utils.c
index 2a05624..634edc5 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -691,6 +691,8 @@ xsymlinks(char *s)
zsfree(*pp);
if (!strcmp(xbuf, "/"))
continue;
+ if (!*xbuf)
+ continue;
p = xbuf + strlen(xbuf);
while (*--p != '/');
*p = '\0';
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-02 21:06 ` Mikael Magnusson
@ 2009-11-02 21:30 ` Mikael Magnusson
2009-11-03 10:10 ` Peter Stephenson
0 siblings, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-11-02 21:30 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
2009/11/2 Mikael Magnusson <mikachu@gmail.com>:
> 2009/11/2 Mikael Magnusson <mikachu@gmail.com>:
>
>> Here we are,
>>
>> (gdb) watch lextok2['/']
>> Hardware watchpoint 9: lextok2['/']
>> Continuing.
>> Hardware watchpoint 9: lextok2['/']
>>
>> Old value = 47 '/'
>> New value = 0 '\000'
>> 0x080c962f in xsymlinks (s=0x818e8f1 "..") at utils.c:696
>> 696 *p = '\0';
>>
>> (start over and add breakpoints (i'm not in reversible now just to be safe))
>>
>> 692 if (!strcmp(xbuf, "/"))
>> (gdb) print xbuf
>> $22 = '\000' <repeats 8191 times>
>> (gdb) n
>> 694 p = xbuf + strlen(xbuf);
>> (gdb)
>> 695 while (*--p != '/');
>> (gdb) n
>> at this point i pressed ctrl-c because i was curious what was taking
>> so long. Apparently this loops over all memory until it finds a slash,
>> which takes a while under gdb. Also apparently, the first / it finds
>> is in lextok2.
>
> And this seems to fix it. Not sure if checking that inside the loop is the best
> place, but there seems to be some freeing of *pp going on that I
> didn't feel like
> investigating. Maybe checking xbuf before slashsplit would work?
And of course, I have no idea if *xbuf being '\0' is indicative of
some other bug.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-02 21:30 ` Mikael Magnusson
@ 2009-11-03 10:10 ` Peter Stephenson
2009-11-03 11:24 ` Frank Terbeck
0 siblings, 1 reply; 102+ messages in thread
From: Peter Stephenson @ 2009-11-03 10:10 UTC (permalink / raw)
To: zsh-workers
On Mon, 2 Nov 2009 22:30:08 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> > And this seems to fix it. Not sure if checking that inside the loop is the best
> > place, but there seems to be some freeing of *pp going on that I
> > didn't feel like
> > investigating. Maybe checking xbuf before slashsplit would work?
>
> And of course, I have no idea if *xbuf being '\0' is indicative of
> some other bug.
Thanks, it looks like that really is the problem and I think the fix is
fine.
We only call xsymlinks() in two places (apart from recursive calls), and in
both we initialise xbuf to contain '\0' as the first character. So on the
first time through (!*xbuf) is the standard case, and this was being
handled by searching backwards through memory for the first "/". So the
bug you've found is a major one.
The code around here is dependent (without checking) on paths being shorter
than PATH_MAX (the factor of 2 is for metafication), which is another
long-standing peeve of mine, but that's a completely separate problem.
--
Peter Stephenson <pws@csr.com> Software Engineer
Tel: +44 (0)1223 692070 Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-03 10:10 ` Peter Stephenson
@ 2009-11-03 11:24 ` Frank Terbeck
0 siblings, 0 replies; 102+ messages in thread
From: Frank Terbeck @ 2009-11-03 11:24 UTC (permalink / raw)
To: zsh-workers
Peter Stephenson <pws@csr.com>:
> On Mon, 2 Nov 2009 22:30:08 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
> > > And this seems to fix it. Not sure if checking that inside the loop is the best
> > > place, but there seems to be some freeing of *pp going on that I
> > > didn't feel like
> > > investigating. Maybe checking xbuf before slashsplit would work?
> >
> > And of course, I have no idea if *xbuf being '\0' is indicative of
> > some other bug.
>
> Thanks, it looks like that really is the problem and I think the fix is
> fine.
[...]
My colleague just tried this and it seems like it fixes the bug for
him, too. Thanks for wading through the code and coming up with a fix,
Mikael! :-)
Regards, Frank
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-11-02 0:57 ` Mikael Magnusson
2009-11-02 1:26 ` Mikael Magnusson
@ 2009-11-02 10:06 ` Peter Stephenson
1 sibling, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2009-11-02 10:06 UTC (permalink / raw)
To: zsh-workers
Mikael Magnusson wrote:
> 2009/11/1 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> > Mikael Magnusson wrote:
> >> I did the break after typing ./ and before pressing tab.
> >
> > Nothing leapt out of that. "tokstr" seems to be coming back as "." and
> > then the "." gets removed, as far as I can see, but looking at it out of
> > context I might be wrong.
> >
> > Might be worth checking zlemetacs as well as zlemetaline and zlemetall
> > on entry to get_comp_string() in case they're already wrong, otherwise
> > it's probably down in the bowels of gettok() where it puts together the
> > string it's returning. I suspect a counting error---one of the myriad
> > substractions to correct for all the things that need fixing on the way
> > through the position calculations is wrong.
>
> Before I actually have to start thinking and looking at the code, does
> this tell you anything? In the working trace, I didn't do the cd ..
> (this is from break zshlex)
> I tried breaking on gettok but it appears my modules don't have debug
> symbols, I might get back in a while if anything interesting shows up there.
I think the diff there is simply telling us it's already broken by that
point because the first loop is ending early, i.e the tt string which
has come from tokstr (copied just before the first difference) is zero
bytes long. This definitely points at the lexical analyser (but the
interaction between that and zle is murky so there could still be some
information from zle that's wrong).
--
Peter Stephenson <pws@csr.com> Software Engineer
Tel: +44 (0)1223 692070 Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-30 16:10 zsh eats 100% CPU with completion in / Frank Terbeck
2009-10-30 21:14 ` Mikael Magnusson
@ 2009-10-30 21:29 ` Benjamin R. Haskell
2009-10-30 23:01 ` Frank Terbeck
1 sibling, 1 reply; 102+ messages in thread
From: Benjamin R. Haskell @ 2009-10-30 21:29 UTC (permalink / raw)
To: zsh-workers
On Fri, 30 Oct 2009, Frank Terbeck wrote:
> The subject is not entirely correct.
>
> Zsh eats 100% CPU on a colleague's laptop if you do this:
> % cd /
> % cd ..
> % ./<tab>
>
> Happens with the most recent CVS HEAD, too.
> That's on linux (debian stable, actually).
>
> The problem is reproducible on his machine. And it doesn't happen if
> you do this:
> % cd /
> % ./<tab>
>
> The 'cd ..' is required.
>
> I cannot reproduce the problem on my system with exactly the same
> configuration and with exactly the same zsh version. And I am in fact
> using the very same OS (debian stable), too.
>
> As you can imagine, I am quite puzzled and I don't know how to tackle
> this.
>
> Any hints or suggestions are highly welcome.
As a potential source of differences, it might be interesting to see
what's mounted in the root directory of each machine. This reminds me of
Zsh scanning drives under /cygdrive/ when running under Cygwin.
Though I'm not sure how the 'cd ..' plays into it... something relating to
changes that dealt with current-directory tracking? (sorry, can't recall
specifics -- it was on here or zsh-users within the past three months or
so)
Best,
Ben
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: zsh eats 100% CPU with completion in /
2009-10-30 21:29 ` Benjamin R. Haskell
@ 2009-10-30 23:01 ` Frank Terbeck
2009-10-30 23:15 ` Frank Terbeck
0 siblings, 1 reply; 102+ messages in thread
From: Frank Terbeck @ 2009-10-30 23:01 UTC (permalink / raw)
To: zsh-workers
Benjamin R. Haskell <zsh@benizi.com>:
> On Fri, 30 Oct 2009, Frank Terbeck wrote:
> > Zsh eats 100% CPU on a colleague's laptop if you do this:
> > % cd /
> > % cd ..
> > % ./<tab>
[...]
> As a potential source of differences, it might be interesting to see
> what's mounted in the root directory of each machine. This reminds me of
> Zsh scanning drives under /cygdrive/ when running under Cygwin.
>
> Though I'm not sure how the 'cd ..' plays into it... something relating to
> changes that dealt with current-directory tracking? (sorry, can't recall
> specifics -- it was on here or zsh-users within the past three months or
> so)
Ah, I just remembered something:
After doing:
% cd /
% cd ..
My colleague could hang the shell by doing this:
% cd ~
% ./<tab>
I don't know if he could reproduce that every time, but I saw him do
it at least once.
So, I'm not to sure if what's mounted to / comes into play...
For my machine, I can tell you that I got an encrypted partition
mounted at /mnt and non-encrypted partitions at /home, /var, /tmp and
/usr.
On my colleague's system, / homes everything with respect to his linux
system, except for /boot which is a separate partition at the start of
the hard drive. I think he had a cdrom mounted at /media/cdrom0 which
/cdrom was a symlink to, IIRC. He also had his NTFS windows partition
mounted read-only at /media/something (where something is the actual
directory name, which I cannot recall right now).
Too bad this doesn't happen on a system of mine... But the fact that
Mikael can reproduce it at least means I'm not seeing things. :)
Regards, Frank
^ permalink raw reply [flat|nested] 102+ messages in thread
* Add completion suffix highlighting
@ 2009-12-04 21:53 Mikael Magnusson
2009-12-04 22:00 ` Mikael Magnusson
2009-12-05 19:31 ` Peter Stephenson
0 siblings, 2 replies; 102+ messages in thread
From: Mikael Magnusson @ 2009-12-04 21:53 UTC (permalink / raw)
To: zsh workers
I have a friend who just started using zsh a while ago, and she
complained about / being removed when she pressed enter after
completing a directory to rsync. Apparently rsync behaves differently
with and without the / in place. We tried setting
ZLE_REMOVE_SUFFIX_CHARS to nothing and using zle auto-suffix-retain in
an accept-line hook, but then we realized it would be better if you
could tell more easily if the suffix would be removed or not. So I
whipped this up and somehow it worked on the first try (no guarantees
though :). Given my poor track record of posting patches via email,
here is a link to the patch, and I'll just paste in the essence of the
patch for reference.
http://mika.l3ib.org/0001-Add-region-highlighting-for-completion-suffixes.patch
I'm not sure if the reference in the man page needs to be more
specific, or point somewhere completely different. Or if this is at
all a good idea, it seems to work and be useful though.
diff --git a/Src/Zle/zle_refresh.c b/Src/Zle/zle_refresh.c
index c849d9b..e99cbb3 100644
--- a/Src/Zle/zle_refresh.c
+++ b/Src/Zle/zle_refresh.c
@@ -247,8 +247,9 @@ struct region_highlight *region_highlights;
* for the first few elements of region_highlights.
* 0: region between point and mark
* 1: isearch region
+ * 2: suffix
*/
-#define N_SPECIAL_HIGHLIGHTS (2)
+#define N_SPECIAL_HIGHLIGHTS (3)
/*
* Number of elements in region_highlights.
* This includes the special elements above.
@@ -340,6 +341,7 @@ zle_set_highlight(void)
int special_atr_on_set = 0;
int region_atr_on_set = 0;
int isearch_atr_on_set = 0;
+ int suffix_atr_on_set = 0;
struct region_highlight *rhp;
special_atr_on = default_atr_on = 0;
@@ -373,6 +375,9 @@ zle_set_highlight(void)
} else if (strpfx("isearch:", *atrs)) {
match_highlight(*atrs + 8, &(region_highlights[1].atr));
isearch_atr_on_set = 1;
+ } else if (strpfx("suffix:", *atrs)) {
+ match_highlight(*atrs + 7, &(region_highlights[2].atr));
+ suffix_atr_on_set = 1;
}
}
}
@@ -384,6 +389,8 @@ zle_set_highlight(void)
region_highlights[0].atr = TXTSTANDOUT;
if (!isearch_atr_on_set)
region_highlights[1].atr = TXTUNDERLINE;
+ if (!suffix_atr_on_set)
+ region_highlights[2].atr = TXTBOLDFACE;
allocate_colour_buffer();
}
@@ -1076,6 +1083,13 @@ zrefresh(void)
} else {
region_highlights[1].start = region_highlights[1].end = -1;
}
+ /* check for an active completion suffix */
+ if (suffixnoinslen) {
+ region_highlights[2].start = zlecs - suffixnoinslen;
+ region_highlights[2].end = zlecs;
+ } else {
+ region_highlights[2].start = region_highlights[2].end = -1;
+ }
if (clearlist && listshown > 0) {
if (tccan(TCCLEAREOD)) {
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread* Re: Add completion suffix highlighting
2009-12-04 21:53 Add completion suffix highlighting Mikael Magnusson
@ 2009-12-04 22:00 ` Mikael Magnusson
2009-12-05 19:31 ` Peter Stephenson
1 sibling, 0 replies; 102+ messages in thread
From: Mikael Magnusson @ 2009-12-04 22:00 UTC (permalink / raw)
To: zsh workers
2009/12/4 Mikael Magnusson <mikachu@gmail.com>:
> @@ -384,6 +389,8 @@ zle_set_highlight(void)
> region_highlights[0].atr = TXTSTANDOUT;
> if (!isearch_atr_on_set)
> region_highlights[1].atr = TXTUNDERLINE;
> + if (!suffix_atr_on_set)
> + region_highlights[2].atr = TXTBOLDFACE;
Feel free to change this to default to nothing if you want of course,
I imagine some people would be annoyed if every / they completed was
bolded before they continued typing.
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Add completion suffix highlighting
2009-12-04 21:53 Add completion suffix highlighting Mikael Magnusson
2009-12-04 22:00 ` Mikael Magnusson
@ 2009-12-05 19:31 ` Peter Stephenson
2009-12-05 19:36 ` Peter Stephenson
2009-12-05 19:49 ` Mikael Magnusson
1 sibling, 2 replies; 102+ messages in thread
From: Peter Stephenson @ 2009-12-05 19:31 UTC (permalink / raw)
To: zsh workers
On Fri, 4 Dec 2009 22:53:58 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> I have a friend who just started using zsh a while ago, and she
> complained about / being removed when she pressed enter after
> completing a directory to rsync. Apparently rsync behaves differently
> with and without the / in place. We tried setting
> ZLE_REMOVE_SUFFIX_CHARS to nothing and using zle auto-suffix-retain in
> an accept-line hook, but then we realized it would be better if you
> could tell more easily if the suffix would be removed or not. So I
> whipped this up and somehow it worked on the first try (no guarantees
> though :). Given my poor track record of posting patches via email,
> here is a link to the patch, and I'll just paste in the essence of the
> patch for reference.
I'll commit this as it is with the following documentation and see
what happens---I think highlighting removable suffixes is actually quite
intuitive. Note, however, there are a lot of cases in which bold fonts
aren't supported.
Thanks.
Index: Doc/Zsh/zle.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/zle.yo,v
retrieving revision 1.80
diff -u -r1.80 zle.yo
--- Doc/Zsh/zle.yo 17 Apr 2009 15:26:46 -0000 1.80
+++ Doc/Zsh/zle.yo 5 Dec 2009 19:29:55 -0000
@@ -2163,6 +2163,17 @@
representation but are shown in a special manner by the line editor.
These characters are described below.
)
+cindex(completion removable suffix, highlighting)
+cindex(suffix, highlighting removable, in completion)
+cindex(removable suffix, highlighting in completino)
+item(tt(suffix)))(
+This context is used in completion for characters that are
+marked as suffixes that will be removed if the completion ends
+at that point, the most obvious example being a slash (tt(/)) after
+a directory name. Note that suffix removal is configurable; the
+circumstances under which the suffix will be removed may differ
+for different completions.
+)
enditem()
tt(zle_highlight) may contain additional fields for controlling how
@@ -2235,6 +2246,7 @@
)
item(tt(bold))(
The characters in the given context are shown in a bold font.
+Not all terminals distinguish bold fonts.
)
item(tt(standout))(
The characters in the given context are shown in the terminal's standout
@@ -2280,7 +2292,7 @@
context, the defaults applied are equivalent to
example(zle_highlight=LPAR()region:standout special:standout
-isearch:underline+RPAR())
+suffix:bold isearch:underline+RPAR())
i.e. both the region and special characters are shown in standout mode.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Add completion suffix highlighting
2009-12-05 19:31 ` Peter Stephenson
@ 2009-12-05 19:36 ` Peter Stephenson
2009-12-05 19:49 ` Mikael Magnusson
1 sibling, 0 replies; 102+ messages in thread
From: Peter Stephenson @ 2009-12-05 19:36 UTC (permalink / raw)
To: zsh workers
On Sat, 5 Dec 2009 19:31:41 +0000
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> +item(tt(suffix)))(
I've fixed the extra parenthesis, too.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: Add completion suffix highlighting
2009-12-05 19:31 ` Peter Stephenson
2009-12-05 19:36 ` Peter Stephenson
@ 2009-12-05 19:49 ` Mikael Magnusson
2009-12-05 20:11 ` Peter Stephenson
1 sibling, 1 reply; 102+ messages in thread
From: Mikael Magnusson @ 2009-12-05 19:49 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh workers
2009/12/5 Peter Stephenson <p.w.stephenson@ntlworld.com>:
> On Fri, 4 Dec 2009 22:53:58 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
>> I have a friend who just started using zsh a while ago, and she
>> complained about / being removed when she pressed enter after
>> completing a directory to rsync. Apparently rsync behaves differently
>> with and without the / in place. We tried setting
>> ZLE_REMOVE_SUFFIX_CHARS to nothing and using zle auto-suffix-retain in
>> an accept-line hook, but then we realized it would be better if you
>> could tell more easily if the suffix would be removed or not. So I
>> whipped this up and somehow it worked on the first try (no guarantees
>> though :). Given my poor track record of posting patches via email,
>> here is a link to the patch, and I'll just paste in the essence of the
>> patch for reference.
>
> I'll commit this as it is with the following documentation and see
> what happens---I think highlighting removable suffixes is actually quite
> intuitive. Note, however, there are a lot of cases in which bold fonts
> aren't supported.
>
> Thanks.
Ah, I think I meant to add at least the example too, but obviously
forgot, thanks. I also meant to ask if the variable I check for the
suffix length was the correct one, I was somewhat confused when I
discovered the suffix list. Is there any completion that actually uses
the multiple suffix feature?
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 102+ messages in thread
end of thread, other threads:[~2014-11-23 21:53 UTC | newest]
Thread overview: 102+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-04-13 16:54 PATCH: (large) initial support for combining characters in ZLE Peter Stephenson
2008-04-13 17:32 ` Bart Schaefer
2008-04-14 9:02 ` Peter Stephenson
2008-04-14 12:00 ` Peter Stephenson
2008-04-14 13:34 ` Mikael Magnusson
2008-04-14 13:54 ` Peter Stephenson
2008-04-15 13:58 ` Mikael Magnusson
2008-04-15 16:46 ` Peter Stephenson
2008-04-16 1:28 ` Mikael Magnusson
2008-04-16 8:47 ` Peter Stephenson
2008-04-17 9:28 ` Stephane Chazelas
2008-09-22 18:16 ` Mikael Magnusson
2008-09-22 18:36 ` Peter Stephenson
2008-09-22 18:39 ` Mikael Magnusson
2008-04-17 18:33 ` Jun T.
2008-04-18 9:40 ` Peter Stephenson
2008-04-18 15:48 ` Jun T.
2008-04-18 16:05 ` Peter Stephenson
2008-04-19 15:04 ` Jun T.
2008-05-04 0:52 Identify "active" region? Bart Schaefer
2008-05-04 7:16 ` Mikael Magnusson
2008-05-04 12:21 ` Peter Stephenson
2008-05-04 12:33 ` Mikael Magnusson
2008-05-04 12:35 ` Mikael Magnusson
2008-05-04 13:28 ` Mikael Magnusson
2008-05-04 18:05 ` Peter Stephenson
2008-05-04 19:10 ` Mikael Magnusson
2008-05-04 16:38 ` Bart Schaefer
2008-05-04 17:52 ` Mikael Magnusson
[not found] <mikachu@gmail.com>
2008-02-06 15:02 ` Completion lockup Mikael Magnusson
2008-02-06 15:07 ` Ismail Dönmez
2008-02-09 17:21 ` Peter Stephenson
2008-02-09 18:04 ` Mikael Magnusson
2008-02-09 19:13 ` Peter Stephenson
2008-08-30 12:02 ` Who is sorting my completion results? Mikael Magnusson
2008-08-30 12:16 ` Peter Stephenson
2008-08-30 12:39 ` Mikael Magnusson
[not found] ` <237967ef0902140622s7389d2c8h5a0c786dcf207422@mail.gmail.com>
[not found] ` <200902141801.n1EI1E2l003603@pws-pc.ntlworld.com>
[not found] ` <237967ef0902141019t30118690m30116c9413015d96@mail.gmail.com>
[not found] ` <090214111316.ZM15188@torch.brasslantern.com>
[not found] ` <237967ef0902141141y609b61d3i154546f6f6886c65@mail.gmail.com>
[not found] ` <090214133904.ZM15383@torch.brasslantern.com>
[not found] ` <20090216094632.30502fe9@news01>
2009-02-16 9:55 ` Problem with fake-files style and cd Mikael Magnusson
2011-05-27 1:25 ` Mikael Magnusson
2011-05-27 4:41 ` Bart Schaefer
2011-05-27 4:57 ` Mikael Magnusson
2011-05-27 5:36 ` Bart Schaefer
2011-05-27 14:24 ` Mikael Magnusson
2011-05-27 14:39 ` Peter Stephenson
2011-05-27 15:06 ` Bart Schaefer
2011-05-27 15:16 ` Mikael Magnusson
2011-05-27 15:44 ` Bart Schaefer
2011-01-06 19:22 ` Infinite loop, can't reproduce with zsh -f Mikael Magnusson
2011-01-06 20:03 ` Peter Stephenson
2014-11-23 21:07 ` PATCH: Fix leaks of desthost in ztcp Mikael Magnusson
2014-11-23 21:53 ` Peter Stephenson
[not found] <okiddle@yahoo.co.uk>
2008-10-30 21:20 ` another bug: zsh_directory_name Oliver Kiddle
2008-10-30 21:26 ` Mikael Magnusson
2008-10-30 22:13 ` Peter Stephenson
2008-10-30 23:44 ` Vincent Lefevre
2008-10-31 0:14 ` Mikael Magnusson
2008-10-31 9:44 ` Oliver Kiddle
2008-10-31 9:58 ` Peter Stephenson
2008-10-30 22:16 ` Peter Stephenson
2008-10-31 11:10 ` PATCH: bug with hash builtin Oliver Kiddle
2008-10-31 21:07 ` Peter Stephenson
2008-12-16 15:38 PATCH: edit-command-line with spaces in EDITOR Clint Adams
2008-12-16 17:07 ` Mikael Magnusson
2008-12-16 19:22 ` Peter Stephenson
2008-12-16 19:27 ` Mikael Magnusson
2008-12-16 20:51 ` Richard Hartmann
2008-12-16 21:45 ` Clint Adams
2008-12-16 22:31 ` Mikael Magnusson
2008-12-17 12:16 ` Romain Francoise
[not found] ` <237967ef0812170448n11bd34f8y2c98b6484c8c0024@mail.gmail.com>
[not found] ` <87oczb9d1j.fsf@elegiac.orebokech.com>
2008-12-17 13:17 ` Mikael Magnusson
2008-12-17 14:44 ` Greg Klanderman
2008-12-17 4:04 ` Bart Schaefer
2009-10-30 16:10 zsh eats 100% CPU with completion in / Frank Terbeck
2009-10-30 21:14 ` Mikael Magnusson
2009-10-30 21:25 ` Mikael Magnusson
2009-10-30 21:33 ` Mikael Magnusson
2009-10-31 20:00 ` Peter Stephenson
2009-10-31 22:43 ` Mikael Magnusson
2009-10-31 23:00 ` Peter Stephenson
2009-11-01 1:50 ` Mikael Magnusson
2009-11-01 18:31 ` Peter Stephenson
2009-11-01 19:33 ` Bart Schaefer
2009-11-01 21:12 ` Mikael Magnusson
2009-11-01 22:20 ` Peter Stephenson
2009-11-02 0:57 ` Mikael Magnusson
2009-11-02 1:26 ` Mikael Magnusson
2009-11-02 16:38 ` Peter Stephenson
2009-11-02 20:58 ` Mikael Magnusson
2009-11-02 21:06 ` Mikael Magnusson
2009-11-02 21:30 ` Mikael Magnusson
2009-11-03 10:10 ` Peter Stephenson
2009-11-03 11:24 ` Frank Terbeck
2009-11-02 10:06 ` Peter Stephenson
2009-10-30 21:29 ` Benjamin R. Haskell
2009-10-30 23:01 ` Frank Terbeck
2009-10-30 23:15 ` Frank Terbeck
2009-12-04 21:53 Add completion suffix highlighting Mikael Magnusson
2009-12-04 22:00 ` Mikael Magnusson
2009-12-05 19:31 ` Peter Stephenson
2009-12-05 19:36 ` Peter Stephenson
2009-12-05 19:49 ` Mikael Magnusson
2009-12-05 20:11 ` Peter Stephenson
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).