From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 14904 invoked from network); 30 Oct 2008 15:26:52 -0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 30 Oct 2008 15:26:52 -0000 Received-SPF: none (ns1.primenet.com.au: domain at sunsite.dk does not designate permitted sender hosts) Received: (qmail 22192 invoked from network); 30 Oct 2008 15:26:41 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 30 Oct 2008 15:26:41 -0000 Received: (qmail 2234 invoked by alias); 30 Oct 2008 15:26:32 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 25969 Received: (qmail 2215 invoked from network); 30 Oct 2008 15:26:30 -0000 Received: from bifrost.dotsrc.org (130.225.254.106) by sunsite.dk with SMTP; 30 Oct 2008 15:26:30 -0000 Received: from cluster-d.mailcontrol.com (cluster-d.mailcontrol.com [217.69.20.190]) by bifrost.dotsrc.org (Postfix) with ESMTPS id E53E980524C0 for ; Thu, 30 Oct 2008 16:26:25 +0100 (CET) Received: from rly32d.srv.mailcontrol.com (localhost.localdomain [127.0.0.1]) by rly32d.srv.mailcontrol.com (MailControl) with ESMTP id m9UFQL7F012809 for ; Thu, 30 Oct 2008 15:26:21 GMT Received: from submission.mailcontrol.com (submission.mailcontrol.com [86.111.216.190]) by rly32d.srv.mailcontrol.com (MailControl) id m9UFQ9Jg012467 for zsh-workers@sunsite.dk; Thu, 30 Oct 2008 15:26:09 GMT Received: from cameurexb01.EUROPE.ROOT.PRI ([193.128.72.68]) by rly32d-eth0.srv.mailcontrol.com (envelope-sender Peter.Stephenson@csr.com) (MIMEDefang) with ESMTP id m9UFPi9G011771; Thu, 30 Oct 2008 15:26:09 +0000 (GMT) Received: from news01 ([10.103.143.38]) by cameurexb01.EUROPE.ROOT.PRI with Microsoft SMTPSVC(6.0.3790.3959); Thu, 30 Oct 2008 15:25:40 +0000 Date: Thu, 30 Oct 2008 15:25:35 +0000 From: Peter Stephenson To: "Kazuo Teramoto" Cc: zsh-workers@sunsite.dk Subject: Re: Problem with unicode string length zsh segfaults Message-ID: <20081030152535.038bc541@news01> In-Reply-To: <5992e17f0810300703l1856332cx6ef4e749a5194e1b@mail.gmail.com> References: <5992e17f0810300703l1856332cx6ef4e749a5194e1b@mail.gmail.com> Organization: CSR X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.8; i386-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-OriginalArrivalTime: 30 Oct 2008 15:25:40.0581 (UTC) FILETIME=[C4651950:01C93AA3] X-Scanned-By: MailControl A-08-50-15 (www.mailcontrol.com) on 10.68.1.142 X-Virus-Scanned: ClamAV 0.92.1/8542/Thu Oct 30 15:23:43 2008 on bifrost X-Virus-Status: Clean On Thu, 30 Oct 2008 12:03:22 -0200 "Kazuo Teramoto" wrote: > Hello! > (My first email to list, I subscribed to workers, but I send this type > of msg to workers or users?) Yes, zsh-workers is correct. > I'm having a problem with string size and unicode chars. I'm using > string size in my prompt theme, I'm using the following code to > calculate the string size of pwd: >=20 > ${#${(%):-%~}} >=20 > But sometimes this got the number wrong e.g. I have a dir named > '~/Music/=E6=A2=B6=E6=B5=A6=E7=94=B1=E8=A8=98' (Yuki Kajiura composer of = Noir anime soundtrack) the > code above give 13 chars but it have 12 chars (count or using "print > -Pn '%~'|wc -m" and wc -c give 20 bytes). >=20 > And if I cd on this dir my prompt get wrong by 3 chars (e.g. on a > prompt like adams2 the first line is bigger by 3 chars) >=20 > The problem is greater for a dir named '=D0=9F=D1=91=D1=82=D1=80 =D0=98= =D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0=D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA= =D0=B8=D0=B9' (Pyotr > Il'yich Tchaikovsky, Russian composer) if I cd on it and execute the > code above I get >=20 > zsh: unmatched ' > zsh: parse error in command substitution >=20 > and if I try this several times (repeat this command) my zsh segfaults: >=20 > 1: subst.c:222: BUG: parse error in command substitution Thanks for the report. Same bug in each case; it's taken surprisingly long to find and I can't understand how we thought the code should look like that. I found a similar bug in new code at the same time. Index: Src/subst.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /cvsroot/zsh/zsh/Src/subst.c,v retrieving revision 1.89 diff -u -r1.89 subst.c --- Src/subst.c 27 Sep 2008 20:58:28 -0000 1.89 +++ Src/subst.c 30 Oct 2008 15:22:34 -0000 @@ -2738,8 +2738,8 @@ char *tmps; unmetafy(*ap, &len); untokenize(*ap); - tmps =3D unmetafy(promptexpand(metafy(*ap, len, META_NOALLOC), - 0, NULL, NULL, NULL), &len); + tmps =3D promptexpand(metafy(*ap, len, META_NOALLOC), + 0, NULL, NULL, NULL); *ap =3D dupstring(tmps); free(tmps); } @@ -2749,8 +2749,8 @@ val =3D dupstring(val), copied =3D 1; unmetafy(val, &len); untokenize(val); - tmps =3D unmetafy(promptexpand(metafy(val, len, META_NOALLOC), - 0, NULL, NULL, NULL), &len); + tmps =3D promptexpand(metafy(val, len, META_NOALLOC), + 0, NULL, NULL, NULL); val =3D dupstring(tmps); free(tmps); } Index: Src/utils.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /cvsroot/zsh/zsh/Src/utils.c,v retrieving revision 1.205 diff -u -r1.205 utils.c --- Src/utils.c 30 Oct 2008 14:04:31 -0000 1.205 +++ Src/utils.c 30 Oct 2008 15:22:34 -0000 @@ -857,7 +857,9 @@ scanhashtable(nameddirtab, 0, 0, 0, finddir_scan, 0); =20 if (func) { - char **ares =3D subst_string_by_func(func, "d", finddir_full); + char *dir_meta =3D metafy(finddir_full, strlen(finddir_full), + META_ALLOC); + char **ares =3D subst_string_by_func(func, "d", dir_meta); int len; if (ares && arrlen(ares) >=3D 2 && (len =3D (int)zstrtol(ares[1], NULL, 10)) > finddir_best) { @@ -868,6 +870,8 @@ finddir_last->diff =3D len - strlen(finddir_last->node.nam); finddir_best =3D len; } + if (dir_meta !=3D finddir_full) + zsfree(dir_meta); } =20 return finddir_last; Index: Test/D07multibyte.ztst =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /cvsroot/zsh/zsh/Test/D07multibyte.ztst,v retrieving revision 1.27 diff -u -r1.27 D07multibyte.ztst --- Test/D07multibyte.ztst 18 Oct 2008 16:03:18 -0000 1.27 +++ Test/D07multibyte.ztst 30 Oct 2008 15:22:34 -0000 @@ -403,3 +403,10 @@ 0:Backslashes and metafied characters in patterns >Matched against glob/()=C4=84/* >Matched against glob/()=C4=80/* + + mkdir =E6=A2=B6=E6=B5=A6=E7=94=B1=E8=A8=98 '=D0=9F=D1=91=D1=82=D1=80 =D0= =98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0=D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0= =BA=D0=B8=D0=B9' + (cd =E6=A2=B6=E6=B5=A6=E7=94=B1=E8=A8=98; print ${${(%):-%~}:t}) + (cd '=D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0= =B0=D0=B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9'; print ${${(%):-%~}:t}) +0:Metafied characters in prompt expansion +>=E6=A2=B6=E6=B5=A6=E7=94=B1=E8=A8=98 +>=D0=9F=D1=91=D1=82=D1=80 =D0=98=D0=BB=D1=8C=D0=B8=D1=87 =D0=A7=D0=B0=D0= =B9=D0=BA=D0=BE=D0=B2=D1=81=D0=BA=D0=B8=D0=B9 --=20 Peter Stephenson Software Engineer CSR PLC, Churchill House, Cambridge Business Park, Cowley Road Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070