* BUG: tied parameters with NUL-separator
@ 2004-09-13 18:25 Matthias B.
2004-09-16 20:15 ` PATCH: incorrect tied param splitting/joining when imeta(separator) is true Matthias B.
0 siblings, 1 reply; 3+ messages in thread
From: Matthias B. @ 2004-09-13 18:25 UTC (permalink / raw)
To: zsh-workers
The following works as expected in zsh 4.2.1
(NOTE: hexd is a hexdump script that displays
offset: hex correspondingchars)
pmt> unset foo
pmt> unset bar
pmt> sep=:
pmt> foo='a:b:c'
pmt> typeset -T foo bar "$sep"
pmt> echo -n "${bar[1]}" | hexd
00000: 61 a
pmt> echo -n "${bar[2]}" | hexd
00000: 62 b
pmt> echo -n "${bar[3]}" | hexd
00000: 63 c
pmt> bar[3]='1 2'
pmt> echo -n "${bar[3]}" | hexd
00000: 31 20 32 1 2
pmt> echo -n "$foo" | hexd
00000: 61 3a 62 3a 31 20 32 a:b:1 2
Now look what happens when I use NUL as separator
pmt> unset foo
pmt> unset bar
pmt> sep=$'\0'
pmt> foo=$'a\0b\0c'
pmt> typeset -T foo bar "$sep"
pmt> echo -n "${bar[1]}" | hexd
00000: 61 a
pmt> echo -n "${bar[2]}" | hexd
00000: 20 62 b
pmt> echo -n "${bar[3]}" | hexd
00000: 20 63 c
pmt> bar[3]='1 2'
pmt> echo -n "${bar[3]}" | hexd
00000: 31 20 32 1 2
pmt> echo -n "$foo" | hexd
00000: 61 20 62 31 20 a b1
So we have 2 bugs here:
1) When typeset -T splits foo it inserts a space in front of all elements
but the 1st one. This only happens when splitting apparently. Assigning to
an element directly does not insert a space.
2) Re-assembling the scalar doesn't work. The separators are not inserted
at all and the part after the space in the string '1 2' assigned to
element 3 gets lost somehow.
I know that I probably shouldn't complain. I'm actually quite impressed
that embedding NULs in strings works at all. Few C-programs have
binary-clean string-handling.
But it's such a useful feature and since it works almost perfectly already
I'm hopeful that this can be made to work fully. I'm really looking
forward to this, because I want to parse output from find into an array
and since pathnames can in theory contain every character but NUL, using
NUL as separator seems to be the only way to achieve this.
Bug 1) is the important one for me as I'm only interested in splitting, so
a fix for that would be appreciated even if Bug 2) would take too much
effort to fix.
MSB
--
pragma est dogma.
^ permalink raw reply [flat|nested] 3+ messages in thread
* PATCH: incorrect tied param splitting/joining when imeta(separator) is true
2004-09-13 18:25 BUG: tied parameters with NUL-separator Matthias B.
@ 2004-09-16 20:15 ` Matthias B.
2004-09-17 9:28 ` Peter Stephenson
0 siblings, 1 reply; 3+ messages in thread
From: Matthias B. @ 2004-09-16 20:15 UTC (permalink / raw)
To: zsh-workers
[-- Attachment #1: Type: text/plain, Size: 971 bytes --]
Since no one seemed to be interested in the NUL-separator bugs I've taken
a look at the code myself and I've managed to locate the bugs.
The function tiedarrsetfn() does not convert characters for which imeta()
is true into separator strings properly. This means that the bug is not
limited to NUL-characters but affects all characters for which imeta() is
true (e.g. 0x83).
There are also 2 bugs in zjoin():
1. zjoin() doesn't put the NUL-terminator in the right place when delim is
a meta-character.
2. zjoin() refuses to add 0-delimiters (although it does count them when
computing the length). This behaviour does not seem to be used anywhere in
zsh code AFAICS and even if it were, that would be a bug in the respective
call.
The attached patch fixes both problems and adds 2 regression tests to
B02typeset.ztst (both tests are necessary, since 0 and 0x83 are treated
differently in some places, so that one case could regress independent of
the other).
MSB
[-- Attachment #2: zsh-4.2.1-split-join-with-meta-sep-fix.patch --]
[-- Type: text/plain, Size: 4287 bytes --]
diff -r --unified=23 zsh-4.2.1/Src/params.c zsh-4.2.1-split-join-with-meta-sep-fix/Src/params.c
--- zsh-4.2.1/Src/params.c Fri Aug 13 12:22:45 2004
+++ zsh-4.2.1-split-join-with-meta-sep-fix/Src/params.c Thu Sep 16 19:34:59 2004
@@ -2623,47 +2623,47 @@
}
/**/
char *
tiedarrgetfn(Param pm)
{
struct tieddata *dptr = (struct tieddata *)pm->u.data;
return *dptr->arrptr ? zjoin(*dptr->arrptr, dptr->joinchar, 1) : "";
}
/**/
void
tiedarrsetfn(Param pm, char *x)
{
struct tieddata *dptr = (struct tieddata *)pm->u.data;
if (*dptr->arrptr)
freearray(*dptr->arrptr);
if (x) {
char sepbuf[3];
if (imeta(dptr->joinchar))
{
sepbuf[0] = Meta;
- sepbuf[1] = dptr->joinchar;
+ sepbuf[1] = dptr->joinchar ^ 32;
sepbuf[2] = '\0';
}
else
{
sepbuf[0] = dptr->joinchar;
sepbuf[1] = '\0';
}
*dptr->arrptr = sepsplit(x, sepbuf, 0, 0);
if (pm->flags & PM_UNIQUE)
uniqarray(*dptr->arrptr);
} else
*dptr->arrptr = NULL;
if (pm->ename)
arrfixenv(pm->nam, *dptr->arrptr);
zsfree(x);
}
/**/
void
tiedarrunsetfn(Param pm, UNUSED(int exp))
{
/*
* Special unset function because we allocated a struct tieddata
diff -r --unified=23 zsh-4.2.1/Src/utils.c zsh-4.2.1-split-join-with-meta-sep-fix/Src/utils.c
--- zsh-4.2.1/Src/utils.c Fri Aug 13 12:22:46 2004
+++ zsh-4.2.1-split-join-with-meta-sep-fix/Src/utils.c Thu Sep 16 21:25:56 2004
@@ -1853,56 +1853,54 @@
} else {
if (ztrftimebuf(&bufsize, 1))
return 0;
*buf++ = *fmt++;
}
*buf = '\0';
return buf - origbuf;
}
/**/
mod_export char *
zjoin(char **arr, int delim, int heap)
{
int len = 0;
char **s, *ret, *ptr;
for (s = arr; *s; s++)
len += strlen(*s) + 1 + (imeta(delim) ? 1 : 0);
if (!len)
return heap? "" : ztrdup("");
ptr = ret = (heap ? (char *) hcalloc(len) : (char *) zshcalloc(len));
for (s = arr; *s; s++) {
strucpy(&ptr, *s);
- if (delim) {
if (imeta(delim)) {
*ptr++ = Meta;
*ptr++ = delim ^ 32;
}
else
*ptr++ = delim;
- }
}
- ptr[-1] = '\0';
+ ptr[-1 - (imeta(delim) ? 1 : 0)] = '\0';
return ret;
}
/* Split a string containing a colon separated list *
* of items into an array of strings. */
/**/
char **
colonsplit(char *s, int uniq)
{
int ct;
char *t, **ret, **ptr, **p;
for (t = s, ct = 0; *t; t++) /* count number of colons */
if (*t == ':')
ct++;
ptr = ret = (char **) zalloc(sizeof(char **) * (ct + 2));
t = s;
do {
s = t;
/* move t to point at next colon */
for (; *t && *t != ':'; t++);
diff -r --unified=23 zsh-4.2.1/Test/B02typeset.ztst zsh-4.2.1-split-join-with-meta-sep-fix/Test/B02typeset.ztst
--- zsh-4.2.1/Test/B02typeset.ztst Fri Aug 13 12:22:54 2004
+++ zsh-4.2.1-split-join-with-meta-sep-fix/Test/B02typeset.ztst Thu Sep 16 21:58:41 2004
@@ -163,46 +163,61 @@
?(eval):3: read-only variable: r
typeset r=success
readonly r
print $r
r=failure
1:Convert to readonly
>success
?(eval):4: read-only variable: r
typeset -gU array
print $array
0:Uniquified arrays and non-local scope
>a r y
typeset -T SCALAR=l:o:c:a:l array
print $array
typeset -U SCALAR
print $SCALAR $array
0:Tied parameters and uniquified colon-arrays
>l o c a l
>l:o:c:a l o c a
+ typeset -T SCALAR=$'l\x83o\x83c\x83a\x83l' array $'\x83'
+ print $array
+ typeset -U SCALAR
+ print $SCALAR $array
+0:Tied parameters and uniquified arrays with meta-character as separator
+>l o c a l
+>loca l o c a
+
+ typeset -T SCALAR=$'l\000o\000c\000a\000l' array $'\000'
+ typeset -U SCALAR
+ print $array
+ [[ $SCALAR == $'l\000o\000c\000a' ]]
+0:Tied parameters and uniquified arrays with NUL-character as separator
+>l o c a
+
typeset -T SCALAR array
typeset +T SCALAR
1:Untying is prohibited
?(eval):typeset:2: use unset to remove tied variables
OUTER=outer
scope13
print $OUTER
0:Export of tied parameters
>i:n:n:e:r
>outer
typeset -TU MORESTUFF=here-we-go-go-again morestuff '-'
print -l $morestuff
0:Tied arrays with separator specified
>here
>we
>go
>again
typeset -T THIS will not work
1:Tied array syntax
?(eval):typeset:1: -T requires names of scalar and array
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: PATCH: incorrect tied param splitting/joining when imeta(separator) is true
2004-09-16 20:15 ` PATCH: incorrect tied param splitting/joining when imeta(separator) is true Matthias B.
@ 2004-09-17 9:28 ` Peter Stephenson
0 siblings, 0 replies; 3+ messages in thread
From: Peter Stephenson @ 2004-09-17 9:28 UTC (permalink / raw)
To: zsh-workers
"Matthias B." wrote:
> Since no one seemed to be interested in the NUL-separator bugs I've taken
> a look at the code myself and I've managed to locate the bugs.
Thanks, this one passed me by. I've committed it (trying not to commit
anything to do with ignore_eof).
It will probably need updating for Unicode, but no one's holding their
breath.
--
Peter Stephenson <pws@csr.com> Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK Tel: +44 (0)1223 692070
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.
This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.
www.mimesweeper.com
**********************************************************************
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2004-09-17 9:29 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-09-13 18:25 BUG: tied parameters with NUL-separator Matthias B.
2004-09-16 20:15 ` PATCH: incorrect tied param splitting/joining when imeta(separator) is true Matthias B.
2004-09-17 9:28 ` Peter Stephenson
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).