* "${*:offset:length}" and ksh93/bash compatibility @ 2022-03-10 7:50 Stephane Chazelas 2022-03-10 12:48 ` Mikael Magnusson 0 siblings, 1 reply; 11+ messages in thread From: Stephane Chazelas @ 2022-03-10 7:50 UTC (permalink / raw) To: Zsh hackers list Hello, From workers/28418 I understand the ${param:offset:length} was introduced only for compatibility with bash (though the feature was initially from ksh93) as zsh already had $param[first,last] (long before bash had arrays or ksh93 even existed). Still, there's a difference with ksh/bash when ${*:offset:length} (same for ${array[*]}) is quoted: $ zsh --emulate ksh -c 'printf "<%s>\n" ${*:1:2}' zsh foo bar baz <foo> <bar> $ zsh --emulate ksh -c 'printf "<%s>\n" "${*:1:2}"' zsh foo bar baz <oo> $ ksh -c 'printf "<%s>\n" ${*:1:2}' ksh foo bar baz <foo> <bar> $ ksh -c 'printf "<%s>\n" "${*:1:2}"' ksh foo bar baz <foo bar> $ bash -c 'printf "<%s>\n" ${*:1:2}' ksh foo bar baz <foo> <bar> $ bash -c 'printf "<%s>\n" "${*:1:2}"' ksh foo bar baz <foo bar> Should zsh align with bash/ksh there. -- Stephane ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-10 7:50 "${*:offset:length}" and ksh93/bash compatibility Stephane Chazelas @ 2022-03-10 12:48 ` Mikael Magnusson 2022-03-10 14:25 ` Peter Stephenson 0 siblings, 1 reply; 11+ messages in thread From: Mikael Magnusson @ 2022-03-10 12:48 UTC (permalink / raw) To: Zsh hackers list On 3/10/22, Stephane Chazelas <stephane@chazelas.org> wrote: > Hello, > > From workers/28418 I understand the ${param:offset:length} was > introduced only for compatibility with bash (though the feature > was initially from ksh93) as zsh already had $param[first,last] > (long before bash had arrays or ksh93 even existed). > > Still, there's a difference with ksh/bash when > ${*:offset:length} (same for ${array[*]}) is quoted: > > $ zsh --emulate ksh -c 'printf "<%s>\n" ${*:1:2}' zsh foo bar baz > <foo> > <bar> > $ zsh --emulate ksh -c 'printf "<%s>\n" "${*:1:2}"' zsh foo bar baz > <oo> > > $ ksh -c 'printf "<%s>\n" ${*:1:2}' ksh foo bar baz > <foo> > <bar> > $ ksh -c 'printf "<%s>\n" "${*:1:2}"' ksh foo bar baz > <foo bar> > > $ bash -c 'printf "<%s>\n" ${*:1:2}' ksh foo bar baz > <foo> > <bar> > $ bash -c 'printf "<%s>\n" "${*:1:2}"' ksh foo bar baz > <foo bar> > > Should zsh align with bash/ksh there. If we do, it would probably look something like this (does not handle all cases), unless someone has a better idea... diff --git i/Src/subst.c w/Src/subst.c index 465fe970ff..a34ad5f571 100644 --- i/Src/subst.c +++ w/Src/subst.c @@ -2718,7 +2718,7 @@ paramsubst(LinkList l, LinkNode n, char **str, int qt, int pf_flags, } } /* See if this is a reference to the positional parameters. */ - if (v && v->pm && v->pm->gsu.a == &vararray_gsu && + if (v && v->pm && (char ***)v->pm->u.data == &pparams) horrible_offset_hack = 1; else @@ -3441,6 +3441,17 @@ colonsubscript: } if (offset < 0) offset = 0; + } else if (offset > 0 && horrible_offset_hack) { + offset--; + if (length_set && length > 0) { + char **elem; + int newlen = 0; + for (elem = aval; *elem && length; elem++) { + newlen += strlen(*elem) + 1; + length--; + } + length = newlen - 1; + } } given_offset = offset; MB_METACHARINIT(); -- Mikael Magnusson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-10 12:48 ` Mikael Magnusson @ 2022-03-10 14:25 ` Peter Stephenson 2022-03-13 15:22 ` Mikael Magnusson 0 siblings, 1 reply; 11+ messages in thread From: Peter Stephenson @ 2022-03-10 14:25 UTC (permalink / raw) To: Zsh hackers list On 10 March 2022 at 12:48 Mikael Magnusson <mikachu@gmail.com> wrote: > On 3/10/22, Stephane Chazelas <stephane@chazelas.org> wrote: > > Should zsh align with bash/ksh there. > > If we do, it would probably look something like this (does not handle > all cases), unless someone has a better idea... I think it should change --- the documentation is already clear this is here to please users of other shells, not native zsh. Otherwise we end up with what's (as we say where I come from) neither nowt nor something. This syntax gives effects similar to parameter subscripting in the form $name[start,end], but is compatible with other shells; note that both off‐ set and length are interpreted differently from the components of a sub‐ script. pws ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-10 14:25 ` Peter Stephenson @ 2022-03-13 15:22 ` Mikael Magnusson 2022-03-13 18:21 ` Bart Schaefer 2022-03-16 5:57 ` Jun T 0 siblings, 2 replies; 11+ messages in thread From: Mikael Magnusson @ 2022-03-13 15:22 UTC (permalink / raw) To: Peter Stephenson; +Cc: Zsh hackers list On 3/10/22, Peter Stephenson <p.w.stephenson@ntlworld.com> wrote: > On 10 March 2022 at 12:48 Mikael Magnusson <mikachu@gmail.com> wrote: >> On 3/10/22, Stephane Chazelas <stephane@chazelas.org> wrote: >> > Should zsh align with bash/ksh there. >> >> If we do, it would probably look something like this (does not handle >> all cases), unless someone has a better idea... > > I think it should change --- the documentation is already clear this is > here to please users of other shells, not native zsh. Otherwise we end > up with what's (as we say where I come from) neither nowt nor something. > > This syntax gives effects similar to parameter subscripting in the > form > $name[start,end], but is compatible with other shells; note that both > off‐ > set and length are interpreted differently from the components of a > sub‐ > script. I realized that this is not limited to $*, zsh% a=(one two three); echo "${a[@]:0:2}" one two zsh% a=(one two three); echo "${a[*]:0:2}" on zsh% bash -c 'a=(one two three); echo "${a[@]:0:2}"' one two zsh% bash -c 'a=(one two three); echo "${a[*]:0:2}"' one two So perhaps this should be attacked in a different place/way than my test patch (though i think the horrible offset hack is still needed for $*:0 specifically (and i actually didn't even handle that in the patch anyway.).) -- Mikael Magnusson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-13 15:22 ` Mikael Magnusson @ 2022-03-13 18:21 ` Bart Schaefer 2022-03-13 19:23 ` Mikael Magnusson 2022-03-16 5:57 ` Jun T 1 sibling, 1 reply; 11+ messages in thread From: Bart Schaefer @ 2022-03-13 18:21 UTC (permalink / raw) To: Mikael Magnusson; +Cc: Peter Stephenson, Zsh hackers list Arriving to this a bit late, have been traveling. On Sun, Mar 13, 2022 at 8:22 AM Mikael Magnusson <mikachu@gmail.com> wrote: > > zsh% a=(one two three); echo "${a[*]:0:2}" > on > zsh% bash -c 'a=(one two three); echo "${a[*]:0:2}"' > one two The [*] subscript in double quotes has always been treated differently in zsh. You need [@] or the (@) modifier to preserve array-ness. I'm not sure the presence of the :off:len suffix should change that as a side-effect, at least not in native zsh mode ... and it might be difficult to "notice" the suffix early enough in the expansion code to cause that effect anyway. Particularly without also having the effect that the end result is not joined into a single string. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-13 18:21 ` Bart Schaefer @ 2022-03-13 19:23 ` Mikael Magnusson 2022-03-13 20:57 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Mikael Magnusson @ 2022-03-13 19:23 UTC (permalink / raw) To: Bart Schaefer; +Cc: Peter Stephenson, Zsh hackers list On 3/13/22, Bart Schaefer <schaefer@brasslantern.com> wrote: > Arriving to this a bit late, have been traveling. > > On Sun, Mar 13, 2022 at 8:22 AM Mikael Magnusson <mikachu@gmail.com> wrote: >> >> zsh% a=(one two three); echo "${a[*]:0:2}" >> on >> zsh% bash -c 'a=(one two three); echo "${a[*]:0:2}"' >> one two > > The [*] subscript in double quotes has always been treated differently > in zsh. You need [@] or the (@) modifier to preserve array-ness. I'm > not sure the presence of the :off:len suffix should change that as a > side-effect, at least not in native zsh mode ... and it might be > difficult to "notice" the suffix early enough in the expansion code to > cause that effect anyway. Particularly without also having the effect > that the end result is not joined into a single string. The way it works in bash is it does become a scalar, but the subscript works on the array elements before joining (just like [1,3] does in zsh in the same context). I'm reasonably sure that there would be no difference in the zsh code for handling $*:X or $a[*]:X, other than having to add the extra faked element for X=0. I think the correct answer is to always have :off:len work on the array indices if they exist, regardless of double-quoted-ness... But I also just took a look at getindex() and don't want to replicate the appropriate parts of that logic (which would require understanding what any of the parts do). I'm also not sure if it matters too much since it's been some years before anyone even noticed it, I just (somewhat naively) thought that maybe it wouldn't be too difficult to add... -- Mikael Magnusson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-13 19:23 ` Mikael Magnusson @ 2022-03-13 20:57 ` Bart Schaefer 0 siblings, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2022-03-13 20:57 UTC (permalink / raw) To: Mikael Magnusson; +Cc: Peter Stephenson, Zsh hackers list On Sun, Mar 13, 2022 at 12:23 PM Mikael Magnusson <mikachu@gmail.com> wrote: > > The way it works in bash is it does become a scalar, but the subscript > works on the array elements before joining (just like [1,3] does in > zsh in the same context). Ah, OK. You're thinking of "${a[*][1,3]}" and I'm thinking of "${${a[*]}[1,3]}". Empirically, :off:len is (currently) implemented as the latter rather than as the former. > I also just took a look > at getindex() and don't want to replicate the appropriate parts of > that logic (which would require understanding what any of the parts > do). Indeed. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-13 15:22 ` Mikael Magnusson 2022-03-13 18:21 ` Bart Schaefer @ 2022-03-16 5:57 ` Jun T 2022-03-16 15:57 ` Mikael Magnusson 2022-03-23 8:08 ` Jun T 1 sibling, 2 replies; 11+ messages in thread From: Jun T @ 2022-03-16 5:57 UTC (permalink / raw) To: zsh-workers > 2022/03/14 0:22, Mikael Magnusson <mikachu@gmail.com> wrote: > > I realized that this is not limited to $*, > > zsh% a=(one two three); echo "${a[@]:0:2}" > one two > zsh% a=(one two three); echo "${a[*]:0:2}" > on The following patch seems to work for "${*:1:2}" "${@:1:2}" "$a[@]:0:2}" "${a[*]:0:2}" For example (with or without KSH_ARRAYS): zsh% printf '<%s>\n' "${a[@]:0:2}" <one> <two> If KSH_ARRAYS is not set, "$a:0:2" behaves like "${a[*]:0:2}". I don't know if there are better/simpler solutions. 'make check' passes, but please do some tests to see if there are any bad side effects. diff --git a/Src/subst.c b/Src/subst.c index 465fe970f..0f98e6ea3 100644 --- a/Src/subst.c +++ b/Src/subst.c @@ -1854,6 +1854,12 @@ paramsubst(LinkList l, LinkNode n, char **str, int qt, int pf_flags, * nested (P) flags. */ int fetch_needed; + /* + * If an array parameter is quoted but has :offset:length (as in + * "${array:off:len}"), we apply :off:len as array index before + * joining the array into a string (for compatibility with ksh/bash). + */ + int quoted_array_with_offset = 0; *s++ = '\0'; /* @@ -3377,7 +3383,16 @@ colonsubscript: return NULL; } } - if (isarr) { + /* + * We've got :OFFSET (and :LENGTH). + * If aval is non-NULL but isarr is 0, PARAM is (probably) + * an array but quoted like "${PARAM:OFFSET}". We apply + * :OFFSET as array index (as if it is not quoted). We will + * join them later (search for quoted_array_with_offset). + */ + if (aval && !isarr) + quoted_array_with_offset = 1; + if (isarr || quoted_array_with_offset) { int alen, count; char **srcptr, **dstptr, **newarr; @@ -3622,9 +3637,9 @@ colonsubscript: * exception is that ${name:-word} and ${name:+word} will have already * done any requested splitting of the word value with quoting preserved. */ - if (ssub || spbreak || spsep || sep) { + if (ssub || spbreak || spsep || sep || quoted_array_with_offset) { int force_split = !ssub && (spbreak || spsep); - if (isarr) { + if (isarr || quoted_array_with_offset) { /* sep non-null here means F or j flag, force join */ if (nojoin == 0 || sep) { val = sepjoin(aval, sep, 1); ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-16 5:57 ` Jun T @ 2022-03-16 15:57 ` Mikael Magnusson 2022-03-23 8:08 ` Jun T 1 sibling, 0 replies; 11+ messages in thread From: Mikael Magnusson @ 2022-03-16 15:57 UTC (permalink / raw) To: Jun T; +Cc: zsh-workers On 3/16/22, Jun T <takimoto-j@kba.biglobe.ne.jp> wrote: > >> 2022/03/14 0:22, Mikael Magnusson <mikachu@gmail.com> wrote: >> >> I realized that this is not limited to $*, >> >> zsh% a=(one two three); echo "${a[@]:0:2}" >> one two >> zsh% a=(one two three); echo "${a[*]:0:2}" >> on > > The following patch seems to work for > "${*:1:2}" "${@:1:2}" "$a[@]:0:2}" "${a[*]:0:2}" > > For example (with or without KSH_ARRAYS): > > zsh% printf '<%s>\n' "${a[@]:0:2}" > <one> > <two> > > If KSH_ARRAYS is not set, "$a:0:2" behaves like "${a[*]:0:2}". > > I don't know if there are better/simpler solutions. > 'make check' passes, but please do some tests to see if there > are any bad side effects. It's certainly a lot simpler than my idea, and also seems to work in the cases I could come up with. I guess if KSH_ARRAYS is not set, then there is no expectation of bash compatibility anyway so whatever we do there is fine. -- Mikael Magnusson ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-16 5:57 ` Jun T 2022-03-16 15:57 ` Mikael Magnusson @ 2022-03-23 8:08 ` Jun T 2022-03-23 10:23 ` Mikael Magnusson 1 sibling, 1 reply; 11+ messages in thread From: Jun T @ 2022-03-23 8:08 UTC (permalink / raw) To: zsh-workers > 2022/03/16 14:57, I wrote: > > The following patch seems to work for > "${*:1:2}" "${@:1:2}" "$a[@]:0:2}" "${a[*]:0:2}" This is a test for the patch. diff --git a/Test/D04parameter.ztst b/Test/D04parameter.ztst index b6b1f2e33..20f731dcb 100644 --- a/Test/D04parameter.ztst +++ b/Test/D04parameter.ztst @@ -1693,6 +1693,30 @@ >b >c + () { + emulate -L sh + local a=( one two three ) + printf '<%s><%s>\n' ${a[*]:0:2} + printf '<%s><%s>\n' "${a[*]:0:2}" + printf '<%s><%s>\n' ${a[@]:0:2} + printf '<%s><%s>\n' "${a[@]:0:2}" + printf '<%s><%s>\n' "${a:0:2}" + printf '<%s><%s>\n' ${*:1:2} + printf '<%s><%s>\n' "${*:1:2}" + printf '<%s><%s>\n' ${@:1:2} + printf '<%s><%s>\n' "${@:1:2}" + } one two three +0:Bash-style offsets, quoted array +><one><two> +><one two><> +><one><two> +><one><two> +><on><> +><one><two> +><one two><> +><one><two> +><one><two> + printf "%n" '[0]' 1:Regression test for identifier test ?(eval):1: not an identifier: [0] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: "${*:offset:length}" and ksh93/bash compatibility 2022-03-23 8:08 ` Jun T @ 2022-03-23 10:23 ` Mikael Magnusson 0 siblings, 0 replies; 11+ messages in thread From: Mikael Magnusson @ 2022-03-23 10:23 UTC (permalink / raw) To: Jun T; +Cc: zsh-workers On 3/23/22, Jun T <takimoto-j@kba.biglobe.ne.jp> wrote: > >> 2022/03/16 14:57, I wrote: >> >> The following patch seems to work for >> "${*:1:2}" "${@:1:2}" "$a[@]:0:2}" "${a[*]:0:2}" > > This is a test for the patch. > > > diff --git a/Test/D04parameter.ztst b/Test/D04parameter.ztst > index b6b1f2e33..20f731dcb 100644 > --- a/Test/D04parameter.ztst > +++ b/Test/D04parameter.ztst > @@ -1693,6 +1693,30 @@ > >b > >c > > + () { > + emulate -L sh > + local a=( one two three ) > + printf '<%s><%s>\n' ${a[*]:0:2} > + printf '<%s><%s>\n' "${a[*]:0:2}" > + printf '<%s><%s>\n' ${a[@]:0:2} > + printf '<%s><%s>\n' "${a[@]:0:2}" > + printf '<%s><%s>\n' "${a:0:2}" > + printf '<%s><%s>\n' ${*:1:2} > + printf '<%s><%s>\n' "${*:1:2}" > + printf '<%s><%s>\n' ${@:1:2} > + printf '<%s><%s>\n' "${@:1:2}" > + } one two three > +0:Bash-style offsets, quoted array > +><one><two> > +><one two><> > +><one><two> > +><one><two> > +><on><> > +><one><two> > +><one two><> > +><one><two> > +><one><two> > + > printf "%n" '[0]' > 1:Regression test for identifier test > ?(eval):1: not an identifier: [0] I think the test should include + printf '<%s><%s>\n' ${*:0:2} + printf '<%s><%s>\n' "${*:0:2}" + printf '<%s><%s>\n' ${@:0:2} + printf '<%s><%s>\n' "${@:0:2}" as well, since 0 is handled specially (the horrible offset hack). -- Mikael Magnusson ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-03-23 10:23 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-10 7:50 "${*:offset:length}" and ksh93/bash compatibility Stephane Chazelas 2022-03-10 12:48 ` Mikael Magnusson 2022-03-10 14:25 ` Peter Stephenson 2022-03-13 15:22 ` Mikael Magnusson 2022-03-13 18:21 ` Bart Schaefer 2022-03-13 19:23 ` Mikael Magnusson 2022-03-13 20:57 ` Bart Schaefer 2022-03-16 5:57 ` Jun T 2022-03-16 15:57 ` Mikael Magnusson 2022-03-23 8:08 ` Jun T 2022-03-23 10:23 ` Mikael Magnusson
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).