[BUG]builtin echo error doing arguments parsing

[BUG]builtin echo error doing arguments parsing

[wumingxwk]

if there is only a '-' as argument，builtin echo won't print it

just like that:

localhost# echo -

localhost# /bin/echo -
-
localhost# bash -c 'echo -'
-
localhost# tcsh -c 'echo -'
-

so I think that's a bug

Sent via t.me/gmailbot

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On Wed, 21 Feb 2018 23:23:09 -0800
wumingxwk@gmail.com wrote:
> if there is only a '-' as argument，builtin echo won't print it

It's entirely deliberate and documented, but you're quite right that it
plays merry hell with compatibility.

You might have thought it should at least be turned off in some or most
of the emulation modes...

pws


SHELL BUILTIN COMMANDS

...

All  builtin  commands other than precommand modifiers, even those that
have no options, can be given the argument  `--'  to  terminate  option
processing.   This  indicates  that  the following words are non-option
arguments, but is otherwise ignored.  This is  useful  in  cases  where
arguments  to  the command may begin with `-'.  For historical reasons,
most builtin commands also recognize a single `-' in  a  separate  word
for  this  purpose;  note that this is less standard and use of `--' is
recommended.

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On Thu, 22 Feb 2018 09:37:11 +0000
Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Wed, 21 Feb 2018 23:23:09 -0800
> wumingxwk@gmail.com wrote:
> > if there is only a '-' as argument，builtin echo won't print it
>
> You might have thought it should at least be turned off in some or most
> of the emulation modes...

Would look something like this.  Probably a good idea if there aren't
subtleties?

pws

diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo
index 6c7ec4b..098b989 100644
--- a/Doc/Zsh/builtins.yo
+++ b/Doc/Zsh/builtins.yo
@@ -61,7 +61,9 @@ arguments, but is otherwise ignored.  This is useful in cases where
 arguments to the command may begin with `tt(-)'.  For historical
 reasons, most builtin commands also recognize a single `tt(-)' in a
 separate word for this purpose; note that this is less standard and
-use of `tt(-)tt(-)' is recommended.
+use of `tt(-)tt(-)' is recommended.  Use of a single `tt(-)' to
+terminate option processing is turned off if the option
+tt(POSIX_BUILTINS) is set.
 
 startitem()
 prefix(-)
diff --git a/Doc/Zsh/options.yo b/Doc/Zsh/options.yo
index 25b3d57..5217e62 100644
--- a/Doc/Zsh/options.yo
+++ b/Doc/Zsh/options.yo
@@ -2183,6 +2183,9 @@ Furthermore, the tt(getopts) builtin behaves in a POSIX-compatible
 fashion in that the associated variable tt(OPTIND) is not made
 local to functions.
 
+In addition, a single dash (`tt(-)') does not cause the termination of option
+processing: a double dash (`tt(-)tt(-)') is required.
+
 Moreover, the warning and special exit code from
 tt([[ -o )var(non_existent_option)tt( ]]) are suppressed.
 )
diff --git a/Src/builtin.c b/Src/builtin.c
index fb59738..d2c6ec4 100644
--- a/Src/builtin.c
+++ b/Src/builtin.c
@@ -304,7 +304,8 @@ execbuiltin(LinkList args, LinkList assigns, Builtin bn)
 		if (!(flags & BINF_KEEPNUM) && idigit(arg[1]))
 		    break;
 		/* For cd and friends, a single dash is not an option. */
-		if ((flags & BINF_SKIPDASH) && !arg[1])
+		if (((flags & BINF_SKIPDASH) || isset(POSIXBUILTINS)) &&
+		    !arg[1])
 		    break;
 		if ((flags & BINF_DASHDASHVALID) && !strcmp(arg, "--")) {
 		    /*

Re: [BUG]builtin echo error doing arguments parsing

[Mikael Magnusson]

On Thu, Feb 22, 2018 at 11:32 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
> On Thu, 22 Feb 2018 09:37:11 +0000
> Peter Stephenson <p.stephenson@samsung.com> wrote:
>> On Wed, 21 Feb 2018 23:23:09 -0800
>> wumingxwk@gmail.com wrote:
>> > if there is only a '-' as argument，builtin echo won't print it
>>
>> You might have thought it should at least be turned off in some or most
>> of the emulation modes...
>
> Would look something like this.  Probably a good idea if there aren't
> subtleties?

If we change it now, people would have to add a version check in
addition to their check for zsh to be able to echo things portably (to
know if they should add an extra - or not).

-- 
Mikael Magnusson

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On Thu, 22 Feb 2018 17:50:48 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> On Thu, Feb 22, 2018 at 11:32 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> > On Thu, 22 Feb 2018 09:37:11 +0000
> > Peter Stephenson <p.stephenson@samsung.com> wrote:
> >> On Wed, 21 Feb 2018 23:23:09 -0800
> >> wumingxwk@gmail.com wrote:
> >> > if there is only a '-' as argument，builtin echo won't print it
> >>
> >> You might have thought it should at least be turned off in some or most
> >> of the emulation modes...
> >
> > Would look something like this.  Probably a good idea if there aren't
> > subtleties?
> 
> If we change it now, people would have to add a version check in
> addition to their check for zsh to be able to echo things portably (to
> know if they should add an extra - or not).

The portable option is "--", which has worked for a long time.  Adding -
would be a zsh kludge, nothing to do with portability.

pws

Re: [BUG]builtin echo error doing arguments parsing

[Mikael Magnusson]

On Thu, Feb 22, 2018 at 6:26 PM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
> On Thu, 22 Feb 2018 17:50:48 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
>> On Thu, Feb 22, 2018 at 11:32 AM, Peter Stephenson
>> <p.stephenson@samsung.com> wrote:
>> > On Thu, 22 Feb 2018 09:37:11 +0000
>> > Peter Stephenson <p.stephenson@samsung.com> wrote:
>> >> On Wed, 21 Feb 2018 23:23:09 -0800
>> >> wumingxwk@gmail.com wrote:
>> >> > if there is only a '-' as argument，builtin echo won't print it
>> >>
>> >> You might have thought it should at least be turned off in some or most
>> >> of the emulation modes...
>> >
>> > Would look something like this.  Probably a good idea if there aren't
>> > subtleties?
>>
>> If we change it now, people would have to add a version check in
>> addition to their check for zsh to be able to echo things portably (to
>> know if they should add an extra - or not).
>
> The portable option is "--", which has worked for a long time.  Adding -
> would be a zsh kludge, nothing to do with portability.

My point is that if people are already doing "echo - $whatever" when
they detect zsh, and we stop parsing it, their stuff will break
(output their extra dash). I didn't try the patch but currently echo
-- just outputs --, only - terminates options for echo.

-- 
Mikael Magnusson

Re: [BUG]builtin echo error doing arguments parsing

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

On Feb 22, 2018 11:01 AM, "Mikael Magnusson" <mikachu@gmail.com> wrote:


My point is that if people are already doing "echo - $whatever" when
they detect zsh, and we stop parsing it, their stuff will break
(output their extra dash).


That would only matter if they are both detecting zsh AND setting
POSIX_BUILTINS.

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On Thu, 22 Feb 2018 20:00:58 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> I didn't try the patch but currently echo
> -- just outputs --, only - terminates options for echo.

Yes, the -- behaviour appears to be general behaviour, in fact,
so not something that should be changed.  So indeed it's hard to
do this at the moment in a shell script without some kind of kludge
for zsh.  "disable echo" and use /bin/echo might be the best bet.

However, I'm not really sure if that makes it less or actually more
useful to align with other shells (with POSIXBUILTINS) from now on...
it's not obvious perpetuating the need for a kludge for ever more
is the best bet.

pws

Re: [BUG]builtin echo error doing arguments parsing

[Stephane Chazelas]

2018-02-22 19:34:23 +0000, Peter Stephenson:
> On Thu, 22 Feb 2018 20:00:58 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
> > I didn't try the patch but currently echo
> > -- just outputs --, only - terminates options for echo.
> 
> Yes, the -- behaviour appears to be general behaviour, in fact,
> so not something that should be changed.  So indeed it's hard to
> do this at the moment in a shell script without some kind of kludge
> for zsh.  "disable echo" and use /bin/echo might be the best bet.
> 
> However, I'm not really sure if that makes it less or actually more
> useful to align with other shells (with POSIXBUILTINS) from now on...
> it's not obvious perpetuating the need for a kludge for ever more
> is the best bet.
[...]

IMO, the best thing to do here is to do nothing. Leave it as it
is.

The fact that - marks the end of options in zsh is documented
and relatively well known.

See
https://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo

That makes it one of the very few echo implementations that can
actually output arbitrary strings reliably (the only one  if you
consider that zsh is the only shell that can store NULs in its
variables)

echo -E - $var

The only other modern implementation I'm aware of that can do
that (in a Bourne-like shell) is yash's with its:

ECHO_STYLE=raw echo "$var"

To be POSIX compliant, echo -- *must* output --<nl> (-- as an
end-of-option marker must *not* be supported), and support for
-, -e and -E should be disabled. Also echo -nn should output
-nn<nl>.

However doing that would certainly break many scripts as the sh
emulation is often used to interpret code written for bash and
bash is also not POSIX compliant in that regard even in POSIX
mode (unless the xpg_echo option is also enabled) as "echo -e"
doesn't output "-e<nl>" there.

To be UNIX compliant, no option should be recognised and -e
should be the default.

People already know or should already know that echo cannot be
used for portability/reliability. It's too late to fix it and
zsh's implementation is actually the least broken of them (for
the very reason that it supports a way to mark the end of
options)..

zsh does support the POSIX printf and the ksh print which have a
more reliable and portable API (at least when limited to the
basic usage of echo, with the caveat that print '\01234' behaves
differently in zsh than in other ksh implementations).

See also https://github.com/att/ast/issues/370 for ksh93, where
they considered changing the behaviour of echo and eventually
backed down when considering the backward compatibility risk.

Note that pdksh was another shell that skipped "-" arguments.
But the "-" didn't mark the end of options, it was a bug in the
option parsing.

-- 
Stephane

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On 24 February 2018 08:20:40 GMT+00:00, Stephane Chazelas <stephane.chazelas@gmail.com> wrote:
>2018-02-22 19:34:23 +0000, Peter Stephenson:
>> On Thu, 22 Feb 2018 20:00:58 +0100
>> Mikael Magnusson <mikachu@gmail.com> wrote:
>> > I didn't try the patch but currently echo
>> > -- just outputs --, only - terminates options for echo.
>> 
>> Yes, the -- behaviour appears to be general behaviour, in fact,
>> so not something that should be changed.  So indeed it's hard to
>> do this at the moment in a shell script without some kind of kludge
>> for zsh.  "disable echo" and use /bin/echo might be the best bet.
>> 
>> However, I'm not really sure if that makes it less or actually more
>> useful to align with other shells (with POSIXBUILTINS) from now on...
>> it's not obvious perpetuating the need for a kludge for ever more
>> is the best bet.
>[...]
>
>IMO, the best thing to do here is to do nothing. Leave it as it
>is.

OK, this all sounds rational.

Perhaps the difference could be flagged up better in the do.

Thanks
pws

Re: [BUG]builtin echo error doing arguments parsing

[Peter Stephenson]

On Sat, 24 Feb 2018 09:51:51 +0000
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> Perhaps the difference could be flagged up better in the do.

(or "doc" as we say, if we're not sending replies from our mobile
phones.)

I'm sure somebody can object to aspects of the following.

pws

diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo
index 6c7ec4b..9014c78 100644
--- a/Doc/Zsh/builtins.yo
+++ b/Doc/Zsh/builtins.yo
@@ -504,6 +504,15 @@ pindex(BSD_ECHO, use of)
 The tt(-E) flag, or the tt(BSD_ECHO) option, can be used to disable
 these escape sequences.  In the latter case, tt(-e) flag can be used to
 enable them.
+
+Note that for standards compliance a double dash does not terminate
+option processing; instead, it is printed directly.  However, a
+single dash does terminate option processing, so the first dash,
+possibly following options, is not printed, but everything following it
+is printed as an argument.  The single dash behaviour is different
+from other shells.  For a more portable way of printing text, see
+tt(printf), and for a more controllable way of printing text within zsh,
+see tt(print).
 )
 module(echotc)(zsh/termcap)
 module(echoti)(zsh/terminfo)

Re: [BUG]builtin echo error doing arguments parsing

[Stephane Chazelas]

(full quote below for reference as that was over a year ago)

I had requested the Austin Group amend the POSIX specification
to allow for echo -e, echo -E, echo -:

http://austingroupbugs.net/view.php?id=1222

But (as kind of anticipated) it looks like they will add -e and
-E, but not zsh's -

http://austingroupbugs.net/view.php?id=1222#c4373
https://posix@posix.rhansen.org/p/2019-04-25 (password in
https://www.mail-archive.com/austin-group-l@opengroup.org/msg03909.html)

So it may be worth disabling the special handling of "echo -" in
sh emulation (and keep the rest as-is). Also, maybe add a
xpg_echo alias to no_bsd_echo for bash compatibility.

-- 
Stephane

2018-02-24 08:20:40 +0000, Stephane Chazelas:
> 2018-02-22 19:34:23 +0000, Peter Stephenson:
> > On Thu, 22 Feb 2018 20:00:58 +0100
> > Mikael Magnusson <mikachu@gmail.com> wrote:
> > > I didn't try the patch but currently echo
> > > -- just outputs --, only - terminates options for echo.
> > 
> > Yes, the -- behaviour appears to be general behaviour, in fact,
> > so not something that should be changed.  So indeed it's hard to
> > do this at the moment in a shell script without some kind of kludge
> > for zsh.  "disable echo" and use /bin/echo might be the best bet.
> > 
> > However, I'm not really sure if that makes it less or actually more
> > useful to align with other shells (with POSIXBUILTINS) from now on...
> > it's not obvious perpetuating the need for a kludge for ever more
> > is the best bet.
> [...]
> 
> IMO, the best thing to do here is to do nothing. Leave it as it
> is.
> 
> The fact that - marks the end of options in zsh is documented
> and relatively well known.
> 
> See
> https://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo
> 
> That makes it one of the very few echo implementations that can
> actually output arbitrary strings reliably (the only one  if you
> consider that zsh is the only shell that can store NULs in its
> variables)
> 
> echo -E - $var
> 
> The only other modern implementation I'm aware of that can do
> that (in a Bourne-like shell) is yash's with its:
> 
> ECHO_STYLE=raw echo "$var"
> 
> To be POSIX compliant, echo -- *must* output --<nl> (-- as an
> end-of-option marker must *not* be supported), and support for
> -, -e and -E should be disabled. Also echo -nn should output
> -nn<nl>.
> 
> However doing that would certainly break many scripts as the sh
> emulation is often used to interpret code written for bash and
> bash is also not POSIX compliant in that regard even in POSIX
> mode (unless the xpg_echo option is also enabled) as "echo -e"
> doesn't output "-e<nl>" there.
> 
> To be UNIX compliant, no option should be recognised and -e
> should be the default.
> 
> People already know or should already know that echo cannot be
> used for portability/reliability. It's too late to fix it and
> zsh's implementation is actually the least broken of them (for
> the very reason that it supports a way to mark the end of
> options)..
> 
> zsh does support the POSIX printf and the ksh print which have a
> more reliable and portable API (at least when limited to the
> basic usage of echo, with the caveat that print '\01234' behaves
> differently in zsh than in other ksh implementations).
> 
> See also https://github.com/att/ast/issues/370 for ksh93, where
> they considered changing the behaviour of echo and eventually
> backed down when considering the backward compatibility risk.
> 
> Note that pdksh was another shell that skipped "-" arguments.
> But the "-" didn't mark the end of options, it was a bug in the
> option parsing.