DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Rocky Bernstein]

On Wed, Aug 6, 2008 at 3:49 PM, Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
> "Rocky Bernstein" wrote:
>> But given the choice of adding
>>    1) an option in the return statement everywhere that is specific to
>> just "trap DEBUG" or
>>    2) specifying what specific numbers do when on a return from "trap DEBUG",
>>
>> isn't 2) simpler and more consistent with programming in shell
>> languages work? I take it as a given that every function or command is
>> going to have error codes that are somewhat arbitrary and that I'm not
>> going to intuit.
>
> I'm in two minds about this.

Ok. I'm sure you'll do what's best and let us know what is decided.

> If we didn't have the existing rule that
> any non-zero return status from TRAPDEBUG, or any explicit return from
> within an inline trap, forced an immediate return, then I'd agree simply
> adding to the set of useful statuses was cleaner and more natural.  As
> it is, we're now forced to pick some numeric value the user wouldn't
> naturally want to return from am enclosing context (since the return
> value is propagated).
>
> The option to return seems to me natural enough, because as "return"
> means "just return", so "return with an option" means "return but with
> slightly different semantics".  Having a different return status meaning
> to execute different code (rather than simply provide a different test
> for the caller, as normal) is an unusual enough effect that it doesn't
> strike me as the unequivocal answer.
>
> But, whatever.  If there's a number we all really like to mean "skip",
> we can go with that.  It's already working, is easy to document, and
> doesn't need any changes to parsing.
>
> (By the way, it wouldn't be too hard, if not completely trivial, to pass
> down the code about to be executed in a variable, say DEBUG_CMD_LINE, as
> reconstructed text, i.e. the same sort of format as what you get if you
> get the shell to output a shell function that's already loaded.  But
> it's messy enough that I won't unless it's definitely useful.)

My secret plan was to get the debugger working enough for folks to realize this
would be useful. In bash the variable $BASH_COMMAND holds the command that
is up for execution next. It is not the "line"  (as suggested in the
name DEBUG_CMD_LINE)
 but the last command or statement.

And  although DEBUG_CMD_LINE can be used in debugging, there's nothing
about it that
strictly needs to be so. For example suppose I wanted to get statistics on how
many commands are run that start with the letter A.

Anyway, in debugging where it's most useful are places where there are
statement
lines for example

   [[ -z $FOO ]] && bar || baz
   temp=x; x=y; y=temp

If one is is stepping along it can be useful to know in which part of
the line you
are about to run. The hueristic that bashdb uses is that if the line
number is the
same as the last line number that it stopped at but the command string changes,
then it's probably a good idea to show not just the line but also the command
that is about to be executed. In practice this means that one can assume that
whenever you see a line with more than one statement and no statement is
explicitly printed, then one can assume that the statement that is about to be
run is the first one shown. And if one is not sure one can just run a
print $BASH_COMMAND
debugger command.

>
> --
> Peter Stephenson <p.w.stephenson@ntlworld.com>
> Web page now at http://homepage.ntlworld.com/p.w.stephenson/
>

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Peter Stephenson]

On Wed, 6 Aug 2008 17:19:14 -0400
"Rocky Bernstein" <rocky.bernstein@gmail.com> wrote:
> > (By the way, it wouldn't be too hard, if not completely trivial, to pass
> > down the code about to be executed in a variable, say DEBUG_CMD_LINE, as
> > reconstructed text, i.e. the same sort of format as what you get if you
> > get the shell to output a shell function that's already loaded.  But
> > it's messy enough that I won't unless it's definitely useful.)
> 
> My secret plan was to get the debugger working enough for folks to
> realize this would be useful. In bash the variable $BASH_COMMAND holds
> the command that is up for execution next. It is not the "line"  (as
> suggested in the name DEBUG_CMD_LINE)
> but the last command or statement.

That's not quite how it works in zsh at the moment.  The DEBUG trap is run
at the level of a "sublist", which is anything separated by ;, &, end of
line, or special terminators in certain cases (e.g. end of case markers) so
in your examples

>    [[ -z $FOO ]] && bar || baz
>    temp=x; x=y; y=temp

the sublists are

[[ -z $FOO ]] && bar || baz
tmp=x
x=y
y=temp

and you get one execution of the trap for each group.  So DEBUG_CMD_LINE
would include the whole of each sublist.

Without having looked at it in depth, it sounds is if zsh is being a bit
more structure-based rather than line-based compared with bash.

It would be possible with a bit of fiddling to get the DEBUG trap to happen
at the level of elements of a sublist, i.e. commands separated by && and
||, but I'm not sure if that's worth it.  Beyond that you run into the
hideous complexity of pipelines and I don't think it can be pushed that far
down.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Stephane Chazelas]

On Thu, Aug 07, 2008 at 09:54:41AM +0100, Peter Stephenson wrote:
[...]
> It would be possible with a bit of fiddling to get the DEBUG trap to happen
> at the level of elements of a sublist, i.e. commands separated by && and
> ||, but I'm not sure if that's worth it.  Beyond that you run into the
> hideous complexity of pipelines and I don't think it can be pushed that far
> down.
[...]

But see how PS4 is expanded for every command:
$ zsh -o PROMPT_SUBST -c 'PS4="\$(od -vAn -N2 -x /dev/urandom) "
                          set -x; : a | : b; : c && : d'
 4542 : a
 e866 : b
 a7ad : c
 6051 : d

BTW, shouldn't PROMPT_SUBST be enabled by default in sh and ksh
emulation?

Is there a way to run shell code (other than assignments and
arithmetic expansion) upon the expansion of PS4? I thought of
/(e:code:), but in scalar context I couldn't find a way for
globbing to be performed.

-- 
Stéphane

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Peter Stephenson]

On Thu, 7 Aug 2008 10:31:40 +0100
Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> On Thu, Aug 07, 2008 at 09:54:41AM +0100, Peter Stephenson wrote:
> [...]
> > It would be possible with a bit of fiddling to get the DEBUG trap to happen
> > at the level of elements of a sublist, i.e. commands separated by && and
> > ||, but I'm not sure if that's worth it.  Beyond that you run into the
> > hideous complexity of pipelines and I don't think it can be pushed that far
> > down.
> [...]
> 
> But see how PS4 is expanded for every command:
> $ zsh -o PROMPT_SUBST -c 'PS4="\$(od -vAn -N2 -x /dev/urandom) "
>                           set -x; : a | : b; : c && : d'
>  4542 : a
>  e866 : b
>  a7ad : c
>  6051 : d

Yes, there are printprompt4()'s scattered all over the code.  However, PS4
doesn't interact with code execution as debug traps do.

> BTW, shouldn't PROMPT_SUBST be enabled by default in sh and ksh
> emulation?

If they do that, then presumably yes.

> Is there a way to run shell code (other than assignments and
> arithmetic expansion) upon the expansion of PS4? I thought of
> /(e:code:), but in scalar context I couldn't find a way for
> globbing to be performed.

I don't see how this differs from the stuff in $(...), which is treated as
a complete command line.  Do you mean you need code to run within
the current shell?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Stephane Chazelas]

On Thu, Aug 07, 2008 at 11:14:38AM +0100, Peter Stephenson wrote:
[...]
> > BTW, shouldn't PROMPT_SUBST be enabled by default in sh and ksh
> > emulation?
> 
> If they do that, then presumably yes.
[...]

It's required by SUSv3 for non-priviledged users for systems
supporting the User Portability Utilities option (so include the
sh of Unix conformant systems, but not necessarily POSIX
conformant ones I suppose).

> > Is there a way to run shell code (other than assignments and
> > arithmetic expansion) upon the expansion of PS4? I thought of
> > /(e:code:), but in scalar context I couldn't find a way for
> > globbing to be performed.
> 
> I don't see how this differs from the stuff in $(...), which is treated as
> a complete command line.  Do you mean you need code to run within
> the current shell?
[...]

Yes, sorry, I forgot the most important part of my sentence,
"without forking a subshell", or as you put it: "within
the current shell" but I suppose it's not easy to determine what
the current shell is when PS4 is expanded (especially in
pipe-lines).

The idea behind it being another way to have a DEBUG trap.

-- 
Stéphane

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Peter Stephenson]

On Thu, 7 Aug 2008 11:40:30 +0100
Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> On Thu, Aug 07, 2008 at 11:14:38AM +0100, Peter Stephenson wrote:
> [...]
> > > BTW, shouldn't PROMPT_SUBST be enabled by default in sh and ksh
> > > emulation?
> > 
> > If they do that, then presumably yes.
> [...]
> 
> It's required by SUSv3 for non-priviledged users for systems
> supporting the User Portability Utilities option (so include the
> sh of Unix conformant systems, but not necessarily POSIX
> conformant ones I suppose).

Looks like it's already set up for ksh.  It should probably be turned on
for other Bourne-style shells.  I suppose it wasn't done originally because
at the time that meant old-fashioned, fairly limited Bourne shells, but
nowadays it doesn't.  "promptvars" is on by default in bash.

Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.61
diff -u -r1.61 options.yo
--- Doc/Zsh/options.yo	12 Jun 2008 13:45:05 -0000	1.61
+++ Doc/Zsh/options.yo	7 Aug 2008 12:04:46 -0000
@@ -1009,7 +1009,7 @@
 )
 pindex(PROMPT_SUBST)
 cindex(prompt, parameter expansion)
-item(tt(PROMPT_SUBST) <K>)(
+item(tt(PROMPT_SUBST) <K> <S>)(
 If set, em(parameter expansion), em(command substitution) and
 em(arithmetic expansion) are performed in prompts.  Substitutions
 within prompts do not affect the command status.
Index: Src/options.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/options.c,v
retrieving revision 1.43
diff -u -r1.43 options.c
--- Src/options.c	31 Jul 2008 08:44:21 -0000	1.43
+++ Src/options.c	7 Aug 2008 12:04:47 -0000
@@ -198,7 +198,7 @@
 {{NULL, "promptcr",	      OPT_ALL},			 PROMPTCR},
 {{NULL, "promptpercent",      OPT_NONBOURNE},		 PROMPTPERCENT},
 {{NULL, "promptsp",	      OPT_ALL},			 PROMPTSP},
-{{NULL, "promptsubst",	      OPT_KSH},			 PROMPTSUBST},
+{{NULL, "promptsubst",	      OPT_BOURNE},		 PROMPTSUBST},
 {{NULL, "pushdignoredups",    OPT_EMULATE},		 PUSHDIGNOREDUPS},
 {{NULL, "pushdminus",	      OPT_EMULATE},		 PUSHDMINUS},
 {{NULL, "pushdsilent",	      0},			 PUSHDSILENT},

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Rocky Bernstein]

On Thu, Aug 7, 2008 at 8:06 AM, Peter Stephenson <pws@csr.com> wrote:
> On Thu, 7 Aug 2008 11:40:30 +0100
> Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
>> On Thu, Aug 07, 2008 at 11:14:38AM +0100, Peter Stephenson wrote:
>> [...]
>> > > BTW, shouldn't PROMPT_SUBST be enabled by default in sh and ksh
>> > > emulation?
>> >
>> > If they do that, then presumably yes.
>> [...]
>>
>> It's required by SUSv3 for non-priviledged users for systems
>> supporting the User Portability Utilities option (so include the
>> sh of Unix conformant systems, but not necessarily POSIX
>> conformant ones I suppose).
>
> Looks like it's already set up for ksh.  It should probably be turned on
> for other Bourne-style shells.  I suppose it wasn't done originally because
> at the time that meant old-fashioned, fairly limited Bourne shells, but
> nowadays it doesn't.  "promptvars" is on by default in bash.

Perhaps it is time to change the default in zsh as well. I don't think
it  ikely someone sets PS4 with backticks and $() and so on and
expects to see exactly those strings in trace output.

However if older zsh compatibility is a concern here, the usual thing
to do is warn that the option is changing, and after a release or so
make the change.

>
> Index: Doc/Zsh/options.yo
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
> retrieving revision 1.61
> diff -u -r1.61 options.yo
> --- Doc/Zsh/options.yo  12 Jun 2008 13:45:05 -0000      1.61
> +++ Doc/Zsh/options.yo  7 Aug 2008 12:04:46 -0000
> @@ -1009,7 +1009,7 @@
>  )
>  pindex(PROMPT_SUBST)
>  cindex(prompt, parameter expansion)
> -item(tt(PROMPT_SUBST) <K>)(
> +item(tt(PROMPT_SUBST) <K> <S>)(
>  If set, em(parameter expansion), em(command substitution) and
>  em(arithmetic expansion) are performed in prompts.  Substitutions
>  within prompts do not affect the command status.
> Index: Src/options.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/options.c,v
> retrieving revision 1.43
> diff -u -r1.43 options.c
> --- Src/options.c       31 Jul 2008 08:44:21 -0000      1.43
> +++ Src/options.c       7 Aug 2008 12:04:47 -0000
> @@ -198,7 +198,7 @@
>  {{NULL, "promptcr",          OPT_ALL},                  PROMPTCR},
>  {{NULL, "promptpercent",      OPT_NONBOURNE},           PROMPTPERCENT},
>  {{NULL, "promptsp",          OPT_ALL},                  PROMPTSP},
> -{{NULL, "promptsubst",       OPT_KSH},                  PROMPTSUBST},
> +{{NULL, "promptsubst",       OPT_BOURNE},               PROMPTSUBST},
>  {{NULL, "pushdignoredups",    OPT_EMULATE},             PUSHDIGNOREDUPS},
>  {{NULL, "pushdminus",        OPT_EMULATE},              PUSHDMINUS},
>  {{NULL, "pushdsilent",       0},                        PUSHDSILENT},
>
> --
> Peter Stephenson <pws@csr.com>                  Software Engineer
> CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
> Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070
>

Re: DEBUG_CMD_LINE (Was Re: PATCH: skip command from debug trap)

[Bart Schaefer]

On Aug 7,  9:37am, Rocky Bernstein wrote:
}
} Perhaps it is time to change the default in zsh as well. I don't think
} it  ikely someone sets PS4 with backticks and $() and so on and
} expects to see exactly those strings in trace output.

I strongly disagree.  Exporting PS4 (or any other PS* variable) is Just
Plain Wrong and shouldn't be considered as a reason for such a change,
and if you're putting those things in a variable from within zsh it's
negligible extra effort to set the option.
 
} However if older zsh compatibility is a concern here, the usual thing
} to do is warn that the option is changing, and after a release or so
} make the change.

Heh.  If you want to do it that way, I've probably got nothing to worry
about, as I'll be retired by the time the shell goes through two more
"stable" releases at the current rate.