prexec never gets empty string?

prexec never gets empty string?

[Derek Gleim]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

Am I misunderstanding the docs, or is this a bug?

> If the history mechanism is active (and the line was not discarded from
the history buffer), the string that the user typed is passed as the first
argument, otherwise it is an empty string.

Yet, whether I "discard" (as I understand it) a command because of
histnostore, histignorespace, histnodups, or by returning non-zero from
zshaddhistory(), I never get an empty string as the first argument in
preexec().

That is:

    % setopt histignorespace
    % preexec() { echo $1 }
    %   true ignore me
     true ignore me

zsh 5.0.2 (x86_64-pc-linux-gnu)
Ubuntu 14.04.1

By the way, for me this is related to a question I had opened
<http://superuser.com/questions/877206/zsh-prompts-history-number-sometimes-increments-after-ignored-commands>
on Stack Overflow, trying to get the prompt expansion `%!` to truly be "the
number that is to be assigned to the next command".

Thanks in advance.

Re: prexec never gets empty string?

[Peter Stephenson]

On Fri, 13 Feb 2015 11:19:42 -0600
Derek Gleim <mail@dcgleim.com> wrote:
>     % setopt histignorespace
>     % preexec() { echo $1 }
>     %   true ignore me
>      true ignore me

I think there must be some other ingredient involved (and it doesn't
appear to be histignorespace, which was my first guess).  Could you try
it with "zsh -f" and see what other options you need to set to produce
it?

> zsh 5.0.2 (x86_64-pc-linux-gnu)

That's really rather old now, though I've got a close version around and
it's not showing the problem either.

pws

Re: prexec never gets empty string?

[Derek Gleim]

[-- Attachment #1: Type: text/plain, Size: 507 bytes --]

Thanks for checking.  That actual was with `-f`, and I've now
double-checked all cases (setting only histignorespace, only histnostore,
etc) and still see the same behavior, even after upgrading to 5.0.7.

Couple more examples:

    % zshaddhistory() { return 1 }
    % preexec() { echo $1 }
    % true
    true

    % setopt histignorespace
    % alias test=' true'
    % preexec() { echo "$1,$2" }
    test,true

I feel like I should provide some more information, but I'm not sure what
would be helpful.

Re: prexec never gets empty string?

[Peter Stephenson]

On Fri, 13 Feb 2015 17:35:35 +0000
Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Fri, 13 Feb 2015 11:19:42 -0600
> Derek Gleim <mail@dcgleim.com> wrote:
> >     % setopt histignorespace
> >     % preexec() { echo $1 }
> >     %   true ignore me
> >      true ignore me
> 
> I think there must be some other ingredient involved (and it doesn't
> appear to be histignorespace, which was my first guess).

Bleugh.  I meant "it doesn't appear to be histreduceblanks".

pws

Re: prexec never gets empty string?

[Mikael Magnusson]

On Fri, Feb 13, 2015 at 6:19 PM, Derek Gleim <mail@dcgleim.com> wrote:
> Am I misunderstanding the docs, or is this a bug?
>
>> If the history mechanism is active (and the line was not discarded from
> the history buffer), the string that the user typed is passed as the first
> argument, otherwise it is an empty string.
>
> Yet, whether I "discard" (as I understand it) a command because of
> histnostore, histignorespace, histnodups, or by returning non-zero from
> zshaddhistory(), I never get an empty string as the first argument in
> preexec().
>
> That is:
>
>     % setopt histignorespace
>     % preexec() { echo $1 }
>     %   true ignore me
>      true ignore me

I certainly would never want preexec to be disabled just because a
line isn't stored in history.

-- 
Mikael Magnusson

Re: prexec never gets empty string?

[Derek Gleim]

[-- Attachment #1: Type: text/plain, Size: 402 bytes --]

> I certainly would never want preexec to be disabled just because a
line isn't stored in history.

I don't follow.  Me neither.  My understanding is if the line isn't going to
be stored, preexec is still called, but it is supposed to be given an empty
string as the first argument when the command will be ignored by history
(while the second and third arguments give you the command regardless). No?

Re: prexec never gets empty string?

[Mikael Magnusson]

On Fri, Feb 13, 2015 at 11:41 PM, Derek Gleim <mail@dcgleim.com> wrote:
>> I certainly would never want preexec to be disabled just because a
> line isn't stored in history.
>
> I don't follow.  Me neither.  My understanding is if the line isn't going to
> be stored, preexec is still called, but it is supposed to be given an empty
> string as the first argument when the command will be ignored by history
> (while the second and third arguments give you the command regardless). No?

Judging by http://www.zsh.org/mla/workers/2001/msg00605.html , the
empty string was never intended as a feature, just a workaround for
not having access to the string. I guess a subsequent change made the
string always be available. Possibly the feature that always keeps the
last command in history until you run another command is newer than
this change and is why it always works now.

-- 
Mikael Magnusson

Re: prexec never gets empty string?

[Mikael Magnusson]

On Sat, Feb 14, 2015 at 12:04 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
> On Fri, Feb 13, 2015 at 11:41 PM, Derek Gleim <mail@dcgleim.com> wrote:
>>> I certainly would never want preexec to be disabled just because a
>> line isn't stored in history.
>>
>> I don't follow.  Me neither.  My understanding is if the line isn't going to
>> be stored, preexec is still called, but it is supposed to be given an empty
>> string as the first argument when the command will be ignored by history
>> (while the second and third arguments give you the command regardless). No?
>
> Judging by http://www.zsh.org/mla/workers/2001/msg00605.html , the
> empty string was never intended as a feature, just a workaround for
> not having access to the string. I guess a subsequent change made the
> string always be available. Possibly the feature that always keeps the
> last command in history until you run another command is newer than
> this change and is why it always works now.

Yep, here we are, the month after:
http://www.zsh.org/mla/workers/2001/msg01028.html
So this part of the manual was only accurate for about 21 days in 2001.

-- 
Mikael Magnusson

Re: prexec never gets empty string?

[Peter Stephenson]

On Fri, 13 Feb 2015 16:41:24 -0600
Derek Gleim <mail@dcgleim.com> wrote:
> > I certainly would never want preexec to be disabled just because a
> line isn't stored in history.
> 
> I don't follow.  Me neither.  My understanding is if the line isn't going to
> be stored, preexec is still called, but it is supposed to be given an empty
> string as the first argument when the command will be ignored by history
> (while the second and third arguments give you the command regardless). No?

Sorry, no.  I was misunderstanding completely --- I thought your error
was that spaces were wrong in the line that was being output.

As Mikael says, it's certainly not the case that preexec is supposed to
veto lines based on the history.  It's purpose is unrelated to the
recording of history: it's there to pass information about the line
about to be executed.

pws

Re: prexec never gets empty string?

[Derek Gleim]

[-- Attachment #1: Type: text/plain, Size: 1537 bytes --]

I understood the real intent of preexec, I hoped I could use it as part of
some work-around for my real goal: to /detect/ when a command was (or, is
about to be, seems to be the more accurate way of putting it?) ignored from
history.  Something like "if preexex sees an command destined to be ignored
by
history, add one to the normal event count".  Just to re-cap, I thought I
could because of this bit in the manual:

> If the history mechanism is active (and the line was not discarded from
the
> history buffer), the string that the user typed is passed as the first
> argument, otherwise it is an empty string.

An impression that was reinforced for me when I also found the first post
Mikael cited:

> when the current command is discarded from the history (as is often the
case
> when the user is ignoring functions, history commands, and/or lines that
> start with a space) ... so I decided to set the first parameter to an
empty
> string when this happens.

... But, I missed the second post linked talking about leaving it off.  So,
thanks, not a bug, looks like the docs just need an update (21 days in 2001,
ha!).

As an aside, I'd still like to be able to get the prompt expansion `%!` to
always show the number to be assigned to the next command...

    % PROMPT='%! >' zsh --no-rcs --histignorespace
    1 > true ignore me
    2 >this command will really be event 1
    2 >

... maybe it's just me, but it trips me up sometimes (and histnostore has
the
same behavior).  But, looks like re-purposing preexec is a dead-end.

Re: prexec never gets empty string?

[Derek Gleim]

[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]

Sorry, I meant "minus one from the normal event count" in my
half-baked example.

On Mon, Feb 16, 2015 at 9:11 AM, Derek Gleim <mail@dcgleim.com> wrote:

> I understood the real intent of preexec, I hoped I could use it as part of
> some work-around for my real goal: to /detect/ when a command was (or, is
> about to be, seems to be the more accurate way of putting it?) ignored from
> history.  Something like "if preexex sees an command destined to be
> ignored by
> history, add one to the normal event count".  Just to re-cap, I thought I
> could because of this bit in the manual:
>
> > If the history mechanism is active (and the line was not discarded from
> the
> > history buffer), the string that the user typed is passed as the first
> > argument, otherwise it is an empty string.
>
> An impression that was reinforced for me when I also found the first post
> Mikael cited:
>
> > when the current command is discarded from the history (as is often the
> case
> > when the user is ignoring functions, history commands, and/or lines that
> > start with a space) ... so I decided to set the first parameter to an
> empty
> > string when this happens.
>
> ... But, I missed the second post linked talking about leaving it off.  So,
> thanks, not a bug, looks like the docs just need an update (21 days in
> 2001,
> ha!).
>
> As an aside, I'd still like to be able to get the prompt expansion `%!` to
> always show the number to be assigned to the next command...
>
>     % PROMPT='%! >' zsh --no-rcs --histignorespace
>     1 > true ignore me
>     2 >this command will really be event 1
>     2 >
>
> ... maybe it's just me, but it trips me up sometimes (and histnostore has
> the
> same behavior).  But, looks like re-purposing preexec is a dead-end.
>

Re: prexec never gets empty string?

[Peter Stephenson]

On Mon, 16 Feb 2015 09:11:07 -0600
Derek Gleim <mail@dcgleim.com> wrote:
> > If the history mechanism is active (and the line was not discarded from
> the
> > history buffer), the string that the user typed is passed as the first
> > argument, otherwise it is an empty string.

Yes, it looks like that's been wrong for a long time.

> As an aside, I'd still like to be able to get the prompt expansion `%!` to
> always show the number to be assigned to the next command...
> 
>     % PROMPT='%! >' zsh --no-rcs --histignorespace
>     1 > true ignore me
>     2 >this command will really be event 1
>     2 >
> 
> ... maybe it's just me, but it trips me up sometimes (and histnostore has
> the same behavior).

It's perfectly possible you've hit a conflict that no one else has --- I
don't think %! is that commonly used.  Without some kind of internal
workaround it's hard to see what you can do without simply preforming
the same check as the history doesl.  One way of doing it would be
always to use zshaddhistory with appropriate logic and note there what
whether the line was to be ignored --- but it wouldn't be that surprsing
if there were corresponding gotchas there, too.

pws


diff --git a/Doc/Zsh/func.yo b/Doc/Zsh/func.yo
index 6e9cfee..ace0a40 100644
--- a/Doc/Zsh/func.yo
+++ b/Doc/Zsh/func.yo
@@ -247,10 +247,10 @@ findex(preexec)
 vindex(preexec_functions)
 item(tt(preexec))(
 Executed just after a command has been read and is about to be
-executed.  If the history mechanism is active (and the line was not
-discarded from the history buffer), the string that the user typed is
-passed as the first argument, otherwise it is an empty string.  The
-actual command that will be executed (including expanded aliases) is
+executed.  If the history mechanism is active (regardless of whether
the +line was discarded from the history buffer), the string that the
user +typed is passed as the first argument, otherwise it is an empty
string. +The actual command that will be executed (including expanded
aliases) is passed in two different forms: the second argument is a
single-line, size-limited version of the command (with things like
function bodies elided); the third argument contains the full text that
is being

-- 
Peter Stephenson | Principal Engineer Samsung Cambridge Solution Centre
Email: p.stephenson@samsung.com | Phone: +44 1223 434724 |
www.samsung.com

Re: prexec never gets empty string?

[Bart Schaefer]

On Feb 16,  9:11am, Derek Gleim wrote:
}
} As an aside, I'd still like to be able to get the prompt expansion `%!` to
} always show the number to be assigned to the next command...
} 
}     % PROMPT='%! >' zsh --no-rcs --histignorespace
}     1 > true ignore me
}     2 >this command will really be event 1
}     2 >

Hrm.  The history number isn't ajusted until the hend() call of the input
command following the one to be ignored.  That's because the previous
line actually CAN be referenced as (in the example above) !1 while you
are at the prompt that says "2 > ".

So there's an inherent conflict here; for a short while, there effectively
are two history events both numbered the same.  You're either going to be
misled about what the last command was numbered, or about what the current
command will be numbered.  This actually works well for HIST_IGNORE_DUPS
because the two identical commands are collapsed onto the same number, but
in other cases cases you end up with !N references where N finds something
that doesn't match your terminal scrollback.

Possible approaches:

(0) Do nothing.

(1) Make the numbering match the "what it will be" case.
    (a) Introduce a second history counter used only in prompt expansion
        that skips incrementing at hbegin() time [rather than hend()].
    (b) Do a significant rework of hist.c so the "curhist" variable can
        serve that role correctly.

(2) Rework history to leave holes in the number sequence when an event
    is skipped.  I believe this is already done for HIST_IGNORE_ALL_DUPS
    and HIST_EXPIRE_DUPS_FIRST when the older duplicate is dropped.  The
    history is stored in a hash table, so event numbers are just keys in
    the hash and holes are (I think) not an issue.

I'm not going to jump in on implementing any of the above any time soon,
but perhaps we can reach consensus on which is the best thing and then
somebody else can tackle it.