zsh-users
 help / color / mirror / code / Atom feed
* .zsh_history
@ 2023-04-08 16:16 Perry Smith
  2023-04-08 16:53 ` .zsh_history Roman Perepelitsa
  2023-04-14 14:51 ` .zsh_history Felipe Contreras
  0 siblings, 2 replies; 24+ messages in thread
From: Perry Smith @ 2023-04-08 16:16 UTC (permalink / raw)
  To: zsh-users

[-- Attachment #1: Type: text/plain, Size: 1749 bytes --]

I don’t have anything special set up other than setting HISTFILE, HISTSIZE=2097152, and SAVEHIST=1048576.  I also have histignoredups and histignorealldups on.

The reason for the huge numbers is partially due to some confusion I had (see below) but basically I want to save unique commands that I did for very long periods of time so I don’t need to figure them out again.

I’m on macOS and they have this thing for “sessions” which I’ve turned off now (a few weeks ago).  The reason is that the session history files are limited to two weeks and so commands I did long ago are getting deleted and my .zsh_history file was only growing to be about 1000 lines.

My question is, if I start multiple terminals and run various commands and eventually they each log out or exit, I’m wondering what happens to the .zsh_history file.  Does the last shell to exit rewrite the file using its own history which (I’m guessing) is the contents of .zsh_history when the shell started up plus the commands that were executed in that shell?  This would overwrite all the commands done in the other shells that have just exited.

Alternatively, the shell could re-read .zsh_history at the time it is exiting and append the commands that were executed within that shell and write the result out — thus preserving the commands done by other shells.  I see in the /etc/zsh_Apple_Terminal file some dancing around that appears to be trying to re-read the history file at the time the shell is exiting.  I haven’t looked closely at what precisely it is doing.  I thought I would ask first what is the default behavior.

TL; DR: question: Does zsh re-read the .zsh_history file at the time it is exiting by default?

Thank you,
Perry


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 16:16 .zsh_history Perry Smith
@ 2023-04-08 16:53 ` Roman Perepelitsa
  2023-04-08 17:23   ` .zsh_history Ray Andrews
  2023-04-14 14:00   ` .zsh_history Felipe Contreras
  2023-04-14 14:51 ` .zsh_history Felipe Contreras
  1 sibling, 2 replies; 24+ messages in thread
From: Roman Perepelitsa @ 2023-04-08 16:53 UTC (permalink / raw)
  To: Perry Smith; +Cc: zsh-users

On Sat, Apr 8, 2023 at 6:17 PM Perry Smith <pedz@easesoftware.com> wrote:
>
> I don’t have anything special set up other than setting HISTFILE, HISTSIZE=2097152, and SAVEHIST=1048576.  I also have histignoredups and histignorealldups on.
>
>
> TL; DR: question: Does zsh re-read the .zsh_history file at the time it is exiting by default?

Yes. If you exit all shells, you'll have history from all of them
saved within $HISTFILE.

FWIW, I personally dislike all global rc files and especially those
that try to do non-trivial things (like on macOS). I disable global rc
files on all machines and all operating systems that I use.

Roman.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 16:53 ` .zsh_history Roman Perepelitsa
@ 2023-04-08 17:23   ` Ray Andrews
  2023-04-08 17:27     ` .zsh_history Roman Perepelitsa
  2023-04-14 14:28     ` .zsh_history Felipe Contreras
  2023-04-14 14:00   ` .zsh_history Felipe Contreras
  1 sibling, 2 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-08 17:23 UTC (permalink / raw)
  To: zsh-users


On 2023-04-08 09:53, Roman Perepelitsa wrote:
>
> Yes. If you exit all shells, you'll have history from all of them
> saved within $HISTFILE.
>
>
I can't remember if it's stock or something I cobbled together for 
myself, but my history is updated in real time, I don't need to exit.  
All shells have all commands available from all other shells in real 
time.  Only thing is that I have to press ENTER to update each shell to 
the current state.  Tho my history selection function doesn't even need 
that, it's up to the second all the time.  Mind, I've sometimes wanted 
to be able to isolate a shell from history and I expect that's easy, tho 
I don't know how.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 17:23   ` .zsh_history Ray Andrews
@ 2023-04-08 17:27     ` Roman Perepelitsa
  2023-04-08 17:35       ` .zsh_history Perry Smith
  2023-04-08 17:54       ` .zsh_history Ray Andrews
  2023-04-14 14:28     ` .zsh_history Felipe Contreras
  1 sibling, 2 replies; 24+ messages in thread
From: Roman Perepelitsa @ 2023-04-08 17:27 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sat, Apr 8, 2023 at 7:23 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2023-04-08 09:53, Roman Perepelitsa wrote:
> >
> > Yes. If you exit all shells, you'll have history from all of them
> > saved within $HISTFILE.
> >
> I can't remember if it's stock or something I cobbled together for
> myself, but my history is updated in real time, I don't need to exit.

Indeed, there are options that do this. All options are described in
`man zshoptions`.

Perry's options (which I've quoted in my original reply) require him
to exit zsh before history is written.

Roman.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 17:27     ` .zsh_history Roman Perepelitsa
@ 2023-04-08 17:35       ` Perry Smith
  2023-04-08 17:54       ` .zsh_history Ray Andrews
  1 sibling, 0 replies; 24+ messages in thread
From: Perry Smith @ 2023-04-08 17:35 UTC (permalink / raw)
  To: Roman Perepelitsa; +Cc: Ray Andrews, zsh-users

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]



> On Apr 8, 2023, at 12:27, Roman Perepelitsa <roman.perepelitsa@gmail.com> wrote:
> 
> On Sat, Apr 8, 2023 at 7:23 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> 
>> On 2023-04-08 09:53, Roman Perepelitsa wrote:
>>> 
>>> Yes. If you exit all shells, you'll have history from all of them
>>> saved within $HISTFILE.
>>> 
>> I can't remember if it's stock or something I cobbled together for
>> myself, but my history is updated in real time, I don't need to exit.
> 
> Indeed, there are options that do this. All options are described in
> `man zshoptions`.
> 
> Perry's options (which I've quoted in my original reply) require him
> to exit zsh before history is written.

Yea.  I’ve seen those options.  Generally that isn’t my pain point.  It is (was) Apple’s session stuff and not having history from long ago.

Thank you all


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 17:27     ` .zsh_history Roman Perepelitsa
  2023-04-08 17:35       ` .zsh_history Perry Smith
@ 2023-04-08 17:54       ` Ray Andrews
  2023-04-14 14:36         ` .zsh_history Felipe Contreras
  1 sibling, 1 reply; 24+ messages in thread
From: Ray Andrews @ 2023-04-08 17:54 UTC (permalink / raw)
  To: zsh-users


On 2023-04-08 10:27, Roman Perepelitsa wrote:
>
> Perry's options (which I've quoted in my original reply) require him
> to exit zsh before history is written.

I do remember that one's control of history is essentially perfect, you 
can get just about anything you want.  History is one of those features 
of zsh that is tractable and understandable without too much head 
scratching and without any weird gotchas. Only thing missing IIRC is 
being able to pull apart: "$ echo command one; echo command two; echo 
command three" into it's constituent commands.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 16:53 ` .zsh_history Roman Perepelitsa
  2023-04-08 17:23   ` .zsh_history Ray Andrews
@ 2023-04-14 14:00   ` Felipe Contreras
  1 sibling, 0 replies; 24+ messages in thread
From: Felipe Contreras @ 2023-04-14 14:00 UTC (permalink / raw)
  To: Roman Perepelitsa; +Cc: Perry Smith, zsh-users

On Sat, Apr 8, 2023 at 11:54 AM Roman Perepelitsa
<roman.perepelitsa@gmail.com> wrote:
>
> On Sat, Apr 8, 2023 at 6:17 PM Perry Smith <pedz@easesoftware.com> wrote:
> >
> > I don’t have anything special set up other than setting HISTFILE, HISTSIZE=2097152, and SAVEHIST=1048576.  I also have histignoredups and histignorealldups on.
> >
> >
> > TL; DR: question: Does zsh re-read the .zsh_history file at the time it is exiting by default?
>
> Yes. If you exit all shells, you'll have history from all of them
> saved within $HISTFILE.

That isn't true.

That's true *only* if APPEND_HISTORY is set, which yeah, is the
default, but still, let's be clear.

When APPEND_HISTORY isn't set, only the history of the last shell is saved.

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 17:23   ` .zsh_history Ray Andrews
  2023-04-08 17:27     ` .zsh_history Roman Perepelitsa
@ 2023-04-14 14:28     ` Felipe Contreras
  2023-04-14 15:18       ` .zsh_history Ray Andrews
  1 sibling, 1 reply; 24+ messages in thread
From: Felipe Contreras @ 2023-04-14 14:28 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sat, Apr 8, 2023 at 12:23 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2023-04-08 09:53, Roman Perepelitsa wrote:
> >
> > Yes. If you exit all shells, you'll have history from all of them
> > saved within $HISTFILE.
> >
> >
> I can't remember if it's stock or something I cobbled together for
> myself, but my history is updated in real time, I don't need to exit.
> All shells have all commands available from all other shells in real
> time.  Only thing is that I have to press ENTER to update each shell to
> the current state.  Tho my history selection function doesn't even need
> that, it's up to the second all the time.

You have SHARE_HISTORY on. It's not on by default.

Why not give a try to some minimal defauts?

  mkdir -p /tmp/zsh
  touch /tmp/zsh/.zshrc
  ZDOTDIR=/tmp/zsh HISTFILE=/tmp/zsh/.history SAVEHIST=30 zsh

It seems nobody knows what vanilla zsh actually does.

> Mind, I've sometimes wanted to be able to isolate a shell from history
> and I expect that's easy, tho I don't know how.

Quite easy:

  fc -p

It creates a new history stack (no $HISTFILE). To return to the previous one:

  fc -P

Cheers.

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 17:54       ` .zsh_history Ray Andrews
@ 2023-04-14 14:36         ` Felipe Contreras
  2023-04-14 15:27           ` .zsh_history Ray Andrews
  0 siblings, 1 reply; 24+ messages in thread
From: Felipe Contreras @ 2023-04-14 14:36 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sat, Apr 8, 2023 at 12:54 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2023-04-08 10:27, Roman Perepelitsa wrote:
> >
> > Perry's options (which I've quoted in my original reply) require him
> > to exit zsh before history is written.
>
> I do remember that one's control of history is essentially perfect, you
> can get just about anything you want.  History is one of those features
> of zsh that is tractable and understandable without too much head
> scratching and without any weird gotchas.

I couldn't disagree more.

Even zsh experts don't understand what HISTSIZE and SAVEHIST are for.
Nevermind their interactions with APPEND_HISTORY. Maybe the other
options are more understandable, but certainly not the basic ones.

Or you tell me, do you think everyone understands what this does?

  HISTSIZE=100
  SAVEHIST=200

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-08 16:16 .zsh_history Perry Smith
  2023-04-08 16:53 ` .zsh_history Roman Perepelitsa
@ 2023-04-14 14:51 ` Felipe Contreras
  1 sibling, 0 replies; 24+ messages in thread
From: Felipe Contreras @ 2023-04-14 14:51 UTC (permalink / raw)
  To: Perry Smith; +Cc: zsh-users

On Sat, Apr 8, 2023 at 11:17 AM Perry Smith <pedz@easesoftware.com> wrote:
>
> I don’t have anything special set up other than setting HISTFILE, HISTSIZE=2097152, and SAVEHIST=1048576.  I also have histignoredups and histignorealldups on.
>
> The reason for the huge numbers is partially due to some confusion I had (see below) but basically I want to save unique commands that I did for very long periods of time so I don’t need to figure them out again.

I do the same, but hopefully you are not relying on zsh to do that,
because you have to grep the history file yourself.

> I’m on macOS and they have this thing for “sessions” which I’ve turned off now (a few weeks ago).  The reason is that the session history files are limited to two weeks and so commands I did long ago are getting deleted and my .zsh_history file was only growing to be about 1000 lines.
>
> My question is, if I start multiple terminals and run various commands and eventually they each log out or exit, I’m wondering what happens to the .zsh_history file.  Does the last shell to exit rewrite the file using its own history which (I’m guessing) is the contents of .zsh_history when the shell started up plus the commands that were executed in that shell?  This would overwrite all the commands done in the other shells that have just exited.

It writes all the current history of the shell (`fc -l 0`), but by
default APPEND_HISTORY is on, so it would append the current history
to the file. So in practice the history is not overridden.

But is this realistic? Do you actually have more than 1048576 lines in
your history? What is $HISTCMD?

> Alternatively, the shell could re-read .zsh_history at the time it is exiting and append the commands that were executed within that shell and write the result out — thus preserving the commands done by other shells.

That's exactly what it does (unless you set NO_APPEND_HISTORY).

Cheers.

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-14 14:28     ` .zsh_history Felipe Contreras
@ 2023-04-14 15:18       ` Ray Andrews
  0 siblings, 0 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-14 15:18 UTC (permalink / raw)
  To: zsh-users


On 2023-04-14 07:28, Felipe Contreras wrote:
> Mind, I've sometimes wanted to be able to isolate a shell from history
>> and I expect that's easy, tho I don't know how.
> Quite easy:
>
>    fc -p
>
> It creates a new history stack (no $HISTFILE). To return to the previous one:
>
>    fc -P

Thanks, I'll file that.  I'm not doing any shelling right now, but next 
time I am, I'll play with that.  You hafta love the level of control.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-14 14:36         ` .zsh_history Felipe Contreras
@ 2023-04-14 15:27           ` Ray Andrews
  2023-04-15  4:49             ` .zsh_history Felipe Contreras
  0 siblings, 1 reply; 24+ messages in thread
From: Ray Andrews @ 2023-04-14 15:27 UTC (permalink / raw)
  To: zsh-users


On 2023-04-14 07:36, Felipe Contreras wrote:

> I do remember that one's control of history is essentially perfect, you
>> can get just about anything you want.  History is one of those features
>> of zsh that is tractable and understandable without too much head
>> scratching and without any weird gotchas.
> I couldn't disagree more.
>
It could very well be that a closer examination than I've made would 
reveal various mysteries and gotchas.  All I can say is that I've never 
wanted something from history that it couldn't provide.  One exception: 
"$ echo one; echo two; echo three" ... you can't, AFAICT, break up the 
line into it's component commands.  One of the first brick walls I 
crashed into learning zsh was realizing that a command has no access to 
it's own unexpanded, 'raw' tail -- which still strikes me as 
incomprehensible.   Pulling a command back from history comes closest, 
but again there's that 'many commands on one line' issue.  I'd sure love 
to be able to pull just the text of a single command.





^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-14 15:27           ` .zsh_history Ray Andrews
@ 2023-04-15  4:49             ` Felipe Contreras
  2023-04-15 15:29               ` .zsh_history Ray Andrews
  0 siblings, 1 reply; 24+ messages in thread
From: Felipe Contreras @ 2023-04-15  4:49 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Fri, Apr 14, 2023 at 10:28 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2023-04-14 07:36, Felipe Contreras wrote:
>
> > I do remember that one's control of history is essentially perfect, you
> >> can get just about anything you want.  History is one of those features
> >> of zsh that is tractable and understandable without too much head
> >> scratching and without any weird gotchas.
> > I couldn't disagree more.
> >
> It could very well be that a closer examination than I've made would
> reveal various mysteries and gotchas.  All I can say is that I've never
> wanted something from history that it couldn't provide.

Yes, that is true.

My current history configuration is *exactly* what I want. I have a
large ~/.history file that is updated every time I type a command, but
it doesn't affect any currently running shell (inc_append_history).

In addition to that, every command I type is stored in a separate
historical file that I use to generate statistics (top 20 commands and
top 20 directories).

So let me clarify my statement: the zsh history configuration is very
powerful and simple, and it allows everyone to configure zsh the way
they want it.

*But* that doesn't mean everyone understands all the configurations.
So I think a more accurate statement would be that the configuration
is unidirectional: everyone understands their own configuration,
because they had an idea of what made sense, looked for a way to
achieve it, and found it. But other people have other ideas, and their
configuration isn't necessarily understood by others.

> One exception: "$ echo one; echo two; echo three" ... you can't,
> AFAICT, break up the line into it's component commands.

Is that really a zsh thing? I think the same happens in all shells.

> I'd sure love to be able to pull just the text of a single command.

I'm 100% certain you can do that, but not with history expansion, you'll
need to use zle for that.

To me (and I'm not sure if this analogy will land), it's like trying to
update CSS dynamically using CSS, when JavaScript is the tool for that.

This is not something I generally use, but I checked the documentation
of default zle widgets, and you can copy the previous word with
CTRL+ALT+_ and you can switch back and forth the previous two words with
ALT+t.

So you can write zle widgets to do anything you want to do with the
current line, but that's significantly different from history
substitution (e.g. !!0).

Cheers.

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-15  4:49             ` .zsh_history Felipe Contreras
@ 2023-04-15 15:29               ` Ray Andrews
  2023-04-15 19:47                 ` "Pull just the text of a single command" (was Re: .zsh_history) Bart Schaefer
  2023-04-16  7:38                 ` .zsh_history Roman Perepelitsa
  0 siblings, 2 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-15 15:29 UTC (permalink / raw)
  To: zsh-users


On 2023-04-14 21:49, Felipe Contreras wrote:
> *But* that doesn't mean everyone understands all the configurations. 


That's for sure.  Like anything that grew by accretion, not design, as 
new features were tacked on, the documentation became more and more 
incoherent.  I don't even attempt to understand completion now, I threw 
myself at it a couple of times and failed miserably.  complete completer 
completion completed completable ... only Satan really understands it.

> So I think a more accurate statement would be that the configuration
> is unidirectional: everyone understands their own configuration,
> because they had an idea of what made sense, looked for a way to
> achieve it, and found it. But other people have other ideas, and their
> configuration isn't necessarily understood by others.

Yup.  Write-only configuration.

I'm 100% certain you can do that, but not with history expansion, you'll
> need to use zle for that.
I'd sure like to know how!  Back in the day nobody seemed to know how to 
do it, not even Bart.  I came pretty close using 'noglob' and my 
functions work OK, but it would be vastly simpler if I could recover 
unexpanded command tails in some direct way.   As it is I run the 
function as an alias preceded by 'noglob' then save the literal text of 
the command tail as needed then 'eval' it.  Quite dumb, but it's the 
only way that I've found.
> So you can write zle widgets to do anything you want to do with the
> current line, but that's significantly different from history
> substitution (e.g. !!0).

Never played with that.  Don't know anything about it.  All I want is to 
be able to recover a command tail *exactly* as it was typed.

my_function ()

{

grab_tail # writes tail to 'my_tail'.

echo "my tail, exactly as typed, is: $my_tail"

}

$ my_function $path $(eval 'ls *') one two three ! < > ``.."" &>^!

my tail, exactly as typed, is: $path $(eval 'ls *') one two three ! < > 
``.."" &>^!

... no simple thing, since it is the nature of shells to expand and 
process command lines, not save them as typed.  But history does almost 
do it.



>
> Cheers.
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-15 15:29               ` .zsh_history Ray Andrews
@ 2023-04-15 19:47                 ` Bart Schaefer
  2023-04-15 22:47                   ` Ray Andrews
  2023-04-16  7:38                 ` .zsh_history Roman Perepelitsa
  1 sibling, 1 reply; 24+ messages in thread
From: Bart Schaefer @ 2023-04-15 19:47 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Sat, Apr 15, 2023 at 8:30 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I'm 100% certain you can do that, but not with history expansion, you'll
> > need to use zle for that.
> I'd sure like to know how!  Back in the day nobody seemed to know how to
> do it, not even Bart.

I'm not sure which past discussion you're referencing, but it's
possible you're mixing your metaphors here.

"History expansion" is what happens when you use !! or !-2 or similar
references at the prompt.  That happens before pretty much anything
else as soon as you hit enter (or even sooner if you have setopt
magic-space or similar).  In this case you can use !# to refer
backwards to any words already entered:

% echo first !#:1 last !#:3
first first last last
% echo everything so far !# and more
everything so far echo everything so far and more

References to the stored history with "fc" and related commands like
"history" accesses events (don't think in terms of "lines" -- a
multi-line command like if/while/for is usually one event) that have
previously been executed.  This doesn't have access to the event
currently being executed, and those events are stored with history
expansions (like !# in the foregoing, etc.) already replaced.

ZLE has access to what you're entering as you are entering it and
before either of the above things (or any other expansions like
globbing etc.) happen.  So if you want the unaltered text as you work
with it, ZLE is where you get it.  The best place to grab it after
hitting enter is probably in the zle-line-finish hook, but that won't
always give you full multi-line events (e.g., when the PS2 prompt is
active, you get it a line at a time).

Finally, the preexec function (and hook) gives access to the full
event with only history expansions already done (in $1), before it is
entered into the "fc"-able history, along with the event with all the
other expansions done (in $3).  There's nowhere other than ZLE that
can give you the raw ! references, and preexec can't give you $1 in a
shell script where history isn't available at all.

What you can't do is defer expansions (other than globbing, via
noglob) until "inside" the execution of the command (e.g., your shell
function).  I have a vague recollection that this may relate to you
being accustomed to DOS/Windows where expansion of the arguments is
done by calling a library after the fact, as opposed to the shell
doing all of it first.

Anyway, everything you asked about is there, you just need to adjust
your thinking about the sequence of events to find the right place to
accomplish what you want.

Of course this has NOTHING to do with how SAVING the history is configured.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-15 19:47                 ` "Pull just the text of a single command" (was Re: .zsh_history) Bart Schaefer
@ 2023-04-15 22:47                   ` Ray Andrews
  2023-04-15 23:26                     ` Bart Schaefer
  0 siblings, 1 reply; 24+ messages in thread
From: Ray Andrews @ 2023-04-15 22:47 UTC (permalink / raw)
  To: zsh-users


On 2023-04-15 12:47, Bart Schaefer wrote:
> I'm not sure which past discussion you're referencing, but it's 


This goes back to when I first started with Linux.  It was one of my 
first issues here.  You weighed in on the topic but 'noglob' was the 
best we did at the time.  Quite possible I was not communicating very well.

> So if you want the unaltered text as you work
> with it, ZLE is where you get it.

That's what I want.  I only refer to history because:

$ print -rS "$@"

... as I have it now, does write the command with its tail unexpanded, 
and, combined with 'noglob' I get pretty much what I want.  But it's a 
huge labor to do something that should be very simple.

> The best place to grab it after
> hitting enter is probably in the zle-line-finish hook, but that won't
> always give you full multi-line events (e.g., when the PS2 prompt is
> active, you get it a line at a time).
Sounds worth trying.
>
> Finally, the preexec function (and hook) gives access to the full
> event with only history expansions already done (in $1), before it is
> entered into the "fc"-able history, along with the event with all the
> other expansions done (in $3).

That sounds exactly right.  Come to that, an expansion of a call to 
history -- the ! stuff -- would be the one and only thing I would want 
pre-expanded.  It's sorta the obvious exception because it's a meta 
command -- a command to call a command.  Nuts, I'm trying to remember if 
I ever tried something like that.  If you're recommending it now, you 
would have recommended it then, and I'd have tried it then ... but I 
can't remember.  But I know I didn't try anything with zle.

> There's nowhere other than ZLE that
> can give you the raw ! references, and preexec can't give you $1 in a
> shell script where history isn't available at all.
>
> What you can't do is defer expansions (other than globbing, via
> noglob) until "inside" the execution of the command (e.g., your shell
> function).  I have a vague recollection that this may relate to you
> being accustomed to DOS/Windows where expansion of the arguments is
> done by calling a library after the fact, as opposed to the shell
> doing all of it first.
Yeah.  We did conclude that what I was trying to do was a fundamental 
violation of basic Unix/Linux philosophy.
>
> Anyway, everything you asked about is there, you just need to adjust
> your thinking about the sequence of events to find the right place to
> accomplish what you want.

I'm not fussy.  Just so long as I can access the tail from within a 
function, I don't care how that tail gets stored.   It's hard to explain 
why this matters but most of my wrapper functions are ever more complex 
massaging of the output of some basic command -- and lots of color.  If 
I need a modification, rather than edit my own code, what I do is first 
edit the 'real' command to make the change I want, then when I know what 
I want, I edit my own code so as to achieve that result.  For example 
'l' wraps 'ls':

$ l

LISTING of "*": All file types, INsensitive. Sorting upside down by: 
Mod. Time:

  18432 [2022-10-09--16:43] 2021-08-12-backup/
   1024 [2022-11-12--09:43] Boneyard/
   4096 [2022-11-20--10:38] 2022-11-20-backup/
  18432 [2022-11-20--14:12] Empty/
   4096 [2022-11-28--06:55] Znt/

...

   8649 [2023-03-22--19:54] miscfunctions,8,local files only with changed
   8649 [2023-03-22--19:54] miscfunctions
  19831 [2023-03-22--20:28] l,4,BUG  x colorized in l X
  19831 [2023-03-22--20:28] l

Items found: 169
Total bytes in directory "/aWorking/Zsh/Source/Wk": 1.7M
Total including subdirs: 25M

... a listing the way I like it to look.  What 'l' actually executes is:

$ ls --time-style='+[%F--%H:%M]' --group-directories-first -AFrGgdt 
--color=always (#i)* 2> /dev/null | sed -r "s/^(.{10} 
{1,3}[[:digit:]]{1,3} )/ /" | egrep -v '^total' | sed 
'/;34m\.\x1b\[0m\/$/d' | perl -pe "s|\] (.*?)()|\] \1^[[31;1m\2^[[0m|i"

So, using my convoluted method:

$ l ,H

... writes the above command to history and then up-arrow retrieves it 
for editing.  Once I've made whatever change I want to the 'real' 
command, then I edit my own code so as to implement the modification.  
Basically I can edit in two stages.  And the thing is that the '*' and 
various other expandable things must *not* expand, otherwise the command 
written to history can become almost infinitely long.  Clear as mud?  
Anyway it works marvelously but is convoluted ... unless I can grab the 
tail, unexpanded, with less trouble.  That way, instead of storing it 
via history, I can access it some other way.  IOW the only reason I use 
history is because it shows unexpanded commands. Show me how to do this 
with zle or preexec and I'll be in heaven.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-15 22:47                   ` Ray Andrews
@ 2023-04-15 23:26                     ` Bart Schaefer
  2023-04-16 15:53                       ` Bart Schaefer
  0 siblings, 1 reply; 24+ messages in thread
From: Bart Schaefer @ 2023-04-15 23:26 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sat, Apr 15, 2023 at 3:47 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> IOW the only reason I use
> history is because it shows unexpanded commands. Show me how to do this
> with zle or preexec and I'll be in heaven.

I get the feeling something very like this has come through zsh-users
before, but:

autoload add-zsh-hook
hist-keep-raw-event() { print -rS -- "$1" }
hist-skip-expanded-event() { return 1 }
add-zsh-hook preexec hist-keep-raw-event
add-zsh-hook zshaddhistory hist-skip-expanded-event

This could be extended so that hist-skip-expanded-event only returns 1
if hist-keep-raw-event previously ran.  Or you could leave out
hist-skip-expanded-event entirely if you want access to both forms.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-15 15:29               ` .zsh_history Ray Andrews
  2023-04-15 19:47                 ` "Pull just the text of a single command" (was Re: .zsh_history) Bart Schaefer
@ 2023-04-16  7:38                 ` Roman Perepelitsa
  2023-04-16 14:53                   ` .zsh_history Ray Andrews
  1 sibling, 1 reply; 24+ messages in thread
From: Roman Perepelitsa @ 2023-04-16  7:38 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sat, Apr 15, 2023 at 5:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> All I want is to be able to recover a command tail *exactly* as it was typed.
>
> my_function ()
> {
> grab_tail # writes tail to 'my_tail'.
> echo "my tail, exactly as typed, is: $my_tail"
> }
>
> $ my_function $path $(eval 'ls *') one two three ! < > ``.."" &>^!
>
> my tail, exactly as typed, is: $path $(eval 'ls *') one two three ! < >
> ``.."" &>^!

What would this do?

    % list=(my_function arg)
    % $list

What I typed is `$list`, but what is "tail"?

Roman.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-16  7:38                 ` .zsh_history Roman Perepelitsa
@ 2023-04-16 14:53                   ` Ray Andrews
  2023-04-16 15:28                     ` .zsh_history Bart Schaefer
  2023-04-16 15:29                     ` .zsh_history Roman Perepelitsa
  0 siblings, 2 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-16 14:53 UTC (permalink / raw)
  To: zsh-users


On 2023-04-16 00:38, Roman Perepelitsa wrote:
> On Sat, Apr 15, 2023 at 5:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> $ my_function $path $(eval 'ls *') one two three ! < > ``.."" &>^!
>>
>> my tail, exactly as typed, is: $path $(eval 'ls *') one two three ! < >
>> ``.."" &>^!
> What would this do?
>
>      % list=(my_function arg)
>      % $list
>
> What I typed is `$list`, but what is "tail"?
>
> Roman.

I don't understand.  You're assigning a variable no?  In that case 'arg' 
.... hmmmm .... good question.  In practice the problem wouldn't arise, 
I'm concerned with newly typed keystrokes.  In theory tho that's  quite 
mind stretching.  Is there a logical necessity or are options 
available?  My first stab at it would be that any syntax enclosing 
'my_function' must needs be executed as normal since there's no way of 
'pre-knowing' that my_function wants special access to its tail as raw 
keystrokes.  Or, not? Naively the tail is just 'arg'.  Dunno, whatever 
history does is what I want.  History stores commands as typed and 
that's what I want.  There's just that: "% echo one; echo two; echo 
three" problem -- I'd like the middle 'echo' to know that it's tail is 
'two' and nothing more or nothing less but of course recall from history 
gives all three commands in one serving.

  % my_function one; my_function $PATH; my_function three > filename

my tail, exactly as typed, is: one

my tail, exactly as typed, is: $PATH

my tail, exactly as typed, is: three > filename

... I guess the semicolon is the character that must logically end the 
tail and can't be part of the tail for obvious reasons. Or in practice 
the ENTER key cuz I'm never actually going to chain these commands which 
is why in practice just writing to history and recalling from history 
works fine (along with 'noglob') ... but it's laborious.  And one 
intuitively sees that zle will be a good candidate for a solution 
because once ENTER is pressed, what it has in it's buffer must needs be 
a sequence of raw keystrokes. Give me those keystrokes!  Raw.  On the 
half-shell.  But, broken by semicolons so that chained commands know 
where their own tail ends.  Thing is that the shell obviously does all 
this internally -- chained commands are obviously broken down into their 
individual units at the semicolon.  The information is obviously there I 
just want to see it prior to expansion.

I'm not communicating any of this well.  If I wasn't me I'd not be 
understanding these posts myself, they are not clear.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-16 14:53                   ` .zsh_history Ray Andrews
@ 2023-04-16 15:28                     ` Bart Schaefer
  2023-04-16 15:29                     ` .zsh_history Roman Perepelitsa
  1 sibling, 0 replies; 24+ messages in thread
From: Bart Schaefer @ 2023-04-16 15:28 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sun, Apr 16, 2023 at 7:53 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I don't understand.  You're assigning a variable no?  In that case 'arg'
> .... hmmmm .... good question.  In practice the problem wouldn't arise,

Let's stop following this up on THIS thread, as it has wandered way
off the subject-line topic.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: .zsh_history
  2023-04-16 14:53                   ` .zsh_history Ray Andrews
  2023-04-16 15:28                     ` .zsh_history Bart Schaefer
@ 2023-04-16 15:29                     ` Roman Perepelitsa
  1 sibling, 0 replies; 24+ messages in thread
From: Roman Perepelitsa @ 2023-04-16 15:29 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sun, Apr 16, 2023 at 4:54 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2023-04-16 00:38, Roman Perepelitsa wrote:
> > On Sat, Apr 15, 2023 at 5:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
> >> $ my_function $path $(eval 'ls *') one two three ! < > ``.."" &>^!
> >>
> >> my tail, exactly as typed, is: $path $(eval 'ls *') one two three ! < >
> >> ``.."" &>^!
> > What would this do?
> >
> >      % list=(my_function arg)
> >      % $list
> >
> > What I typed is `$list`, but what is "tail"?
> >
> > Roman.
>
> Naively the tail is just 'arg'.

If in the example above the tail is 'arg', then it should also be
'arg' in the following example:

    % list=(arg)
    % my_function $list

This contradicts what you described earlier. If this is not enough,
consider yet another example:

    % ls
    my_function xyz
    % *

What is tail here?

>  And one intuitively sees that zle will be a good candidate for a solution
> because once ENTER is pressed, what it has in it's buffer must needs be
> a sequence of raw keystrokes.

Indeed, if you need the raw command as it was typed, you can get it from zle.

> But, broken by semicolons so that chained commands know
> where their own tail ends.

Let's see if this makes sense.

    % alias -g sep='hi ; echo'
    % echo sep bye

What do you want to capture when the last line gets executed?

Roman.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-15 23:26                     ` Bart Schaefer
@ 2023-04-16 15:53                       ` Bart Schaefer
  2023-04-16 16:37                         ` Ray Andrews
  2023-04-16 19:24                         ` Ray Andrews
  0 siblings, 2 replies; 24+ messages in thread
From: Bart Schaefer @ 2023-04-16 15:53 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

[Bringing in other responses from the "was Re:" thread]

On Sun, Apr 16, 2023 at 7:53 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2023-04-16 00:38, Roman Perepelitsa wrote:
> > On Sat, Apr 15, 2023 at 5:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
> >> $ my_function $path $(eval 'ls *') one two three ! < > ``.."" &>^!
> >>
> >> my tail, exactly as typed, is: $path $(eval 'ls *') one two three ! < >
> >> ``.."" &>^!
> > What would this do?
> >
> >      % list=(my_function arg)
> >      % $list
> >
> > What I typed is `$list`, but what is "tail"?
> >
> > Roman.
>
> I don't understand.  You're assigning a variable no?

Right, you don't understand.  Roman's point is that

% $list

is going to run "my_function arg" as a command, so there's nothing in
"its tail as raw keystrokes" that corresponds to what "my_function"
will receive in its positional parameters.  The assignment is just
setting up the case of interest.

If Roman had written

% list=$(my_function arg)

then your follow-up would be closer to the mark, because then
my_function would actually run during the assignment instead of during
the expansion of $list.

> There's just that: "% echo one; echo two; echo> three" problem -- I'd like the middle 'echo' to know that it's tail is
> 'two' and nothing more or nothing less but of course recall from history
> gives all three commands in one serving.

It's much worse than that, consider

% my_function one && my_function $PATH || my_function three > filename

Or pipelines, or backgrounding, or if/then/else/fi, etc.  You will
need to define what you intend in each case (please, not by writing it
in prose here).

The ${(z)...} expansion does a best-case job of mimicking the parser,
so you can start by using ${(z)1} in preexec and work forward from
there.  The simple case with semicolons can be handled by

chain=( ${(s:; :)${(z)1}} )


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-16 15:53                       ` Bart Schaefer
@ 2023-04-16 16:37                         ` Ray Andrews
  2023-04-16 19:24                         ` Ray Andrews
  1 sibling, 0 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-16 16:37 UTC (permalink / raw)
  To: zsh-users


On 2023-04-16 08:53, Bart Schaefer wrote:
> It's much worse than that, consider 
You know, as a logician I'm quite aware that there are things one might 
ask for that are logically not possible to deliver.  It's Godel's 
Theorem applied to tokens.  Heavy duty stuff.  In my case close will count.
> % my_function one && my_function $PATH || my_function three > filename
>
> Or pipelines, or backgrounding, or if/then/else/fi, etc.  You will
> need to define what you intend in each case (please, not by writing it
> in prose here).
>
> The ${(z)...} expansion does a best-case job of mimicking the parser,
> so you can start by using ${(z)1} in preexec and work forward from
> there.  The simple case with semicolons can be handled by
>
> chain=( ${(s:; :)${(z)1}} )

That all sounds quite promising.  What I have now works 99% of the time 
and I'm not interested -- except as a brain-bending puzzle -- in trying 
to break it, but I think what you're suggesting will be a possible 
simplification and that will be nice.  Some virgin territory here tho, 
so I'll be getting stuck most likely.


>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: "Pull just the text of a single command" (was Re: .zsh_history)
  2023-04-16 15:53                       ` Bart Schaefer
  2023-04-16 16:37                         ` Ray Andrews
@ 2023-04-16 19:24                         ` Ray Andrews
  1 sibling, 0 replies; 24+ messages in thread
From: Ray Andrews @ 2023-04-16 19:24 UTC (permalink / raw)
  To: zsh-users


On 2023-04-16 08:53, Bart Schaefer wrote:
> chain=( ${(s:; :)${(z)1}} )

in preexec ():

chain=( ${(s:; :)${(z)1}} )
print -lr $chain

try it:

0 /aWorking/Zsh/Boot 1 $ echo *; echo howdy; echo $path; echo $path > 
nowhere; echo "Now is the time" | grep 'time'
echo *
echo howdy
echo $path
echo $path > nowhere
echo "Now is the time" | grep 'time'

... so that's about it.  I'm not going to be throwing anything more 
troublesome at it than globs and variables and redirections and pipes, 
and those are remaining as raw text so that's fit for purpose.  Only 
thing that would break it is chained commands which isn't really a 
problem.  However, even there, I can't help but notice that if there was 
a way of matching each one of the chained commands to its appropriate 
member of the array, then even that would be solvable.  Some sort of 
counter for each one of the chained commands?  'echo $path' is command 
#3 above so its match is $chain[3] and its tail is '$path'.  Easy.





^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2023-04-16 19:26 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-08 16:16 .zsh_history Perry Smith
2023-04-08 16:53 ` .zsh_history Roman Perepelitsa
2023-04-08 17:23   ` .zsh_history Ray Andrews
2023-04-08 17:27     ` .zsh_history Roman Perepelitsa
2023-04-08 17:35       ` .zsh_history Perry Smith
2023-04-08 17:54       ` .zsh_history Ray Andrews
2023-04-14 14:36         ` .zsh_history Felipe Contreras
2023-04-14 15:27           ` .zsh_history Ray Andrews
2023-04-15  4:49             ` .zsh_history Felipe Contreras
2023-04-15 15:29               ` .zsh_history Ray Andrews
2023-04-15 19:47                 ` "Pull just the text of a single command" (was Re: .zsh_history) Bart Schaefer
2023-04-15 22:47                   ` Ray Andrews
2023-04-15 23:26                     ` Bart Schaefer
2023-04-16 15:53                       ` Bart Schaefer
2023-04-16 16:37                         ` Ray Andrews
2023-04-16 19:24                         ` Ray Andrews
2023-04-16  7:38                 ` .zsh_history Roman Perepelitsa
2023-04-16 14:53                   ` .zsh_history Ray Andrews
2023-04-16 15:28                     ` .zsh_history Bart Schaefer
2023-04-16 15:29                     ` .zsh_history Roman Perepelitsa
2023-04-14 14:28     ` .zsh_history Felipe Contreras
2023-04-14 15:18       ` .zsh_history Ray Andrews
2023-04-14 14:00   ` .zsh_history Felipe Contreras
2023-04-14 14:51 ` .zsh_history Felipe Contreras

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).