How to make APPENDHISTORY useful?

How to make APPENDHISTORY useful?

[Bart Schaefer]

`APPEND_HISTORY'
     If this is set, zsh sessions will append their history list to the
     history file, rather than overwrite it.  Thus, multiple parallel
     zsh sessions will all have their history lists added to the
     history file, in the order they are killed.

This sounds useful, but in practice it's not very helpful.  Here's why:

Suppose I set HISTSIZE to be more lines than SAVEHIST.  Now every time a
shell exits, it appends its history to the file and then truncates the
file to SAVEHIST lines.  Result: Same as if history weren't appended.

Now suppose I set SAVEHIST to be more lines than HISTSIZE.  Every time a
shell exits, it appends its history to the file, leaving some lines from
other shells at the top.  So far so good.  But when I start a new shell,
it reads in all that history and then truncates it to HISTSIZE.  Result:
Same as if history hadn't been appended.

You get the illusion that something good is happening if you start with
an empty file and run a few shells that don't put SAVEHIST lines into the
history before exiting; but once the file gets full you're back to one of
the states described above.

A logical thought would be to start out with HISTSIZE set to at least as
large as SAVEHIST, and then just before saving the history file, reset
HISTSIZE to something less than SAVEHIST, so that the tail end of this
shell's history would mix with whatever is already in the file.  But that
doesn't work because the history file is written *before* the exit traps
execute and before zlogout files are read, so there's no place to put the
re-assignment to HISTSIZE.

To get the *effect* of reducing HISTSIZE before saving, you have to put
a bunch of `fc' commands in your exit trap or zlogout.  But if you have
to do that anyway, then you can *explicitly* append, so APPEND_HISTORY is
of no benefit in that case.

What am I missing here?

Re: How to make APPENDHISTORY useful?

[Greg Badros]

I've been thinking about this too.  What would be useful is to have an
option or mechanism to grep out only the "interesting" lines from the
history before saving them.  "Interesting" might mean a regular
expression; even more useful would be a way to keep the last couple of
executions of each command name (sometimes if I haven't used a command,
a good example from an earlier use [while I was looking at the man page]
is all I need).  Also "interesting" could be any pipeline, or multi-part
command.

This might be almost doable within the current framework, though still
having the problem about when to do the trimming down of the history.

Greg

Re: How to make APPENDHISTORY useful?

[Wayne Davison]

"Bart Schaefer" writes:
> but once the file gets full you're back to one of
> the states described above.

Not quite.  Since savehistfile() only appends newly entered lines, the
above only occurs if SAVEHIST new lines were entered since starting the
shell.

One simple solution is to have zsh call savehistfile() every N lines when
APPEND_HISTORY is set.  This would intersperse the lines from multiple
shells nicely, and doesn't require you to exit an old shell before its
history can be used in new shells.  It might be nice if the history file
was not rewritten for every auto-append, allowing the file to grow a
while before going to the extra work of truncating it.

..wayne..

Re: How to make APPENDHISTORY useful?

[Bart Schaefer]

On Feb 13,  3:53pm, Wayne Davison wrote:
> Subject: Re: How to make APPENDHISTORY useful?
> "Bart Schaefer" writes:
> > but once the file gets full you're back to one of
> > the states described above.
> 
> Not quite.  Since savehistfile() only appends newly entered lines

Ah, that's what I'm missing.

Still, I typically leave my shells running for days or even weeks at a
time; the odds of my not entering SAVEHIST new lines in any given shell
are vanishingly small.

> One simple solution is to have zsh call savehistfile() every N lines when
> APPEND_HISTORY is set.

Another, even simpler, solution would be to separate variables for "how big
is the history file" and "how many lines do I append to it" -- rather than
have SAVEHIST mean both things.