Re: search through alternative HISTFILE

Re: search through alternative HISTFILE

[Maarten Grachten]

Hi Peter,

Thanks a lot for the explanation, and the patch, it works perfectly! 
Based on your explanation however, I tried the following as well:

ACC_HISTFILE="${HOME}/.accumulated-history"

function accumulated-history-incremental-search-backward () {
     fc -p $ACC_HISTFILE
     HISTNO=$(( $( wc -l < $ACC_HISTFILE ) + 1 ))
     zle history-incremental-search-backward
     fc -P
}

i.e. before doing the search on the accumulated history, set the 
variable HISTNO to just beyond the last line of that history file. This 
also seems to work without the patch you provided, however I cannot 
really judge if there are any unintended side-effects with this approach.

Many thanks again for helping me out! I appreciate the effort.

Maarten

Re: search through alternative HISTFILE

[Maarten Grachten]

Hi, last update, hopefully.

I found out that when I just append several history files to generate 
ACC_HISTFILE, setting HISTNO to the number of lines (+1) in 
ACC_HISTFILE, does not necessarily point zle to the end of the history. 
I am not sure why this happens (maybe it has to do with duplicate 
removals in the internal history representation?). If the value of 
HISTNO set in this way turns out to be invalid, it seems to be reset to 
its original value. I think the correct way to deal with this is to use 
the end-of-history widget to move to the end of the newly loaded history.

The resulting widget is then like this:

function accumulated-history-incremental-search-backward () {
     fc -p $ACC_HISTFILE
     zle .end-of-history
     zle .history-incremental-search-backward
     fc -P
     zle .end-of-history
}

Maarten

Re: search through alternative HISTFILE

[Peter Stephenson]

On Tue, 06 Aug 2013 12:11:12 +0200
Maarten Grachten <maarten.grachten@ofai.at> wrote:
> Just for the record. I figured out that it is necessary to reset HISTNO 
> to its old value after the search on accumulated history is done 
> (otherwise the normal search will be confused):
> 
> function accumulated-history-incremental-search-backward () {
>      fc -p $ACC_HISTFILE
>      HISTNO=$(( $( wc -l < $ACC_HISTFILE ) + 1 ))
>      zle history-incremental-search-backward
>      fc -P
>      HISTNO=$(( $( wc -l < $HISTFILE ) + 1 ))
> }

Yes, you've now basically got something equivalent to the patch.

pws

Re: search through alternative HISTFILE

[Maarten Grachten]

Just for the record. I figured out that it is necessary to reset HISTNO 
to its old value after the search on accumulated history is done 
(otherwise the normal search will be confused):

function accumulated-history-incremental-search-backward () {
     fc -p $ACC_HISTFILE
     HISTNO=$(( $( wc -l < $ACC_HISTFILE ) + 1 ))
     zle history-incremental-search-backward
     fc -P
     HISTNO=$(( $( wc -l < $HISTFILE ) + 1 ))
}

Maarten

Re: search through alternative HISTFILE

[Peter Stephenson]

On Mon, 05 Aug 2013 12:05:56 +0200
Maarten Grachten <maarten.grachten@ofai.at> wrote:
> # Define shell function to be used as widget:
> 
> function accumulated-history-incremental-search-backward () {
>      fc -p "${HOME}/.accumulated-history"
>      zle .history-incremental-search-backward
> }
> 
> Is there any particular reason why calling fc -p inside the widget does 
> not affect the history search?

It seems this is because reading the new history line doesn't update the
record of the history line number in ZLE, so it tries to start from an
invalid number.

ZLE has its own history line number from the main shell because you can
go up or down the history list regardless of what's already been stored.
It's initialised when the line editor starts and after that is
maintained internally.  I think the answer is to check if ZLE is already
active when pushing history, popping history or reading a line and if it
is then update the internal history line directly to the current line
number in the main history code.

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index a1d54dd..756ff11 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -1934,6 +1934,13 @@ zle_main_entry(int cmd, va_list ap)
 	break;
     }
 
+    case ZLE_CMD_SET_HIST_LINE:
+    {
+	histline = va_arg(ap, zlong);
+
+	break;
+    }
+
     default:
 #ifdef DEBUG
 	    dputs("Bad command %d in zle_main_entry", cmd);
diff --git a/Src/hist.c b/Src/hist.c
index 5e962e9..f78c97d 100644
--- a/Src/hist.c
+++ b/Src/hist.c
@@ -76,6 +76,9 @@ mod_export int excs, exlast;
  * and a temporary history entry is inserted while the user is editing.
  * If the resulting line was not added to the list, a flag is set so
  * that curhist will be decremented in hbegin().
+ *
+ * Note curhist is passed to zle on variable length argument list:
+ * type must match that retrieved in zle_main_entry.
  */
  
 /**/
@@ -2414,6 +2417,9 @@ readhistfile(char *fn, int err, int readflags)
 	zerr("can't read history file %s", fn);
 
     unlockhistfile(fn);
+
+    if (zleactive)
+	zleentry(ZLE_CMD_SET_HIST_LINE, curhist);
 }
 
 #ifdef HAVE_FCNTL_H
@@ -3339,6 +3345,8 @@ pushhiststack(char *hf, zlong hs, zlong shs, int level)
     }
     hist_ring = NULL;
     curhist = histlinect = 0;
+    if (zleactive)
+	zleentry(ZLE_CMD_SET_HIST_LINE, curhist);
     histsiz = hs;
     savehistsiz = shs;
     inithist(); /* sets histtab */
@@ -3378,6 +3386,8 @@ pophiststack(void)
     histtab = h->histtab;
     hist_ring = h->hist_ring;
     curhist = h->curhist;
+    if (zleactive)
+	zleentry(ZLE_CMD_SET_HIST_LINE, curhist);
     histlinect = h->histlinect;
     histsiz = h->histsiz;
     savehistsiz = h->savehistsiz;
diff --git a/Src/zsh.h b/Src/zsh.h
index d7b130c..e6f0f65 100644
--- a/Src/zsh.h
+++ b/Src/zsh.h
@@ -2756,7 +2756,8 @@ enum {
     ZLE_CMD_RESET_PROMPT,
     ZLE_CMD_REFRESH,
     ZLE_CMD_SET_KEYMAP,
-    ZLE_CMD_GET_KEY
+    ZLE_CMD_GET_KEY,
+    ZLE_CMD_SET_HIST_LINE
 };
 
 /***************************************/

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: search through alternative HISTFILE

[Maarten Grachten]

Dear Peter,

Thank you very much for your answer. Your use case is indeed similar, if 
slightly more complex. I had hoped I could reuse the existing 
history-incremental-search-backward widget, only changing the history 
before calling it. The relevant command (which you also use in your 
widget) seems to be "fc -p <NEWHISTFILE>". As I understand it, this 
pushes the current history and replaces it with a new history, read from 
<NEWHISTFILE>. This approach actually works when I use the fc -p command 
on the command line before hitting Ctrl-R, but when I wrap it in a 
widget it does not work:

# Define shell function to be used as widget:

function accumulated-history-incremental-search-backward () {
     fc -p "${HOME}/.accumulated-history"
     zle .history-incremental-search-backward
}

# Tell zsh to use the function as a widget:

zle -N accumulated-history-incremental-search-backward

# Bind Ctrl-i to the widget:

bindkey "^I" accumulated-history-incremental-search-backward

Is there any particular reason why calling fc -p inside the widget does 
not affect the history search?

Maarten

Re: search through alternative HISTFILE

[Peter Stephenson]

[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]

On Fri, 02 Aug 2013 12:38:43 +0200
Maarten Grachten <maarten.grachten@gmail.com> wrote:
> As part of an attempt to archive my command line history, I have a need 
> for bck-i-search (as normally invoked by Ctrl-R), but with a history 
> file that is not $HISTFILE, that is, a different file from that which 
> the current shell uses for saving its history.
> 
> As I envision it, Ctrl-R would still search through the current shell's 
> history (i.e. through $HISTFILE), and another key binding would search 
> through an alternative but constant histfile (say 
> $HOME/.accumulated-history).
> 
> Is this possible, and if so, can any of you give me a hint as to how to 
> achieve it?

It's not quite what you want but it might provide a start since the bits
underneath are pretty similar...  I've attached a ZLE widget I've been
using for a few years that I never quite made neat enough to post.
It uses the history-beginning-search-backward interface rather than the
isearch interface, but that shouldn't be too hard to change.

The idea is it can search backwards both through the main history and a
local history file with the name defined by a style; as an optional
extra it decides by means of what's on the command line so far whether
to search the local history.  Your use is actually slightly simpler.

For good measure, I've also attached a hook function for use with the
zshaddhistory hook that decides whether to save a command locally
(again, this is more than you actually asked for but might be useful).

pws

[-- Attachment #2: history-beginning-local --]
[-- Type: text/plain, Size: 4783 bytes --]

# history-beginning-local
#
#   Mostly a drop-in replacement for history-beginning-search-backward
#   or -forward, depending on the name.
#
#   Allows a local history file to be searched for certain commands to execute.
#   The style local-history-file should be set to a list of files that
#   can contain the local history, in the same format as $HISTFILE.
#   The files are searched in order until the first is find.  They
#   can contain a relative or absolute path; clearly an absolute
#   path to an existing file will always match.
#
#   local-history-commands should be set to a list of commands
#   (or patterns matching commands) that should use the local history
#   file; this can be "*" to handle all commands.  Both styles be set.
#
#   Alternatively (or in addition), local-history-pattern is a scalar
#   that gives a pattern that must match the command line exactly to
#   initiate use of the mechanism.  Typically this will end in a "*".
#
#   If the style local-history-only is not set the global history
#   will be searched if there is no match in the local history.
#   The global history is tried again from the most recent entry;
#   no ordering is implied between the two histories.  (Note
#   this style is also checked by zshaddhistory-local to decide
#   whether to save a history entry to the global history.)
#
#   If the style local-history-verbose is set a notice is printed
#   below the command line if the local history was searched.
#
#   Styles use the standard form of components separated by colons.
#   The components are
#    - The string "zhist": used rather than "zle" to make this consistent
#      across different functions handling history in different parts of
#      the shell.
#    - The widget name
#    - The current directory as given by $PWD.
#   There is a terminating colon.  It is recommended that a wildcard be
#   used at the end to protect against future enhancements.
#
#   For example,
#    zstyle ':zhist:*' local-history-file .zsh-local-history
#    zstyle ':zhist:*' local-history-commands make gcc gdb

emulate -L zsh
setopt extendedglob

local w f new lhp curline
local -a lhf lhc
integer lhv lho restore

typeset -gA __history_beginning_matches
if [[ $WIDGET != $LASTWIDGET ]]; then
  __history_beginning_matches=()
fi
integer oldcursor=CURSOR

zstyle -a ":zhist:${WIDGET}:${PWD}:" local-history-file lhf || return 1
zstyle -a ":zhist:${WIDGET}:${PWD}:" local-history-commands lhc || return 1
zstyle -s ":zhist:${WIDGET}:${PWD}:" local-history-pattern lhp
zstyle -t ":zhist:${WIDGET}:${PWD}:" local-history-verbose && lhv=1
zstyle -t ":zhist:${WIDGET}:${PWD}:" local-history-only && lho=1
# try / always block for restoring history
{
  for f in $lhf; do
    if [[ -f $f ]]; then
      integer iline
      local -a words found

      words=(${(z)BUFFER})
      if [[ ${(Q)words[1]} = (${(j.|.)~lhc}) || \
	    ( -n $lhp && ${BUFFER} = ${~lhp} ) ]]; then
	(( restore = 1 ))
	fc -p $f

	# Search history for pattern.
	# As $history is an associative array we can get all matches.
	if [[ $WIDGET = *forw* ]]; then
	  # Searching forward.  Look back through matches until we
	  # get back to the current history number.
	  found=(${(kOn)history[(R)${LBUFFER}*]})
	  for iline in $found; do
	    (( $iline <= HISTNO )) && break
	    # Skip duplicates.
	    curline=$history[$iline]
	    [[ $curline = $BUFFER ]] && continue
	    [[ -n ${__history_beginning_matches[$curline]} ]] && continue
	    new=$iline
	  done
	else
	  # Searching backward.  Look forward through matches until we
	  # reach the current history number.
	  found=(${(kon)history[(R)${LBUFFER}*]})
	  for iline in $found; do
	    (( $iline >= HISTNO )) && break
	    # Skip duplicates.
	    curline=$history[$iline]
	    [[ $curline = $BUFFER ]] && continue
	    [[ -n ${__history_beginning_matches[$curline]} ]] && continue
	    new=$iline
	  done
	fi
	[[ -n $new ]] && break
	fc -P
	(( restore = 0 ))
      fi
    fi
  done
  if [[ -n $new ]]; then
    # Match found.  Move to line.
    HISTNO=$new
    __history_beginning_matches[$history[$HISTNO]]=1
    if [[ $WIDGET = *-end* ]]; then
      zle end-of-line
    else
      (( CURSOR = oldcursor ))
    fi
    (( lhv )) && zle -M "Matched in local history"
    return 0
  elif (( lho )); then
    (( lhv )) && zle -M "No match in local history"
    return 1
  else
    (( lhv )) && zle -M "No match in local history; falling through"
  fi
} always {
  (( restore )) && fc -P
}

if [[ $WIDGET = *forw* ]]; then
  zle history-beginning-search-forward
else
  zle history-beginning-search-backward
fi
local stat=$?

if (( $stat == 0 )); then
  __history_beginning_matches[$history[$HISTNO]]=1
fi

if [[ $WIDGET = *-end* ]]; then
  zle end-of-line
else
  (( CURSOR = oldcursor ))
fi

return $stat
# end

[-- Attachment #3: zshaddhistory-local --]
[-- Type: text/plain, Size: 1658 bytes --]

# This function is an adjunct to the history-beginning-local
# zle widget.  It saves any history entries that would be
# found by that widget in the local history file, provided that
# already exists.  It also saves the history globally unless
# the local-history-only style is set in the context.
#
# The context is :zhist: followed by the function name, a colon, the
# current directory from $PWD, and another colon.

emulate -L zsh
setopt extendedglob warncreateglobal

zmodload -i zsh/parameter

local name=${funcstack[1]} lhp f
local -a lhf lhc
integer lho

zstyle -a ":zhist:${name}:${PWD}:" local-history-file lhf || return 1
zstyle -a ":zhist:${name}:${PWD}:" local-history-commands lhc || return 1
zstyle -s ":zhist:${WIDGET}:${PWD}:" local-history-pattern lhp
zstyle -t ":zhist:${name}:${PWD}:" local-history-only && lho=1

for f in $lhf; do
  if [[ -f $f ]]; then
    local -a words

    words=(${(z)1})
    if [[ ${(Q)words[1]} = (${(j.|.)~lhc}) || \
      ( -n $lhp && $1 = ${~lhp} ) ]]; then
      # Save on the global history unless we're told not to.
      # If we define multiple zshaddhistory hooks we want a
      # a way of signalling that we've done this.  One way
      # of doing this would be to set a global parameter to $HISTCMD,
      # although that doesn't change if we've ignored the previous line.
      # Another way would be to have a zshaddhistoryhook that reset
      # a global parameter (since this is called first) and rely
      # on all the other hooks being in zshaddhistory_functions,
      # as they should be for neatness.
      (( lho )) || print -Sr -- ${1%%$'\n'}
      fc -p -- $f
      break
    fi
  fi
done

search through alternative HISTFILE

[Maarten Grachten]

Dear zsh users,

As part of an attempt to archive my command line history, I have a need 
for bck-i-search (as normally invoked by Ctrl-R), but with a history 
file that is not $HISTFILE, that is, a different file from that which 
the current shell uses for saving its history.

As I envision it, Ctrl-R would still search through the current shell's 
history (i.e. through $HISTFILE), and another key binding would search 
through an alternative but constant histfile (say 
$HOME/.accumulated-history).

Is this possible, and if so, can any of you give me a hint as to how to 
achieve it?

Many thanks in advance!

Maarten