multiline ZLE w/ bash-like single line .zsh_history...?

multiline ZLE w/ bash-like single line .zsh_history...?

[zsh]

Hi,

I like the multiline capability of ZLE, but when history lines are
appended to the .zsh_history file, I would like for multiline commands
to be morphed into a single line (similar to what bash does).

This is because I sometimes like to grep (or even edit) .zsh_history,
and if I find the command I'm looking for, I want to be able copy and
paste it into a new shell w/o having to make any changes to it.

For example, if I enter a multiline command like so:

  % echo \
  `> foo

... And then use 'up-line-or-history', I'll get to see it still as a
multiline in ZLE (which is good).  The line is not yet in my
.zsh_history file, because I have not set the INC_APPEND_HISTORY
option.

If I exit the shell and inspect .zsh_history, I see these two lines:

  echo \\
  foo

In bash (which does not have the excellent multiline capability), I
would see a single line, like so:

  echo foo

If I simply copy and paste the multiline entry, it will not behave the
way my original command did because of the double slash '\\'...

Is there a way to get zsh to be bash-like when it writes to the
history file...?  I'm willing to lose the multiline niceness for old
commands in new shells.

I have zsh setup to keep a smaller internal history and a larger
external history with these settings:

  HISTSIZE=9999  # internal
  SAVEHIST=99999 # external
  HISTFILE=$HOME/.zsh_history

And I have these helper aliases defined:

  alias history="fc -IA ; cat $HISTFILE" # access external history too
  alias h=history                        # a shortcut...

So when I want to find a really old command (usually because
history-incremental-search-backward failed to find it), I may do
something like this:

  h | grep 'some old cmd'

And here, again, having the history saved as a single line would be
helpful, because the output would be ready for copy and paste...

Maybe I do not know the "zsh way" to best do what I'm hoping to be
able to do...  Can I keep multiline in ZLE but lose it in HISTFILE?

Thanks for your time and consideration!

- Tor

Re: multiline ZLE w/ bash-like single line .zsh_history...?

[Bart Schaefer]

On Dec 6, 11:02am, zsh@noid.net wrote:
}
} I like the multiline capability of ZLE, but when history lines are
} appended to the .zsh_history file, I would like for multiline commands
} to be morphed into a single line (similar to what bash does).

In a sufficiently new zsh, there is a zshaddhistory hook function
(like precmd, chpwd, etc.).  You can manipulate the history any way
you like; e.g., the following saves each line of the multiline as a
single line in the history:

    zshaddhistory() { print -sr "${(z)1%%$'\n'}"; return 1 }

(And I finally get another chance to point out to PWS why it's a good
thing that (z) converts newlines into semicolons ... but it also
reveals a bug in the handling of here-documents ... moving that over
to zsh-workers.)  [Bash appears to ignore here-documents entirely?]

As was discussed in a lengthy thread earlier this year, the full multi-
line command "lingers" in the interactive history for one cycle even
if zshaddhistory returns nonzero.

} Is there a way to get zsh to be bash-like when it writes to the
} history file...?  I'm willing to lose the multiline niceness for old
} commands in new shells.

There is not a way to get the interactive history and the history file
to behave differently from one another, except by using more than one
history file (the "fc -p" example for zshaddhistory in the manual).
You could, for example, do

    zshaddhistory() {
      fc -p ${HISTFILE}_bash_format $HISTSIZE $SAVEHIST
      print -sr "${(z)1%%$'\n'}"
      return 0
    }

which would still save in zsh native format in $HISTFILE, but keep a
bash-format copy in a parallel file.

Re: multiline ZLE w/ bash-like single line .zsh_history...?

[zsh]

Thank you very much for your excellent reply!

I tried your second zshaddhistory example to create a parallel history
file that has the bash-format:

  zshaddhistory() {
    fc -p ${HISTFILE}_bash_format $HISTSIZE $SAVEHIST
    print -sr "${(z)1%%$'\n'}"
    return 0
  }

Unfortunately, it's not working for me...  It creates the file but
never writes any lines to it.  Even worse, the normal history file
($HISTFILE) is no longer written to either.

As an experiment, I tried this:

  zshaddhistory() {
    print -r "${(z)1%%$'\n'}" >>${HISTFILE}_bash_format
    return 0
  }

This works like a champ (and normal history is OK too).  The problem
with this solution is that the new file does not get the benefit of
ZSH's "history smarts".  It does show that the hook and the z-flag
expansion are working though...

On my normal system I'm running this version of ZSH:

  print $ZSH_VERSION $ZSH_PATCHLEVEL 

    4.3.10 Debian

Since I feared a debian bug, I tried it on OpenBSD too:

  print $ZSH_VERSION $ZSH_PATCHLEVEL

    4.3.10 1.4705

Both systems were the same...  I even tried it with a "stripped down"
.zshrc file, and that was also to no avail.

Because maintaining a parallel history file is not super-optimal to
me, and based on inspiration from the above, I took an alternate route
and it seems to be working for me at the moment.  It uses a little
perl hack (sorry it's not in ZSH!):

---------------------------------

#!/usr/bin/perl
#
# zsh_cook_history
#
# This script accepts a zsh format history file on STDIN and will
# print all multiline commands as single line commands (bash-like).
#
# Activate this script in ZSH (.zshrc) like so:
#
#   if ( which zsh_cook_history >/dev/null 2>&1 ) ; then
#     alias history="fc -IA ; cat $HISTFILE | zsh_cook_history"
#   else
#     alias history="fc -IA ; cat $HISTFILE"
#   fi
#   alias h=history
#

while (<>) {
  chomp;                       # remove the newline
  s/(?<=^\s)\s+// if $l;       # allow a lead space if multiline
  s/\s+(?=\\+$)//;             # ... but no trailing white space
  $l .= $_;                    # start building a command line
  unless( $l =~ s{\\\\$}{} ||  # a double slash is merely removed
          $l =~ s{\\$}{;} ) {  # a single slash becomes a ';'...
    print "$l\n";              # no more slashes: print the result
    undef $l;                  # start over
  }
}






On Mon, Dec 06, 2010 at 08:47:48PM -0800, Bart Schaefer wrote:
> On Dec 6, 11:02am, zsh@noid.net wrote:
> }
> } I like the multiline capability of ZLE, but when history lines are
> } appended to the .zsh_history file, I would like for multiline commands
> } to be morphed into a single line (similar to what bash does).
> 
> In a sufficiently new zsh, there is a zshaddhistory hook function
> (like precmd, chpwd, etc.).  You can manipulate the history any way
> you like; e.g., the following saves each line of the multiline as a
> single line in the history:
> 
>     zshaddhistory() { print -sr "${(z)1%%$'\n'}"; return 1 }
> 
> (And I finally get another chance to point out to PWS why it's a good
> thing that (z) converts newlines into semicolons ... but it also
> reveals a bug in the handling of here-documents ... moving that over
> to zsh-workers.)  [Bash appears to ignore here-documents entirely?]
> 
> As was discussed in a lengthy thread earlier this year, the full multi-
> line command "lingers" in the interactive history for one cycle even
> if zshaddhistory returns nonzero.
> 
> } Is there a way to get zsh to be bash-like when it writes to the
> } history file...?  I'm willing to lose the multiline niceness for old
> } commands in new shells.
> 
> There is not a way to get the interactive history and the history file
> to behave differently from one another, except by using more than one
> history file (the "fc -p" example for zshaddhistory in the manual).
> You could, for example, do
> 
>     zshaddhistory() {
>       fc -p ${HISTFILE}_bash_format $HISTSIZE $SAVEHIST
>       print -sr "${(z)1%%$'\n'}"
>       return 0
>     }
> 
> which would still save in zsh native format in $HISTFILE, but keep a
> bash-format copy in a parallel file.

Re: multiline ZLE w/ bash-like single line .zsh_history...?

[Bart Schaefer]

On Dec 7, 11:46am, zsh@noid.net wrote:
}
} I tried your second zshaddhistory example to create a parallel history
} file that has the bash-format:
} 
}   zshaddhistory() {
}     fc -p ${HISTFILE}_bash_format $HISTSIZE $SAVEHIST
}     print -sr "${(z)1%%$'\n'}"
}     return 0
}   }
} 
} Unfortunately, it's not working for me...  It creates the file but
} never writes any lines to it.  Even worse, the normal history file
} ($HISTFILE) is no longer written to either.

Ah; I copied/edited the example from the zshaddhistory entry in the
manual page, but it has a small bug.  Change "fc -p" to "fc -ap".
 
} Because maintaining a parallel history file is not super-optimal to
} me, and based on inspiration from the above, I took an alternate route
} and it seems to be working for me at the moment.  It uses a little
} perl hack (sorry it's not in ZSH!):
} 
} ---------------------------------
} 
} #!/usr/bin/perl
} #
} # zsh_cook_history

This won't work if you have the extended_history option set.

Here's the same in zsh:

zsh_cook_history() {
  # Save values because fc -p will reset them
  local histfile=$HISTFILE histsize=$HISTSIZE
  # Start a new temporary history we won't save anywhere
  fc -ap
  HISTSIZE=$histsize
  SAVEHIST=0
  # Read in the file
  fc -R $histfile
  # Now print it all out again
  zmodload -i zsh/parameter
  local entry
  for entry in ${(nok)history}
  do print -r -- "${(z)history[$entry]}"
  done
}

As previously noted, (z) has a small problem with here-documents
(it converts the embedded newlines to semicolons).  I'm not sure
what your perl solution does with here-documents.  Neither the perl
version nor this version will deal well with history written with
interactivecomments ...

-- 

Re: multiline ZLE w/ bash-like single line .zsh_history...?

[zsh]

Thanks for showing me the "zsh way" (very cool!)...

You are right about the "here document" issue.  When I enter a
command that has one, like so:

  cat <<EOF
  one
  two
  EOF

... and then inspect my .zsh_history file, I see:

  cat <<EOF\
  one\
  two\
  EOF

This is not consistent with what the perl version currently looks for.
It thinks a single terminating slash should become ';' in the new
single line version of the command it wants to generate.  For the perl
zsh_cook_history implementation to work, there needs to be special
'<<' handling that triggers an exception to the normal handling of the
terminating '\'.  Without that extra logic, it results in this single
line as output (which is wrong):

  cat <<EOF;one;two;EOF

But maybe the zsh version of zsh_cook_history could be made to handle
it properly with a tweak to (z), as you alluded to previously...  At
the moment, it's results are similarly not correct:

  cat << EOF ; one ; two ; EOF

Because I think the zsh way is better than the perl way, and because
this issue does not really affect my intended use of zsh_cook_history,
I will not bother fixing my perl version but rather just use the zsh
version and be happy.  Should (z) get updated in the future, then
that's a bonus!

As for INTERACTIVE_COMMENTS, I do have that option set, but I'm not
sure how to enter a multiline interactive comment, so unless there is
a method I'm unaware of to do that, it's a non-issue (single line
interactive comments seem OK in both zsh_cook_history implementations).

This will not work as the first line in a multiline w/ interactive
comments:

  echo # comment \

For me, the '#' char effectively turns off the '\' char...

- Tor

PS -

I'm very pleased with my new zsh_cook_history function (that you
provided) and I thank you for your help and expertise!

Re: multiline ZLE w/ bash-like single line .zsh_history...?

[Bart Schaefer]

On Dec 9, 11:12am, zsh@noid.net wrote:
}
} As for INTERACTIVE_COMMENTS, I do have that option set, but I'm not
} sure how to enter a multiline interactive comment, so unless there is
} a method I'm unaware of to do that, it's a non-issue

The problem isn't with multi-line comments, it's with comments that
contain an odd number of quote marks.  (z) won't know it's in a
comment and will therefore begin behaving as if part of the line
is a quoted string.