zsh-workers
 help / color / Atom feed
* zsh: corrupt history file
@ 2020-07-29 14:32 Perette Barella
  2020-07-29 14:39 ` Roman Perepelitsa
  0 siblings, 1 reply; 7+ messages in thread
From: Perette Barella @ 2020-07-29 14:32 UTC (permalink / raw)
  To: zsh-workers

Research on Google suggests the 'corrupt history file' problem is rare. 
In my experience, it is not; I get these _all_ _the_ _time_.

I've identified one source of corruption: $HISTFILE.  Korn shell *uses* 
but does not *require* this variable be set.  If unset, it uses 
~/.sh_history as a default.  `zsh` *requires* this variable be set, or 
there is no persistent history feature.  However, once set, if the user 
enters `ksh` (shelling out from an editor, for instance; or starting 
another shell to run a few experiments to compare behavior and increase 
understanding, or try commands to ensure a solution is portable for a 
script), then Korn shell does what it's told and appends its commands to 
the .zsh_history file.  And a `corrupt history file` ensues.

This does not happen with bash; although bash uses $HISTFILE it seems to 
use a similar history format to zsh.  ksh does some unusual encoding.

A solution to this is to add to .kshrc a `HISTFILE=$HOME/.sh_history` 
and to .zshrc a `HISTFILE=$HOME/.zsh_history`, so that if invoking 
different shells they do not corrupt each other's history files.

This issue should be noted in the man page, either with $HISTFILE's 
explanation, or in the BUGS section.


When corruption *does* occur, it is compounded by zsh's behavior: all 
sessions sitting at prompts, will ignore their next command and display 
the corruption error message.  Then they starts working normally again, 
history and all.  It sucks that I lost the past, but having every window 
drop one command gets super frustrating; essentially, this error isn't 
just effecting the history, it's effecting the future.

Even if the past has to be lost, it would be much less frustrating to 
have zsh *execute the inputted command*, then display the error before 
prompting for the next command.  For me, this is the real irritation, 
usually more than losing any history.

Perette Barella


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-29 14:32 zsh: corrupt history file Perette Barella
@ 2020-07-29 14:39 ` Roman Perepelitsa
  2020-07-29 15:15   ` Perette Barella
  0 siblings, 1 reply; 7+ messages in thread
From: Roman Perepelitsa @ 2020-07-29 14:39 UTC (permalink / raw)
  To: Perette Barella; +Cc: zsh-workers


[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

On Wed, 29 Jul 2020 at 16:33, Perette Barella <perette@barella.org> wrote:

> Research on Google suggests the 'corrupt history file' problem is rare.
> In my experience, it is not


Are you perchance exporting HISTFILE? You shouldn’t.

Roman.

>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-29 14:39 ` Roman Perepelitsa
@ 2020-07-29 15:15   ` Perette Barella
  2020-07-29 15:21     ` Eric Cook
  0 siblings, 1 reply; 7+ messages in thread
From: Perette Barella @ 2020-07-29 15:15 UTC (permalink / raw)
  To: Roman Perepelitsa; +Cc: zsh-workers

I tracked down the export, removing it works.  Must have put that in the 
.zshrc 5 years ago, when I first looked at it, before deciding it was 
broken because the history kept getting corrupt.

Perhaps update the man page for $HISTFILE:

HISTFILE
     The file to save the history in when an interactive shell exits.  If
     unset, the history is not saved.

     Do not export this parameter, as doing so may cause other shells to
     corrupt the history file.

Thanks Roman,

Perette

On 2020-07-29 10:39, Roman Perepelitsa wrote:
> On Wed, 29 Jul 2020 at 16:33, Perette Barella <perette@barella.org 
> <mailto:perette@barella.org>> wrote:
> 
>     Research on Google suggests the 'corrupt history file' problem is rare.
>     In my experience, it is not
> 
> 
> Are you perchance exporting HISTFILE? You shouldn’t.
> 
> Roman.
> 

-- 
Perette Barella • 176 Middlesex Road, Rochester NY 14610 • 585-317-3013
"Once we realize that imperfect understanding is the human condition 
there is no shame in being wrong, only in failing to correct our 
mistakes." -- George Soros

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-29 15:15   ` Perette Barella
@ 2020-07-29 15:21     ` Eric Cook
  2020-07-29 23:07       ` Daniel Shahaf
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Cook @ 2020-07-29 15:21 UTC (permalink / raw)
  To: zsh-workers

On 7/29/20 11:15 AM, Perette Barella wrote:
> I tracked down the export, removing it works.  Must have put that in the .zshrc 5 years ago, when I first looked at it, before deciding it was broken because the history kept getting corrupt.
>
> Perhaps update the man page for $HISTFILE:

exporting most shell variables commonly used among shells result in that kind of weirdness.

another one people (mis)commonly export is PS1. i don't think adding such an warning to each
variable would yield much of a result when people seem to think that using export is required
to set variables, ignorant of the differences in environment variables and shell variables.


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-29 15:21     ` Eric Cook
@ 2020-07-29 23:07       ` Daniel Shahaf
  2020-07-30  3:56         ` Eric Cook
  0 siblings, 1 reply; 7+ messages in thread
From: Daniel Shahaf @ 2020-07-29 23:07 UTC (permalink / raw)
  To: zsh-workers

Eric Cook wrote on Wed, 29 Jul 2020 11:21 -0400:
> another one people (mis)commonly export is PS1. i don't think adding such an warning to each
> variable

I guess you mean adding such warnings to zshparam(1)?  In which case,
perhaps adding them to bin_typeset() would be more effective?  As in:

% export HISTORY=foo
zsh:1: warning: $HISTORY should not be exported; use 'typeset HISTORY=foo' instead

(and if someone actually needs to export $HISTORY for whatever reason,
we can make -h suppress the warning)

> would yield much of a result when people seem to think that using export is required
> to set variables, ignorant of the differences in environment variables and shell variables.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-29 23:07       ` Daniel Shahaf
@ 2020-07-30  3:56         ` Eric Cook
  2020-07-30  4:30           ` Daniel Shahaf
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Cook @ 2020-07-30  3:56 UTC (permalink / raw)
  To: zsh-workers

On 7/29/20 7:07 PM, Daniel Shahaf wrote:
> I guess you mean adding such warnings to zshparam(1)?  In which case,
> perhaps adding them to bin_typeset() would be more effective?  As in:
>
> % export HISTORY=foo
> zsh:1: warning: $HISTORY should not be exported; use 'typeset HISTORY=foo' instead
>
> (and if someone actually needs to export $HISTORY for whatever reason,
> we can make -h suppress the warning)

That is indeed what i meant, but your solution has two problems in the example of PS1.
if it is in the environment when zsh starts and the user doesn't set it, you experience
the issue of another shell's escape sequences being used that typeset won't warn of since
it isn't being used. the second issue being software like python's virtualenv being written
under the assumption that exporting PS1 won't cause an warning message in the shells their
activation script is written for, would need to now account for that.

I know i have offered critiques yet no solutions, but i don't think either of the suggestions
is ideal personally.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: zsh: corrupt history file
  2020-07-30  3:56         ` Eric Cook
@ 2020-07-30  4:30           ` Daniel Shahaf
  0 siblings, 0 replies; 7+ messages in thread
From: Daniel Shahaf @ 2020-07-30  4:30 UTC (permalink / raw)
  To: Eric Cook, zsh-workers

Eric Cook wrote on Thu, 30 Jul 2020 03:56 +00:00:
> On 7/29/20 7:07 PM, Daniel Shahaf wrote:
> > I guess you mean adding such warnings to zshparam(1)?  In which case,
> > perhaps adding them to bin_typeset() would be more effective?  As in:
> >
> > % export HISTORY=foo
> > zsh:1: warning: $HISTORY should not be exported; use 'typeset HISTORY=foo' instead
> >
> > (and if someone actually needs to export $HISTORY for whatever reason,
> > we can make -h suppress the warning)
>
> That is indeed what i meant, but your solution has two problems in the
> example of PS1. if it is in the environment when zsh starts and the
> user doesn't set it, you experience the issue of another shell's
> escape sequences being used that typeset won't warn of since it isn't
> being used.

What I proposed would address the issue of zsh exporting $PS1 and some
other shell then using zsh's value (e.g., «export PS1='%# '; dash»).  What
you're describing is the converse problem and would need to be fixed
separately.

> the second issue being software like python's virtualenv
> being written under the assumption that exporting PS1 won't cause an
> warning message in the shells their activation script is written for,
> would need to now account for that.

Isn't that a bug in their code that they should fix?  PS1 shouldn't be exported.

Incidentally, if they do the equivalent of «env PS1=foo zsh», then they
won't be affected because that doesn't go through bin_typeset() in the
first place.  They'd only be affected if they somehow cause the 'export'
or 'typeset' builtins to be executed (e.g., by setting up their own
$ZDOTDIR).

> I know i have offered critiques yet no solutions, but i don't think
> either of the suggestions is ideal personally.

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-29 14:32 zsh: corrupt history file Perette Barella
2020-07-29 14:39 ` Roman Perepelitsa
2020-07-29 15:15   ` Perette Barella
2020-07-29 15:21     ` Eric Cook
2020-07-29 23:07       ` Daniel Shahaf
2020-07-30  3:56         ` Eric Cook
2020-07-30  4:30           ` Daniel Shahaf

zsh-workers

Archives are clonable: git clone --mirror http://inbox.vuxu.org/zsh-workers

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.zsh.workers


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git