histroy file being truncated (konsole with multiple shells context)

histroy file being truncated (konsole with multiple shells context)

[Samuel Krempp]

I noticed some people already noticed that kind of unreproduced
behaviour ("vanishing history", 
http://www.zsh.org/mla/users/2002/msg00105.html )

It's becoming very frequent for me, with 3 or 4 zsh shells inside a
konsole (kde's advanced console for those that are not familiar with
kde), and zsh history set with 
HISTSIZE=99000
SAVEHIST=90000

I get truncated $HISTFILE files from time to time, and then there are
several
$HISTFILE.<number> empty files created (with the numbers certainly being
the PIDs of the shells whose history was being written, I guess)


konsole gives the shell 1s to shut down before killing it.
So I guess zsh starts writing a new history file when shutting down, and
if it is stopped before the end, the file ends-up truncated.

Shouldn't zsh be cautious about that, and use some safe commit idiom ?
(i.e. write the new file $HISTFILE.new, then commit the swap of it with
the old $HISTFILE in an atomic way)

BTW, how does incappendhistory works ?  is the history written at each
and every command, even if it means rewriting it completely every time ?

I'm using zsh a lot, and I'd really like to be able to rely on its
history for automatically archiving complicated commands and parameter
sets, with no risk of loosing any command.

In the case of very big $HISTFILE, it seems to me the best solution to
insure all commands end-up in the $HISTFILE even in case of brutal
shutdown is for each zsh to write incremental temporary histfiles (by
plain appending), then merging them (with a safe commit method) into the
final $HISTFILE at regular intervals.
zsh can then check for these files at startup, and do the merging of any
temporary files pending from last session.

A scheme like that should be both efficient and reliable, right ?

-- 
Sam

Re: histroy file being truncated (konsole with multiple shells context)

[Peter Stephenson]

Samuel Krempp wrote:
> Shouldn't zsh be cautious about that, and use some safe commit idiom ?

It uses a lockfile, in fact, so in principle it ought to be safe.
(Whether it's bug free in practice is another matter, of course.)  It
would be less safe if you were accessing files remotely via NFS or
[makes sign of evil eye] Samba.

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: histroy file being truncated (konsole with multiple shells context)

[Wayne Davison]

[I had meant to comment on this earlier and forgot.]

On Sun, Jul 04, 2004 at 12:40:10PM +0200, Samuel Krempp wrote:
> konsole gives the shell 1s to shut down before killing it.  So I guess
> zsh starts writing a new history file when shutting down, and if it is
> stopped before the end, the file ends-up truncated.

The current scheme uses a lock file to ensure that only one shell at a
time can write out the history file, but the file is updated in place,
so it is possible for it to be truncated if zsh is killed during this
update.

> Shouldn't zsh be cautious about that, and use some safe commit idiom ?
> (i.e. write the new file $HISTFILE.new, then commit the swap of it
> with the old $HISTFILE in an atomic way)

Updating a .new file and then moving it into place would be a safer way
to ensure that the history file never gets truncated.  I would worry a
little about compatibility changes, such as the breaking of hard-links
to the history file, so this is something that we should discuss and
consider the side effects.

> BTW, how does incappendhistory works ?  is the history written at each
> and every command, even if it means rewriting it completely every time ?

No, the INC_APPEND_HISTORY and SHARE_HISTORY add new lines onto the end
of the history file via a normal appending write.  When the file gets to
be a certain percent larger than its configured size, it gets rewritten
to bring its size back down.

I think the easiest improvement would be for zsh to avoid doing a
history-file rewrite when performing an exit due to a signal.  With an
incremental-append, zsh wouldn't have to do any history updating on
abnormal exit (making it very safe).  For those that use the more basic
APPEND_HISTORY setting, each shell would just append their history lines
and leave the history file too big (it could then be resized on load).
For those that don't use an appending-history setting, I suppose we'd
just leave their algorithm alone for now.

..wayne..

Re: histroy file being truncated (konsole with multiple shells context)

[Bart Schaefer]

[Further discussion should probably move to zsh-workers.]

On Wed, 29 Sep 2004, Wayne Davison wrote:

> The current scheme uses a lock file to ensure that only one shell at a
> time can write out the history file, but the file is updated in place,
> so it is possible for it to be truncated if zsh is killed during this
> update.

We could do something sneaky like fork a child with most signals blocked 
that does nothing but write the history file and exit, while the parent 
exits immediately to fake out _its_ parent.  That's not a very resource- 
friendly idea, though, because (among other things) it might fail due to 
inability to fork.  However, it's probably the only one that covers all 
cases short of catastrophe.

> Updating a .new file and then moving it into place would be a safer way 
> to ensure that the history file never gets truncated.

On the other hand, in that case the changes get lost entirely, which might
be just as confusing.

> ... the INC_APPEND_HISTORY and SHARE_HISTORY add new lines onto the end
> of the history file via a normal appending write.  When the file gets to
> be a certain percent larger than its configured size, it gets rewritten
> to bring its size back down.

So a signal in the middle of that rewrite would still be an issue, even if
we change the exit-time behavior.

Re: history file being truncated (konsole with multiple shells context)

[Samuel Krempp]

I'm really glad this subject is finally discussed, whatever the result
will be I'm very grateful to all zsh workers who take some time to think
over this issue.
I was beginning to feel like I were the only one concerned about that..

On Thu, 2004-09-30 at 06:39, Bart Schaefer wrote:
> We could do something sneaky like fork a child with most signals blocked 
> that does nothing but write the history file and exit, while the parent 
> exits immediately to fake out _its_ parent.  That's not a very resource- 
> friendly idea, though, because (among other things) it might fail due to 
> inability to fork.  However, it's probably the only one that covers all 
> cases short of catastrophe.

In the cases of computer crash / brutal reset, the user should know bad
things can happen anyway. but still, maybe zsh can even cover those
safely.

I think the history update can not be completely safe without some kind
of "atomic transaction" technique, to avoid any problem when zsh gets
stopped at any time.

> > Updating a .new file and then moving it into place would be a safer way 
> > to ensure that the history file never gets truncated.
> 
> On the other hand, in that case the changes get lost entirely, which might
> be just as confusing.

well, it's already better than the risk of losing the entire history.
And in fact, I think it's possible to design a scheme where you can not
lose more than the few last history items.
For instance :
1. every second save in a file the latest commands, in the simplest
append way, without any processing. Its only purpose is crash-recovery.
Let's call this the temp files
(I guess we need one temp file per zsh instance in order to do plain,
safe appends)
2.a every minute (and on exit), do the whole history processing and
create a ${HISTFILE}.new file, 
  2.b then erase the tempfile
  2.c then move ${HISTFILE}.new to ${HISTFILE}

This way, it should be possible to recover all hanging modifications
even after a crash, whatever zsh was doing at the time :
at zsh startup, 
(0 : parse the history file)
1. check for temp files. If there are any, ignore the .new file if it
exists (it wasn't completely written, else the temp files would be
cleared), process the temp files into memory, and start over at step 2.a
 Else, if a .new file exists start at 2.c 
Else, it means zsh had time to finish its history update on last exit,
and just start at step 1.

Such a scheme should be safe whatever happens, its only vulnerable parts
are its pseudo-atomic transactions, 2.c (i.e. the move was not completed
on the filesystem, and as a result the old histfile is unlinked and the
new one is not yet relinked. maybe that's not even possible with modern
filesystems though) and 2.b (i.e. not all files were erased - though by
using a dummy temp file to be erased first, partial erase of the temp
files is detected. this step might in fact be made safe).  
A filesystem expert can probably make even those steps safe. (and in any
case they're so fast that their being interrupted is unprobable)

Now, I agree it's not the easiest solution, and easier ones might very
well be satisfactory enough.


Hmm, I'm realizing this kind of scheme is not really specific to shell
history updates anyway, there might already be such a library/tool
somewhere that does everything the right way.
Or, at least, it's available with any database.
If zsh can afford dependancies on a database library, storing the
history in a database could be a solution  :-)

> > ... the INC_APPEND_HISTORY and SHARE_HISTORY add new lines onto the end
> > of the history file via a normal appending write.  When the file gets to
> > be a certain percent larger than its configured size, it gets rewritten
> > to bring its size back down.
> 
> So a signal in the middle of that rewrite would still be an issue, even if
> we change the exit-time behavior.

ah yes. This would make the issue less probable, but not solve it
completely.
-- 
Samuel