[BUG] APPEND_HISTORY leads to complete reread + rewrite of histfile

[Zeshy <ml.zsh@gmx.es>]

Hi all!

In search of the reason why zsh (5.4.2 and fairly recent git) is
so much slower reading/writing larger history files (13 MB, 330k lines)
than bash (4.4.12), I think I found a bug. If I missed something (like
some option), and this is not a bug, but just misconfiguration, or if
you need more information, please let me know.


When configured to *append* the history on exit with the following
config:

---------------- BEGIN .zshrc -----------------
#!/usr/bin/zsh
HISTFILE="/var/tmp/.histfile"
HISTSIZE=9223372036854775807 # LONG_MAX (limits.h)
SAVEHIST=9223372036854775807 # LONG_MAX (limits.h)
setopt append_history
setopt extended_history
unsetopt hist_save_by_copy
unsetopt inc_append_history_time
---------------- END .zshrc -----------------

zsh (with "-d" or without) seems to do the following:

Step 1) append to history file
Step 2) read complete history file
Step 3) write complete history file

This seems to be invariant to
- option HIST_SAVE_BY_COPY,
- option INC_APPEND_HISTORY_TIME,
- values of HISTSIZE and SAVEHIST (tried with 500000 for both).

With "unsetopt APPEND_HISTORY", only step 3 is executed, as expected.


I interspersed some debug printf's in Src/hist.c and saved the output
of simply starting zsh and hitting Ctrl-D (exit). Absolute and relative
times are in milliseconds.

URLs:

https://pastebin.com/BzJhrr69  .zshrc-appendhistory
https://pastebin.com/ULCt7XcR  .zshrc-noappendhistory
https://pastebin.com/bHTqQu0C  hist.c
https://pastebin.com/QUjZ98Zn  zsh-appendhistory--debug-output.txt
https://pastebin.com/VUpGf6Fa  zsh-noappendhistory--debug-output.txt


Zeshy