From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28425 invoked by alias); 6 Dec 2017 19:36:45 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 42087 Received: (qmail 14674 invoked by uid 1010); 6 Dec 2017 19:36:45 -0000 X-Qmail-Scanner-Diagnostics: from mout.gmx.net by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(212.227.15.15):SA:0(-2.6/5.0):. Processed in 7.006556 secs); 06 Dec 2017 19:36:45 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=ham autolearn_force=no version=3.4.1 X-Envelope-From: ml.zsh@gmx.es X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Date: Wed, 6 Dec 2017 20:31:23 +0100 From: Zeshy To: zsh-workers@zsh.org Subject: [BUG] APPEND_HISTORY leads to complete reread + rewrite of histfile Message-ID: <20171206193122.w4r3dhkotlhdtrah@local> Mail-Followup-To: zsh-workers@zsh.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: NeoMutt/20170609 (1.8.3) X-Provags-ID: V03:K0:KZ4vXrhPs80a0E/ROgA+ApGDqXRSn1vGK6oDpVf6D6UzqrPi0b5 Uw9lQZuCBojrlineOtJ/avTAKUEoUWGK+2duADvmxC5NCyCJ/kway6SRChHURKqcphywRdd KL2Ffqo4DL8KAsb/VFltWunVL0UeamU1VeOpTLipW+V84czjirVRvMI86lELQuoxR62486d WMcvYJjkhjE8Cp4zYHw2Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:gQVFW0hUOeE=:qb9i/Ec0iMgfXcxnT5cMv7 r5bN/EEm9ukfUa2ina134NYsKdH/m54vDEvRc810xoS6dTBzRxwzAiHTMS27305KImnWcWpea Ha/es07iUwF/jnOAZg8WEh/NDx+eTVYAItXlcLx2DN7VwUUW0NBLynzifeB0uglq8IIxT4Cyk Dkdx9ZWR4yHwV9al5o3s1EZzvvCftXWy7cZ5eQCTlvv2S0nFSRcpCnf+93kGmmpBgTW+zWPZi jdyxHDaA2z/Tkcebif1aJRwBK50nZIkpB841Oim1JuUaD6YAd8MXiVNzQW4Bh1fxiG8BE1Rnk lZ1Zs8aUzRzdxrlcTIH4zlt6oRDkZEcGW+/+vjtyDfz8EFbeE0NMadTXA7mx5oi2XIXK3c4kE piuwdV01fVzZvhFLMWyzy8a62xbzp+3SeEEB57UVM7XS/1fAJ0ARY8Ab0SIua31KYU1v15hIy TfyMwQZvctxV1TS1YegzsZwp6VJBqdyYqhyhuOqaEPfwsN5YWXfQx8cY5X/LEfYSAGTcVXdB7 6HzoRLRZj1m5MfyVqjD5ICCAqMR03aQBXaL3wnrA+NgXs22O6NsSYTzbGkvRaABbGS1MIqUfd VORPOK+Wyf21py2gwNQza2jsvzLukqFQUSmk7N9VeCR47U7iaBogSKNjS/G/x0FQn6faNiIRg JxtPrdOfv5N1LNS27DyvRQ7N45K7tI1CCGsvtBhrCEMKfFoGh8zzXoCt6qqXVw5mBjrbwNraf ZfEzZuyS1GrNdNXFvkZz5VrPUZe83lzBNct6ANE0tiRApyYdbhek0wPZVqnt4H7cXxkoN71l7 zRvqSj5Y6Ca6O/6GpMYl6JrUw0mYQ== Hi all! In search of the reason why zsh (5.4.2 and fairly recent git) is so much slower reading/writing larger history files (13 MB, 330k lines) than bash (4.4.12), I think I found a bug. If I missed something (like some option), and this is not a bug, but just misconfiguration, or if you need more information, please let me know. When configured to *append* the history on exit with the following config: ---------------- BEGIN .zshrc ----------------- #!/usr/bin/zsh HISTFILE="/var/tmp/.histfile" HISTSIZE=9223372036854775807 # LONG_MAX (limits.h) SAVEHIST=9223372036854775807 # LONG_MAX (limits.h) setopt append_history setopt extended_history unsetopt hist_save_by_copy unsetopt inc_append_history_time ---------------- END .zshrc ----------------- zsh (with "-d" or without) seems to do the following: Step 1) append to history file Step 2) read complete history file Step 3) write complete history file This seems to be invariant to - option HIST_SAVE_BY_COPY, - option INC_APPEND_HISTORY_TIME, - values of HISTSIZE and SAVEHIST (tried with 500000 for both). With "unsetopt APPEND_HISTORY", only step 3 is executed, as expected. I interspersed some debug printf's in Src/hist.c and saved the output of simply starting zsh and hitting Ctrl-D (exit). Absolute and relative times are in milliseconds. URLs: https://pastebin.com/BzJhrr69 .zshrc-appendhistory https://pastebin.com/ULCt7XcR .zshrc-noappendhistory https://pastebin.com/bHTqQu0C hist.c https://pastebin.com/QUjZ98Zn zsh-appendhistory--debug-output.txt https://pastebin.com/VUpGf6Fa zsh-noappendhistory--debug-output.txt Zeshy