From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 7081 invoked from network); 19 Mar 2023 11:57:21 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 19 Mar 2023 11:57:21 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1679227041; b=iACsF2Pr8GKYhFr4DpBOteBWKbH2uyOEN2pquj8rdQEIFzvABzQvxhFTwZqNTgeukYkeyKaefv WyBngBfFpjUphF36LmAcMj3wMpqgB9r0xOTVBPe0za747hJeTVhMIdjR3DZ3s5FoRK0web6tXK s5eu/qv9/U55nxcV4sExYS2lEzW11ePwoR30jd7tYLk7nFSuH2JU3hxk3oeniAVyxwJPCvPa6L RpfbDN0ukd4dbnXIFxRYmPxanV83VZi1gUHWkJYz/koVlGKZ7fYhL9nrdowhDzCgx/JsQqu2z/ g3BT2WP/wxRSuurvwVreQL8hmgahIDjsTgdrwCnoK4ypkQ==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-4316.protonmail.ch) smtp.remote-ip=185.70.43.16; dkim=pass header.d=protonmail.ch header.s=protonmail3 header.a=rsa-sha256; dmarc=pass header.from=protonmail.ch; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1679227041; bh=z3iQKs/Cet0xPGMrJIVb3Rkc3HkqWXr96fOISzcEHk8=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Transfer-Encoding:Content-Type:MIME-Version: Message-ID:Subject:From:To:DKIM-Signature:Date:DKIM-Signature; b=JU5Dc/1o40/SmmBpyg0ILnoHUcPg8LBVHxDl/e4DnZjtXxQfbojL+CwRYJtBvLNzHiE3OR0Fos sSgIaOgNAp9x8Xn7lko5ayiIj2u0KIG4ymCzFAo7phxxslAusb5cGesKVsQI9gm7p6pXyISaDA oNKog0VcnwsSpyqTgRNrfnYQ+A26vssF/1/s3vwI8KONXWd013sZYypqz2478ndklv0S70jg1A M/QI0bSNnuYAVjLnK9RfCOdeUySM7Ifjbo8I5HB4woWLGBIhDrQR3mXxIugbk0KUFru5LpXbzU eAilV3bhfAGJvU/RKda4JVncwlSm5avGF1kD6qa0j1hn8w==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Message-ID:Subject:From:To:Date:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References; bh=7ulLb39mqbhfMYOmlD+iWqHjxnpZU+xctwL+Jg3H7sA=; b=ZBHY40TMopqZmVDwAcJN5nfaj9 gLwm3f0jsxdDvEgoDeK97A2+e1f5gB6j1I63F1oFBxQYx7qQ+jJjpelrRG5xT0y+WmZGcpQW8ers0 WnS6ExtXYXxVLHa3F9JFhAMb20gTtL/+lwNbzDQfLOKIDTbBYgdF7MlvHQQ0t6Ets1sJphFiGFlab 0YBEDOWhuq5mk+VaJkjm2MR0SdP85B0M7oaUU8Iy8BlGQJtokAqS42rndFUZ7G4Quj4PodhI9YGo9 IwhZis51KYnpIrxZoi6o1IIDjJ8zPP/Zb/lOHBfU1pe5XCklnccl1B/05FjPeRtRsi9++JfR87BWv gQopoXhA==; Received: by zero.zsh.org with local id 1pdrf5-000I7h-CX; Sun, 19 Mar 2023 11:57:19 +0000 Authentication-Results: zsh.org; iprev=pass (mail-4316.protonmail.ch) smtp.remote-ip=185.70.43.16; dkim=pass header.d=protonmail.ch header.s=protonmail3 header.a=rsa-sha256; dmarc=pass header.from=protonmail.ch; arc=none Received: from mail-4316.protonmail.ch ([185.70.43.16]:17169) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_256_GCM_SHA384:256) id 1pdreT-000HP3-H1; Sun, 19 Mar 2023 11:56:42 +0000 Date: Sun, 19 Mar 2023 11:56:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.ch; s=protonmail3; t=1679227000; x=1679486200; bh=7ulLb39mqbhfMYOmlD+iWqHjxnpZU+xctwL+Jg3H7sA=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=gX0kD8pvtwmyrnBxsn1FJpXvJSEMtyd+XspyiY8E9ABNRUm0xTLEY8cS8VH5VFF+1 dJ8GJQG8XqgmJOji8ppSgofYC2ZgUEV3GaA7zVXdjUO6Y6YrT6xgeWN/M+1vTBbRWR kxuZAc2dBW4G+hOVbJ1brmm1Jzw4C678D+Ct1vHoJacJYFmXM+mm0jQj1uI+AyLyE9 0EuEC43ym2iTd9CW5mbxnT5A/FzFqfACbQWVEOhvVwrSkDELQMza+wrzdduI0p6XtI sQiMNYCvTkOKFub8nRTTShMCqp2v9rtz8KSuujPH8OG8I0ed8DhFuWB2eD2GI640/D PtelqGrXj6HXA== To: zsh-users@zsh.org From: Piotr Karbowski Subject: The HIST_EXPIRE_DUPS_FIRST might corrupt and wipe partially history file if many shells exit at the same time Message-ID: <33a75d10-d765-3d79-a179-943c57659111@protonmail.ch> Feedback-ID: 7579861:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Seq: 28972 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: Hi, Recently I went ahead and enabled HIST_EXPIRE_DUPS_FIRST on my systems in order to get the most out of zsh_history. It worked great however I on two separated ocasions had a large chunk of my history file being lost. I narrowed it down to HIST_EXPIRE_DUPS_FIRST They way I can reproduce it sometimes is if multiple zsh shells exit at the very same time. It happens when I terminate tmux session that have 10+ zsh instances,or when I just reboot my system while I have dozens of urxvt instances open. This leads to all of the zsh instances to exit and do it's magic. This however happens only if I intentionally litter my .zsh_history to make the zsh actually want to run this logic. meaning the steps to reproduce would be: - Exit multiple zsh instances at the very same time. - Have .zsh_history big enough (HISTSIZE, SAVEHIST) so it trigger this logic. I had it happen to me (outside of trying to reproduce it) twice, on one system as root, on another as non-root user. Before I reported it I checked all the other things that could lead to this corruption, and I found a single user reporting the very same problem year and half ago on stackoverflow[1], he does indeed have histexpiredupsfirst enabled. The relevant configuration change that I did to enable this (among few other) HISTSIZE=3D'128000' -SAVEHIST=3D'128000' +SAVEHIST=3D'96000' +setopt hist_expire_dups_first setopt hist_ignore_dups +setopt hist_ignore_all_dups +setopt hist_find_no_dups +setopt hist_save_no_dups I have disabled this feature since due those corruptions. Would love to get back to it though, perhaps adding some locking mechanism would help here? [1] https://stackoverflow.com/questions/69434630/closing-multiple-iterm2-tabs-m= akes-zsh-history-lose-most-of-the-history -- Piotr.