From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 3611 invoked from network); 11 Aug 2021 20:30:03 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 11 Aug 2021 20:30:03 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1628713803; b=UlMntgcTGbyIZw7umMtHEHeKw87crTM0NeVP5oKdmZzW4s4/oyJx2NZmpJoF0oNaN7fcEBY4UT RqR7qmYKbUYsT7Fb9fYiKVwn0RtWcx6Wjy+2aKlVDs8/FayO8VjQ/JsMrjlQI4VrE/9WTXf1dL hRiEbp3kXy9NFG9XNSgwWE12xnWkqTym3nAfzcqwbaS9kixRbhOtfEMldpJV6o2XVcy0V9iCGg ew06tkzwK3qK0lzrKML824BCQJh8ZgA9MTgQBa0ylaqNGd7MACZBMKx5Q888xYdVoSina8de+1 NcMxrIyQZByi0DccDpjgDe8af8U9N3heEdZYsZWHXvbfiw==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-wr1-f44.google.com) smtp.remote-ip=209.85.221.44; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1628713803; bh=3uVYR5p5fcDE5csU6V7TjHi2isiNujBwbf/HQOFHmg8=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Transfer-Encoding:Content-Type:To:Subject:Message-ID: Date:From:MIME-Version:DKIM-Signature:DKIM-Signature; b=hth2JeuJ7EsdYcqr9LlM4rZiUxPakEfx3346hTROvj69G5ujZ/Q9LlZWshKRYMbbXY8AwO0nG7 6Ec+sHHRWkoLWvCHXs3Wko7ME5dEp7xeJmEBPXz6T2raCcKDOPKomGfxrLsJRklCkmpS79Ti3q ZrpyqqoCJ91j48qiPgqF2Q4D7rm+dvSjWiDH/jjJ4VZFWpAW0eEmlob9666KVRIqcdn9yultPy 0EowQvpWBXjlcD52m0Pxmp46wVjc3Inaq+anhJNobiNhxPb4zkFtG8c+iOuYN/dezP7g3NC8UP 6H6Kx6k6+TWXNcwPJeHtPVsqb9kOP9bqj+gfxhu5LpmLOA==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Transfer-Encoding: Content-Type:To:Subject:Message-ID:Date:From:MIME-Version:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References; bh=T0zQp5CI5i6T1EoF0F5Ji2LReV4ucooQ/dPR2PeqgzA=; b=XfGjLZucL4CBD79hsbDX6fSvYe 9iWotncpvAGYy+6QtW3plXe5tvDHOjQfUOTz2SEfIMH7Ob2TrMDgPtIteVGsrMEl8DqsMAK3iXyX1 HzzXO893mJC2KZFqsbHNANnZjBt7ectAL54M0yK8rq3Tog4XZpdW/jSAR0mH9lV9YJgbj84IdLtC8 SwExBGs1fMiONZ+zbKboNwG601p0VPZXenA0Y+TRL1JT33+dcCgKX9dpU/TieMBWOSDEj99OR8p91 IAa+pFvqSf4nFV0CoVgOtU5j+8m3R2XcyhZ2EgAWc/s9rpUCaeA31bzAiNJshmNYsQtBW8NSMQRIA oVycR8MA==; Received: from authenticated user by zero.zsh.org with local id 1mDurS-0000pt-PC; Wed, 11 Aug 2021 20:30:02 +0000 Authentication-Results: zsh.org; iprev=pass (mail-wr1-f44.google.com) smtp.remote-ip=209.85.221.44; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-wr1-f44.google.com ([209.85.221.44]:38734) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1mDuql-00005P-53; Wed, 11 Aug 2021 20:29:19 +0000 Received: by mail-wr1-f44.google.com with SMTP id l18so4819256wrv.5 for ; Wed, 11 Aug 2021 13:29:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=T0zQp5CI5i6T1EoF0F5Ji2LReV4ucooQ/dPR2PeqgzA=; b=ni4uXU34orzycNR0rCszjCIaGXkKdJp0O+agg8LMlbhJGfkv9JKAkjKe9rteN4ItBH WXylW8tKPHO5tF3oXEO0ogm+6or5bR9PGmH7hDq5iXCgCSG1aWQ5zMxmBawk1QVm3CDG WLO1ut0KGOJJGjIaKvZdx7Z5chDJkzE7ZQPqYCmsSVYbH26tBzpCfYJBqrjLjNgClZ/C EAIvZH7PTMo6u2gWQ9yZCwSW3VnmEs4v+LiKSeR4EdSK18XXiNrT8JK/ZZxrt79KrdEy 5jks3HTNvUGQupKy69xGsJL48bOJAv+JVWa98yiR9qEi9sPOoBoIkOFYbUXMoNyIOd+h L3KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=T0zQp5CI5i6T1EoF0F5Ji2LReV4ucooQ/dPR2PeqgzA=; b=mgGaGm6oUJXkS2Y+0X8v1Z3UKFBYRpC9kNuAj+8j0DIijTAB5WgRkaPV9xztFh0Wzu 9//we5nAlA3YHW5W7CqXsg2okJeP9fr0GEjiyq5NRgvc3AToGtuMulco8ngkzmwIcSA8 6icAMJevPzgAdyxgw2iABD8ktswJukbAa6I3pwAzyUPmPDRN0f7MhL5hxh3+JL2nlDR/ AFqf3Da8n+ndTBnPL8oHBWd0ApYXUTmt61Y2IBre0PvnsRUeYXEX5zQCwKAueYgFML55 nSsevuy+HbHP4xrMFZDHaHF0uCZRaZoSw0R/eX8JTV5DNCKs3AjekQg7dfZMfWjN/+c9 C3nA== X-Gm-Message-State: AOAM531ius2yIyZmBrPDsfq+r0gAfXdDJKRlceBIO3ESDE3gKGDDxKcB D7Paf8bwNhNcfo7ehoUmfZlMmKC0b3YK3sNxoY+ey5pBNVs= X-Google-Smtp-Source: ABdhPJxhJvrP17fjvLip3/2uLcyI+f+qqSxs/MCVLQypnSgmk/U/UngLAg6f6Ij3r/bX9OLMU9JG4AMfabPMoF7nxfo= X-Received: by 2002:a05:6000:92:: with SMTP id m18mr235435wrx.277.1628713758536; Wed, 11 Aug 2021 13:29:18 -0700 (PDT) MIME-Version: 1.0 From: Zach Riggle Date: Wed, 11 Aug 2021 15:29:07 -0500 Message-ID: Subject: Odd behavior with various (q) array modifiers and non-printable characters (backspace, newline) To: Zsh Users Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Seq: 26917 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: Hello all! It's me again, hopefully with a useful question. TLDR: (q-) behaves unexpectedly for \010 aka "\b", and there are other corner cases that may be unintended. I love the (q-) modifier for array expansion, since it allows printing out e.g. a command and then running it. run() { echo '$' "${(q-)@}" "$@" } The docs (https://zsh.sourceforge.io/Doc/Release/Expansion.html, search for "q-" to jump to the right area) state: > If a q- is given (only a single q may appear), a minimal form of single q= uoting is used that only quotes the string if needed to protect special cha= racters. Typically this form gives the most readable output. > If a q+ is given, an extended form of minimal quoting is used that causes= unprintable characters to be rendered using $=E2=80=99...=E2=80=99. This q= uoting is similar to that used by the output of values by the typeset famil= y of commands. However, it appears that half of the q modifiers handle 0x08 / \010 / "\b" aka "backspace" in an unexpected manner. Notably, (q) is incorrect, (qq) (q+) and (q-) embed a literal backspace, which causes unexpected output (it's there if you pipe to e.g. xxd, but it should be escaped). $ mysed=3D( gsed "s/\ba\b/x/" ) $ echo "aaaa a aaa" | "${mysed[@]}" aaaa x aaa $ echo ${(q)mysed} gsed s/\ba\b/x/ $ echo ${(qq)mysed} 'gsed' 's/x/' $ echo ${(qqq)mysed} "gsed" "s/\ba\b/x/" $ echo ${(qqqq)mysed} $'gsed' $'s/\ba\b/x/' $ echo ${(q+)mysed} gsed 's/x/' $ echo ${(q-)mysed} gsed 's/x/' This issue also appears with newlines, but for these one could argue it's entirely legitimate to have a *literal* newline that breaks the quoted statement onto multiple lines. Once again, the same culprits are (qq), (q-), and (q+). Interestingly, bare (q) is actually incorrect. $ newline=3D( aa "b\nb" cc ) $ echo ${(q)newline} aa b\nb cc $ echo ${(qq)newline} 'aa' 'b b' 'cc' $ echo ${(qqq)newline} "aa" "b\nb" "cc" $ echo ${(qqqq)newline} $'aa' $'b\nb' $'cc' $ echo ${(q-)newline} aa 'b b' cc $ echo ${(q+)newline} aa 'b b' cc IMHO both newline and backspace should be escaped at LEAST for (q-) and (q+), and newline behavior for (q) is incorrect. What's your opinion? Zach Riggle