From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 24877 invoked from network); 11 Aug 2021 23:26:23 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 11 Aug 2021 23:26:23 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1628724383; b=Iuf4SNlZkCrXRtTqumgJkiqwYTxf11Anhj3/6X9gVB6ShzowpqosgIz/jEDQD/cxUXlKVINqxp Z/g1Qd3uTPNHem9biK7rs58BJcGy8qdI2b1pTIrmwBQfPrvJmPH/Gj4ZTmPi4HbNqoiMh9m9sx Rshev4X2wCp5YueDLWhF1YMWSYueSnzeeGiKWYBBvU53SDbPZ1EMKvGp+37HbeJotI435zoGy+ sRIye5rD2kc1n07cufKHBh+6sKPlcaJdKLPDutKDmOLHxGGlVQt/yI0G1omUBnNOx+y2hrTnFc yTlmnHiZGqum6dw24LfP0iRM0MCa2w8TUUHFe06ljV2NqA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-wr1-f51.google.com) smtp.remote-ip=209.85.221.51; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1628724383; bh=Yh3XxOS1uZHghLVERfyVMTr0QswiGbTFaBDoM3rw8DM=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Transfer-Encoding:Content-Type:Cc:To:Subject: Message-ID:Date:From:In-Reply-To:References:MIME-Version:DKIM-Signature: DKIM-Signature; b=LUrcfmDaHMF6Ng1OXQla9C52knqEwg+fgkHQyQgAufABuJl6MAHxJAELz/A8rOUk34IWowykuj 3QOgZWlCpSXUXC3mTcvZmE65ueb83i/IV/JaZAqPQRq5qYoSGIzuz3XV33VkdSkjtVLvJazWne 0mf1NPZh/Dch4xpv2s06hEoW5ufmv9M5UmCUeQiOIkY2helU/JeRMF2bE3one4B60Xtrrd0mJQ oPouRtDDVoduI82G3gjRsDInMW5lrJWgMD6bBGNkgPEN+V+bMXEMhiitOxh3RBUaGeq42zSVLF bGFL6+quaZFINhNXb/LDKXVJtN6RpRR3SRUYEztW0aY6hw==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Transfer-Encoding: Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To:References: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=APZAo1WMgipCMQaWQGqNJEUDRGZybqpT/1JCR+xM55w=; b=LmBoutB0FPVnRUSY72mkTK4OyG ByBtHQS20mp9XAjJElju92fr3ZJuKCViF9t2BuHWKquNcXxkH9jLM9pLW8a/JhpRYMMj2kcI3LX3G KqZrmKjyk5DQnVWdTppdiNoIfIy13vYw1famWw5ocxLgzIuvRfgkGABtW755I2B7rb4iNF05PE4NR AHH6VMcwKJPWSnUB4xCNg27dZHAOYsouMUQ6CCb5ktAg0STmyYk1QIlCCIP7dxEffVU9fGwNiJHes scYdHLMz3sdG3JzX2BA3IuCNF95W6fmwXnQHR8/C9QTkx91z6wSfeezQuSRlOLxIbMXUJGOOEdKyl IKp0xkPg==; Received: from authenticated user by zero.zsh.org with local id 1mDxc7-000EQY-1V; Wed, 11 Aug 2021 23:26:23 +0000 Authentication-Results: zsh.org; iprev=pass (mail-wr1-f51.google.com) smtp.remote-ip=209.85.221.51; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-wr1-f51.google.com ([209.85.221.51]:40536) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1mDxbG-000DkE-Nz; Wed, 11 Aug 2021 23:25:31 +0000 Received: by mail-wr1-f51.google.com with SMTP id k29so5371387wrd.7; Wed, 11 Aug 2021 16:25:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=APZAo1WMgipCMQaWQGqNJEUDRGZybqpT/1JCR+xM55w=; b=Bh4s0nckRpXoIQ2nJLHY0opw1bwuOWCuEAF85FoTG7Td2cA9UnUj8nSWMr27nR6/FA yiivhkRlnZVsg8eX5X+uyTRGzHaIpl5hRlmFuXvrUwa340FyHmrrcTYAkuqUDThu21AN gW1iaDGX0RXXijozGpgRQllCKX3dpgantjvbGT2ZMTXkpFKbnbU19OHRvjdw/OCel9Go G3fhXFHiJSpXecoWpaJOwShO82VJ0L53H8m1KQyacnkjC7z4mfaQL7ImSM+qppIciE2H 1+FAwa5ABjaQqkGkZA26cOd+gnMVmaNJTCblv4kQfV1WLLGokR0u1E0bhRacFS4EuREZ lcWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=APZAo1WMgipCMQaWQGqNJEUDRGZybqpT/1JCR+xM55w=; b=RD7z4GariWmJmcbwTpWxlYwd1DB3OuUr5JrmAfU1+4fqrEkWfjY/6ytxHVvJIfyC/U Y5ecuPrHOseX/oAz/Uw9q/5o4l+6J3YR2K+6nIOFCEh+I/BSiPFvVAmpaL4kkNuxr9QT 20Rf9IQ+WYVfbE7nx1DL1eLowPM+6HHtHNq452WHkv9cPUkWgZLMISEdFIhB0xJ1w0l7 5cr+jSA0pCpm2itKKhtaprom1Phv2wWqaGqssaioevvDHBsuOvk6LEZ1AVLmkuuVWH+T EhtfiDxZQ9roe9bNycGJI6E+f/XcKiFA6yO9qmFLuuKpAMB95nBxNwJurUnmtnhH+gWl qS6Q== X-Gm-Message-State: AOAM532RhJ8wWSEM3ZEw+cK0V6YktAzjOtZk/AG2LJF+W0tb7LmWE9X0 RwOgz1qJCzHlTgKUiaf47+frVtNIm//nZZ7oqMg0RisCLMY= X-Google-Smtp-Source: ABdhPJxpDquDgSTA/TL+vnhOFquabto0DEt3EMlcReJNpSTSBxfJoMLRBUu3goMFrj7xWzjT9fn2dz6KgfQXt/hJG6g= X-Received: by 2002:a5d:4e4f:: with SMTP id r15mr785538wrt.346.1628724329812; Wed, 11 Aug 2021 16:25:29 -0700 (PDT) MIME-Version: 1.0 References: <27bc215c-3916-4afe-b8b3-7946dd0cb4bc@www.fastmail.com> In-Reply-To: <27bc215c-3916-4afe-b8b3-7946dd0cb4bc@www.fastmail.com> From: Zach Riggle Date: Wed, 11 Aug 2021 18:25:18 -0500 Message-ID: Subject: Re: Odd behavior with various (q) array modifiers and non-printable characters (backspace, newline) To: =?UTF-8?Q?Lawrence_Vel=C3=A1zquez?= Cc: Bart Schaefer , Zsh Users Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Seq: 26922 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: TLDR 1: I learned/realized that zsh builtin echo automatically interprets escape codes, unlike bash echo and /bin/echo. TLDR 2: I also learned that (q+) is my new favorite, and my code should use "command echo" to avoid the above behavior. TLDR 3: Thank you all for taking the time to write back. > You need to stop testing things with "echo". The "echo" builtin interpre= ts some backslash escapes itself... Ah, the real culprit. For some reason Bart's comments didn't land in my inbox, but I am a mixture of disappointed in myself (for forgetting the builtin evaluates escape codes) and surprised (bash's builtin echo requires "-e" to do this, /bin/echo does not support this at all on BSD). Indeed, avoiding the builtin echo's interpretation of escape codes by default (not sure how I feel about that, but it is what it is) shows that the expansion modifiers work as they say on the tin. $ command echo ${(qq)mysed} 'gsed' 's/\ba\b/x/' $ command echo ${(q+)mysed} gsed 's/\ba\b/x/' $ command echo ${(q-)mysed} gsed 's/\ba\b/x/' > Presumably you thought "..." works like $'...'. Nope, that one I knew pretty well, but I was indeed spoiled by the shell interpretation of the string "\n". Leaning on Python a bit to make it more obvious, but yeah: $ python -c 'import sys; print(sys.argv)' a b "c\n" d ['-c', 'a', 'b', 'c\\n', 'd'] $ python -c 'import sys; print(sys.argv)' a b $'c\n' d ['-c', 'a', 'b', 'c\n', 'd'] Interestingly, the handling of ACTUAL newlines ($'\n') by (q-) is less than ideal, (q) works but is clunky, and (q+) seems to work the best. $ newline=3D( aa $'b\nb' cc ) $ command echo ${(q)newline} aa b$'\n'b cc $ command echo ${(q-)newline} aa 'b b' cc $ command echo ${(q+)newline} aa $'b\nb' cc Thanks for all of the help fielding my silly questions! Zach Riggle On Wed, Aug 11, 2021 at 4:45 PM Lawrence Vel=C3=A1zquez wr= ote: > > On Wed, Aug 11, 2021, at 5:16 PM, Bart Schaefer wrote: > > You need to stop testing things with "echo". The "echo" builtin > > interprets some backslash escapes itself, which will confuse you about > > what the quoting options have done. > > > > Repeat all your tests instead with > > printf "%s\n" ${(q)...} > > and so on, and come back if you still have questions. > > Additionally, \b and \n are not interpreted in double quotes, so > your initial data does not actually contain BS or NL characters. > Presumably you thought "..." works like $'...'. > > % mysed_orig=3D( gsed "s/\ba\b/x/" ) > % typeset -p mysed_orig > typeset -a mysed_orig=3D( gsed 's/\ba\b/x/' ) > > % mysed_fixed=3D(gsed $'s/\ba\b/x/') > % typeset -p mysed_fixed > typeset -a mysed_fixed=3D( gsed $'s/\C-Ha\C-H/x/' ) > > % newline_orig=3D( aa "b\nb" cc ) > % typeset -p newline_orig > typeset -a newline_orig=3D( aa 'b\nb' cc ) > > % newline_fixed=3D( aa $'b\nb' cc ) > % typeset -p newline_fixed > typeset -a newline_fixed=3D( aa $'b\nb' cc ) > > As per the QUOTING section of zshmisc(1): > > Inside double quotes (""), parameter and command substitution > occur, and `\' quotes the characters `\', ``', `"', `$', > and the first character of $histchars (default `!'). > > -- > vq