From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 2622 invoked from network); 15 Oct 2020 07:38:03 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 15 Oct 2020 07:38:03 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20200801; t=1602747483; b=it1TVChfJNaod6kmHA2WbkkmS610O6MYM3bYhxToanqgRsgV9NTYNeabrnJcvBFsjMgsnN7/f3 l3dFi/AjJxYGUxp2MVb4MQpMFXcdDy28hQ9Ul1hJ6IN5twgoY83ts3C+y1q9xomtAAjGqjrdBw +g0pE4few0GY23KIiUa7QPM+WN0A53kVjcEPAZJvyxVnaQFSDmOw5jK9eGlE/0zTB9PH2V9/of aWCy2vA9p4pBw/SJKkpcBhLpElQxkHH2G+jKPReMazn8ZK16d5FxrLEG4nAXTpQXytCqI87A4k N3nJ7i5CniI6FieDZTvVZ70zeIPYRJ0IozWGKGTc4acqBQ==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-io1-f50.google.com) smtp.remote-ip=209.85.166.50; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20200801; t=1602747483; bh=yTdoJ6c9UD7eO2sRnPovG3LXbwVv2PUEa6efO4zq/rs=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:DKIM-Signature:DKIM-Signature; b=rILfNZNR3fNnKZ/KngKVkuxzaRoHuQiA/E4k7sV49bvNScTHGPEFvDUTJoJbr83vOfiR/41e6Z rqpwAgoxO//ITE8vjgLOnV4pJC1+BCnsE1xz+HkoT0mlPo0/VZ4M9K2xPX2uKn0cXIW/Tx+wrt sRHdfZztd9XF/H6nvCGM/YVtXd152HuJu64SP5V25dbejfwLTs77+c70eV40cNkobBUFZ49u25 or4sTKlQrXFUBiMmTD298m+tdOfAFEI3ztuIFcV2VRABaBj8FPxBdg4otlUxD3KeAFmIdXyTwZ yKS16g4LlJo8Q3vekDS91DdIpDxhFN3uJNyhcgbBzoei4w==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20200801; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=Zs9f5YZ2yX/ysvXg0OGMJ4yPhdhWXw8ayCBanAUA6IY=; b=xe0hu/ChVO8+jjut1zHmIlGmft rOWHtgo/CTtb2Hy3N/ce/Ao7GPSjZeGJDOtveyfv5sgjLdyzRlVJhGwy6GRVl6nwxRPlWSfpMCazv pbrdtLQ2fB2rTSHUtJLLSLcgPXWmjaWT6QhiX8d2MCM+iwf4TQiQDDncVMCveX0TlS45hIWr/AHj3 KX9CMYjToj74Py3LC0oBRUG9LKm32b8I4RCC4KMK3iRds2U95TpBmyjXQ2Rb5RzyLAPLjAVZXcBxe a42zQJqwVuu48pcUN3x1F5iNiD8ybEzRRxuRyh6TTxPf0BcPcop3uwOUJhHfj+y0w/srnGT3MM3o9 G/gwSuJw==; Received: from authenticated user by zero.zsh.org with local id 1kSxpi-0001zF-Vx; Thu, 15 Oct 2020 07:37:55 +0000 Authentication-Results: zsh.org; iprev=pass (mail-io1-f50.google.com) smtp.remote-ip=209.85.166.50; dkim=pass header.d=gmail.com header.s=20161025 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-io1-f50.google.com ([209.85.166.50]:35770) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1kSxpN-0001qX-Jh; Thu, 15 Oct 2020 07:37:35 +0000 Received: by mail-io1-f50.google.com with SMTP id k6so3150623ior.2 for ; Thu, 15 Oct 2020 00:37:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Zs9f5YZ2yX/ysvXg0OGMJ4yPhdhWXw8ayCBanAUA6IY=; b=sm1MJPC2fb6Lqt8+MkHD75BRhEeCBgCX/0MDfjT9o5giXwBy3R5zcCOTr5SQd27up0 BhNj2gj6Wjep+I6/TP041uNiJ1RCQP5xxpwFASzcSP7rXKp5CEpMKls3t2Ute6DhGnPs JhN+nIZUs771WvO61FLAF8+ErHmovX8qw5wtJKHRQOeWEIW3l3p45HfBCPEd2o0OEeZA vi5GjUADym4VELi6ZEmZjpzNK+Ua+Gc4+Gb8B4shaTFT5hh/5/UGIdTLFmfjtVkSkAPm n6VrU0FlRh0i3KvJIaWW1Wc82OVrqSkPrEQaYcjb3fOYWCLUqFWVm2m/NdfHk2I59oij NbQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Zs9f5YZ2yX/ysvXg0OGMJ4yPhdhWXw8ayCBanAUA6IY=; b=KOcGBqqVNK97HdJiU1WUv44TkAKWQbqBU50Ez82ZvevXzoJ5uMxaEfCmtTHCJZ4S79 WL1WbC3ZuR7375P9DJoc4saAtK1/KTHZPQbeWhw6Sll2ibgNg/K/6SJ7UxyTquRv/vwq EgxktN09whw+vy4oIxy24eD6/2wbxMK8hQc+YpbBJGN3F01LwtHl5M14CP6DYrQEe3Sj 2/mUNhT/xKdmaTw0sQsctoIMKy6zaJ/e3fatM1z+cY2HVDFkfNy+4DZ+WyHGIbj7epJO GNByOR/mmT6C+1GG22yvQ014oix6Smm6YDNQzBMtCotf0tIDB1nTfcRqcrnrw1ukIylV rx3w== X-Gm-Message-State: AOAM530eResy6eHox1hkMr6uezciSQ9V6r3LihinYRJxJt/1tCtUhnNG zBTvmiTG7kWJ6xrP3mVLkRh/xw9cNUX/kbzSIZM= X-Google-Smtp-Source: ABdhPJxZ5nUwdhNNbCg6qUgC3C4gaDWBz4gEfw+ahs779oDMBUj9ROp1kP51ZSdTQmblP/jysD5wppTjsfpJxDcarcA= X-Received: by 2002:a05:6638:2494:: with SMTP id x20mr2472722jat.83.1602747452207; Thu, 15 Oct 2020 00:37:32 -0700 (PDT) MIME-Version: 1.0 References: <20201014204621.4cf5b2b0@tarpaulin.shahaf.local2> In-Reply-To: <20201014204621.4cf5b2b0@tarpaulin.shahaf.local2> From: Roman Perepelitsa Date: Thu, 15 Oct 2020 09:37:21 +0200 Message-ID: Subject: Re: region_highlight converts `fg=default` to `none`, which is not the same To: Daniel Shahaf Cc: Marlon Richert , Zsh hackers list Content-Type: multipart/mixed; boundary="0000000000008e100b05b1b0b82d" X-Seq: 47458 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: Archived-At: --0000000000008e100b05b1b0b82d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Oct 14, 2020 at 10:46 PM Daniel Shahaf wro= te: > > z-sy-h does =C2=ABregion_highlight_copy=3D("${region_highlight[@]}"); =E2= =80=A6 > region_highlight=3D("${region_highlight_copy[@]}");=C2=BB around invoking > a highlighter, so the serialization bug could explain the observed ZLE > behaviour, couldn't it? I haven't tried reproducing the problem with z-sy-h. Instead, I've used the following code from `zsh -f`: zle-line-pre-redraw() { region_highlight=3D('0 5 fg=3D1' '1 2 fg=3Ddefault' '3 4 none') } zle -N zle-line-pre-redraw If you type "12345", odd and only odd numbers are supposed to be red. The actual behavior is that all numbers are red. I now took a look at how highlighting is applied (function zrefresh in Src/Zle/zle_refresh.c), then read the docs again, and I think there is a problem. If the same character is affected by two highlight specifications, how should it be highlighted? For example, if the first spec sets fg=3D1 and the second sets bg=3D2, how should the character be highlighted? What about fg=3D1 plus underline? Or underline plus fg=3D1? I could imagine two simple merging strategies: 1. All attributes are merged, so fg=3D1 + bg=3D2 + underline would result in underlined red text on green background. 2. The second highlight completely overrides the first. fg=3D1 + bg=3D2 + underline results in underlined text with the default color and no background. The meaning of "none" naturally follows from the choice of merging strategy. In the first case region highlight with "none" spec has no effect (X + none =3D> X). In the second case such a region is displayed without any highlighting (X + none =3D> no highlighting). The actual code does something else. If a spec has fg or bg with any value other than "default", then the spec completely overrides the previous spec. Otherwise the spec is merged with the previous spec with one exception: fg=3Ddefault and bg=3Ddefault have no effect (fg=3D1 + fg=3Ddefault,underline =3D> fg=3D1,underline). Note: "special" highlight is merged with a different algorithm. All other highlights, including "region", "isearch" and "paste", are merged the same way as the elements of region_highlights. A few examples of what the current code does: - fg=3D1 + bg=3D2 =3D> bg=3D2 * the second spec completely overrides the first - fg=3D1 + underline =3D> fg=3D1,underline * specs are merged - underline + fg=3D1 =3D> fg=3D1 * the second spec completely overrides the first - fg=3D1 + none =3D> fg=3D1 * specs are merged - fg=3D1 + fg=3Ddefault,underline =3D> fg=3D1,underline * specs are merged except that fg=3Ddefault has no effect This doesn't look ideal. So, how can we fix it? "The second highlight completely overrides the first" will change the meaning of "none" from "ignore this spec completely" to "display text with no highlighting". For example, if you set zle_highlight=3D(special:none), special characters will have no highlighting whatsoever, while currently they would get highlighted by region_highlight. "All attributes are merged" makes it impossible to disable underline, bold, etc. We have fg=3Ddefault and bg=3Ddefault to disable colors but there is no equivalent syntax for disabling underline. I think there is one change to the current algorithm that would be an improvement and is relatively safe to do -- make fg=3Ddefault and bg=3Ddefault work similarly to fg=3D1 and bg=3D1 as far as merging goes. Since fg=3D1 in a spec causes a complete override (disabling underline, bold, etc.), fg=3Ddefault should also do that. Patch attached (only for zrefresh; singlerefresh should be updated similarly). With this patch, if a spec has fg or bg, then it completely overrides the previous spec; otherwise the spec is merged with the previous spec. >From the examples above, only the following works differently with the provided patch: fg=3D1 + fg=3Ddefault,underline used to produce fg=3D1,underline but now it gives just underline without foreground color. Going forward, we can extend the spec syntax to give users more flexibility. I can see two extensions. 1. In addition to "underline", one can use "underline=3Don" and "underline= =3Doff". 2. If the first character of the spec is "+", it's merged with the current spec for the region; if the character is "=3D", it overrides; otherwise the current behavior is preserved (specs with fg and bg override, other specs merge). Thoughts? Roman. P.S. singlerefresh() doesn't use default_atr_on and thus zle_highlight=3D(default:fg=3D1) has no effect if SINGLE_LINE_ZLE is set. Is this intended? --0000000000008e100b05b1b0b82d Content-Type: text/plain; charset="US-ASCII"; name="highlight-patch.txt" Content-Disposition: attachment; filename="highlight-patch.txt" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_kgaieylq0 ZGlmZiAtLWdpdCBhL1NyYy9abGUvemxlX3JlZnJlc2guYyBiL1NyYy9abGUvemxlX3JlZnJlc2gu YwppbmRleCBkOWQ5NTAzZTIuLjFiMjQ2MTkxZiAxMDA2NDQKLS0tIGEvU3JjL1psZS96bGVfcmVm cmVzaC5jCisrKyBiL1NyYy9abGUvemxlX3JlZnJlc2guYwpAQCAtMTI3OCwyNyArMTI3OCwyMiBA QCB6cmVmcmVzaCh2b2lkKQogCQlvZmZzZXQgPSBwcmVkaXNwbGF5bGVuOyAvKiBpbmNyZW1lbnQg b3ZlciBpdCAqLwogCSAgICBpZiAocmhwLT5zdGFydCArIG9mZnNldCA8PSB0bXBwb3MgJiYKIAkJ dG1wcG9zIDwgcmhwLT5lbmQgKyBvZmZzZXQpIHsKLQkJaWYgKHJocC0+YXRyICYgKFRYVEZHQ09M T1VSfFRYVEJHQ09MT1VSKSkgewotCQkgICAgLyogb3ZlcnJpZGUgY29sb3VyIHdpdGggbGF0ZXIg ZW50cnkgKi8KLQkJICAgIGJhc2VfYXRyX29uID0gKGJhc2VfYXRyX29uICYgflRYVF9BVFRSX09O X1ZBTFVFU19NQVNLKSB8Ci0JCQlyaHAtPmF0cjsKLQkJfSBlbHNlIHsKLQkJICAgIC8qIG5vIGNv bG91ciBzZXQgeWV0ICovCisJCWlmIChyaHAtPmF0ciAmIChUWFRGR0NPTE9VUnxUWFROT0ZHQ09M T1VSfFRYVEJHQ09MT1VSfFRYVE5PQkdDT0xPVVIpKQorCQkgICAgYmFzZV9hdHJfb24gPSByaHAt PmF0cjsKKwkJZWxzZQogCQkgICAgYmFzZV9hdHJfb24gfD0gcmhwLT5hdHI7Ci0JCX0KIAkJaWYg KHRtcHBvcyA9PSByaHAtPmVuZCArIG9mZnNldCAtIDEgfHwKIAkJICAgIHRtcHBvcyA9PSB0bXBs bCAtIDEpCiAJCSAgICBiYXNlX2F0cl9vZmYgfD0gVFhUX0FUVFJfT0ZGX0ZST01fT04ocmhwLT5h dHIpOwogCSAgICB9CiAJfQotCWlmIChzcGVjaWFsX2F0cl9vbiAmIChUWFRGR0NPTE9VUnxUWFRC R0NPTE9VUikpIHsKLQkgICAgLyoga2VlcCBjb2xvdXJzIGZyb20gc3BlY2lhbCBhdHRyaWJ1dGVz ICovCi0JICAgIGFsbF9hdHJfb24gPSBzcGVjaWFsX2F0cl9vbiB8Ci0JCShiYXNlX2F0cl9vbiAm IH5UWFRfQVRUUl9DT0xPVVJfT05fTUFTSyk7Ci0JfSBlbHNlIHsKLQkgICAgLyoga2VlcCBjb2xv dXJzIGZyb20gc3RhbmRhcmQgYXR0cmlidXRlcyAqLwotCSAgICBhbGxfYXRyX29uID0gc3BlY2lh bF9hdHJfb24gfCBiYXNlX2F0cl9vbjsKLQl9CisKKwlhbGxfYXRyX29uID0gYmFzZV9hdHJfb247 CisJaWYgKHNwZWNpYWxfYXRyX29uICYgKFRYVEZHQ09MT1VSfFRYVE5PRkdDT0xPVVIpKQorCSAg ICBhbGxfYXRyX29uICY9IH4oVFhUX0FUVFJfRkdfT05fTUFTS3xUWFROT0ZHQ09MT1VSKTsKKwlp ZiAoc3BlY2lhbF9hdHJfb24gJiAoVFhUQkdDT0xPVVJ8VFhUTk9CR0NPTE9VUikpCisJICAgIGFs bF9hdHJfb24gJj0gfihUWFRfQVRUUl9CR19PTl9NQVNLfFRYVE5PQkdDT0xPVVIpOworCWFsbF9h dHJfb24gfD0gc3BlY2lhbF9hdHJfb247CiAJYWxsX2F0cl9vZmYgPSBUWFRfQVRUUl9PRkZfRlJP TV9PTihhbGxfYXRyX29uKTsKIAogCWlmICh0ID09IHNjcykJCQkvKiBpZiBjdXJzb3IgaXMgaGVy ZSwgcmVtZW1iZXIgaXQgKi8K --0000000000008e100b05b1b0b82d--