Re: [BUG?] If true-color is used, overlapping colors do not work

[Sebastian Gniazdowski <sgniazdowski@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2347 bytes --]

On Fri, 9 Nov 2018 at 02:28, Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
>(...)
>The only hope would be to strip the output
> down to just the section actually being tested (which comptest manages),
> also using something like:
>   zle_highlight=( fg_start_code:FGSTART fg_end_code:FGEND fg_default_code:FGDEF )
> and perhaps zle -T somehow to map escape sequences to something readable
> somehow contriving to set TERM to something that claims 256 colour support.
> And, only relying on the portable subset of sed syntax.
> It's not without reason that I was too lazy to attempt this in the first
> place.

I have new fresh test file which I hope brings the X04 test to some
new stage or level. The lines sent to zpty now aren't repeated,
because zsh is started with +Z option, and `setopt zle' is emitted
just before the "zpty -w zsh $'\C-a'" line, and the shell / zpty is
closed via Ctrl-D, so luckily only single one, the test line
(BUFFER="true", region_highlight+=...) is emitted and read.

I've used zle -T tc tcfunc, which sets REPLY="", i.e. discards the
codes. Should I change them to something? Because I'm only getting LE
termcap command, moving cursor to the left (after printing POSTDISPLAY
with zsh-autosuggestions text), some cd, ce, up commands and no codes
for colors. Do you think I should not strip the termcap codes and
specify them in test's expected output?

I'm currently removing following escapes from all zpty output:
- ^[[27m, ^[[24m, etc.
- ^[[J
- ^[[K
- ^[[?2004h
- e^Mexit -> exit
- ^[[?2004l

You seem to retrieve some other Zle-managing-output code (I think it's
this) and the tests fail? Maybe it is already fixed by the late-Zle
enabling, so you could try running the new test file?

There are 6 tests currently:

0:basic region_highlight with 8 colors
0:basic region_highlight with true-color (hex-triplets)
0:basic region_highlight with near-color (hex-triplets at input)
0:overlapping region_highlight with 8 colors
0:overlapping region_highlight with true-color
0:overlapping region_highlight with near-color (hex-triplets at input)

PS. Didn't yet search for terminfo file, just did export
TERM=xterm-256, not actually sure how to search for the definition
-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

[-- Attachment #2: X04highlight.ztst.txt --]
[-- Type: text/plain, Size: 5469 bytes --]

# Tests for region_highlight, true-color support, near-color support

%prep

  if [[ $OSTYPE == cygwin ]]; then
    ZTST_unimplemented='the zsh/zpty module does not work on Cygwin'
  elif zmodload zsh/zpty 2> /dev/null; then
    export TERM=xterm-256color
    zpty_start() {
      export PS1= PS2=
      zpty -d
      zpty zsh "${(q)ZTST_testdir}/../Src/zsh -fiV +Z"
    }
    zpty_input() {
      zpty ${${(M)2:#nonl}:+-n} -w zsh "$1"
    }
    zpty_enable_zle() {
      zpty -w zsh "tcfunc() { REPLY=""; }"
      zpty -w zsh "setopt zle; zle -T tc tcfunc"
    }
    zpty_line() {
      setopt localoptions extendedglob noshwordsplit
      local REPLY cm=$'\r'
      integer i
      for (( i = 0; i < ${1:-1}; ++i )); do
        zpty -r zsh REPLY
        # P is for "preserve", induces keeping some
        # color codes to test region_highlight, etc.
        # The color codes are then made regular text
        [[ "$2" = "p" ]] && {
            print -rl -- "$REPLY" >> /tmp/output # THE LINE
            REPLY=${REPLY//$'\x1b'\[([2][0-9;]m|[JK]|\?[0-9]##(h|l))/}
            REPLY=${REPLY//(#b)$'\x1b'\[([0-9;]##m)/${match[1]}}
        } || {
            REPLY=${REPLY//$'\x1b'\[([0-9;]##m|[JK]|\?[0-9]##(h|l))/}
        }
        # Fix e^Mexit - match ((?)\r(?)), if \2 == \3, then replace with \2
        # otherwise replace with \1 stripped out of leading/trailing [[:space:]]
        REPLY=${REPLY//(#b)((?(#c0,1))$cm(?(#c0,1)))/${${${(M)match[2]:#${match[3]}}:+${match[2]}}:-${${match[1]##[[:space:]]##}%%[[:space:]]##}}}
        [[ -n "$REPLY" ]] && print -r -- ${${REPLY%%[[:space:]]##}##[[:space:]]##}
      done
    }
    zpty_stop() {
      setopt localoptions extendedglob
      local REPLY cm=$'\r'
      # Zle is active, can use Ctrl-D to exit
      zpty -n -w zsh $'\C-d'
      # zpty gives no output when piped without these braces (?)
      # The while loop with // substitution is to convert `e^Mexit'
      # into `exit' (see zpty_line). The sed commands remove escapes
      { zpty -r zsh } | sed $'/[^[:space:]]/!d; s/\r$//; s/\x1b\\[[0-9;]*m//g; s/\x1b\\[[JK]//g; s/\x1b\\[?[0-9]*[hl]//g' | while read REPLY; do REPLY=${REPLY//(#b)((?(#c0,1))$cm(?(#c0,1)))/${${${(M)match[2]:#${match[3]}}:+${match[2]}}:-${${match[1]##[[:space:]]##}%%[[:space:]]##}}}; print -rn -- "$REPLY"; done
      zpty -d
      :
    }
  else
    ZTST_unimplemented='the zsh/zpty module is not available'
  fi

%test

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=green" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with 8 colors
>0m32mtrue39m

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#040810" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with true-color (hex-triplets)
>0m38;2;4;8;16mtrue39m

  zpty_start
  zpty_input 'zmodload zsh/nearcolor'
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#040810" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with near-color (hex-triplets at input)
>0m38;5;232mtrue39m

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=green" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=red" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with 8 colors
>0m32mt31mr39m32mue39m

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#00cc00" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=#cc0000" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with true-color
>0m38;2;0;204;0mt38;2;204;0;0mr39m38;2;0;204;0mue39m

  zpty_start
  zpty_input 'zmodload zsh/nearcolor'
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#00cc00" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=#cc0000" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with near-color (hex-triplets at input)
>0m38;5;40mt38;5;160mr39m38;5;40mue39m

%clean

  zmodload -ui zsh/zpty

# vim:ft=zsh