Re: highlight test cases (was Re: [BUG?] If true-color is used, overlapping colors do not work)

[Sebastian Gniazdowski <sgniazdowski@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2817 bytes --]

On Sat, 24 Nov 2018 at 18:32, Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
>
> On 11 Nov, Sebastian Gniazdowski wrote:
>
> > I've used zle -T tc tcfunc, which sets REPLY="", i.e. discards the
> > codes. Should I change them to something? Because I'm only getting LE
>
> If removing them works for the particular tests then I'd have thought
> this was fine.

Ok, so I've also removed the deleting of \[[J and \[[K codes (result:
simpler patterns and regexes), which are covered by the "zle -T ..."
sink.

> > I'm currently removing following escapes from all zpty output:
> > - ^[[?2004h
> > - ^[[?2004l
>
> unsetting zle_bracketed_paste will prevent zsh from generating these
> which is better than removing them afterwards.

I'm using this now, the codes are indeed not generated now. One thing,
I've had to call `unset zle_brack...' after `setopt zle', not before
it.

So the sed command is now quite simple, only three "-e" arguments:

{ zpty -r zsh } | sed -e $'/[^\t\r ]/!d' -e $'s/\r$//' -e
$'s/\x1b\\[[0-9;]*m//g' | while ...

> > PS. Didn't yet search for terminfo file, just did export
> > TERM=xterm-256, not actually sure how to search for the definition
>
> Searching probably isn't easy to do portably. Perhaps guessing like this
> is better, but it would be good to check $termcap[Co] afterwards and
> skip tests as appropriate.

This is now done, i.e. termcap[Co] is being check:

  export TERM=xterm-256color
  if [[ ${+termcap} != 1 || ${termcap[Co]} != <-> || ${termcap[Co]}
-lt 256 ]]; then
    ZTST_unimplemented="no termcap module OR termcap doesn't support
256 or more colors"
  elif [[ $OSTYPE == cygwin ]]; then
    ZTST_unimplemented='the zsh/zpty module does not work on Cygwin'
  elif zmodload zsh/zpty 2> /dev/null; then
    zpty_start() {
    ...

> This use of sed isn't portable. In particular, the use of semi-colons to
> separate commands and the [[:space:]] character class are extensions.
> You can use literal spaces and multiple -e options. Either that
> or do the substitutions in zsh.

The current sed invocation is pasted above – sed is now using literal
spaces instead of [[:space:]], multiple -e commands instead of
semicolon and is simplified.


Now, the bug with zle_highlight + zsh/nearcolor:

-0mCDE|38;5;232|trueCDE|39|
+0m38;5;232mtrueCDE|39|
Test ./X04zlehighlight.ztst failed: output differs from expected as
shown above for:
...
Was testing: basic region_highlight with near-color (hex-triplets at input)

The output doesn't follow zsh_highlight replacements for start-code
and end-code and still emits raw codes.

Could this be fixed?

--
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

[-- Attachment #2: X04zlehighlight.ztst.txt --]
[-- Type: text/plain, Size: 5915 bytes --]

# Tests for region_highlight, true-color support, near-color support
# Version 0.5
%prep

  export TERM=xterm-256color
  if [[ ${+termcap} != 1 || ${termcap[Co]} != <-> || ${termcap[Co]} -lt 256 ]]; then
    ZTST_unimplemented="no termcap module OR termcap doesn't support 256 or more colors"
  elif [[ $OSTYPE == cygwin ]]; then
    ZTST_unimplemented='the zsh/zpty module does not work on Cygwin'
  elif zmodload zsh/zpty 2> /dev/null; then
    zpty_start() {
      export PS1= PS2=
      zpty -d
      zpty zsh "${(q)ZTST_testdir}/../Src/zsh -fiV +Z"
      zpty -w zsh 'zle_highlight=( fg_start_code:"CDE|3" fg_end_code:"|" bg_start_code:"BCDE|4" bg_end_code:"|" )'
    }
    zpty_input() {
      zpty ${${(M)2:#nonl}:+-n} -w zsh "$1"
    }
    zpty_enable_zle() {
      zpty -w zsh "tcfunc() { REPLY=""; }"
      # This line will not be echoed back, behaving like ! -o zle
      zpty -w zsh "setopt zle; zle -T tc tcfunc; unset zle_bracketed_paste"
    }
    zpty_line() {
      setopt localoptions extendedglob noshwordsplit
      local REPLY cm=$'\r'
      integer i
      for (( i = 0; i < ${1:-1}; ++i )); do
        zpty -r zsh REPLY
        # P is for "preserve", and induces keeping some
        # expected color codes to test region_highlight,
        # etc. - the color codes are made a regular text.
        [[ "$2" = "p" ]] && {
            REPLY=${REPLY//$'\x1b'\[([2][0-9;]m|\?[0-9]##(h|l))/}  # remove only 2...m codes
            REPLY=${REPLY//(#b)$'\x1b'\[([0-9;]##m)/${match[1]}}
        } || {
            REPLY=${REPLY//$'\x1b'\[([0-9;]##m|\?[0-9]##(h|l))/}   # remove all [0-9]...m codes
        }
        # Fix e^Mexit - match ((?)\r(?)), if \2 == \3, then replace with \2
        # otherwise replace with \1 stripped out of leading/trailing [[:space:]]
        REPLY=${REPLY//(#b)((?(#c0,1))$cm(?(#c0,1)))/${${${(M)match[2]:#${match[3]}}:+${match[2]}}:-${${match[1]##[[:space:]]##}%%[[:space:]]##}}}
        [[ -n "$REPLY" ]] && print -r -- ${${REPLY%%[[:space:]]##}##[[:space:]]##}
      done
    }
    zpty_stop() {
      setopt localoptions extendedglob
      local REPLY cm=$'\r'
      # Zle is active, can use Ctrl-D to exit
      zpty -n -w zsh $'\C-d'
      # zpty gives no output when piped without these braces (?)
      # The while loop with // substitution is to convert `e^Mexit'
      # into `exit' (see zpty_line). The sed commands remove escapes
      { zpty -r zsh } | sed -e $'/[^\t\r ]/!d' -e $'s/\r$//' -e $'s/\x1b\\[[0-9;]*m//g' | while read REPLY; do REPLY=${REPLY//(#b)((?(#c0,1))$cm(?(#c0,1)))/${${${(M)match[2]:#${match[3]}}:+${match[2]}}:-${${match[1]##[[:space:]]##}%%[[:space:]]##}}}; print -rn -- "$REPLY"; done
      zpty -d
      :
    }
  else
    ZTST_unimplemented='the zsh/zpty module is not available'
  fi

%test

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=green" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with 8 colors
>0mCDE|32|trueCDE|39|

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#040810" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with true-color (hex-triplets)
>0mCDE|38;2;4;8;16|trueCDE|39|

  zpty_start
  zpty_input 'zmodload zsh/nearcolor'
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#040810" ); }'
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:basic region_highlight with near-color (hex-triplets at input)
>0mCDE|38;5;232|trueCDE|39|

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=green" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=red" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with 8 colors
>0mCDE|32|tCDE|31|rCDE|39|CDE|32|ueCDE|39|

  zpty_start
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#00cc00" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=#cc0000" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with true-color
>0mCDE|38;2;0;204;0|tCDE|38;2;204;0;0|rCDE|39|CDE|38;2;0;204;0|ueCDE|39|

  zpty_start
  zpty_input 'zmodload zsh/nearcolor'
  zpty_input 'rh_widget() { BUFFER="true"; region_highlight+=( "0 4 fg=#00cc00" ); rh2; }'
  zpty_input 'rh2() { region_highlight+=( "1 2 fg=#cc0000" ); }' # `r' in red; the above line would be too long
  zpty_input 'zle -N rh_widget'
  zpty_input 'bindkey "\C-a" rh_widget'
  zpty_enable_zle
  zpty_input $'\C-a'  # emits newline, which executes BUFFER="true" command
  zpty_line 1 p       # the line of interest, preserving escapes ("p")
  zpty_stop
0:overlapping region_highlight with near-color (hex-triplets at input)
>0mCDE|38;5;40|tCDE|38;5;160|rCDE|39|CDE|38;5;40|ueCDE|39|

%clean

  zmodload -ui zsh/zpty

# vim:ft=zsh