End boldface also ends background color

End boldface also ends background color

[Dominik Ritter]

Hi List!

I have a Problem regarding boldface mode. If I change the background
color of a string, and want to set a part of that string in boldface,
the background color gets ended as well.

Example:
print -P '%K{black} %F{white}%Bhello%b world'

The boldface ending sequence, also ends the background color. So only
"hello" gets printed white on black, "world" gets printed in white
(without background).

This seems like a bug to me.. Anyone else can confirm that?
Btw. I am on ZSH 5.2 (latest homebrew).

Of course, there are some workarounds:

- Start the background color again, after ending boldface.
Well. Possible, but quite verbose.

- Use $fg_bold[white] and $fg_no_bold[white] instead of %B and %b.
Also quite verbose..

Best regards,
Dominik Ritter

Re: End boldface also ends background color

[Mikael Magnusson]

On Tue, Mar 22, 2016 at 8:44 PM, Dominik Ritter
<dritter03@googlemail.com> wrote:
> Hi List!
>
> I have a Problem regarding boldface mode. If I change the background
> color of a string, and want to set a part of that string in boldface,
> the background color gets ended as well.
>
> Example:
> print -P '%K{black} %F{white}%Bhello%b world'
>
> The boldface ending sequence, also ends the background color. So only
> "hello" gets printed white on black, "world" gets printed in white
> (without background).
>
> This seems like a bug to me.. Anyone else can confirm that?
> Btw. I am on ZSH 5.2 (latest homebrew).
>
> Of course, there are some workarounds:
>
> - Start the background color again, after ending boldface.
> Well. Possible, but quite verbose.
>
> - Use $fg_bold[white] and $fg_no_bold[white] instead of %B and %b.
> Also quite verbose..

The code for ending bold sequences is usually ^[[22m but %b sends
^[[0m which resets most things. As far as I know, termcap
unfortunately doesn't include the code that only ends bold, so there's
no way for zsh to know what it is. Zsh doesn't hardcode any of these
sequences because they can vary across terminals.

-- 
Mikael Magnusson

Re: End boldface also ends background color

[Bart Schaefer]

On Mar 22, 10:00pm, Mikael Magnusson wrote:
}
} The code for ending bold sequences is usually ^[[22m but %b sends
} ^[[0m which resets most things. As far as I know, termcap
} unfortunately doesn't include the code that only ends bold, so there's
} no way for zsh to know what it is. Zsh doesn't hardcode any of these
} sequences because they can vary across terminals.

The code for this in prompt.c is interesting:

	    case 'b':
		txtchangeset(txtchangep, TXTNOBOLDFACE, TXTBOLDFACE);
		txtchangeset(txtchangep, TXTNOSTANDOUT, TXTSTANDOUT);
		txtchangeset(txtchangep, TXTNOUNDERLINE, TXTUNDERLINE);
		txtunset(TXTBOLDFACE);
		tsetcap(TCALLATTRSOFF, TSC_PROMPT|TSC_DIRTY);
		break;

If you look at zsh.h where the TC* flags are defined:

#define TCBOLDFACEBEG  18
#define TCSTANDOUTBEG  19
#define TCUNDERLINEBEG 20
#define TCALLATTRSOFF  21
#define TCSTANDOUTEND  22
#define TCUNDERLINEEND 23

There's no TCBOLDFACEEND, which I presume is why %b resorts to turning
off everything.  But I don't know much about how the txt* macros work.

In any case this code pre-dates the ability to set colors with %F by
several years, and probably wasn't updated when %F was added.  Given
that the color attributes are stored in the txtchangep set, it should
be possible to turn them back on after TCALLATTRSOFF disables bold, if
somebody who *does* know how this works wants to dive in.

Aside:

The doc for %F references zle_highlight which implies that you should
be able to do %F{bold} but that doesn't work, and the numeric color
values supported are not the ANSI color attributes.  The doc should
probably be tightened up to reflect this.

Re: End boldface also ends background color

[Peter Stephenson]

On Wed, 23 Mar 2016 18:53:56 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> There's no TCBOLDFACEEND, which I presume is why %b resorts to turning
> off everything.  But I don't know much about how the txt* macros work.

The bottom level of this, where it feeds into termcap, is
settextattributes() in zle_referesh.c:

    if (txtchangeisset(atr, TXTNOBOLDFACE))
	tsetcap(TCALLATTRSOFF, 0);

Even in terminfo there doesn't seem to be exit_bold_mode, unlike
standout, reverse and underline (though it looks like standout maps to
reverse here).

If you look at the definitions, you'll see that \e[1m turns on bold,
\e[7m reverse, \e[4m underline.  The definitions mapping to exit reverse
and exit underline are \e[27m and \e[24m.  Nothing maps to \e[21m that I
can see.  It works on some terminals, I used gnome-terminal:

print -P "first%Ufoo%{\e[1m%}bar%{\e[21m%}more%ulast"

However, that didn't seem to work on xterm (a few years old), so
apparently we can't assume it.  We could make it an option, but that's a
bit of a pain.  Short of that, it's not going to work without rewriting
the code substantially to track the current mode as well as the modes
that need changing.

> The doc for %F references zle_highlight which implies that you should
> be able to do %F{bold} but that doesn't work, and the numeric color
> values supported are not the ANSI color attributes.  The doc should
> probably be tightened up to reflect this.

No, they're not the same thing, though we do make use of termcap for
colours where available.

pws

Re: End boldface also ends background color

[Danek Duvall]

On Thu, Mar 24, 2016 at 10:38:37AM +0000, Peter Stephenson wrote:

> On Wed, 23 Mar 2016 18:53:56 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > There's no TCBOLDFACEEND, which I presume is why %b resorts to turning
> > off everything.  But I don't know much about how the txt* macros work.
> 
> The bottom level of this, where it feeds into termcap, is
> settextattributes() in zle_referesh.c:
> 
>     if (txtchangeisset(atr, TXTNOBOLDFACE))
> 	tsetcap(TCALLATTRSOFF, 0);
> 
> Even in terminfo there doesn't seem to be exit_bold_mode, unlike
> standout, reverse and underline (though it looks like standout maps to
> reverse here).
> 
> If you look at the definitions, you'll see that \e[1m turns on bold,
> \e[7m reverse, \e[4m underline.  The definitions mapping to exit reverse
> and exit underline are \e[27m and \e[24m.  Nothing maps to \e[21m that I
> can see.  It works on some terminals, I used gnome-terminal:
> 
> print -P "first%Ufoo%{\e[1m%}bar%{\e[21m%}more%ulast"
> 
> However, that didn't seem to work on xterm (a few years old), so
> apparently we can't assume it.

Worth at least pinging Thomas Dickey to add the capability to xterm, and to
create a new capability for ncurses -- or at least find out if there's a
reason it doesn't exist?  Once you build zsh against the new version of
both, it would work, and you'd get historical behavior until then.

Danek

Re: End boldface also ends background color

[Mikael Magnusson]

On Thu, Mar 24, 2016 at 11:38 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
> On Wed, 23 Mar 2016 18:53:56 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
>> There's no TCBOLDFACEEND, which I presume is why %b resorts to turning
>> off everything.  But I don't know much about how the txt* macros work.
>
> The bottom level of this, where it feeds into termcap, is
> settextattributes() in zle_referesh.c:
>
>     if (txtchangeisset(atr, TXTNOBOLDFACE))
>         tsetcap(TCALLATTRSOFF, 0);
>
> Even in terminfo there doesn't seem to be exit_bold_mode, unlike
> standout, reverse and underline (though it looks like standout maps to
> reverse here).
>
> If you look at the definitions, you'll see that \e[1m turns on bold,
> \e[7m reverse, \e[4m underline.  The definitions mapping to exit reverse
> and exit underline are \e[27m and \e[24m.  Nothing maps to \e[21m that I
> can see.  It works on some terminals, I used gnome-terminal:
>
> print -P "first%Ufoo%{\e[1m%}bar%{\e[21m%}more%ulast"
>
> However, that didn't seem to work on xterm (a few years old), so
> apparently we can't assume it.  We could make it an option, but that's a
> bit of a pain.  Short of that, it's not going to work without rewriting
> the code substantially to track the current mode as well as the modes
> that need changing.

Like I wrote in my first reply, 22 turns off bold in most terminals,
but termcap doesn't include it.
ctlseqs.txt says
            Ps = 2 2  -> Normal (neither bold nor faint)
            Ps = 2 4  -> Not underlined
            Ps = 2 5  -> Steady (not blinking)
            Ps = 2 7  -> Positive (not inverse)
            Ps = 2 8  -> Visible, i.e., not hidden (VT300)
and as you say, nothing maps to 21. I don't know why termcap doesn't
include the bold-off code.

>> The doc for %F references zle_highlight which implies that you should
>> be able to do %F{bold} but that doesn't work, and the numeric color
>> values supported are not the ANSI color attributes.  The doc should
>> probably be tightened up to reflect this.
>
> No, they're not the same thing, though we do make use of termcap for
> colours where available.
>
> pws



-- 
Mikael Magnusson

Re: End boldface also ends background color

[Bart Schaefer]

On Mar 24,  7:53am, Danek Duvall wrote:
}
} Worth at least pinging Thomas Dickey to add the capability to xterm

This would have to be a new termcap/terminfo library entry (or whatever
stands in for them, maybe it's ncurses); it's not something missing from
xterm.

} Once you build zsh against the new version of
} both, it would work, and you'd get historical behavior until then.

We'd still have to reprogram zle_refresh.c to know about and do the
run-time test for the new capability, which would take years to make it
into everyone's terminfo databases, etc.

I think we already have enough information to reprogram %b to first
turn everything off, and then turn back on the stuff that shouldn't
have been turned off.  That'll potentially emit a bunch more escape
codes at the point of %b but would solve the problem without having to
wait for hundreds of OS distributions to update terminfo.

Re: End boldface also ends background color

[Bart Schaefer]

On Mar 24, 10:38am, Peter Stephenson wrote:
} Subject: Re: End boldface also ends background color
}
} If you look at the definitions, you'll see that \e[1m turns on bold,
} \e[7m reverse, \e[4m underline.  The definitions mapping to exit reverse
} and exit underline are \e[27m and \e[24m.  Nothing maps to \e[21m that I
} can see.

21m is double-underline, according to comments in Functions/Misc/colors,
but it's not widely supported.

The right thing for bold-off would be "normal" i.e. 22m, but as noted
there is no termcap code for that.

} Short of that, it's not going to work without rewriting
} the code substantially to track the current mode as well as the modes
} that need changing.

Aren't we already tracking the current mode in a bitfield?

Replies probably ought to start going to zsh-workers.

Re: End boldface also ends background color

[Peter Stephenson]

On Thu, 24 Mar 2016 10:33:47 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mar 24, 10:38am, Peter Stephenson wrote:
> } Subject: Re: End boldface also ends background color
> }
> } If you look at the definitions, you'll see that \e[1m turns on bold,
> } \e[7m reverse, \e[4m underline.  The definitions mapping to exit reverse
> } and exit underline are \e[27m and \e[24m.  Nothing maps to \e[21m that I
> } can see.
> 
> 21m is double-underline, according to comments in Functions/Misc/colors,
> but it's not widely supported.
> 
> The right thing for bold-off would be "normal" i.e. 22m, but as noted
> there is no termcap code for that.

As long as it generally works when the other sequences do, we could use
a special case.

pws

Re: End boldface also ends background color

[Bart Schaefer]

On Mar 24,  5:39pm, Peter Stephenson wrote:
} Subject: Re: End boldface also ends background color
}
} On Thu, 24 Mar 2016 10:33:47 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > The right thing for bold-off would be "normal" i.e. 22m, but as noted
} > there is no termcap code for that.
} 
} As long as it generally works when the other sequences do, we could use
} a special case.

Yes, we could do something like (obviously psuedo-code):

    if (termcap("md") == "\e[1m")
       emit("\e[22m")

but then what is the "else"?  emit(termcap("me")) like we do now?  Does
that really help anybody who wants to write a portable prompt string?

I suspect "consist" here is better than "often correct".

Re: End boldface also ends background color

[Danek Duvall]

On Thu, Mar 24, 2016 at 10:26:33AM -0700, Bart Schaefer wrote:

> On Mar 24,  7:53am, Danek Duvall wrote:
> }
> } Worth at least pinging Thomas Dickey to add the capability to xterm
> 
> This would have to be a new termcap/terminfo library entry (or whatever
> stands in for them, maybe it's ncurses); it's not something missing from
> xterm.

You said that the expected escape sequence, which worked on gnome-terminal,
didn't work on xterm.  So it's both, AFAICT, which was what I was trying to
say.

> } Once you build zsh against the new version of
> } both, it would work, and you'd get historical behavior until then.
> 
> We'd still have to reprogram zle_refresh.c to know about and do the
> run-time test for the new capability, which would take years to make it
> into everyone's terminfo databases, etc.

Sure.  It may not be useful for the current problem, and something you'll
still want to work around, but unless there's a particularly good reason
why exit_bold_mode doesn't exist, I don't see why adding it wouldn't be
generally useful down the line.

I can ping Thomas about it, if he isn't already listening (I forget whether
he lurks here or not).

Danek

Re: End boldface also ends background color

[Stephane Chazelas]

2016-03-24 10:38:37 +0000, Peter Stephenson:
[...]
> Even in terminfo there doesn't seem to be exit_bold_mode, unlike
> standout, reverse and underline (though it looks like standout maps to
> reverse here).
[...]

See also:

https://groups.google.com/forum/#!msg/comp.unix.programmer/IPIB4VWAK8M/BfJM-CbNVzUJ

In short there's no guarantee that resetting one attribute will
not reset others.

A solution could be to keep track of what attributes are on and
set all attributes for each of the %b/%B... 

on %B (if so was previously on): \e[0;1;3m (bold + so) (tput sgr
1 0 0 0 1 0 0 0 0), on %b: \e[0;3m (so) (tput sgr 1 0 0 0 0 0 0
0 0).

-- 
Stephane

Re: End boldface also ends background color

[Bart Schaefer]

On Thu, Mar 24, 2016 at 11:20 AM, Stephane Chazelas
<stephane.chazelas@gmail.com> wrote:
>
> In short there's no guarantee that resetting one attribute will
> not reset others.

Perhaps what we should do is simply document that %B %U %S %F{} are
not guaranteed to nest.  For example:

print -P '%F{red} red %F{blue} blue %f oops, not red %f'

Then people know that they might need to write
%U%Bbold-underline%b%Uunderline%u if that's what they mean (but they
might not get the underline in bold-underline because %B might reset
it).

Or just use $fg et al. from colors; so what if it's a bit verbose?
How often are you re-typing it?

Re: End boldface also ends background color

[Mikael Magnusson]

On Thu, Mar 24, 2016 at 10:34 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Thu, Mar 24, 2016 at 11:20 AM, Stephane Chazelas
> <stephane.chazelas@gmail.com> wrote:
>>
>> In short there's no guarantee that resetting one attribute will
>> not reset others.
>
> Perhaps what we should do is simply document that %B %U %S %F{} are
> not guaranteed to nest.  For example:
>
> print -P '%F{red} red %F{blue} blue %f oops, not red %f'
>
> Then people know that they might need to write
> %U%Bbold-underline%b%Uunderline%u if that's what they mean (but they
> might not get the underline in bold-underline because %B might reset
> it).
>
> Or just use $fg et al. from colors; so what if it's a bit verbose?
> How often are you re-typing it?

Using $fg from colors doesn't buy you anything over using the builtin
prompt sequences though. They also do not have an un-bold sequence.

-- 
Mikael Magnusson

Re: End boldface also ends background color

[Bart Schaefer]

On Thu, Mar 24, 2016 at 3:43 PM, Mikael Magnusson <mikachu@gmail.com> wrote:
>
> Using $fg from colors doesn't buy you anything over using the builtin
> prompt sequences though. They also do not have an un-bold sequence.

Strictly speaking, $fg doesn't have a simple bold sequence either --
you have to use e.g. $fg_bold[red] and $fg_no_bold[red], so if you
stick to the intended model you do have un-bold.

There is $bold_color but it's only thrown in there because of the
comment about black and white being less than pure in some terminal
color palettes.  There aren't escape codes created by the colors
function for standout, underline, faint, etc. either, but we could add
e.g. $normal_color if that's the only issue.