[TUHS] Termcap vs terminfo

[TUHS] Termcap vs terminfo

[David Barto]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

<much discussion about quadratic search removed>

All I remember (and still support to this day) is that I’ve got a TERMCAP=‘string’ in my login scripts to set termcap to the specific terminal I’m logging in with.

Long ago this made things much faster. Today I think that it is just a holdover that I’m not changing due to inertia, rather than any real need for it.

	David
— 
	David Barto
	david at kdbarto.org

[TUHS] Termcap vs terminfo

[Lyndon Nerenberg]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]

On Jan 10, 2015, at 2:15 PM, David Barto <david at kdbarto.org> wrote:

> All I remember (and still support to this day) is that I’ve got a TERMCAP=‘string’ in my login scripts to set termcap to the specific terminal I’m logging in with.
> 
> Long ago this made things much faster. Today I think that it is just a holdover that I’m not changing due to inertia, rather than any real need for it.

There is still a need for this.

Most modern curses capability entries for 'xterm' and friends use the memory buffer windowing capability (a term I made up) such that when you - say - run less to display a file, it switches to a dedicated region in the terminal memory buffer while printing its output, then restores the buffer to back where you were to begin with when you exit the pager.

This drives me insane!  When I 'man foo' and find the relevant bits in the document, when I quit out of the pager I want those bits to stay on the screen so I can refer to them, dammit!  There are two shortcuts to this, both involving custom termcap/terminfo entries.

In the termcap case, you can define a $TERMCAP capability that describes an 'xterm' without the memory buffer hopping.  In the terminfo case, you tic(1) your custom 'xterm' definition into $HOME/.terminfo/...

These days - naturally - everyone knows the universe exists inside an ANSI terminal window, so who gives a fsck about term{cap.info}?  Well, I do, for this very reason.  A pox on everyone who has not 'setenv TERM aaa-48-s' !!!

--lyndon

P.S.  Terminfo entry attached for your enjoyment. (It's a text/plain. I have no idea what the hell Apple's Mail.app will do with it.)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: xterm.terminfo
Type: application/octet-stream
Size: 1339 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150110/582ebbd1/attachment.obj>
-------------- next part --------------


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150110/582ebbd1/attachment.sig>

[TUHS] Termcap vs terminfo

[Lyndon Nerenberg]

On Jan 10, 2015, at 2:43 PM, Lyndon Nerenberg <lyndon at orthanc.ca> wrote:

> Most modern curses capability entries for 'xterm' and friends use the memory buffer windowing capability (a term I made up)

Umm ... I meant 'just made up' as in "I can't think of a better name this second."  I'm pretty sure somebody else invented memory buffers and sliding pointers.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150110/b0b8aa7b/attachment.sig>

[TUHS] Termcap vs terminfo

[Christian Neukirchen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

Lyndon Nerenberg <lyndon at orthanc.ca> writes:

> On Jan 10, 2015, at 2:15 PM, David Barto <david at kdbarto.org> wrote:
>
>> All I remember (and still support to this day) is that I’ve got a
>> TERMCAP=‘string’ in my login scripts to set termcap to the specific
>> terminal I’m logging in with.
>> 
>> Long ago this made things much faster. Today I think that it is just
>> a holdover that I’m not changing due to inertia, rather than any
>> real need for it.
>
> There is still a need for this.
>
> Most modern curses capability entries for 'xterm' and friends use the
> memory buffer windowing capability (a term I made up) such that when
> you - say - run less to display a file, it switches to a dedicated
> region in the terminal memory buffer while printing its output, then
> restores the buffer to back where you were to begin with when you exit
> the pager.
>
> This drives me insane!  When I 'man foo' and find the relevant bits in
> the document, when I quit out of the pager I want those bits to stay
> on the screen so I can refer to them, dammit!  There are two shortcuts
> to this, both involving custom termcap/terminfo entries.

Or just use: xterm -xrm '*titeInhibit: true'
Just for less: export LESS="-X"
Just for vim: set t_ti= t_te=

-- 
Christian Neukirchen  <chneukirchen at gmail.com>  http://chneukirchen.org

[TUHS] Termcap vs terminfo

[Lyndon Nerenberg]

On Jan 10, 2015, at 3:02 PM, Christian Neukirchen <chneukirchen at gmail.com> wrote:

> Or just use: xterm -xrm '*titeInhibit: true'
> Just for less: export LESS="-X"
> Just for vim: set t_ti= t_te=

Please provide the list of workarounds for all other curses consumers used worldwide and forever.

Wouldn't it be nice if I could just do this in one place?  Oh, wait ...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150110/525f7f9a/attachment.sig>

[TUHS] Termcap vs terminfo

[Dave Horsfall]

On Sat, 10 Jan 2015, Lyndon Nerenberg wrote:

> This drives me insane!  When I 'man foo' and find the relevant bits in 
> the document, when I quit out of the pager I want those bits to stay on 
> the screen so I can refer to them, dammit!  There are two shortcuts to 
> this, both involving custom termcap/terminfo entries.

I'm glad I'm not the only one annoyed by this "feature".

-- 
Dave Horsfall DTM (VK2KFU)  "Bliss is a MacBook with a FreeBSD server."
http://www.horsfall.org/spam.html (and check the home page whilst you're there)

[TUHS] Termcap vs terminfo

[Diomidis Spinellis]

On 11/01/2015 04:06, Dave Horsfall wrote:
> On Sat, 10 Jan 2015, Lyndon Nerenberg wrote:
>
>> This drives me insane!  When I 'man foo' and find the relevant bits in
>> the document, when I quit out of the pager I want those bits to stay on
>> the screen so I can refer to them, dammit!  There are two shortcuts to
>> this, both involving custom termcap/terminfo entries.
>
> I'm glad I'm not the only one annoyed by this "feature".

I agree it's maddening!  This is what I have on my .bashrc file to avoid 
this behavior.

# Don't restore screen, quit at EOF, more-like prompt, pass color 
control chars
export LESS=-XEmR

This problem, and the desire to use the vi key bindings instead of the 
Emacs ones for command-line editing, are the reason I invested effort to 
place my login configuration under Git control, and replicate it on the 
tens of hosts I find myself logged in over a working day.  Now a simple 
"git clone" restores sanity to my working environment.

[TUHS] Termcap vs terminfo

[Christian Neukirchen]

Lyndon Nerenberg <lyndon at orthanc.ca> writes:

> On Jan 10, 2015, at 3:02 PM, Christian Neukirchen <chneukirchen at gmail.com> wrote:
>
>> Or just use: xterm -xrm '*titeInhibit: true'
>> Just for less: export LESS="-X"
>> Just for vim: set t_ti= t_te=
>
> Please provide the list of workarounds for all other curses consumers
> used worldwide and forever.
>
> Wouldn't it be nice if I could just do this in one place?  Oh, wait ...

I did: xterm -xrm '*titeInhibit: true'

-- 
Christian Neukirchen  <chneukirchen at gmail.com>  http://chneukirchen.org

[TUHS] Termcap vs terminfo

[Clem Cole]

+1

On Sat, Jan 10, 2015 at 9:06 PM, Dave Horsfall <dave at horsfall.org> wrote:

> On Sat, 10 Jan 2015, Lyndon Nerenberg wrote:
>
> > This drives me insane!  When I 'man foo' and find the relevant bits in
> > the document, when I quit out of the pager I want those bits to stay on
> > the screen so I can refer to them, dammit!  There are two shortcuts to
> > this, both involving custom termcap/terminfo entries.
>
> I'm glad I'm not the only one annoyed by this "feature".
>
> --
> Dave Horsfall DTM (VK2KFU)  "Bliss is a MacBook with a FreeBSD server."
> http://www.horsfall.org/spam.html (and check the home page whilst you're
> there)
> _______________________________________________
> TUHS mailing list
> TUHS at minnie.tuhs.org
> https://minnie.tuhs.org/mailman/listinfo/tuhs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150112/03579331/attachment.html>

[TUHS] Disable "less" screen swapping was: Re: Termcap vs terminfo

[Russell Rezaian]

On Sat, Jan 10, 2015 at 9:06 PM, Dave Horsfall <dave at horsfall.org 
<mailto:dave at horsfall.org>> wrote:
>
>     On Sat, 10 Jan 2015, Lyndon Nerenberg wrote:
>
>     > This drives me insane!  When I 'man foo' and find the relevant
>     bits in
>     > the document, when I quit out of the pager I want those bits to
>     stay on
>     > the screen so I can refer to them, dammit!  There are two
>     shortcuts to
>     > this, both involving custom termcap/terminfo entries.
>
>     I'm glad I'm not the only one annoyed by this "feature".
>
I am also not happy with this behavior.

There is at least one "fix" I have found for this particular annoyance.

A little background: this behavior uses a pair of termcap entries called 
"ti" and "te" which, on most recent xterm implementations, switch 
between alternate screens.  Many programs on startup switch to the 
alternate screen to keep their displays separate from the normal command 
scroll buffer.

There is an xterm resource "titeInhibit" which, if set to true, will 
disable the remarkably annoying mode switch that is used by less (and 
also any other application that tries to use the screen swap).

Given this started out as a termcap thread, hopefully this will be 
appropriate!

This complete disable of the screen swap may not be what everyone is 
looking for, but it certainly was what I wanted.
--
Russell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150112/5a2613a0/attachment.html>

[TUHS] termcap vs terminfo

[Mary Ann Horton]

Quoting Dan Cross <crossd at gmail.com>:

> On Tue, Jan 6, 2015 at 12:33 PM, Johnny Billquist <bqt at update.uu.se> wrote:
>
>> On 2015-01-06 17:56, Dan Cross wrote:
>>>
>>> I believe that Mary Ann is referring to repeatedly looking up
>>> (presumably different) elements in the entry.  Assuming that e.g. `vi`
>>> looks up O(n) elements, where $n$ is the number of elements, doing a
>>> linear scan for each, you'd end up with quadratic behavior.
>>>
>>
>> Assuming that you'd look up all the elements of the termcap entry at
>> startup, and did each one from scratch, yes.
>
>
> Yes.  Isn't that exactly what Mary Ann said was happening?  :-)

Yes

> But that would beg the question, why is vi doing a repeated scan of the
>> terminal entry at startup, if not to find all the capabilities and store
>> this somewhere? And if doing a look for all of them, why not scan the
>> string from start to finish and store the information as it is found? At
>> which point we move from quadratic to linear time.
>>
>
> I don't think she said it did things intelligently, just that that's how it
> did things.  :-)
>
> But now we're getting into the innards of vi, which I never looked at
> anyway, and I guess is less relevant in this thread anyway.

vi does indeed look up all the various capabilities it will need,
once, when it starts up.  It uses the documented interface, which
is tgetent followed by tgetstr/tgetnum/tgetflag for each capability.
tgetent did a linear search.

>> The short of it (from what I got out of it) is that the move from termcap
>> to terminfo was mostly motivated by attribute name changing away from fixed
>> 2 character names.
>>
>> A secondary motivation would be performance, but I don't really buy that
>> one. Since we only moved to terminfo on systems with plenty of memory,
>> performance of termcap could easily be on par anyway.
>>
>
> I tend to agree with you and I'll go one further: it seems that frequently
> we tend to identify a problem and then go to 11 with the "solution."  I can
> buy that termcap performance was an issue; I don't know that going directly
> to hashed terminfo files was the optimal solution.  A dbm file of termcap
> data and a hash table in whatever library parsed termcap would go a long
> way towards fixing the performance issues.  Did termcap have to be
> discarded just to add longer names?  I kind of tend to doubt it, but I
> wasn't there and don't know what the design criteria were, so my
> very-much-after-the-fact second guessing is just that.

It's been 30+ years, so the memory is a little fuzzy.  But as I recall,
I did measure the performance and that's how I discovered that the
quadratic algorithm was causing a big performance hit on the hardware
available at the time (I think I was on a VAX 11/750, this would have
been about 1982.)

I was making several improvements at the same time.  The biggest one
was rewriting curses to improve the screen update optimization, so it
would use insert/delete line/character on terminals supporting it.
Cleaning up the mess termcap had become (the format had become horrible
to update, and I was spending a lot of time making updates with all
the new terminals coming out) and improving startup time (curses also
had to read in a lot of attributes) were part of an overall cleanup.
IIRC, there may also have been some restrictions on parameters to string
capabilities that needed to be generalized.

Hashing could have been done differently, using DBM or some other method.
In fact, I'd used DBM to hash /etc/aliases in delivermail years earlier
(I have an amusing story about the worlds slowest email I'll tell some
other time) but at the time, it seemed best to break with termcap
and go with a cleaner format.

[TUHS] termcap vs terminfo

[Dan Cross]

On Tue, Jan 6, 2015 at 12:33 PM, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2015-01-06 17:56, Dan Cross wrote:
>>
>> I believe that Mary Ann is referring to repeatedly looking up
>> (presumably different) elements in the entry.  Assuming that e.g. `vi`
>> looks up O(n) elements, where $n$ is the number of elements, doing a
>> linear scan for each, you'd end up with quadratic behavior.
>>
>
> Assuming that you'd look up all the elements of the termcap entry at
> startup, and did each one from scratch, yes.


Yes.  Isn't that exactly what Mary Ann said was happening?  :-)

 Hashing, or storing in some kind of balanced-tree like structure or
>> something, would of course help but would also necessitate doing a copy
>> and would entail some additional memory inefficiency.
>>
>
> Hashing would indeed cause some extra memory, but not necessarily any
> copying.
>

I fail to see how you can avoid copying the data out of the environment
vector (unless you intend to either (a) turn the env var into a hash table,
or (b) store pointers to the datum in the env var, but you'd need to encode
their length somehow.  I don't think environment variables can contain
embedded NULs, can they?).

But that would beg the question, why is vi doing a repeated scan of the
> terminal entry at startup, if not to find all the capabilities and store
> this somewhere? And if doing a look for all of them, why not scan the
> string from start to finish and store the information as it is found? At
> which point we move from quadratic to linear time.
>

I don't think she said it did things intelligently, just that that's how it
did things.  :-)

But now we're getting into the innards of vi, which I never looked at
> anyway, and I guess is less relevant in this thread anyway.
>
> The short of it (from what I got out of it) is that the move from termcap
> to terminfo was mostly motivated by attribute name changing away from fixed
> 2 character names.
>
> A secondary motivation would be performance, but I don't really buy that
> one. Since we only moved to terminfo on systems with plenty of memory,
> performance of termcap could easily be on par anyway.
>

I tend to agree with you and I'll go one further: it seems that frequently
we tend to identify a problem and then go to 11 with the "solution."  I can
buy that termcap performance was an issue; I don't know that going directly
to hashed terminfo files was the optimal solution.  A dbm file of termcap
data and a hash table in whatever library parsed termcap would go a long
way towards fixing the performance issues.  Did termcap have to be
discarded just to add longer names?  I kind of tend to doubt it, but I
wasn't there and don't know what the design criteria were, so my
very-much-after-the-fact second guessing is just that.

Thanks for the insights.
>
>         Johnny
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150106/bd279a0f/attachment.html>

[TUHS] termcap vs terminfo

[Johnny Billquist]

On 2015-01-06 17:56, Dan Cross wrote:
> On Tue, Jan 6, 2015 at 11:51 AM, Johnny Billquist <bqt at update.uu.se
> <mailto:bqt at update.uu.se>> wrote:
>
>     On 2015-01-06 17:32, Mary Ann Horton<mah at mhorton.net
>     <mailto:mah at mhorton.net>> wrote:
>
>         Even with TERMCAP in the environment, there's still that quadratic
>         algorithm every time vi starts up.
>
>
>     I must be stupid or something. What quadratic algorithm?
>     vi gets the "correct" terminal database entry directly from the
>     environment. Admittedly, getting any variable out of the environment
>     means a linear search of the environment, but that's about it.
>
>     What am I missing? And once you have that, any operation still means
>     either searching through the terminal definition for the right
>     function, which in itself is also linear, unless you hash that up in
>     your program. But I fail to see where the quadratic behavior comes in.
>
>
> I believe that Mary Ann is referring to repeatedly looking up
> (presumably different) elements in the entry.  Assuming that e.g. `vi`
> looks up O(n) elements, where $n$ is the number of elements, doing a
> linear scan for each, you'd end up with quadratic behavior.

Assuming that you'd look up all the elements of the termcap entry at 
startup, and did each one from scratch, yes.

> Hashing, or storing in some kind of balanced-tree like structure or
> something, would of course help but would also necessitate doing a copy
> and would entail some additional memory inefficiency.

Hashing would indeed cause some extra memory, but not necessarily any 
copying.
But that would beg the question, why is vi doing a repeated scan of the 
terminal entry at startup, if not to find all the capabilities and store 
this somewhere? And if doing a look for all of them, why not scan the 
string from start to finish and store the information as it is found? At 
which point we move from quadratic to linear time.

But now we're getting into the innards of vi, which I never looked at 
anyway, and I guess is less relevant in this thread anyway.

The short of it (from what I got out of it) is that the move from 
termcap to terminfo was mostly motivated by attribute name changing away 
from fixed 2 character names.

A secondary motivation would be performance, but I don't really buy that 
one. Since we only moved to terminfo on systems with plenty of memory, 
performance of termcap could easily be on par anyway.

Thanks for the insights.

	Johnny

[TUHS] termcap vs terminfo

[arnold]

Mary Ann Horton <mah at mhorton.net> wrote:

> Even with TERMCAP in the environment, there's still that quadratic 
> algorithm every time vi starts up.

Please remind me which one you're talking about? If TERMCAP is
in the environment there's no need to hit /etc/termcap at all.

Or are you talking about something else?

Thanks,

Arnold

[TUHS] termcap vs terminfo

[Dan Cross]

On Tue, Jan 6, 2015 at 11:51 AM, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2015-01-06 17:32, Mary Ann Horton<mah at mhorton.net> wrote:
>
>> Even with TERMCAP in the environment, there's still that quadratic
>> algorithm every time vi starts up.
>>
>
> I must be stupid or something. What quadratic algorithm?
> vi gets the "correct" terminal database entry directly from the
> environment. Admittedly, getting any variable out of the environment means
> a linear search of the environment, but that's about it.
>
> What am I missing? And once you have that, any operation still means
> either searching through the terminal definition for the right function,
> which in itself is also linear, unless you hash that up in your program.
> But I fail to see where the quadratic behavior comes in.


I believe that Mary Ann is referring to repeatedly looking up (presumably
different) elements in the entry.  Assuming that e.g. `vi` looks up O(n)
elements, where $n$ is the number of elements, doing a linear scan for
each, you'd end up with quadratic behavior.

Hashing, or storing in some kind of balanced-tree like structure or
something, would of course help but would also necessitate doing a copy and
would entail some additional memory inefficiency.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20150106/113d08c2/attachment.html>

[TUHS] termcap vs terminfo

[Johnny Billquist]

On 2015-01-06 17:32, Mary Ann Horton<mah at mhorton.net> wrote:
>
> On 01/06/2015 04:22 AM,arnold at skeeve.com  wrote:
>>> >>Peter Jeremy scripsit:
>>>> >>>But you pay for the size of $TERMCAP in every process you run.
>> >John Cowan<cowan at mercury.ccil.org>  wrote:
>>> >>A single termcap line doesn't cost that much, less than a KB in most cases.
>> >In 1981 terms, this has more weight. On a non-split I/D PDP-11 you only
>> >have 32KB to start with.  (The discussion a few weeks ago about cutting
>> >yacc down to size comes to mind...)
>> >
>> >On a Vax with 2 Meg of memory, 512 bytes is a whole page, and it might
>> >even be paged out, and BSD on the vax didn't have copy-on-write.
>> >
>> >ISTR that the /etc/termcap file had a comment saying something like
>> >"you should move the entries needed at your site to the top of this file."
>> >Or am I imagining it?:-)
>> >
>> >In short - today, sure, no problem - back then, carrying around a large
>> >environment made more of a difference.
>> >
>> >Thanks,
>> >
>> >Arnold
> Even with TERMCAP in the environment, there's still that quadratic
> algorithm every time vi starts up.

I must be stupid or something. What quadratic algorithm?
vi gets the "correct" terminal database entry directly from the 
environment. Admittedly, getting any variable out of the environment 
means a linear search of the environment, but that's about it.

What am I missing? And once you have that, any operation still means 
either searching through the terminal definition for the right function, 
which in itself is also linear, unless you hash that up in your program. 
But I fail to see where the quadratic behavior comes in.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] termcap vs terminfo

[Mary Ann Horton]

On 01/06/2015 04:22 AM, arnold at skeeve.com wrote:
>> Peter Jeremy scripsit:
>>> But you pay for the size of $TERMCAP in every process you run.
> John Cowan <cowan at mercury.ccil.org> wrote:
>> A single termcap line doesn't cost that much, less than a KB in most cases.
> In 1981 terms, this has more weight. On a non-split I/D PDP-11 you only
> have 32KB to start with.  (The discussion a few weeks ago about cutting
> yacc down to size comes to mind...)
>
> On a Vax with 2 Meg of memory, 512 bytes is a whole page, and it might
> even be paged out, and BSD on the vax didn't have copy-on-write.
>
> ISTR that the /etc/termcap file had a comment saying something like
> "you should move the entries needed at your site to the top of this file."
> Or am I imagining it? :-)
>
> In short - today, sure, no problem - back then, carrying around a large
> environment made more of a difference.
>
> Thanks,
>
> Arnold
> _______________________________________________
> TUHS mailing list
> TUHS at minnie.tuhs.org
> https://minnie.tuhs.org/mailman/listinfo/tuhs
Even with TERMCAP in the environment, there's still that quadratic 
algorithm every time vi starts up.