Re: Problem with cursor position after wrapped completion lists

Re: Problem with cursor position after wrapped completion lists

[Sven Wischnowsky]

Oliver Kiddle wrote:

> I tried it again and it seemed to work fine in the new xterms and
> aixterms which I created but after looking back in the aixterm where I'd
> had the problems, I realised what the cause was. There was one
> line in the list which was exactly 80 ($COLUMNS) characters wide.
> 
> xterm and aixterm seem to treat this differently: aixterm gives me a blank
> line after it whereas xterm doesn't. Zsh then places the cursor correctly
> only with the xterm.

Hm. I've had a look at this: the only solution I see is to make the
code check if the line it just printed is COLUMNS characters long
whenever it wants to print a newline. Since there are several ways how 
the different things are printed (display-lists, explanation strings, 
matches), fixing this can get quite ugly, so I better ask first: does
anybody see a better solution than to keep track of the number of
characters printed?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

RE: Problem with cursor position after wrapped completion lists

[Andrej Borsenkow]

>
> Hm. I've had a look at this: the only solution I see is to make the
> code check if the line it just printed is COLUMNS characters long
> whenever it wants to print a newline. Since there are several ways how
> the different things are printed (display-lists, explanation strings,
> matches), fixing this can get quite ugly, so I better ask first: does
> anybody see a better solution than to keep track of the number of
> characters printed?
>

No. There are two types of terminals. Some ttys wrap immediately after you
printed exactly COLUMNS lines. Some of them wrap first when COLUMNS+1th
character is printed. It means, that after you printed exactly COLUMNS of
characters you do not know, if your cursor is placed on current or next line. As
I have no idea if terminal description inludes such information, the only clean
way to solve it is to explicitly position cursor after exactly COLUMNS
characters were printed. Have you got a look at 4855? Among other things, it
makes Zsh to output '\r' in this case. Weird as it is, it works in both cases.
In the former case, additional character triggers new line; in the latter, we
are already on the new line so nothing changes. Note, that if you output
COLUMNS-1 or COLUMNS+1 characters, you won't get problem at all.

Note, that it only marginally relates to am attrubute. Both types of terminals I
have seen _did_ have am :-)

I believe, that fixing it may be hard but not ugly. The best would be to have
just one function that prints out all sort of output ... dunno, if this is
currently possible.

/andrej

Correction :-( RE: Problem with cursor position after wrapped completion lists

[Andrej Borsenkow]

                Have you got a look at 4855? Among other things, it
> makes Zsh to output '\r' in this case.

Sorry. I've meant '\n' of course :-( This automagically does the trick on all
three terminal types (no am, am with COLUMNS wrap and am with COLUMNS+1 wrap)

/andrej

4855 does not work in all cases

[Andrej Borsenkow]

>
>                 Have you got a look at 4855? Among other things, it
> > makes Zsh to output '\r' in this case.
>
> Sorry. I've meant '\n' of course :-( This automagically does the trick on all
> three terminal types (no am, am with COLUMNS wrap and am with COLUMNS+1 wrap)
>
> /andrej
>

I looks, like I have not used some terminals for really long time ... That is
even worse. I just found a terminal of type 2 (wrap on COLUMNS) that
unfortunately prints extra newline if prompt is exactly COLUMNS chanrters
-( SO, either we have here fourth type ... or nobody on this list never used
terminals of type 2 after 4855.

To test - just try something like

echo "----(80 characters)\n===" and see if you get extra newline. I believe,
this is exactly what happens with AIX term. It does not happen with all
terminals program I'm using here (xterm/dtterm) :-)

After some tests it appears, that the only actually clean way is to output

"<blank><backspace>"

in this case. Blank will force new line and backspace will move to the first
column again. Again, this is needed *only* if the last line is exactly COLUMNS
(better said, if no printable character follows).

Sigh ...

/andrej

Re: 4855 does not work in all cases

[Oliver Kiddle]

Andrej Borsenkow wrote:
> 
> >
> >                 Have you got a look at 4855? Among other things, it
> > > makes Zsh to output '\r' in this case.
> > Sorry. I've meant '\n' of course :-( This automagically does the trick on all

I was trying to work out what 4855 was doing and how \n was doing anything
'automagically' as I was getting nothing to work in all cases for my tests.

> I looks, like I have not used some terminals for really long time ... That is
> even worse. I just found a terminal of type 2 (wrap on COLUMNS) that
> unfortunately prints extra newline if prompt is exactly COLUMNS chanrters
> -( SO, either we have here fourth type ... or nobody on this list never used
> terminals of type 2 after 4855.

Well I'm using this type (aixterm) and was noticing occaisional mess with the
terminal output but it was only recently that I tried to repeat the problems
and find the cause. Before 4855, I was on a different project and wasn't using
AIX. I wish SGI would open the source to xwsh.

> echo "----(80 characters)\n===" and see if you get extra newline. I believe,
> this is exactly what happens with AIX term. It does not happen with all
> terminals program I'm using here (xterm/dtterm) :-)

Yup, I've been using:
echo "First line\n${(l:79:)}XLast line"
and sticking lots of strange stuff after the X with little success.

> After some tests it appears, that the only actually clean way is to output
> "<blank><backspace>"

I hadn't tried this but it is the first thing which actually works for both xterm
and aixterm. It doesn't work when you turn of line wrapping but I take it that
we can get this out of terminfo/termcap (am?)? This blank,backspace thing is a
bit of a hack so I think we should attempt to find a proper way of handling the
different cases - how does ncurses or something else handle this.

Looking at the man page for terminfo, I found this which might be relevant:

Boolean Capabilities

Variable            Cap Name  I. Code  Description
eat_newline_glitch  xenl      xn       Ignores new-line character after 80
                                       columns

I couldn't find a quick way of determining the value for this capability for
various terminals and I wouldn't be suprised if it isn't widely supported. I
hope by '80' they mean the terminal width.

I suggest that we first attempt to get a small program or shell script working
which we can all test on different terminals before Sven (or someone) launches
into a major fix of the zsh code. I also agree with Andrej that it would seem
wise to have one low-level routine or set of routines for handling terminal
output - only one part of the code would have to worry about different terminal
behaviour.

Oliver Kiddle

RE: 4855 does not work in all cases

[Andrej Borsenkow]

>
> I hadn't tried this but it is the first thing which actually works
> for both xterm
> and aixterm. It doesn't work when you turn of line wrapping but I take it that
> we can get this out of terminfo/termcap (am?)? This blank,backspace thing is a
> bit of a hack so I think we should attempt to find a proper way of
> handling the
> different cases - how does ncurses or something else handle this.
>
> Looking at the man page for terminfo, I found this which might be relevant:
>
> Boolean Capabilities
>
> Variable            Cap Name  I. Code  Description
> eat_newline_glitch  xenl      xn       Ignores new-line character after 80
>                                        columns
>

Yes, ncurses looks at the eat_newline_glitch and basically does the CR-NL in
this case (look at the comments to wrap_cursor() in ncurses/do_update.c - it is
really interesting).

Unfortunately, it is not defined for dtterm (at least on our system). It seems
to be defined for xterm and one more terminal type that do have the feature. But
this is the general problem - how can you be sure that terminfo/termcap reflect
the truth? Users are free to  modify settings on the fly :-(

So, the story looks someting like

if (!am || xenl)
  puts("\r\n");

Not sure, that "\r\n" looks much better than " \n", but this works without
auto-margin as well. Funnily enough, it will work even on my dtterm (that is
configured for am but with terminfo that does not list this :-)

/andrej

Re: 4855 does not work in all cases

[Oliver Kiddle]

Andrej Borsenkow wrote:
> 
> > Variable            Cap Name  I. Code  Description
> > eat_newline_glitch  xenl      xn       Ignores new-line character after 80
> >                                        columns
> Unfortunately, it is not defined for dtterm (at least on our system). It seems
> to be defined for xterm and one more terminal type that do have the feature. But
> this is the general problem - how can you be sure that terminfo/termcap reflect
> the truth? Users are free to modify settings on the fly :-(

I'm no terminfo expert so correct me if I'm wrong but I'm assuming that
this xenl can either be set to true or false or can be not defined? If
this is correct then we should be all right as long as we can make a
good assumption in the case of not defined (the xterm behaviour I
suppose). Is xenl correct in the case of aixterm? When you say 'users
are free to modify settings on the fly', does this include xenl or just
things like am which are on the various mouse menus? If users are
modifying xenl, then surely they deserve everything they get so I
wouldn't see that as a problem. If terminfo/termcap entries are
occasionally wrong it might be better to mention how to fix them in the
FAQ than to write a load of nasty hacks to the code.

> if (!am || xenl) puts("\r\n");
> Not sure, that "\r\n" looks much better than " \n", but this works without
> auto-margin as well. Funnily enough, it will work even on my dtterm (that is
> configured for am but with terminfo that does not list this :-)

What doesn't it work for - only incorrect terminfo/termcap?

Thanks

Oliver Kiddle

RE: 4855 does not work in all cases

[Andrej Borsenkow]

>
> Andrej Borsenkow wrote:
> >
> > > Variable            Cap Name  I. Code  Description
> > > eat_newline_glitch  xenl      xn       Ignores new-line character after 80
> > >                                        columns
> > Unfortunately, it is not defined for dtterm (at least on our
> system). It seems
> > to be defined for xterm and one more terminal type that do have the
> feature. But
> > this is the general problem - how can you be sure that
> terminfo/termcap reflect
> > the truth? Users are free to modify settings on the fly :-(
>
> I'm no terminfo expert so correct me if I'm wrong but I'm assuming that
> this xenl can either be set to true or false or can be not defined?

Sorry. Of course, it is boolean and can either be defined (== TRUE) or be not
defined ( == FALSE).


If
> this is correct then we should be all right as long as we can make a
> good assumption in the case of not defined (the xterm behaviour I
> suppose). Is xenl correct in the case of aixterm?

Exactly for my xterm it is set (defined) and that is correct. I cannot tell
anything about aixterm as I've never seen it .

                                                                          When
you say 'users
> are free to modify settings on the fly', does this include xenl or just
> things like am which are on the various mouse menus?

I meant am and friends. xenl is probably too obscure ...

If users are
> modifying xenl, then surely they deserve everything they get so I
> wouldn't see that as a problem. If terminfo/termcap entries are
> occasionally wrong it might be better to mention how to fix them in the
> FAQ than to write a load of nasty hacks to the code.
>

Agreed (if only system updates won't overwrite fixed entries :-)

> > if (!am || xenl) puts("\r\n");
> > Not sure, that "\r\n" looks much better than " \n", but this works without
> > auto-margin as well. Funnily enough, it will work even on my dtterm (that is
> > configured for am but with terminfo that does not list this :-)
>
> What doesn't it work for - only incorrect terminfo/termcap?
>

Yes. But in this case terminal must do am, do not do xenl and terminfo must have
both wrong. Too much I presume. In mu case terminfo has them both wrong,  my
dtterm does am - but it also does xenl :-)

But actually, as I discovered, the original problem was much simpler. Consider:

bor@itsrm2:~%> foo-list () {
function> compadd -- "axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx" "ayyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyy"
function> }
bor@itsrm2:~%> compdef foo-list foo-list
bor@itsrm2:~%> echo $$
13
bor@itsrm2:~%> foo-list a<TAB>
axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

ayyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

And when I truss it, I see:

bor@itsrm2:/tools/src/zsh-3.1.5-pws-25/Functions/Zle%> truss -w 10 -t write -p 1
3
write(10, " a", 1)                              = 1
    Received signal #18, SIGCLD, in sigsuspend() [caught]
      siginfo: SIGCLD CLD_EXITED pid=1664 uid=0 status=0x0001
write(10, "\r\n", 2)                            = 2
write(10, 0x00000000004E8950, 190)              = 190
   a x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
   x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
   x x x x x x x x x x x x x x x x  \n a y y y y y y y y y y y y y
   y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y
   y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y
   y y  1B [ 4 A1B [ 0 m1B [ 2 2 ; 2 7 m1B [ 2 4 m\r1B [ 1 5 C
write(10, 0x00000000004E8950, 13)               = 13
  1B [ 2 2 ; 2 7 m1B [ 1 0 C

Note the extra "  " at the end of every match that is listed. I have no idea,
where it comes from. Probably, without this extra " " nobody would even notice
this problem.


/andrej

Re: 4855 does not work in all cases

[Geoff Wing]

Andrej Borsenkow <Andrej.Borsenkow@mow.siemens.ru> typed:
:To test - just try something like
:echo "----(80 characters)\n===" and see if you get extra newline. I believe,
:this is exactly what happens with AIX term. It does not happen with all
:terminals program I'm using here (xterm/dtterm) :-)

For am (automargin) terminals: 
  a non-special character must be output at terminal width. 
For non-am terminals:
  a return sequence must be output at terminal width.
 
In each case, any cursor movement must be done after that as its behaviour
cannot be reliably determined (due to certain unmentioned terminal emulators
and not due to dumbterms which tend to have pretty standard behaviour).
I'm pretty sure zle_refresh.c violates this in some special cases (prompt
things I think) so I guess I better fix those soonish.  Printing outside of
zle_refresh.c I've got no idea about.  Sometimes I get the urge to look at
combining all the normal output, e.g. lists, into a more generic
zle_refresh.c  but don't worry, I've kept those urges in check.

Regards,
-- 
Geoff Wing : <gcw@pobox.com>     Work URL: http://www.primenet.com.au/
Rxvt Stuff : <gcw@rxvt.org>      Ego URL : http://pobox.com/~gcw/
Zsh Stuff  : <gcw@zsh.org>       Phone   : (Australia) 0413 431 874

Re: 4855 does not work in all cases

[Oliver Kiddle]

Andrej Borsenkow wrote:
> 
> Exactly for my xterm it is set (defined) and that is correct. I cannot tell
> anything about aixterm as I've never seen it .

I've finally worked out an easy way of working this out having not
managed to work it out from the termcap entries: the tcsh echotc command
is a lot better than zsh's with 'echotc xn' giving me a yes/no answer.
xn is unset in aixterm and set in xterm, so if we handle it, the word
wrapping will atleast then work for aixterm. It certainly all seems to
work well in tcsh. If I have time, I might have a go at rewriting some
of the tcsh echotc/telltc/settc builtins for zsh, probably in a module.

Oliver Kiddle

RE: Problem with cursor position after wrapped completion lists

[Andrej Borsenkow]

>
> Yes, I had a look at 4855 and the solution was clear, of course, but
> we can't simply use one function for all cases because we don't want
> that: explanation strings have to handle %-escapes, display-lists
> should be printed literally, and matches should be printed in a way
> that is called (in the code) `nice' (as in `nicezputs()').
>

Ahem, I meant "one function that handles low level interaction with physical
display". All others should (ideally) simply update virtual screen, and then
call one single "refresh", that finally paints it.

Just midsummer dreams :-)

/andrej

RE: Problem with cursor position after wrapped completion lists

[Sven Wischnowsky]

Andrej Borsenkow wrote:

> I believe, that fixing it may be hard but not ugly. The best would be to have
> just one function that prints out all sort of output ... dunno, if this is
> currently possible.

Yes, I had a look at 4855 and the solution was clear, of course, but
we can't simply use one function for all cases because we don't want
that: explanation strings have to handle %-escapes, display-lists
should be printed literally, and matches should be printed in a way
that is called (in the code) `nice' (as in `nicezputs()').

Ok, so I'll have to bite the bullet...

Bye
 Sven

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

RE: Problem with cursor position after wrapped completion lists

[Andrej Borsenkow]

>
> I tried it again and it seemed to work fine in the new xterms and
> aixterms which I created but after looking back in the aixterm where I'd
> had the problems, I realised what the cause was. There was one
> line in the list which was exactly 80 ($COLUMNS) characters wide.
>

Back in the past there was exactly this problem with Zle - if displayed line was
exactly screen-width long, output was wrong. It was fixed by Geoff in a  patch
4855 (I just looked up in archive).

Sven, could you check if something from this patch is aplicable in this case?

>
> Peter Stephenson wrote:
> > By the way, it's high time I pointed out that sending me a copy of a
> > message going to any of the zsh lists is futile as they get sorted into the
> > same folder and I just have to delete one.
>

Well, I am sorry for this as well ... but you see, to send _reply_ to
zsh-workers that is in To: header I have to hit "reply-to-all" - there is no
other possibility


And exactly for this I'm using procmail with formail filter that eliminates
duplicates (as long, as message-id is preserved) - so I get exactly one copy of
all duplicates.

/andrej

Re: Problem with cursor position after wrapped completion lists

[Oliver Kiddle]

Sven Wischnowsky wrote:
> 
> Err, this works for me -- of course the completion code takes care to
> count the lines (including wrapping) correctly. However, this will

I tried it again and it seemed to work fine in the new xterms and
aixterms which I created but after looking back in the aixterm where I'd
had the problems, I realised what the cause was. There was one
line in the list which was exactly 80 ($COLUMNS) characters wide.

xterm and aixterm seem to treat this differently: aixterm gives me a blank
line after it whereas xterm doesn't. Zsh then places the cursor correctly
only with the xterm.

Another thing which I've noticed in this process is that if my prompt is 79
($COLUMNS-1) characters wide, and I use xterm, the cursor initially appears
over the last character of my prompt and when I type, the text wraps back
over the beginning of the prompt.

> only work if the COLUMNS parameter is correctly set (and the terminal
> doesn't do automatic word-wrapping like your mailer ;-).

Sorry about that wrapped mail, I'll attempt to avoid it next time I send a patch.

Peter Stephenson wrote:
> By the way, it's high time I pointed out that sending me a copy of a
> message going to any of the zsh lists is futile as they get sorted into the
> same folder and I just have to delete one.

This is because my mailer includes both the sender and the mailing list (CC'd)
in the receipients when I press reply. I'll try to remember to cut your
address out in future.

Oliver Kiddle

Re: Problem with cursor position after wrapped completion lists

[Sven Wischnowsky]

Oliver Kiddle wrote:

> The completion lists (as generated from the -Y option) don't work very
> well when lines wrap at the end of the line:
> 
> cd ~+<tab>
> 
> If the list then contains lots of stupidly long pathnames some of which
> wrap at the end of the terminal. For example:

Err, this works for me -- of course the completion code takes care to
count the lines (including wrapping) correctly. However, this will
only work if the COLUMNS parameter is correctly set (and the terminal
doesn't do automatic word-wrapping like your mailer ;-).

Are you sure that your COLUMNS information is correct?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Problem with cursor position after wrapped completion lists

[Oliver Kiddle]

The completion lists (as generated from the -Y option) don't work very
well when lines wrap at the end of the line:

cd ~+<tab>

If the list then contains lots of stupidly long pathnames some of which
wrap at the end of the terminal. For example:
0      
/home/ccm/ccmlib/ccmlib_d_log_file_libraries.ss/b3_stubs.wrk/ccm_test_utilities
1      
/home/native_rs6000/test/ccmcda/ccmlm0_log_manager.ss/okiddle.wrk/ccmlm0
etc...

The cursor is then placed a couple of lines below where it should be
because the wrapping has caused it to be lower than zsh thinks it is.

I suppose the only way of fixing this is by controlling the wrapping
directly from zsh unless there is a way of knowing how the terminal is
dealing with lines wider than the terminal (wrapping or truncating
them).

Note that I am not to blame for the directory structure we have here
and that I could solve the problem for myself by modifying _tilde to
compact the directory's names to use references to various associative
arrays and variables which I have: so don't bother suggesting that I
use a few 'hash -d's or shorter directory names.

Oliver Kiddle