hostname completion for ssh / scp

hostname completion for ssh / scp

[Fourhundred Thecat]

Hello,

when I want to complete hostnames for ssh:

  ssh <TAB>

it offers me only the hostnames listed in ~/.ssh/config and completely
ignores global ones defined in /etc/hosts

Is this intended behavior?
Why not use both files, like in bash_completion?

Tested on zsh 4.3.17

thanks,

Re: hostname completion for ssh / scp

[Bart Schaefer]

On Nov 3,  8:11pm, Fourhundred Thecat wrote:
} Subject: hostname completion for ssh / scp
}
}   ssh <TAB>
} 
} it offers me only the hostnames listed in ~/.ssh/config and completely
} ignores global ones defined in /etc/hosts
} 
} Tested on zsh 4.3.17

What does
    getent hosts
return on your system?  That's where _hosts gets the list of hosts.

Only if getent isn't present does it look directly at /etc/hosts,
and also at "ypcat hosts" if there is ypcat.

If getent isn't returning anything useful, you can replace it:

zstyle ':completion:*:hosts' command 'cat /etc/hosts'

Or explicitly list all the hosts yourself:

zstyle ':completion:*:hosts' hosts $(awk '/^[^#]/ {print $2}' /etc/hosts)

Note there have been several other fixes/improvements to ssh host-gathering
since zsh 5.0.

Re: hostname completion for ssh / scp

[Fourhundred Thecat]

> On 2016-11-05 05:42, Bart Schaefer wrote:
> On Nov 3,  8:11pm, Fourhundred Thecat wrote:
> } Subject: hostname completion for ssh / scp
> }
> }   ssh <TAB>
> } 
> } it offers me only the hostnames listed in ~/.ssh/config and completely
> } ignores global ones defined in /etc/hosts
> } 
> } Tested on zsh 4.3.17
> 
> What does
>     getent hosts
> return on your system?  That's where _hosts gets the list of hosts.

getent hosts only returns first two lines from /etc/hosts, and ignores
the rest:

  127.0.0.1       localhost
  127.0.1.1       laptop

I believe, getent hosts only returns hosts which have ip address
specified in /etc/hosts, such as:

  1.2.3.4	myserver1.com

> If getent isn't returning anything useful, you can replace it:
> 
> zstyle ':completion:*:hosts' command 'cat /etc/hosts'

I would like to keep the completion from ~/.ssh/config and add
completion from /etc/hosts.

I cannot just use

  zstyle ':completion:*:hosts' command 'cat /etc/hosts ~/.ssh/config'

because the syntax in ~/.ssh/config is more tricky (multiline host
definition), and I don't want options to be interpreted as hostnames.

> Or explicitly list all the hosts yourself:
> 
> zstyle ':completion:*:hosts' hosts $(awk '/^[^#]/ {print $2}' /etc/hosts)

same as above

> Note there have been several other fixes/improvements to ssh host-gathering
> since zsh 5.0.

can I simply copy Unix/_ssh from zsh 5.0. and use it on zsh 4.3.17 ?

Re: hostname completion for ssh / scp

[Bart Schaefer]

On Nov 5,  9:55am, Fourhundred Thecat wrote:
}
} I believe, getent hosts only returns hosts which have ip address
} specified in /etc/hosts

Well, yes.  The syntax of the /etc/hosts file is first an IP and then
a hostname.  Somebody will possibly correct me, but I don't recall
any valid use of /etc/hosts that has lines of host names without IPs.

} > If getent isn't returning anything useful, you can replace it:
} > 
} > zstyle ':completion:*:hosts' command 'cat /etc/hosts'
} 
} I would like to keep the completion from ~/.ssh/config and add
} completion from /etc/hosts.

This shouldn't change the behavior with respect to ~/.ssh/config
because that's read in _ssh_hosts rather than in _hosts where the
style above applies.

_hosts reads ~/.ssh/known_hosts whether or not the program named by
the command style returns anything.

However, ~/.ssh/config will be skipped if the users-hosts tag finds
any matching hosts, which is likely to happen if you are starting
from a blank word, because users-hosts calls _hosts which will look
in known_hosts.  If you start with a prefix on the line that matches
a host in ~/.ssh/config and does NOT match one in ~/.ssh/known_hosts,
you should get a completion.

} > Note there have been several other fixes/improvements to ssh host-gathering
} > since zsh 5.0.
} 
} can I simply copy Unix/_ssh from zsh 5.0. and use it on zsh 4.3.17 ?

I don't think so (you'd need at least _hosts as well) but in any case
it wouldn't change anything because even 5.0+ is not expecting you to
have just dumped a bunch of bare host names into /etc/hosts like that.

Re: hostname completion for ssh / scp

[Fourhundred Thecat]

> On 2016-11-05 18:24, Bart Schaefer wrote:
> Well, yes.  The syntax of the /etc/hosts file is first an IP and then
> a hostname.  Somebody will possibly correct me, but I don't recall
> any valid use of /etc/hosts that has lines of host names without IPs.

Not true. IP address is not mandatory in /etc/hosts. You can leave it
out, and only have hostnames. This is useful for host autocompletion.
You don't have to type a long hostname, such as:

  ssh long-or-complicated-host.my-subdomain.my-domain.com

instead, you just type

  ssh long<TAB>

> } > If getent isn't returning anything useful, you can replace it:
> } > 
> } > zstyle ':completion:*:hosts' command 'cat /etc/hosts'
> } 
> } I would like to keep the completion from ~/.ssh/config and add
> } completion from /etc/hosts.
> 
> This shouldn't change the behavior with respect to ~/.ssh/config
> because that's read in _ssh_hosts rather than in _hosts where the
> style above applies.
> _hosts reads ~/.ssh/known_hosts whether or not the program named by
> the command style returns anything.
> 
> However, ~/.ssh/config will be skipped if the users-hosts tag finds
> any matching hosts, which is likely to happen if you are starting
> from a blank word, because users-hosts calls _hosts which will look
> in known_hosts.  If you start with a prefix on the line that matches
> a host in ~/.ssh/config and does NOT match one in ~/.ssh/known_hosts,
> you should get a completion.

yes that is correct.
But this is a terrible feature. The completion jumps to first matching
hostname, completely ignoring other matching hostnames.

  ssh s<TAB>

will complete to

  ssh server1.mydomain.com

and completely disregard

  system1.mydomain.com
  system2.mydomain.com

> 
> } > Note there have been several other fixes/improvements to ssh host-gathering
> } > since zsh 5.0.
> } 
> } can I simply copy Unix/_ssh from zsh 5.0. and use it on zsh 4.3.17 ?
> 
> I don't think so (you'd need at least _hosts as well) but in any case
> it wouldn't change anything because even 5.0+ is not expecting you to
> have just dumped a bunch of bare host names into /etc/hosts like that.
> 

Re: hostname completion for ssh / scp

[Bart Schaefer]

On Nov 5,  7:54pm, Fourhundred Thecat wrote:
} Subject: Re: hostname completion for ssh / scp
}
} > On 2016-11-05 18:24, Bart Schaefer wrote:
} > Well, yes.  The syntax of the /etc/hosts file is first an IP and then
} > a hostname.  Somebody will possibly correct me, but I don't recall
} > any valid use of /etc/hosts that has lines of host names without IPs.
} 
} Not true. IP address is not mandatory in /etc/hosts.

/etc/hosts is a file read by the DNS resolver library to look up IPs
for hosts that either aren't found in DNS, or that locally need to
use a different IP than the one found by a network DNS lookup.  It
is not there for purposes of configuring completions in the shell,
and lines that don't have an IP address are useless for the intended
purpose of the file.

Read "man 5 hosts".  Bash might let you use it the way you say by way
of a non-strict parse of the contents, but that doesn't make it right.

} > However, ~/.ssh/config will be skipped if the users-hosts tag finds
} > any matching hosts
} 
} yes that is correct.
} But this is a terrible feature. The completion jumps to first matching
} hostname, completely ignoring other matching hostnames.

This goes way back to 2007, users/11333 and the thread leading up to it.

The intention is that if the user has the users-hosts style configured
for the my-accounts tag and gets a match from it, then ~/.ssh/config
should not be consulted.

However, because _combination is used by _ssh_hosts to also call the
completion for hosts, the users-hosts lookup can succeed even if the
users-hosts style is unset.  I don't think that was an expected effect.
However, I don't know the best way to address it.

I expected to be able to work around this by:

zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts 'return 1'

and indeed that bypasses the call to _hosts, but the return value from
_combination is not 1, so ~/.ssh/config is still not examined.  So
that's another bug to file.

Re: hostname completion for ssh / scp

[Daniel Shahaf]

Fourhundred Thecat wrote on Sat, Nov 05, 2016 at 19:54:29 +0100:
> > On 2016-11-05 18:24, Bart Schaefer wrote:
> > Well, yes.  The syntax of the /etc/hosts file is first an IP and then
> > a hostname.  Somebody will possibly correct me, but I don't recall
> > any valid use of /etc/hosts that has lines of host names without IPs.
> 
> Not true. IP address is not mandatory in /etc/hosts. You can leave it
> out, and only have hostnames.

The IP address is mandatory on both Linux and FreeBSD, according to
their respective hosts(5) man pages.

Does hosts(5) on your system make the IP address optional?  Or are you
simply saying that you put in /etc/hosts a line with no IP address, and
bash's hostname completion used hostnames from that line?

> This is useful for host autocompletion.
> You don't have to type a long hostname, such as:
> 
>   ssh long-or-complicated-host.my-subdomain.my-domain.com
> 
> instead, you just type
> 
>   ssh long<TAB>

Re: hostname completion for ssh / scp

[Fourhundred Thecat]

> On 2016-11-05 21:31, Daniel Shahaf wrote:
> Fourhundred Thecat wrote on Sat, Nov 05, 2016 at 19:54:29 +0100:
>> Not true. IP address is not mandatory in /etc/hosts. You can leave it
>> out, and only have hostnames.
> 
> The IP address is mandatory on both Linux and FreeBSD, according to
> their respective hosts(5) man pages.
> 
> Does hosts(5) on your system make the IP address optional?  Or are you
> simply saying that you put in /etc/hosts a line with no IP address, and
> bash's hostname completion used hostnames from that line?

no, man hosts on my system (Debian) does not explicitly say that IP is
optional. You are right. I am saying that if you omit the IP address,
everything works fine, and all applications understand that, and
bash-completion completes these hostnames for all networking applications.

Re: hostname completion for ssh / scp

[Fourhundred Thecat]

> On 2016-11-05 21:29, Bart Schaefer wrote:
> On Nov 5,  7:54pm, Fourhundred Thecat wrote:
> 
> /etc/hosts is a file read by the DNS resolver library to look up IPs
> for hosts that either aren't found in DNS, or that locally need to
> use a different IP than the one found by a network DNS lookup.  It
> is not there for purposes of configuring completions in the shell,
> and lines that don't have an IP address are useless for the intended
> purpose of the file.
> 
> Read "man 5 hosts".  Bash might let you use it the way you say by way
> of a non-strict parse of the contents, but that doesn't make it right.

I agree. That it works in bash, does not make it right.
It is a useful feature nevertheless.

Re: hostname completion for ssh / scp

[Bart Schaefer]

On Nov 5,  1:29pm, Bart Schaefer wrote:
}
} However, because _combination is used by _ssh_hosts to also call the
} completion for hosts, the users-hosts lookup can succeed even if the
} users-hosts style is unset.  I don't think that was an expected effect.
} However, I don't know the best way to address it.
} 
} I expected to be able to work around this by:
} 
} zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts 'return 1'

Here's a workaround that actually works:

zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts \
	'_hosts "$@"; trap false EXIT; return 1' 

Re: hostname completion for ssh / scp

[Fourhundred Thecat]

> On 2016-11-05 23:53, Bart Schaefer wrote:
>
> On Nov 5,  1:29pm, Bart Schaefer wrote:
> }
> } However, because _combination is used by _ssh_hosts to also call the
> } completion for hosts, the users-hosts lookup can succeed even if the
> } users-hosts style is unset.  I don't think that was an expected effect.
> } However, I don't know the best way to address it.
> } 
> } I expected to be able to work around this by:
> } 
> } zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts 'return 1'
> 
> Here's a workaround that actually works:
> 
> zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts \
> 	'_hosts "$@"; trap false EXIT; return 1' 

yes,

 ssh s<TAB>

no longer jumps to first matching completion, but offers other possible
matching completions. (server1.foo.com, server2.foo.com, ...).

But it's only using /etc/hosts, and not ~/.ssh/config.

Re: hostname completion for ssh / scp

[Bart Schaefer]

On Nov 6,  9:49am, Fourhundred Thecat wrote:
}
} > On 2016-11-05 23:53, Bart Schaefer wrote:
} > 
} > zstyle -e ':completion:*:(ssh|scp):*:my-accounts' users-hosts \
} > 	'_hosts "$@"; trap false EXIT; return 1' 
} 
} But it's only using /etc/hosts, and not ~/.ssh/config.

There has been a small change to ssh-config parsing in 5.something,
but nothing that would make it miss the file entirely.

Note that _ssh stores the hosts it finds in $_cache_hosts, so it may
not work to add the above style to a running zsh where you have tried
ssh host completions before.  "unset _cache_hosts" or restart zsh.

If it still seems not to be working, try:

    ssh s<c-x ?>

That is, control-x question-mark, which invokes _complete_debug.  This
will write a file in /tmp with a backtrace of the completion process.

If you then do:

    grep '\+_ssh_hosts' /tmp/zsh$$ssh? | less

You should be able to see ~/.ssh/config being parsed.  This will most
likely show you what's going on.