apt autocompletion issue

apt autocompletion issue

[Hauser, Felix (MRT)]

Dear ZSH-community,

just discovered the Z-shell and so far it really makes my life a little 
bit easier! The only issue I have with it is the completion of options 
for the apt command. So e.g.:

apt list --u<tab>

won't autocomplete to

apt list --upgradeable

which is easily done in the normal bash file.
Since in my workflow I often have to use apt, this feature is kinda 
important to me.

Thankful for any help!
Felix


Ubuntu 18.04
zsh 5.4.2 (x86_64-ubuntu-linux-gnu)
oh-my-zsh script from some days ago

Re: apt autocompletion issue

[Oliver Kiddle]

On 26 Jul, "Hauser, Felix (MRT)" wrote:
> just discovered the Z-shell and so far it really makes my life a little 
> bit easier! The only issue I have with it is the completion of options 
> for the apt command. So e.g.:
>
> apt list --u<tab>
>
> won't autocomplete to
>
> apt list --upgradeable
>
> which is easily done in the normal bash file.
> Since in my workflow I often have to use apt, this feature is kinda 
> important to me.

Thanks for the report. I'm afraid the Debian specific completions aren't
as well maintained as they once were.

It's not clear from the man page whether the three options can be
combined with the glob(7) patterns for matching packages (as per the
patch below) or whether they need to be standalone.

The option should probably modify whether it is calling _deb_packages
with "avail", "installed", "uninstalled" or some other package set. If
you let me know what the relevant package set argument should be for
each option, then it is easy to do that.

Oliver

diff --git a/Completion/Debian/Command/_apt b/Completion/Debian/Command/_apt
index 4486aa26d..d36d5240a 100644
--- a/Completion/Debian/Command/_apt
+++ b/Completion/Debian/Command/_apt
@@ -399,7 +399,10 @@ _apt-cmd () {
     -o,--option:arbitem \
     -t,--target-release:release \
     -- \
-    /$'list\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
+    /$'list\0'/ \( \
+      \( // %-% /$'--[^\0]#\0'/ ':options:option:(--installed --upgradable --all-versions)' \| \) \
+      /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \
+    \) \| \
     /$'search\0'/ /$'[^\0]#\0'/ ':strings:pattern:' \| \
     /$'showsrc\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
     /$'show\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \

Re: apt autocompletion issue

[Hauser, Felix (MRT)]

On 30.07.19 01:53, Oliver Kiddle wrote:
> On 26 Jul, "Hauser, Felix (MRT)" wrote:
>> just discovered the Z-shell and so far it really makes my life a little
>> bit easier! The only issue I have with it is the completion of options
>> for the apt command. So e.g.:
>>
>> apt list --u<tab>
>>
>> won't autocomplete to
>>
>> apt list --upgradeable
>>
>> which is easily done in the normal bash file.
>> Since in my workflow I often have to use apt, this feature is kinda
>> important to me.
> 
> Thanks for the report. I'm afraid the Debian specific completions aren't
> as well maintained as they once were.
> 
> It's not clear from the man page whether the three options can be
> combined with the glob(7) patterns for matching packages (as per the
> patch below) or whether they need to be standalone.
> 
> The option should probably modify whether it is calling _deb_packages
> with "avail", "installed", "uninstalled" or some other package set. If
> you let me know what the relevant package set argument should be for
> each option, then it is easy to do that.
> 
> Oliver
> 
> diff --git a/Completion/Debian/Command/_apt b/Completion/Debian/Command/_apt
> index 4486aa26d..d36d5240a 100644
> --- a/Completion/Debian/Command/_apt
> +++ b/Completion/Debian/Command/_apt
> @@ -399,7 +399,10 @@ _apt-cmd () {
>       -o,--option:arbitem \
>       -t,--target-release:release \
>       -- \
> -    /$'list\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
> +    /$'list\0'/ \( \
> +      \( // %-% /$'--[^\0]#\0'/ ':options:option:(--installed --upgradable --all-versions)' \| \) \
> +      /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \
> +    \) \| \
>       /$'search\0'/ /$'[^\0]#\0'/ ':strings:pattern:' \| \
>       /$'showsrc\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
>       /$'show\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
> 


Well, I guess the Debian package sets should be:
--installed "installed"
--all-versions "avail"
--upgradeable Do not know the option for this one. Result should be a 
list of all packages that can be updated by "apt upgrade".

But the problem is not only with "apt list" but with all apt commands.
The apt package brings its own detailed completion file (not via the 
bash-completions package):
https://salsa.debian.org/apt-team/apt/blob/master/completions/bash/apt
This is the one used by bash and is located at 
"/usr/share/bash-completion/completions/apt" on my system.
Maybe zsh can simply access the bash completion file and ignore its 
own?? Not sure if this can be done ...

I hesitated replacing "/usr/share/zsh/functions/Completion/Debian/_apt" 
with the bash file, because in the zsh completion file there are also 
the instructions for apt-get etc.

Felix

Re: apt autocompletion issue

[Daniel Shahaf]

Hauser, Felix (MRT) wrote on Tue, 30 Jul 2019 11:50 +00:00:
> --upgradeable Do not know the option for this one. Result should be a 
> list of all packages that can be updated by "apt upgrade".

«aptitude search -F %p '?upgradable'» would print the right set of
packages, I think, but aptitude isn't part of a minimal installation so
can't be assumed available.

I'm not sure what's the best way to answer this question on a minimal
installation.  One _could_ parse the output of «apt-cache policy \*»,
but I hope there's a better way...

Cc += Debian zsh packagers.  tl;dr: What apt(7) command generates
the right set of packages to be completed after «apt list --upgradable <TAB>»?

Re: apt autocompletion issue

[Axel Beckert]

Hi,

TL;DR: I recommend to just complete all packages upon
"apt list --upgradable <Tab>".

Oliver Kiddle wrote:
> On 26 Jul, "Hauser, Felix (MRT)" wrote:
> > just discovered the Z-shell and so far it really makes my life a little 
> > bit easier! The only issue I have with it is the completion of options 
> > for the apt command. So e.g.:
> >
> > apt list --u<tab>
> >
> > won't autocomplete to
> >
> > apt list --upgradeable
> >
> > which is easily done in the normal bash file.

Should be easy in zsh, too, as the number of long options behind "apt
list" isn't that big IIRC.

> It's not clear from the man page whether the three options can be
> combined with the glob(7) patterns for matching packages (as per the
> patch below) or whether they need to be standalone.

Indeed. Just from testing, there are additional filters possible:

→ apt list --installed 'z*'
Listing... Done
zathura-cb/stable,now 0.1.8-2 amd64 [installed,automatic]
zathura-djvu/stable,now 0.2.8-1 amd64 [installed,automatic]
[…]
zsh/stable,now 5.7.1-1 amd64 [installed,automatic]
zutils/stable,now 1.8-3+b10 amd64 [installed,automatic]
zziplib-bin/stable,now 0.13.62-3.2 amd64 [installed]
zzuf/stable,now 0.15-1+b1 amd64 [installed]
→ 

But I think the primary issue of Felix is that not even the (rather
unambiguous) options are completed.

> The option should probably modify whether it is calling _deb_packages
> with "avail", "installed", "uninstalled" or some other package set. If
> you let me know what the relevant package set argument should be for
> each option, then it is easy to do that.

It makes sense that "apt list --installed <Tab>" only completes
installed packages as it will only list these anyway.

It makes sense that "apt list --upgradable <Tab>" (and "apt list
--upgradeable <Tab>") only completes packages as it will only list
these anyway.

And "apt list --all-versions <Tab>" should probably complete all
packages.

Daniel Shahaf wrote:
> Hauser, Felix (MRT) wrote on Tue, 30 Jul 2019 11:50 +00:00:
> > --upgradeable Do not know the option for this one. Result should be a 
> > list of all packages that can be updated by "apt upgrade".
> 
> «aptitude search -F %p '?upgradable'» would print the right set of
> packages, I think, but aptitude isn't part of a minimal installation so
> can't be assumed available.

Correct. Don't expect it to be there, it's no more installed by
default. (Saying that with my "apt+aptitude team" member hat on,
although I'm mostly only working on aptitude. :-)

It seems obvious that "apt list --installed <Tab>" should be completed
with what "apt list --installed" outputs and "apt list --upgradable
<Tab>" and "apt list --upgradeable <Tab>" with what "apt list
--upgradeable" outputs.

But: "apt" output is not yet considered to be stable and hence emits
the following warning on STDERR when used inside a pipe or similar:

  WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

> I'm not sure what's the best way to answer this question on a minimal
> installation.  One _could_ parse the output of «apt-cache policy \*»,
> but I hope there's a better way...

Currently I don't know any efficient _and_ scriptable method to do
that. (Calling "apt-cache policy \*" to see if it's installed or
upgradeable is the only method which comes to my mind, too, but also
rather slow and inefficient.

> Cc += Debian zsh packagers.

Thanks. I read Felix initial mail, but just adding three hard coded
long options seemed rather trivial...

> tl;dr: What apt(7) command generates the right set of packages to be
> completed after «apt list --upgradable <TAB>»?

"apt list --upgradable", but see above.

Officially, there is currently no scriptable interface which offers
that. aptitude-like pattern matching is planned for
apt/apt-get/apt-cache as well, but not yet there.

And parsing /var/lib/apt/lists/* is explicitly discouraged as the
format might change without notice. (Zsh seems to do that somewhere
else already and I get according warnings when in building the zsh
package for Debian:
https://lintian.debian.org/tags/uses-dpkg-database-directly.html)

So we basically have these options:

* Use "apt list --<something>" and have to adapt quickly when the
  output format changes.

* Parse the output of "apt-cache policy \*". (Probably the best way if
  we want to really only list upgradable packages there.)

* Parse /var/lib/apt/archives/*.packages (format might change over
  time, too), /etc/apt/preferences and /etc/apt/preferences.d/*.

* Completely ignore any "apt list" option and just complete all known
  packages. This seems to be a easily done solution and I consider it
  valid, because it is a valid call to pass not upgradable or
  installed packages even with --installed or --upgradable:

  → apt list --installed systemd zsh
  Listing... Done
  zsh/stable,now 5.7.1-1 amd64 [installed,automatic]
  →

  (i.e. in this example, systemd is not installed and zsh is
  installed.)

Installed packages are easier to generated as they can be queried with
dpkg-query which has options for machine-readable formatting.

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe@debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Re: apt autocompletion issue

[Oliver Kiddle]

Axel Beckert wrote:
> TL;DR: I recommend to just complete all packages upon
> "apt list --upgradable <Tab>".

Thanks, so this updated patch uses installed with --installed and avail
otherwise. After checking the man page, I also added completion of
reinstall and auto-removal for - and + suffixes.

> > On 26 Jul, "Hauser, Felix (MRT)" wrote:
> > > which is easily done in the normal bash file.

Just for note Felix, while technically possible, I'd really recommend
against trying to ever use bash completions with zsh.

> Daniel Shahaf wrote:
> > ??aptitude search -F %p '?upgradable'?? would print the right set of
> > packages, I think, but aptitude isn't part of a minimal installation so
> > can't be assumed available.

A _deb_packages_update_upgradable function could check
(( $+commands[aptitude] )) and offer all packages as a fallback.
Or one of the options Axel mentions - if someone wants to do that.

> > Cc += Debian zsh packagers.
>
> Thanks. I read Felix initial mail, but just adding three hard coded
> long options seemed rather trivial...

It's often not as trivial as it might first seem without a copy of the
command to test combinations against. I probably need to install yet
another VM to cover Debian given a large number of Debian specific
completions. Out of interest, what is the recommended apt* tool these
days? Is aptitude the best or most powerful with apt as the simplified
interface and apt-get just a historic relic?

Oliver

diff --git a/Completion/Debian/Command/_apt b/Completion/Debian/Command/_apt
index 4486aa26d..4d60cd249 100644
--- a/Completion/Debian/Command/_apt
+++ b/Completion/Debian/Command/_apt
@@ -399,7 +399,13 @@ _apt-cmd () {
     -o,--option:arbitem \
     -t,--target-release:release \
     -- \
-    /$'list\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
+    /$'list\0'/ \( \
+      \( \
+        /$'--installed\0'/ ':packages::_deb_packages "$expl_packages[@]" installed' \# /'[]'/ \| \
+        // %-% /$'--[^\0]#\0'/ ':options:option:(--installed --upgradable --all-versions)' \
+      \| \) \
+      /$'[^-\0][^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \
+    \) \| \
     /$'search\0'/ /$'[^\0]#\0'/ ':strings:pattern:' \| \
     /$'showsrc\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
     /$'show\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
@@ -409,15 +415,14 @@ _apt-cmd () {
     /$'update\0'/ \| \
     \( \
       /$'(install|download|source|build-dep)\0'/ -'subcmd=${match%?}' \
-      /$'[^\0]#\0'/ ':packages::_deb_packages -qr "\n\t\- /=" "$expl_packages[@]" avail' \# \
+      /$'[^\0]#\0'/ ':packages::_deb_packages -qr "\n\t\- /=+-" "$expl_packages[@]" avail' \# \
       \( \
 	// '%(.|)/%' -'[[ $subcmd = install ]]' /'[]'/ ':files: :_deb_files -c' \| \
 	/$'[^\0/=]#\/'/ /'[]'/ ':apt-releases:release name:_apt_releases' \| \
 	/$'[^\0/=]#='/ /'[]'/ ':apt-package-versions:package version:_apt_versions_of_binary_package' \| \
       \) \
     \) \| \
-    /$'remove\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" installed' \# \| \
-    /$'purge\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" installed' \# \| \
+    /$'(remove|reinstall|purge)\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" installed' \# \| \
     /$'upgrade\0'/ \| \
     /$'autoclean\0'/ \| \
     /$'changelog\0'/ /$'[^\0]#\0'/ ':packages::_deb_packages "$expl_packages[@]" avail' \# \| \
@@ -425,7 +430,7 @@ _apt-cmd () {
     /$'full-upgrade\0'/ \| \
     /$'dist-upgrade\0'/ \| \
     /$'edit-sources\0'/ \| \
-    /"[]"/	':argument-1::compadd "$expl_action[@]" list search showsrc show depends rdepends policy update install download source build-dep remove upgrade full-upgrade dist-upgrade edit-sources autoclean changelog autoremove purge'
+    /"[]"/ ':argument-1::compadd "$expl_action[@]" list search showsrc show depends rdepends policy update install reinstall download source build-dep remove upgrade full-upgrade dist-upgrade edit-sources autoclean changelog autoremove purge'
 
   _apt-cmd () {
     local expl_action expl_packages subcmd

aptitude vs apt vs apt-get (was: Re: [Pkg-zsh-devel] apt autocompletion issue)

[Axel Beckert]

Hi Oliver,

Oliver Kiddle wrote:
> Out of interest, what is the recommended apt* tool these
> days?

Depends on what you want to achieve or where you want to use it. They
all have their uses. :-)

TL;DR: Use aptitude if you need TUI or patterns, apt for CLI comfort,
apt-get/apt-cache for scripting and a stable CLI.

> Is aptitude the best or most powerful with apt as the simplified
> interface and apt-get just a historic relic?

Disclaimer: This might be biased as I'm a hardcore aptitude user. And
I'm one of the two current aptitude maintainers.

aptitude is still the most powerful sincec it's the only one with a
pattern matching language on packages. But apt and apt-get/apt-cache
might get this in the future, too.

There are still a few more differences which won't change that quickly:

* aptitude is the sole tool with a TUI. (And previously also contained
  a GTK and a never released Qt GUI. Both were removed as they were
  unmaintained and never really stable.)

* aptitude's dependency resolver is optimized on interaction with the
  user, i.e. getting interactively further hints from the user on how
  to solve a non-trivial dependency issue.

* apt and apt-get are optimized for one-shot solutions, i.e. try to
  get the sanest solution first.

  They're also usually faster than aptitude which tracks more metadata
  needed for pattern matching or package displaying.

* apt is (currently) solely meant for commandline user by an admin. It
  has far more comfortable defaults (e.g. color and progress bar) and
  a not completely identical set of subcommands compared to
  apt-get/apt-cache.

* apt-cache/apt-get have a stable commandline interface and are best
  for scripting — for now unless you need the power of aptitude's
  pattern matching.

  But beware: Even though a few years ago there was quite an effort to
  make the exit codes of aptitude in error cases more consistent and
  sane, there are still a few rough edges and inconsistencies in that
  area.

P.S. and JFTR: There is no such war between apt and aptitude. We've
merged both teams a year ago or so and all aptitude developers have
now also commit rights on apt and vice versa:
https://salsa.debian.org/apt-team

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe@debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE