Slow completion when using aptitude

Slow completion when using aptitude

[cheer_xiao]

Hi zsh users and hackers,
I'm running Ubuntu 10.04 on my box. I have found that tab-completion
for package names is slow when I use aptitude; however when I try
apt-get instead, the completion is almost instinct. I Google-ed and
found out that zsh switches to apt-cache policy for package name
completion some years ago, due to slow completion. I suspect that that
switch didn't take aptitude into concern, and it shouldn't be hard to
fix. Anyone willing to look into this? I'm using the version from git
repo.

And another small related issue: the command tab-completion of zsh
doesn't seem to know aptitude build-dep, which has been around for a
few years but (unfortunately) undocumented. The bash that comes with
Ubuntu knows this.

Re: Slow completion when using aptitude

[Richard Hartmann]

Hi,

those questions might be better suited to the -users list.

> And another small related issue: the command tab-completion of zsh
> doesn't seem to know aptitude build-dep, which has been around for a
> few years but (unfortunately) undocumented.

I did not have time to check the first issue, but on zsh 4.3.10, you
can complete to aptitude build-dep. As a general rule of thumb, please
try to use the newest version of zsh (and starting with zsh -f where
applicable/possible) before reporting issues. They might have been
fixed, already.


Richard

Re: Slow completion when using aptitude

[cheer_xiao]

On Wed, Jan 19, 2011 at 8:33 PM, Richard Hartmann
<richih.mailinglist@gmail.com> wrote:
> Hi,
>
> those questions might be better suited to the -users list.
>
>> And another small related issue: the command tab-completion of zsh
>> doesn't seem to know aptitude build-dep, which has been around for a
>> few years but (unfortunately) undocumented.
>
> I did not have time to check the first issue, but on zsh 4.3.10, you
> can complete to aptitude build-dep. As a general rule of thumb, please
> try to use the newest version of zsh (and starting with zsh -f where
> applicable/possible) before reporting issues. They might have been
> fixed, already.

Thanks for your attention. I posted this to zsh-workers because I
found it weird enough to be very likely a bug in source code. Could
you tell me if you are having performance problem with aptitude
tab-complete? Also, let me describe my problems in more detail.

If I type "sudo apt-get install a" and hit Tab, I only get a short
(yet noticeable) delay of about one second. But when I type "sudo
aptitude install a" and hit Tab, zsh freezes, one of the CPU cores get
100% loaded. It will recover after some 10 seconds. However after
this, the tab-completion of aptitude is as quick as apt-get's, until I
log out of zsh.

I am using the version from git repo:

xiaq@~> zsh --version
zsh 4.3.11-dev-1 (i686-pc-linux-gnu)

and my system:

xiaq@~> uname -a
Linux xxx 2.6.32-28-generic #55-Ubuntu SMP Mon Jan 10 21:21:01 UTC
2011 i686 GNU/Linux

After typing "sudo aptitude" and hitting Tab, the output is (build-dep
is absent):

xiaq@~> sudo aptitude
autoclean                 -- erase old downloaded package files
changelog                 -- view a package's changelog
clean                     -- erase downloaded package files
download                  -- download the .deb file for a package
forbid-version            -- forbid aptitude from upgrading to a
specific package ve
forget-new                -- forget what packages are "new"
full-upgrade              -- perform an upgrade, possibly installing
and removing pa
hold                      -- place packages on hold
install                   -- install/upgrade packages
markauto                  -- mark packages as having been
automatically installed
purge                     -- remove packages and their configuration files
reinstall                 -- download and (possibly) reinstall a
currently installed
remove                    -- remove packages
safe-upgrade              -- perform a safe upgrade
search                    -- search for a package by name and/or expression
show                      -- display detailed information about a package
unhold                    -- cancel a hold command for a package
unmarkauto                -- mark packages as having been manually installed
update                    -- download lists of new/upgradable packages
why                       -- show the manually installed packages that
require a pac
why-not                   -- show the manually installed packages that
lead to a con

Re: Slow completion when using aptitude

[Julien Nicoulaud]

[-- Attachment #1: Type: text/plain, Size: 3245 bytes --]

Same here.

2011/1/26 cheer_xiao <xiaqqaix@gmail.com>

> On Wed, Jan 19, 2011 at 8:33 PM, Richard Hartmann
> <richih.mailinglist@gmail.com> wrote:
> > Hi,
> >
> > those questions might be better suited to the -users list.
> >
> >> And another small related issue: the command tab-completion of zsh
> >> doesn't seem to know aptitude build-dep, which has been around for a
> >> few years but (unfortunately) undocumented.
> >
> > I did not have time to check the first issue, but on zsh 4.3.10, you
> > can complete to aptitude build-dep. As a general rule of thumb, please
> > try to use the newest version of zsh (and starting with zsh -f where
> > applicable/possible) before reporting issues. They might have been
> > fixed, already.
>
> Thanks for your attention. I posted this to zsh-workers because I
> found it weird enough to be very likely a bug in source code. Could
> you tell me if you are having performance problem with aptitude
> tab-complete? Also, let me describe my problems in more detail.
>
> If I type "sudo apt-get install a" and hit Tab, I only get a short
> (yet noticeable) delay of about one second. But when I type "sudo
> aptitude install a" and hit Tab, zsh freezes, one of the CPU cores get
> 100% loaded. It will recover after some 10 seconds. However after
> this, the tab-completion of aptitude is as quick as apt-get's, until I
> log out of zsh.
>
> I am using the version from git repo:
>
> xiaq@~> zsh --version
> zsh 4.3.11-dev-1 (i686-pc-linux-gnu)
>
> and my system:
>
> xiaq@~> uname -a
> Linux xxx 2.6.32-28-generic #55-Ubuntu SMP Mon Jan 10 21:21:01 UTC
> 2011 i686 GNU/Linux
>
> After typing "sudo aptitude" and hitting Tab, the output is (build-dep
> is absent):
>
> xiaq@~> sudo aptitude
> autoclean                 -- erase old downloaded package files
> changelog                 -- view a package's changelog
> clean                     -- erase downloaded package files
> download                  -- download the .deb file for a package
> forbid-version            -- forbid aptitude from upgrading to a
> specific package ve
> forget-new                -- forget what packages are "new"
> full-upgrade              -- perform an upgrade, possibly installing
> and removing pa
> hold                      -- place packages on hold
> install                   -- install/upgrade packages
> markauto                  -- mark packages as having been
> automatically installed
> purge                     -- remove packages and their configuration files
> reinstall                 -- download and (possibly) reinstall a
> currently installed
> remove                    -- remove packages
> safe-upgrade              -- perform a safe upgrade
> search                    -- search for a package by name and/or expression
> show                      -- display detailed information about a package
> unhold                    -- cancel a hold command for a package
> unmarkauto                -- mark packages as having been manually
> installed
> update                    -- download lists of new/upgradable packages
> why                       -- show the manually installed packages that
> require a pac
> why-not                   -- show the manually installed packages that
> lead to a con
>

Re: Slow completion when using aptitude

[Frank Terbeck]

Julien Nicoulaud wrote:
> Same here.

+ Fullquote? :)

> 2011/1/26 cheer_xiao <xiaqqaix@gmail.com>
>
>> On Wed, Jan 19, 2011 at 8:33 PM, Richard Hartmann
>> <richih.mailinglist@gmail.com> wrote:
[...]
>> >> And another small related issue: the command tab-completion of zsh
>> >> doesn't seem to know aptitude build-dep, which has been around for a
>> >> few years but (unfortunately) undocumented.
[...]

It does know `build-dep' for a while now; it was first introduced here:
    <http://www.zsh.org/mla/workers/2009/msg00193.html>


>> Thanks for your attention. I posted this to zsh-workers because I
>> found it weird enough to be very likely a bug in source code. Could
>> you tell me if you are having performance problem with aptitude
>> tab-complete? Also, let me describe my problems in more detail.
>>
>> If I type "sudo apt-get install a" and hit Tab, I only get a short
>> (yet noticeable) delay of about one second. But when I type "sudo
>> aptitude install a" and hit Tab, zsh freezes, one of the CPU cores get
>> 100% loaded. It will recover after some 10 seconds. However after
>> this, the tab-completion of aptitude is as quick as apt-get's, until I
>> log out of zsh.

_aptitude does more than _apt-get. If the second <tab> doesn't take as
long as the first, you probably got caching enabled via zstyle.

>> I am using the version from git repo:
>>
>> xiaq@~> zsh --version
>> zsh 4.3.11-dev-1 (i686-pc-linux-gnu)
>>
>> and my system:
>>
>> xiaq@~> uname -a
>> Linux xxx 2.6.32-28-generic #55-Ubuntu SMP Mon Jan 10 21:21:01 UTC
>> 2011 i686 GNU/Linux
>>
>> After typing "sudo aptitude" and hitting Tab, the output is (build-dep
>> is absent):
>>
>> xiaq@~> sudo aptitude
>> autoclean                 -- erase old downloaded package files
>> changelog                 -- view a package's changelog

My installation is also from the git repo:

[snip]
% uname -srm
Linux 2.6.32-5-686 i686

% print $ZSH_VERSION
4.3.11-dev-1

% print $ZSH_PATCHLEVEL
1.5174

% sudo aptitude <tab>
- aptitude command -
autoclean                      -- erase old downloaded package files.
build-dep                      -- install the build-dependencies of packages.
changelog                      -- view a package's changelog.
[snip]

Are you sure you don't have two competing versions of zsh installed?
Then it may be possible that "print $ZSH_VERSION" and "zsh --version"
would yield different results.

Regards, Frank

-- 
In protocol design, perfection has been reached not when there is
nothing left to add, but when there is nothing left to take away.
                                                  -- RFC 1925

Re: Slow completion when using aptitude

[cheer_xiao]

On Wed, Jan 26, 2011 at 4:35 PM, Frank Terbeck <ft@bewatermyfriend.org> wrote:
> [snip]
> _aptitude does more than _apt-get. If the second <tab> doesn't take as
> long as the first, you probably got caching enabled via zstyle.

So the problem is, as I understand, the cache for apt-get is
persistent, while that of aptitude isn't. It seems that I have done
nothing special about them, and the following is a snippet of my
.zshrc:

# The following lines were added by compinstall

zstyle ':completion:*' file-sort name
zstyle ':completion:*' list-colors ${(s.:.)LS_COLORS}
zstyle ':completion:*' matcher-list 'm:{[:lower:]}={[:upper:]}'
'r:|[._-]=* r:|=*' 'l:|=* r:|=*'
zstyle ':completion:*' menu select=1
zstyle ':completion:*' select-prompt '%SScrolling active: current
selection at %p%s'
zstyle :compinstall filename '/home/xiaq/.zshrc'

autoload -Uz compinit
compinit
# End of lines added by compinstall
# Lines configured by zsh-newuser-install

To make things more clear, I tried the following:

xiaq@~> zsh -f
xiaq@~> autoload -Uz compinit && compinit

to get a minimal zsh environment with tab-completion enabled. The
result is still the same.

> [snip]
> Are you sure you don't have two competing versions of zsh installed?
> Then it may be possible that "print $ZSH_VERSION" and "zsh --version"
> would yield different results.

I did install the version from Ubuntu repo, but now I have already removed it.

xiaq@~> which -a zsh
/usr/local/bin/zsh

Could you check whether "aptitude -h" on your system mentions
build-dep? That of my system (Ubuntu 10.04) doesn't; while apt-get
does:

xiaq@~> aptitude -h | grep build
[no result]
xiaq@~> apt-get | grep build
  build-dep - Configure build-dependencies for source packages


I doubt whether zsh's tab-completion of aptitude's(and other
utilities') verbs works by looking at "aptitude -h" first to generate
the verbs list. If so, the problem should be considered a bug of
aptitude that ships with Ubuntu 10.04 instead of zsh. (This has been
confirmed, so this minor bug belongs to aptitude instead of zsh.)

Re: Slow completion when using aptitude

[Frank Terbeck]

cheer_xiao wrote:
> On Wed, Jan 26, 2011 at 4:35 PM, Frank Terbeck <ft@bewatermyfriend.org> wrote:
>> [snip]
>> _aptitude does more than _apt-get. If the second <tab> doesn't take as
>> long as the first, you probably got caching enabled via zstyle.
>
> So the problem is, as I understand, the cache for apt-get is
> persistent, while that of aptitude isn't.
[...]

The difference is, that "aptitude install <tab>" lists uninstalled
packages, whereas "apt-get install <tab>" simply lists all available
packages. The former, while being more appropriate, also takes more
work.

I must admit, that I'm not quite familiar with how compsys' caching is
implemented. I may be giving wrong pointers here. So I should better be
quiet.

>> [snip]
>> Are you sure you don't have two competing versions of zsh installed?
>> Then it may be possible that "print $ZSH_VERSION" and "zsh --version"
>> would yield different results.
[...]
> xiaq@~> aptitude -h | grep build
> [no result]
> xiaq@~> apt-get | grep build
>   build-dep - Configure build-dependencies for source packages
>
>
> I doubt whether zsh's tab-completion of aptitude's(and other
> utilities') verbs works by looking at "aptitude -h" first to generate
> the verbs list. If so, the problem should be considered a bug of
> aptitude that ships with Ubuntu 10.04 instead of zsh. (This has been
> confirmed, so this minor bug belongs to aptitude instead of zsh.)

The aptitude completion *does* in fact look at "aptitude -h" to figure
out which sub-commands to offer. And my aptitude does include it in its
output. Are you sure that your aptitude supports `build-dep'? Because
its author claims the help message was updated the in the same version
the functionality was added:
    <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=501998#10>

(Although I think I remember an aptitude that had `build-dep' and didn't
list it in -h output. But I'm not sure.)

Regards, Frank

-- 
In protocol design, perfection has been reached not when there is
nothing left to add, but when there is nothing left to take away.
                                                  -- RFC 1925

Re: Slow completion when using aptitude

[Bart Schaefer]

On Jan 26,  3:55pm, Frank Terbeck wrote:
} Subject: Re: Slow completion when using aptitude
}
} The difference is, that "aptitude install <tab>" lists uninstalled
} packages, whereas "apt-get install <tab>" simply lists all available
} packages. The former, while being more appropriate, also takes more
} work.

I tried this completion on a system that has neither aptitude nor
apt-get installed at all, so that I could limit the differences to
those attributable directly to the zsh completion system.

Comparing _complete_debug output for the two, the main difference
seems to be that _aptitude passes through _deb_packages repeatedly
if you have a matcher-list, while _apt makes only a single pass.
Both start out by looking at the DEBS_avail cache, but _aptititude
then goes on to examine the DEBS_installed cache as well, which
seems like a missed optimization in the case of this particular
sub-command word.

} I must admit, that I'm not quite familiar with how compsys' caching is
} implemented. I may be giving wrong pointers here. So I should better
} be quiet.

The caching appears to be identical in both cases, they're both going
through the standard mechanism; in fact even with the commands not
installed, if I repeat the aptitude completion and compare the
_complete_debug output, the (empty) cache is used and most of the
work done in _deb_packages on the first attempt is avoided.

If the cache doesn't seem to persist across session exit/restart, it
must have something to do with how cache validity is determined.

-- 

Re: Slow completion when using aptitude

[Bart Schaefer]

On Jan 26,  9:05am, Bart Schaefer wrote:
} Subject: Re: Slow completion when using aptitude
}
} Comparing _complete_debug output for the two, the main difference
} seems to be that _aptitude passes through _deb_packages repeatedly
} if you have a matcher-list, while _apt makes only a single pass.

Oh, forgot to mention -- the _apt completer, at least for apt-get,
seems to suffer from the common bug of returning 0 when it finds no
completions, fooling the higher layers into thinking it succeeded
but there's nothing to do rather than that it failed.  Of course it
may be that this is the right thing in this case, and _aptitude is
instead buggy when it returns 1 in the same circumstance.

That's probably why the number of passes through _deb_packages differs.

Re: Slow completion when using aptitude

[cheer_xiao]

[-- Attachment #1: Type: text/plain, Size: 103 bytes --]

Anyone willing to look further into this issue and possibly come up with a
fix? A lot of thanks ahead.

Re: Slow completion when using aptitude

[Bart Schaefer]

On Jan 30, 10:49am, cheer_xiao wrote:
} 
} Anyone willing to look further into this issue and possibly come up with a
} fix? A lot of thanks ahead.

As you're able to reproduce it ... from the manual:

     When loaded, the zsh/zprof causes shell functions to be profiled.
   The profiling results can be obtained with the zprof builtin command
   made available by this module.  There is no way to turn profiling off
   other than unloading the module.

Load that module, run the slow completion, and get us some output from
the zprof command to look at.  (Somebody else can fix the typo in
"the zsh/zprof causes".)

Re: Slow completion when using aptitude

[cheer_xiao]

On Sun, Jan 30, 2011 at 1:16 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Jan 30, 10:49am, cheer_xiao wrote:
> }
> } Anyone willing to look further into this issue and possibly come up with a
> } fix? A lot of thanks ahead.
>
> As you're able to reproduce it ... from the manual:
>
>     When loaded, the zsh/zprof causes shell functions to be profiled.
>   The profiling results can be obtained with the zprof builtin command
>   made available by this module.  There is no way to turn profiling off
>   other than unloading the module.
>
> Load that module, run the slow completion, and get us some output from
> the zprof command to look at.  (Somebody else can fix the typo in
> "the zsh/zprof causes".)

I check the documentation that apparently there is no instruction on
how to load modules. Any hints?

Re: Slow completion when using aptitude

[Bart Schaefer]

On Jan 30,  1:53pm, cheer_xiao wrote:
}
} I check the documentation that apparently there is no instruction on
} how to load modules. Any hints?

zmodload zsh/zprof

The zmodload command ought to be cross-referenced from the "Zsh Modules"
section, but there is an index entry for "loading modules" that points
to zmodload.

Re: Slow completion when using aptitude

[Julien Nicoulaud]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

Here is zprof after a "sudo apt-get install a<TAB>":
http://paste2.org/p/1219060
<http://paste2.org/p/1219060>And after a "sudo aptitude install a<TAB>":
http://paste2.org/p/1219061

<http://paste2.org/p/1219061>As you said earlier, all the time is spent in
_deb_packages_update_uninstalled...

2011/1/30 Bart Schaefer <schaefer@brasslantern.com>

> On Jan 30,  1:53pm, cheer_xiao wrote:
> }
> } I check the documentation that apparently there is no instruction on
> } how to load modules. Any hints?
>
> zmodload zsh/zprof
>
> The zmodload command ought to be cross-referenced from the "Zsh Modules"
> section, but there is an index entry for "loading modules" that points
> to zmodload.
>

Re: Slow completion when using aptitude

[Julien Nicoulaud]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

By the way, this issue was already known:
http://www.zsh.org/mla/workers/2008/msg00930.html

2011/1/30 Julien Nicoulaud <julien.nicoulaud@gmail.com>

> Here is zprof after a "sudo apt-get install a<TAB>":
> http://paste2.org/p/1219060
> <http://paste2.org/p/1219060>And after a "sudo aptitude install a<TAB>":
> http://paste2.org/p/1219061
>
> <http://paste2.org/p/1219061>As you said earlier, all the time is spent
> in _deb_packages_update_uninstalled...
>
> 2011/1/30 Bart Schaefer <schaefer@brasslantern.com>
>
> On Jan 30,  1:53pm, cheer_xiao wrote:
>> }
>> } I check the documentation that apparently there is no instruction on
>> } how to load modules. Any hints?
>>
>> zmodload zsh/zprof
>>
>> The zmodload command ought to be cross-referenced from the "Zsh Modules"
>> section, but there is an index entry for "loading modules" that points
>> to zmodload.
>>
>
>

Re: Slow completion when using aptitude

[cheer_xiao]

By any means, the inconsistency of apt-get's and aptitude's behavior
should be a bug; either we make them all look for all packages - an
approach not correct enough but doesn't lag; or speed up the procedure
aptitude looks for available packages and apply it to apt-get too.

Re: Slow completion when using aptitude

[Bart Schaefer]

On Jan 30, 11:51am, Julien Nicoulaud wrote:
} 
} By the way, this issue was already known:
} http://www.zsh.org/mla/workers/2008/msg00930.html

So ...

_deb_packages_update_avail uses the cache to populate an array variable
of the same name.

As does _deb_packages_update_installed.

Then _deb_packages_update_uninstalled constructs a massive glob pattern
from the results of the latter, and applies it as a filter against the
former, to set a variable _deb_packages_update_uninstalled.

Thereafter the value of _deb_packages_update_uninstalled is used rather
than go through that filter again, so it *is* caching.  The disk cache
isn't used by _deb*_uninstalled because the files are all on the disk
already from the other two functions. 

All the time is being spent building that glob pattern, and filtering.

So try using this instead:

_deb_packages_update_uninstalled () {
  _deb_packages_update_avail
  _deb_packages_update_installed
  if (( ! $+_deb_packages_cache_uninstalled )); then
    local avail
    for avail in $_deb_packages_cache_avail
    do
	(( ${+_deb_packages_cache_installed[(r)$avail]} )) && continue
	_deb_packages_cache_uninstalledr+=( $avail )
    done
  fi
  cachevar=_deb_packages_cache_uninstalled
}

Sometimes the most zsh-ish way to do something isn't the most efficient
way to do it.  It may even be still faster to replace the "for" loop
with:

    _dep_packages_cache_uninstalled=(
	$( print -l $_deb_packages_cache_avail | 
	   fgrep -vf =(print -l $_deb_packages_cache_installed) )
    )

Re: Slow completion when using aptitude

[cheer_xiao]

On Mon, Jan 31, 2011 at 8:02 AM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Jan 30, 11:51am, Julien Nicoulaud wrote:
> }
> } By the way, this issue was already known:
> } http://www.zsh.org/mla/workers/2008/msg00930.html
>
> So ...
>
> _deb_packages_update_avail uses the cache to populate an array variable
> of the same name.
>
> As does _deb_packages_update_installed.
>
> Then _deb_packages_update_uninstalled constructs a massive glob pattern
> from the results of the latter, and applies it as a filter against the
> former, to set a variable _deb_packages_update_uninstalled.
>
> Thereafter the value of _deb_packages_update_uninstalled is used rather
> than go through that filter again, so it *is* caching.  The disk cache
> isn't used by _deb*_uninstalled because the files are all on the disk
> already from the other two functions.
>
> All the time is being spent building that glob pattern, and filtering.
>
> So try using this instead:
>
> _deb_packages_update_uninstalled () {
>  _deb_packages_update_avail
>  _deb_packages_update_installed
>  if (( ! $+_deb_packages_cache_uninstalled )); then
>    local avail
>    for avail in $_deb_packages_cache_avail
>    do
>        (( ${+_deb_packages_cache_installed[(r)$avail]} )) && continue
>        _deb_packages_cache_uninstalledr+=( $avail )
>    done
>  fi
>  cachevar=_deb_packages_cache_uninstalled
> }

Was I supposed to include this in my .zshrc and that would solve my
problem, or was it meant to be a hint for developers on fixing the
bug? I have tried the former but nothing seems to change.

>
> Sometimes the most zsh-ish way to do something isn't the most efficient
> way to do it.  It may even be still faster to replace the "for" loop
> with:
>
>    _dep_packages_cache_uninstalled=(
>        $( print -l $_deb_packages_cache_avail |
>           fgrep -vf =(print -l $_deb_packages_cache_installed) )
>    )
>

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 3,  6:21pm, cheer_xiao wrote:
} Subject: Re: Slow completion when using aptitude
}
} On Mon, Jan 31, 2011 at 8:02 AM, Bart Schaefer
} <schaefer@brasslantern.com> wrote:
} >
} > So try using this instead:
} >
} > _deb_packages_update_uninstalled () {
} >  _deb_packages_update_avail
} >  _deb_packages_update_installed
} >  if (( ! $+_deb_packages_cache_uninstalled )); then
} >    local avail
} >    for avail in $_deb_packages_cache_avail
} >    do
} >        (( ${+_deb_packages_cache_installed[(r)$avail]} )) && continue
} >        _deb_packages_cache_uninstalledr+=( $avail )
} >    done
} >  fi
} >  cachevar=_deb_packages_cache_uninstalled
} > }
} 
} Was I supposed to include this in my .zshrc and that would solve my
} problem, or was it meant to be a hint for developers on fixing the
} bug? I have tried the former but nothing seems to change.

Sorry, I could have been more specific.

Putting it in your .zshrc is unlikely to work because it'll get
clobbered when the completion system autoloads the other _deb*
functions later.

You need to find the file ${^fpath}/_deb_packages and edit it
to replace _deb_packages_update_uninstalled with the above.

Re: Slow completion when using aptitude

[cheer_xiao]

On Thu, Feb 3, 2011 at 10:56 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Feb 3,  6:21pm, cheer_xiao wrote:
> } Subject: Re: Slow completion when using aptitude
> }
> } On Mon, Jan 31, 2011 at 8:02 AM, Bart Schaefer
> } <schaefer@brasslantern.com> wrote:
> } >
> } > So try using this instead:
> } >
> } > _deb_packages_update_uninstalled () {
> } >  _deb_packages_update_avail
> } >  _deb_packages_update_installed
> } >  if (( ! $+_deb_packages_cache_uninstalled )); then
> } >    local avail
> } >    for avail in $_deb_packages_cache_avail
> } >    do
> } >        (( ${+_deb_packages_cache_installed[(r)$avail]} )) && continue
> } >        _deb_packages_cache_uninstalledr+=( $avail )
> } >    done
> } >  fi
> } >  cachevar=_deb_packages_cache_uninstalled
> } > }
> }
> } Was I supposed to include this in my .zshrc and that would solve my
> } problem, or was it meant to be a hint for developers on fixing the
> } bug? I have tried the former but nothing seems to change.
>
> Sorry, I could have been more specific.
>
> Putting it in your .zshrc is unlikely to work because it'll get
> clobbered when the completion system autoloads the other _deb*
> functions later.
>
> You need to find the file ${^fpath}/_deb_packages and edit it
> to replace _deb_packages_update_uninstalled with the above.

I tried this, but after typing "sudo aptitude install a" and hit Tab,
zsh seems to be trapped in an infinite loop, with one of the CPU cores
fully loaded. Should I turn on something else like caching, or is
there a flaw in the code you provided?

Thanks for your patience.

Re: Slow completion when using aptitude

[cheer_xiao]

On Fri, Feb 4, 2011 at 11:33 AM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Feb 4, 10:49am, cheer_xiao wrote:
> } Subject: Re: Slow completion when using aptitude
> }
> } > } > _deb_packages_update_uninstalled () {
> } > } >  _deb_packages_update_avail
> } > } >  _deb_packages_update_installed
> } > } >  if (( ! $+_deb_packages_cache_uninstalled )); then
> } > } >    local avail
> } > } >    for avail in $_deb_packages_cache_avail
> } > } >    do
> } > } >        (( ${+_deb_packages_cache_installed[(r)$avail]} )) && continue
> } > } >        _deb_packages_cache_uninstalledr+=( $avail )
> } > } >    done
> } > } >  fi
> } > } >  cachevar=_deb_packages_cache_uninstalled
> } > } > }
> } >
> } > You need to find the file ${^fpath}/_deb_packages and edit it
> } > to replace _deb_packages_update_uninstalled with the above.
> }
> } I tried this, but after typing "sudo aptitude install a" and hit Tab,
> } zsh seems to be trapped in an infinite loop, with one of the CPU cores
> } fully loaded. Should I turn on something else like caching, or is
> } there a flaw in the code you provided?
>
> It's probably still doing too much incremental memory allocation when
> growing the _deb_*_uninstalled array, if there are many thousands of
> packages either installed or available.
>
> Try the other way I suggested:
>
> _deb_packages_update_uninstalled () {
>  _deb_packages_update_avail
>  _deb_packages_update_installed
>  if (( ! $+_deb_packages_cache_uninstalled )); then
>    _dep_packages_cache_uninstalled=(
>      $( print -l $_deb_packages_cache_avail |
>         fgrep -vf =(print -l $_deb_packages_cache_installed)

Seems you have missed a closing parenthesis at the end of line.

>    )
>  fi
> }
>

It's a wonderful fix. zsh now respond within 2 seconds. Thanks a lot!
I wish to see the fix committed into the source repo ASAP. Also the
completion for apt-get should behave the same.

Re: Slow completion when using aptitude

[cheer_xiao]

On Fri, Feb 4, 2011 at 11:44 AM, cheer_xiao <xiaqqaix@gmail.com> wrote:
>> Try the other way I suggested:
>>
>> _deb_packages_update_uninstalled () {
>>  _deb_packages_update_avail
>>  _deb_packages_update_installed
>>  if (( ! $+_deb_packages_cache_uninstalled )); then
>>    _dep_packages_cache_uninstalled=(
>>      $( print -l $_deb_packages_cache_avail |
>>         fgrep -vf =(print -l $_deb_packages_cache_installed)
>
> Seems you have missed a closing parenthesis at the end of line.
>
>>    )
>>  fi
>> }
>>
>
> It's a wonderful fix. zsh now respond within 2 seconds. Thanks a lot!
> I wish to see the fix committed into the source repo ASAP. Also the
> completion for apt-get should behave the same.

No... It didn't really work; the completion list was very, very
incomplete. "sudo aptitude install a<Tab>" now brings up a list of
packages current *installed* instead *available*...

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 4, 11:48am, cheer_xiao wrote:
}
} >> Try the other way I suggested:
} >>
} >> _deb_packages_update_uninstalled () {
} >>  _deb_packages_update_avail
} >>  _deb_packages_update_installed
} >>  if (( ! $+_deb_packages_cache_uninstalled )); then
} >>    _dep_packages_cache_uninstalled=(
} >>      $( print -l $_deb_packages_cache_avail |
} >>         fgrep -vf =(print -l $_deb_packages_cache_installed)
} >
} > Seems you have missed a closing parenthesis at the end of line.
} >
} >>    )
} >>  fi
} >> }
} >>
} >
} > It's a wonderful fix. zsh now respond within 2 seconds. Thanks a lot!
} > I wish to see the fix committed into the source repo ASAP. Also the
} > completion for apt-get should behave the same.
} 
} No... It didn't really work; the completion list was very, very
} incomplete. "sudo aptitude install a<Tab>" now brings up a list of
} packages current *installed* instead *available*...

Sorry about the cut'n'paste error with the close paren.

As for the other part, try fgrep -xvf.

If that still doesn't work, examine each of:

print -l $_deb_packages_cache_avail
print -l $_deb_packages_cache_installed
print -l $_dep_packages_cache_uninstalled

Do they appear to contain the lists you expect?

Re: Slow completion when using aptitude

[cheer_xiao]

On Fri, Feb 4, 2011 at 12:29 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> [snap]
> Sorry about the cut'n'paste error with the close paren.
>
> As for the other part, try fgrep -xvf.
>
> If that still doesn't work, examine each of:
>
> print -l $_deb_packages_cache_avail
> print -l $_deb_packages_cache_installed
> print -l $_dep_packages_cache_uninstalled

This three lists all seem to be sensible. BTW is
"dep_packages_cache_uninstalled" a typo?

>
> Do they appear to contain the lists you expect?

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 4,  1:46pm, cheer_xiao wrote:
} 
} is "dep_packages_cache_uninstalled" a typo?

Yes.

I don't actually have apt installed anywhere, so I can't test whether
the output of these functions are correct.  Let's try this again -- I
*believe* I've nixed all the typos but proofread before deployment:

_deb_packages_update_uninstalled () {
  _deb_packages_update_avail
  _deb_packages_update_installed
  if (( ! $+_deb_packages_cache_uninstalled )); then
    # Package lists too large to efficiently diff with zsh expansion
    _deb_packages_cache_uninstalled=(
      $( print -l $_deb_packages_cache_avail |
         fgrep -xvf =(print -l $_deb_packages_cache_installed) )
    )
  fi
}

Re: Slow completion when using aptitude

[cheer_xiao]

On Fri, Feb 4, 2011 at 11:04 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Feb 4,  1:46pm, cheer_xiao wrote:
> }
> } is "dep_packages_cache_uninstalled" a typo?
>
> Yes.
>
> I don't actually have apt installed anywhere, so I can't test whether
> the output of these functions are correct.  Let's try this again -- I
> *believe* I've nixed all the typos but proofread before deployment:
>
> _deb_packages_update_uninstalled () {
>  _deb_packages_update_avail
>  _deb_packages_update_installed
>  if (( ! $+_deb_packages_cache_uninstalled )); then
>    # Package lists too large to efficiently diff with zsh expansion
>    _deb_packages_cache_uninstalled=(
>      $( print -l $_deb_packages_cache_avail |
>         fgrep -xvf =(print -l $_deb_packages_cache_installed) )
>    )
>  fi
> }

Still, it doesn't work. The problem isn't here. When I type "sudo
aptitude install a" and hit Tab, zsh brings up the list of *installed*
packages instead of *uninstalled* ones - exactly the same list when I
type "sudo aptitude remove a" and hit Tab. Perhaps something is wrong
with _aptitude?

Re: Slow completion when using aptitude

[cheer_xiao]

On Sat, Feb 5, 2011 at 5:07 PM, cheer_xiao <xiaqqaix@gmail.com> wrote:
> [snip]
> Still, it doesn't work. The problem isn't here. When I type "sudo
> aptitude install a" and hit Tab, zsh brings up the list of *installed*
> packages instead of *uninstalled* ones - exactly the same list when I
> type "sudo aptitude remove a" and hit Tab. Perhaps something is wrong
> with _aptitude?

After examining _aptitude, I think the cause of the problem is most
likely that, _deb_packages isn't handing "_deb_packages uninstalled"
correctly. I suppose that it does update the corresponding cache by
calling "_deb_packages_update_uninstalled", but returned the wrong
"_deb_packages_cache_installed" as completion list, instead of
"..._uninstalled". I'm not familiar with zsh's completion programming
interface, but if nobody's got the time I'm willing to study it and
come up with a fix(in a few days).

Re: Slow completion when using aptitude

[cheer_xiao]

On Fri, Feb 4, 2011 at 11:04 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Feb 4,  1:46pm, cheer_xiao wrote:
> }
> } is "dep_packages_cache_uninstalled" a typo?
>
> Yes.
>
> I don't actually have apt installed anywhere, so I can't test whether
> the output of these functions are correct.  Let's try this again -- I
> *believe* I've nixed all the typos but proofread before deployment:
>
> _deb_packages_update_uninstalled () {
>  _deb_packages_update_avail
>  _deb_packages_update_installed
>  if (( ! $+_deb_packages_cache_uninstalled )); then
>    # Package lists too large to efficiently diff with zsh expansion
>    _deb_packages_cache_uninstalled=(
>      $( print -l $_deb_packages_cache_avail |
>         fgrep -xvf =(print -l $_deb_packages_cache_installed) )
>    )
>  fi

Well, the fix is here, simpler than I thought. You missed a line here:
 cachevar=_deb_packages_cache_uninstalled

Now "aptitude install a<Tab>" responds quickly(within a second) and
correctly. Thanks for your help, Bart! I have learn quite a bit shell
scripting along the way.
On the other hand, I think the performance can be further improved. A
delay of 1s is good enough for me but not good enough for some.

> }
>

Re: Slow completion when using aptitude

[gi1242+zsh]

On Sat, Feb 05, 2011 at 05:30:13PM +0800, cheer_xiao wrote:

> Well, the fix is here, simpler than I thought. You missed a line here:
> cachevar=_deb_packages_cache_uninstalled

Any chance you can post the complete corrected file (or a diff to
current git)? I haven't been following the full thread, but have the
same issue with aptitude, and will be happy to put the fixed version in
my installation.

Thanks,

GI

-- 
When you dream in color, it's a pigment of your imagination.

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 5, 12:12pm, gi1242+zsh@gmail.com wrote:
}
} Any chance you can post the complete corrected file (or a diff to
} current git)?

Index: Completion/Debian/Type/_deb_packages
===================================================================
diff -c -r1.7 _deb_packages
--- _deb_packages	4 Nov 2008 04:47:52 -0000	1.7
+++ _deb_packages	4 Feb 2011 15:44:35 -0000
@@ -71,8 +71,10 @@
   _deb_packages_update_avail
   _deb_packages_update_installed
   if (( ! $+_deb_packages_cache_uninstalled )); then
+    # Package lists too large to efficiently diff with zsh expansion
     _deb_packages_cache_uninstalled=(
-      ${_deb_packages_cache_avail:#${(j:|:)~${_deb_packages_cache_installed:q}}}
+      $( print -l $_deb_packages_cache_avail |
+         fgrep -xvf =(print -l $_deb_packages_cache_installed) )
     )
   fi
   cachevar=_deb_packages_cache_uninstalled

Re: Slow completion when using aptitude

[gi1242+zsh]

On Sat, Feb 05, 2011 at 09:48:41AM -0800, Bart Schaefer wrote:

>>
>> Any chance you can post the complete corrected file (or a diff to
>> current git)?
> 
> Index: Completion/Debian/Type/_deb_packages

Thanks. I can confirm that this "Works for me (TM)"

GI

-- 
BREAKFAST.COM Halted. Cereal Port Not Responding

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 5,  5:30pm, cheer_xiao wrote:
}
} Well, the fix is here, simpler than I thought. You missed a line here:
}  cachevar=_deb_packages_cache_uninstalled

Yeah, as soon as you said the package list hadn't changed when the
_dep -> _deb typo got fixed, I realized I'd failed to copy that
assignment from the original function.

} On the other hand, I think the performance can be further improved. A
} delay of 1s is good enough for me but not good enough for some.

There are a whole slew of completion functions that take 1s or more, so
if there are people who are unhappy with this one, they're going to be
unhappy in general.  My response would be, consider how long it would
take to get the same information if you didn't have the completion.

Also the second and subsequent times you use it in the same shell, it
should be as fast as it's theoretically possible for it to be without a
C-code level optimization deeper in compsys, because the arrays will
all have been populated already.

Re: Slow completion when using aptitude

[cheer_xiao]

On Sun, Jan 30, 2011 at 3:15 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Jan 30,  1:53pm, cheer_xiao wrote:
> }
> } I check the documentation that apparently there is no instruction on
> } how to load modules. Any hints?
>
> zmodload zsh/zprof
>
> The zmodload command ought to be cross-referenced from the "Zsh Modules"
> section, but there is an index entry for "loading modules" that points
> to zmodload.

Um... Could the documentation maintainer fix this?

Re: Slow completion when using aptitude

[Bart Schaefer]

On Feb 3,  6:23pm, cheer_xiao wrote:
} Subject: Re: Slow completion when using aptitude
}
} On Sun, Jan 30, 2011 at 3:15 PM, Bart Schaefer
} <schaefer@brasslantern.com> wrote:
} >
} > The zmodload command ought to be cross-referenced from the "Zsh Modules"
} > section, but there is an index entry for "loading modules" that points
} > to zmodload.
} 
} Um... Could the documentation maintainer fix this?

At the moment we can't fix anything (in any meaningful way) because
SourceForge was hacked and has shut off access to our code repository
until they're sure they've secured everything.  It's taking them a
lot longer than they predicted to get things back online.

ssh: connect to host zsh.cvs.sourceforge.net port 22: Connection refused