zsh-workers
 help / color / mirror / code / Atom feed
* Re: PATCH: completion caching layer
@ 2000-07-27  8:00 Sven Wischnowsky
  2000-08-01 16:31 ` Adam Spiers
  0 siblings, 1 reply; 16+ messages in thread
From: Sven Wischnowsky @ 2000-07-27  8:00 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> ...
> 
> > Shouldn't there be some way to flush the caches (in _retrieve_cache)?
> > I.e. some style that says when the caches should be rebuilt.
> > 
> > This could be done either by a style with values that describe update
> > policies sensible in enough possible contexts or by using a boolean
> > style and let users use `zstyle -e' to implement their own policy (and 
> > of course users could do that in the first case, too, but there may be 
> > ways to make this more convenient for common cases).
> 
> This sounds very nice, but I'm not sure what sort of policies you have
> in mind, and how they should be implement.  Could you give a few
> examples?

Of course, that's the problem. One that may be useful quite often
would compare the mtime of a file and re-build the cache when it has
changed.

In some cases (when using data that gets updated after regular time
intervals) wall-clock time comparison may be good (with that it would
probably be possible to sensibly cache YP data if we would need that).

However, I don't think there can ever be a complete list. Maybe we
should just give _retrieve_cache an optional argument which is
evaluated or a function name or something that gets called to test if
the cache should be re-build. We probably just need more experience...

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: PATCH: completion caching layer
@ 2000-08-04  6:59 Sven Wischnowsky
  0 siblings, 0 replies; 16+ messages in thread
From: Sven Wischnowsky @ 2000-08-04  6:59 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> ...
> 
> Now, it's not hard to imagine a system which doesn't have any CPAN
> modules installed.  In that case, I was worried about the above test
> failing because _perl_CPAN_modules would be undefined.  However there
> appears to be an easy solution, which is just to ensure that all
> cache-related parameters are always defined after retrieval, even if
> they're defined to the empty array or string.  Does that sound OK?

Yes, I think (we do this thing of `empty is the same as undefined'
elsewhere anyway).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: PATCH: completion caching layer
@ 2000-08-03  7:11 Sven Wischnowsky
  0 siblings, 0 replies; 16+ messages in thread
From: Sven Wischnowsky @ 2000-08-03  7:11 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> ...
>
> > > I agree entirely.  That way also I could make _perl_modules set a
> > > default style (if one is not already set) when the function is loaded,
> > > rather than each time it's invoked.  But where would it appear in the
> > > context?  My knowledge of this stuff is slightly weak, I'm afraid.
> > 
> > After the last colon:
> > 
> >   zstyle -t ":completion:${curcontext}:" ...
> >                                        ^here
> > 
> > I.e., instead of the tag (if you would use tags).
> 
> OK.  Although that doesn't solve the problem of having to set a
> default for each of the commands _perl_modules completes.  Maybe
> 
>   zstyle ':completion:::::RPMs' cache-policy _rpms_caching_policy
> 
> would work?

For use by the completion function itself? We decided some time ago
that they shouldn't set styles themselves, only look them up.

So, instead of adding what you suggest (which most probably wouldn't
work because there are other things in the context, e.g. the command
name), one would use the return value of zstyle:

  zstyle -s ":completion:${curcontext}:RPMs" cache-policy foo ||
      foo=_rpms_caching_policy

I.e., the return value of zstyle is non-zero iff the style is not set
for that context.


> Incidentally I couldn't find anything in the documentation to explain
> the difference between e.g. ::: and :*:*: in contexts.  Did I miss it?

Err. Pattern matching. `:::' matches only itself, `:*:*:' matches
`::perl:' and `::perl:argument-rest' and `:complete:perl:argument-rest' 
and `predict:complete:perl:argument-rest'.

The patterns given when defining styles are really only matched agains 
the context-strings given when looking up styles. Normal shell pattern 
matching, nothing special there. Only the sorting of the patterns is
something not used elsewhere.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: PATCH: completion caching layer
@ 2000-08-02 14:13 Sven Wischnowsky
  2000-08-02 15:01 ` Adam Spiers
  0 siblings, 1 reply; 16+ messages in thread
From: Sven Wischnowsky @ 2000-08-02 14:13 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> Sven Wischnowsky (wischnow@informatik.hu-berlin.de) wrote:
> > It's a bit unfortunate that _cache_invalid can be called twice (in
> > your examples), once directly and once from _retrieve_cache. I think.
> 
> Yes, I didn't like that either, but couldn't think of a better
> design.  The problem is that there are actually two caching layers -
> the parameters, and the cache files on disk, but the _cache_invalid
> check needs to be invoked if either is about to be used.  Suggestions
> for how to avoid this welcome.

Couldn't we stuff everything in _retrieve_cache? So that one only
needs to call:

  if ! _retrieve_cache RPMs _rpms; then
    _rpms=(...)
    _store_cache RPMs _rpms
  fi

> ...
> 
> > About the lookup: I /think/ it would be more convenient if the type of 
> > information cached would appear in the context, so that you could say
> > `zstyle ":completion:*:rpms" cache-policy ...'. Haven't really played
> > with it yet, though.
> 
> I agree entirely.  That way also I could make _perl_modules set a
> default style (if one is not already set) when the function is loaded,
> rather than each time it's invoked.  But where would it appear in the
> context?  My knowledge of this stuff is slightly weak, I'm afraid.

After the last colon:

  zstyle -t ":completion:${curcontext}:" ...
                                       ^here

I.e., instead of the tag (if you would use tags). One problem is your
naming scheme (upper-case) which is different from what we've used so
far.

And then the tags should be documented, or similar tags which are
already used elsewhere could be used for caching.

> > And another thing: with `zstyle -e' one could use a boolean style
> > `cache-invalid' or whatever and let the user do the rest. It's hard to 
> > give arguments to that, though. Other than by documenting
> > $_cache_path, that is.
> 
> I think I understand that, but how would it be better than the current
> system?

Maybe hopefully consistency(?) I'm not too sure about this either,
since there are still other styles which allow to give names of
functions to be called (tag-order, for example).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: PATCH: completion caching layer
@ 2000-08-02  9:36 Sven Wischnowsky
  2000-08-02 13:35 ` Adam Spiers
  0 siblings, 1 reply; 16+ messages in thread
From: Sven Wischnowsky @ 2000-08-02  9:36 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> ...
> 
> OK, I've implemented this via the 'cache-policy' style, which should
> contain the name of a function which returns zero iff the cache needs
> rebuilding.  _cache_invalid is a helper function which makes it easy
> to perform this calculation.
> 
> Here's the patch (with documentation too, woohoo!)  It's quite big,
> mainly because of reindentation.  I'd be very grateful if one or two
> people could give it a quick thrashing to see whether I've
> missed/broken anything (it seems fine for me but ...)  I'll commit it
> if it works for someone else or if I get bored of waiting for people
> to try :-)

It's a bit unfortunate that _cache_invalid can be called twice (in
your examples), once directly and once from _retrieve_cache. I think.

And why do _cache_invalid and _retrieve_cache re-define themselves?
There's nothing else in the files (_retrieve_cache doesn't even call
its new definition). Or have I missed something?

And the styles or not in the style list in compsys.yo. Nor in _zstyle
(I always forget that, too ;-).

About the lookup: I /think/ it would be more convenient if the type of 
information cached would appear in the context, so that you could say
`zstyle ":completion:*:rpms" cache-policy ...'. Haven't really played
with it yet, though.

And another thing: with `zstyle -e' one could use a boolean style
`cache-invalid' or whatever and let the user do the rest. It's hard to 
give arguments to that, though. Other than by documenting
$_cache_path, that is.


Minor comment: `foo && return 0' is both smaller and faster than
`if foo; then return 0; fi'. But since I may have done that myself,
I'll better not say this too loudly ;-}

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: PATCH: completion caching layer
@ 2000-07-19 14:48 Sven Wischnowsky
  2000-07-26 19:53 ` Adam Spiers
  0 siblings, 1 reply; 16+ messages in thread
From: Sven Wischnowsky @ 2000-07-19 14:48 UTC (permalink / raw)
  To: zsh-workers


Adam Spiers wrote:

> OK, I've written the first draft of the completion caching layer I
> mentioned I was thinking about ages ago.
> 
> It uses two styles: use-cache (yes/no/1/0) and cache-path (the
> directory for the cache, defaults to ~/.zsh/cache).  I also included
> patches for _rpm and _perl_modules to show example uses.  I think
> you'll agree the interface is very simple.  Everything else should (!)
> be crystal clear; if not, please ask.
> 
> The patch follows.  I won't commit until someone else has gone through
> it and thinks it looks OK.  I'll also write some documentation at some
> point ...

Shouldn't there be some way to flush the caches (in _retrieve_cache)?
I.e. some style that says when the caches should be rebuilt.

This could be done either by a style with values that describe update
policies sensible in enough possible contexts or by using a boolean
style and let users use `zstyle -e' to implement their own policy (and 
of course users could do that in the first case, too, but there may be 
ways to make this more convenient for common cases).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 16+ messages in thread
* PATCH: completion caching layer
@ 2000-07-17 14:38 Adam Spiers
  0 siblings, 0 replies; 16+ messages in thread
From: Adam Spiers @ 2000-07-17 14:38 UTC (permalink / raw)
  To: zsh workers mailing list

OK, I've written the first draft of the completion caching layer I
mentioned I was thinking about ages ago.

It uses two styles: use-cache (yes/no/1/0) and cache-path (the
directory for the cache, defaults to ~/.zsh/cache).  I also included
patches for _rpm and _perl_modules to show example uses.  I think
you'll agree the interface is very simple.  Everything else should (!)
be crystal clear; if not, please ask.

The patch follows.  I won't commit until someone else has gone through
it and thinks it looks OK.  I'll also write some documentation at some
point ...

Adam

P.S. I know we decided against caching for _rpm, as any use of the rpm
command is likely to change the rpm database anyway.  However, it
really is important that we provide the user the choice on this issue
(as the patch does via styles); for instance running rpm -qa on my
laptop is excrutiatingly slow, and there's little point providing
correct completions when you could type the whole rpm name manually
faster.


Index: Completion/Base/.distfiles
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Base/.distfiles,v
retrieving revision 1.4
diff -u -r1.4 .distfiles
--- Completion/Base/.distfiles	2000/05/30 09:30:08	1.4
+++ Completion/Base/.distfiles	2000/07/17 14:31:30
@@ -2,6 +2,6 @@
     .distfiles 
     _arg_compile _arguments _brace_parameter _combination
     _command_names _condition _default _describe _equal _first _in_vared
-    _jobs _math _parameter _precommand _redirect _regex_arguments _subscript
-    _tilde _value _values
+    _jobs _math _parameter _precommand _redirect _regex_arguments _retrieve_cache
+    _store_cache _subscript _tilde _value _values
 '
Index: Completion/Base/_retrieve_cache
===================================================================
RCS file: _retrieve_cache
diff -N _retrieve_cache
--- /dev/null	Tue May  5 13:32:27 1998
+++ _retrieve_cache	Mon Jul 17 07:31:30 2000
@@ -0,0 +1,29 @@
+#autoload
+#
+# Retrieval component of completions caching layer
+
+local _cache_filename
+_cache_filename="$1"
+
+if zstyle -t ":completion:${curcontext}:" use-cache; then
+  # Decide which directory to retrieve cache from, and ensure it exists
+  zstyle -s ":completion:${curcontext}:" cache-path _cache_dir
+  : ${_cache_dir:=~/.zsh/cache}
+  if [[ ! -d "$_cache_dir" ]]; then
+    if [[ -e "$_cache_dir" ]]; then
+      _message "cache-dir ($_cache_dir) isn't a directory\!"
+      return 1
+    else
+      return 1
+    fi
+  fi
+
+  if [[ -e "$_cache_dir/$_cache_filename" ]]; then
+    . "$_cache_dir/$_cache_filename"
+    return 0
+  else
+    return 1
+  fi
+else
+  return 1
+fi
Index: Completion/Base/_store_cache
===================================================================
RCS file: _store_cache
diff -N _store_cache
--- /dev/null	Tue May  5 13:32:27 1998
+++ _store_cache	Mon Jul 17 07:31:30 2000
@@ -0,0 +1,35 @@
+#autoload
+#
+# Storage component of completions caching layer
+
+local _cache_filename
+_cache_filename="$1"
+
+if zstyle -t ":completion:${curcontext}:" use-cache; then
+  # Decide which directory to cache to, and ensure it exists
+  zstyle -s ":completion:${curcontext}:" cache-path _cache_dir
+  : ${_cache_dir:=~/.zsh/cache}
+  if [[ ! -d "$_cache_dir" ]]; then
+    if [[ -e "$_cache_dir" ]]; then
+      _message "cache-dir style points to a non-directory\!"
+    else
+      mkdir -p "$_cache_dir"
+      if [[ ! -d "$_cache_dir" ]]; then
+        _message "Couldn't create cache-dir $_cache_dir"
+        return 1
+      fi
+    fi
+  fi
+
+  # grr, doesn't work, so we have to roll our own
+#  typeset "$@[2,-1]" > "$_cache_dir/$_cache_filename"
+
+  # only deals with arrays so far
+  for var in "$@[2,-1]"; do
+    print -n "$var=("
+    print -nr - "${(o@P)${var}}"
+    print ")\n"
+  done > "$_cache_dir/$_cache_filename"
+else
+  return 1
+fi
Index: Completion/Linux/_rpm
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Linux/_rpm,v
retrieving revision 1.15
diff -u -r1.15 _rpm
--- Completion/Linux/_rpm	2000/07/12 09:01:41	1.15
+++ Completion/Linux/_rpm	2000/07/17 14:31:30
@@ -203,8 +203,12 @@
     state=package_file
     ;&
   package)
+    if ! _retrieve_cache RPMs; then
+      _rpms=( $(_call packages rpm -qa 2>/dev/null) )
+      _store_cache RPMs _rpms
+    fi
     _wanted packages expl 'RPM package' \
-        compadd -M 'r:|-=* r:|=*' - $(_call packages rpm -qa 2> /dev/null) && ret=0
+        compadd -M 'r:|-=* r:|=*' - "$_rpms[@]" && ret=0
     ;;
   spec_file)
     _wanted specfiles expl 'spec file' \
cvs server: Diffing Completion/User
Index: Completion/User/_perl_modules
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/User/_perl_modules,v
retrieving revision 1.6
diff -u -r1.6 _perl_modules
--- Completion/User/_perl_modules	2000/05/31 09:38:26	1.6
+++ Completion/User/_perl_modules	2000/07/17 14:31:30
@@ -21,42 +21,46 @@
 zparseopts -D -a opts S: q
 
 if [[ ${+_perl_modules} -eq 0 ]]; then
-  if zstyle -t ":completion:${curcontext}:modules" try-to-use-pminst \
-     && (( ${+commands[pminst]} )); then
-    _perl_modules=( $(pminst) )
-  else
-    local inc libdir new_pms
-    if (( ${+commands[perl]} )); then
-      inc=( $( perl -e 'print "@INC"' ) )
+  if ! _retrieve_cache perl_modules; then
+    if zstyle -t ":completion:${curcontext}:modules" try-to-use-pminst \
+       && (( ${+commands[pminst]} )); then
+      _perl_modules=( $(pminst) )
     else
-      # If perl isn't there, one wonders why the user's trying to
-      # complete Perl modules.  Maybe her $path is wrong?
-      _message "Didn't find perl on \$PATH; guessing @INC ..."
-
-      setopt localoptions extendedglob
-      inc=( /usr/lib/perl5{,/{site_perl/,}<5->.([0-9]##)}(N) 
-            ${(s.:.)PERL5LIB} )
-    fi
-
-    typeset -agU _perl_modules  # _perl_modules is global, no duplicates
-    _perl_modules=( )
-
-    for libdir in $inc; do
-      # Ignore cwd - could be too expensive e.g. if we're near /
-      if [[ $libdir == '.' ]]; then break; fi
-
-      # Find all modules
-      if [[ -d $libdir && -x $libdir ]]; then
-      cd $libdir
-      new_pms=( {[A-Z]*/***/,}*.pm~*blib*(N) )
-      cd $OLDPWD
+      local inc libdir new_pms
+      if (( ${+commands[perl]} )); then
+        inc=( $( perl -e 'print "@INC"' ) )
+      else
+        # If perl isn't there, one wonders why the user's trying to
+        # complete Perl modules.  Maybe her $path is wrong?
+        _message "Didn't find perl on \$PATH; guessing @INC ..."
+  
+        setopt localoptions extendedglob
+        inc=( /usr/lib/perl5{,/{site_perl/,}<5->.([0-9]##)}(N) 
+              ${(s.:.)PERL5LIB} )
       fi
-
-      # Convert to Perl nomenclature
-      new_pms=( ${new_pms:r:fs#/#::#} )
+  
+      typeset -agU _perl_modules  # _perl_modules is global, no duplicates
+      _perl_modules=( )
+  
+      for libdir in $inc; do
+        # Ignore cwd - could be too expensive e.g. if we're near /
+        if [[ $libdir == '.' ]]; then break; fi
+  
+        # Find all modules
+        if [[ -d $libdir && -x $libdir ]]; then
+        cd $libdir
+        new_pms=( {[A-Z]*/***/,}*.pm~*blib*(N) )
+        cd $OLDPWD
+        fi
+  
+        # Convert to Perl nomenclature
+        new_pms=( ${new_pms:r:fs#/#::#} )
+  
+        _perl_modules=( $new_pms $_perl_modules )
+      done
+    fi
 
-      _perl_modules=( $new_pms $_perl_modules )
-    done
+    _store_cache perl_modules _perl_modules
   fi
 fi
 


^ permalink raw reply	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2000-08-04  7:00 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2000-07-27  8:00 PATCH: completion caching layer Sven Wischnowsky
2000-08-01 16:31 ` Adam Spiers
2000-08-01 17:03   ` Bart Schaefer
2000-08-01 17:09     ` Bart Schaefer
2000-08-01 23:37     ` Adam Spiers
2000-08-02  3:53       ` Bart Schaefer
  -- strict thread matches above, loose matches on Subject: below --
2000-08-04  6:59 Sven Wischnowsky
2000-08-03  7:11 Sven Wischnowsky
2000-08-02 14:13 Sven Wischnowsky
2000-08-02 15:01 ` Adam Spiers
2000-08-03 12:21   ` Adam Spiers
2000-08-02  9:36 Sven Wischnowsky
2000-08-02 13:35 ` Adam Spiers
2000-07-19 14:48 Sven Wischnowsky
2000-07-26 19:53 ` Adam Spiers
2000-07-17 14:38 Adam Spiers

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).