Avoid LDAP hit during completion

Avoid LDAP hit during completion

[Chris Wong]

[-- Attachment #1: Type: text/plain, Size: 465 bytes --]

For some reason, whenever I start a zsh shell, the first completion  
will trigger many many LDAP lookup.

% cd foo<TAB>

As soon as I hit TAB, the shell appears as frozen, but at the  
background, it's hitting the LDAP server of my company network.  This  
will go one for a couple minutes, at least.  tcpdump and top revealed  
that mystery to me.

Is there any workaround for this?  I'd hate to use bash after years  
of using zsh.

Thanks for any help!
Chris



[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2365 bytes --]

Re: Avoid LDAP hit during completion

[lists]

Hi,
     Is there anything in your $fpath that might cause this?  Is your  
'cd' a function or alias of your own design?  First try cd-ing after  
issuing 'zsh -f'.  That should work.  I don't know offhand why cd  
would make a call to LDAP as 'chown<TAB>' or 'ls -l' would.

Regards,
Ryan

On Jun 6, 2005, at 12:25 PM, Chris Wong wrote:

> For some reason, whenever I start a zsh shell, the first completion  
> will trigger many many LDAP lookup.
>
> % cd foo<TAB>
>
> As soon as I hit TAB, the shell appears as frozen, but at the  
> background, it's hitting the LDAP server of my company network.   
> This will go one for a couple minutes, at least.  tcpdump and top  
> revealed that mystery to me.
>
> Is there any workaround for this?  I'd hate to use bash after years  
> of using zsh.
>
> Thanks for any help!
> Chris
>
>
>

Re: Avoid LDAP hit during completion

[Chris wong]

Thanks for the quick reply, Ryan.

My $fpath has the standard value pointing to 
/usr/share/zsh/site-functions /usr/share/zsh/4.1.1/functions

cd is not aliased or redefined in any way.

I should have been more clear about it.  It doesn't just happen to cd, 
it happened to ls and any other commands that trigger file/dir 
completion.  Just a few minutes ago, I noticed that it doesn't 
necessarily happen at the first time of completion.

My zsh is running on Mac OS X 10.3 and the LDAP directory service it's 
hitting is Active Directory.

Chris

--
Chris
On Jun 6, 2005, at 11:16 AM, lists wrote:

> Hi,
>     Is there anything in your $fpath that might cause this?  Is your 
> 'cd' a function or alias of your own design?  First try cd-ing after 
> issuing 'zsh -f'.  That should work.  I don't know offhand why cd 
> would make a call to LDAP as 'chown<TAB>' or 'ls -l' would.
>
> Regards,
> Ryan
>
> On Jun 6, 2005, at 12:25 PM, Chris Wong wrote:
>
>> For some reason, whenever I start a zsh shell, the first completion 
>> will trigger many many LDAP lookup.
>>
>> % cd foo<TAB>
>>
>> As soon as I hit TAB, the shell appears as frozen, but at the 
>> background, it's hitting the LDAP server of my company network.  This 
>> will go one for a couple minutes, at least.  tcpdump and top revealed 
>> that mystery to me.
>>
>> Is there any workaround for this?  I'd hate to use bash after years 
>> of using zsh.
>>
>> Thanks for any help!
>> Chris
>>
>>
>>
>

Re: Avoid LDAP hit during completion

[Peter Stephenson]

Chris Wong wrote:
> For some reason, whenever I start a zsh shell, the first completion  
> will trigger many many LDAP lookup.
> 
> % cd foo<TAB>
> 
> As soon as I hit TAB, the shell appears as frozen, but at the  
> background, it's hitting the LDAP server of my company network.  This  
> will go one for a couple minutes, at least.  tcpdump and top revealed  
> that mystery to me.

Are you using the new completion system (with compinit)?  The best way
of tracking this down would be to see what the completion scripts are
doing.  You can arrange this with

_comp_setup="$_comp_setup
setopt xtrace"

This produces a huge amount of output so is useless for normal
completion but you should be able to see where the shell is pausing for
a long time.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: Avoid LDAP hit during completion

[Bart Schaefer]

On Jun 7, 10:26am, Peter Stephenson wrote:
} Subject: Re: Avoid LDAP hit during completion
}
} Chris Wong wrote:
} > For some reason, whenever I start a zsh shell, the first completion  
} > will trigger many many LDAP lookup.
} 
} Are you using the new completion system (with compinit)?  The best way
} of tracking this down would be to see what the completion scripts are
} doing.

It's almost certainly the zsh/parameter module looking up all possible
user names on the system to implement the $userdirs variable, which is
used by several of the completion functions including _expand and the
more obvious _users and _tilde_files.

Unfortunately there's no simple way to disable this.

Re: Avoid LDAP hit during completion

[lists]

Bart, this may be true, but I've used OS X 10.3 workstations bound to  
an LDAP domain containing 2000+ users and haven't seen problems like  
this unless I do something like 'chown<TAB>'.  Odd that Chris would  
run into it and I wouldn't if it were strictly due to user lookups.   
Chris, Peter's suggestions (adding xtrace to _comp_setup) sounds like  
a good thing to try.

-Ryan

On Jun 7, 2005, at 5:24 AM, Bart Schaefer wrote:

> On Jun 7, 10:26am, Peter Stephenson wrote:
> } Subject: Re: Avoid LDAP hit during completion
> }
> } Chris Wong wrote:
> } > For some reason, whenever I start a zsh shell, the first  
> completion
> } > will trigger many many LDAP lookup.
> }
> } Are you using the new completion system (with compinit)?  The  
> best way
> } of tracking this down would be to see what the completion scripts  
> are
> } doing.
>
> It's almost certainly the zsh/parameter module looking up all possible
> user names on the system to implement the $userdirs variable, which is
> used by several of the completion functions including _expand and the
> more obvious _users and _tilde_files.
>
> Unfortunately there's no simple way to disable this.
>

Re: Avoid LDAP hit during completion

[William Scott]

One possible approach is to use a modified _users completion function.
I haven't had the problem you are reporting, but I got tired of having
all the fake users like www  added to the completion list, so I have been
using this:

http://xanana.ucsc.edu/Library/init/zsh/local-functions/darwin/_users

HTH,

Bill





On Mon, 6 Jun 2005, Chris Wong wrote:

> For some reason, whenever I start a zsh shell, the first completion
> will trigger many many LDAP lookup.
>
> % cd foo<TAB>
>
> As soon as I hit TAB, the shell appears as frozen, but at the
> background, it's hitting the LDAP server of my company network.  This
> will go one for a couple minutes, at least.  tcpdump and top revealed
> that mystery to me.
>
> Is there any workaround for this?  I'd hate to use bash after years
> of using zsh.
>
> Thanks for any help!
> Chris
>
>
>

Re: Avoid LDAP hit during completion

[Chris Wong]

I'll try that today.  Unfortunately, it doesn't always freeze up at  
the first completion attempt.  I though it did for the first time,  
but found out otherwise.  It may have to do with the caching  
happening at the DirectoryService on Mac. It seems to always happen  
at the worst possible time such as doing a demo/presentation. :-)

Thanks
Chris

On Jun 7, 2005, at 2:26 AM, Peter Stephenson wrote:

> Chris Wong wrote:
>
>> For some reason, whenever I start a zsh shell, the first completion
>> will trigger many many LDAP lookup.
>>
>> % cd foo<TAB>
>>
>> As soon as I hit TAB, the shell appears as frozen, but at the
>> background, it's hitting the LDAP server of my company network.  This
>> will go one for a couple minutes, at least.  tcpdump and top revealed
>> that mystery to me.
>>
>
> Are you using the new completion system (with compinit)?  The best way
> of tracking this down would be to see what the completion scripts are
> doing.  You can arrange this with
>
> _comp_setup="$_comp_setup
> setopt xtrace"
>
> This produces a huge amount of output so is useless for normal
> completion but you should be able to see where the shell is pausing  
> for
> a long time.
>
> -- 
> Peter Stephenson <pws@csr.com>                  Software Engineer
> CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
> Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223  
> 692070
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
>
> **********************************************************************
>
>

Re: Avoid LDAP hit during completion

[Chris wong]

Peter, thanks for the xtrace trick.  Here is the partial output. The  
completion slow down (or appeared to be frozen) at _all_labels:39,  
(boxed up in asterisks below).  Any help would be much appreciated.

+_description:35> zstyle -a :completion::complete:cd::users  
ignored-patterns _comp_ignore
+_description:38> zstyle -s :completion::complete:cd::users ignore-line  
hidden
+_description:49> ((  1  ))
+_description:49> opts=( -F _comp_ignore )
+_description:54> tag=users
+_description:56> shift 2
+_description:57> [[ -z user ]]
+_description:59> [[ -n '' ]]
+_description:63> [[ -n '' ]]
+_description:70> [[ -n '' ]]
+_description:73> set -A expl -F _comp_ignore -J -default-
+_description:77> ((  0  ))
+_description:78> zstyle -a :completion::complete:cd::users fake match
+_description:88> return 0
************************************************************************ 
**
* +_all_labels:39> compadd -F _comp_ignore -J -default- -qS/ -k  
userdirs *
************************************************************************ 
**
+_all_labels:26> comptags -A users curtag __spec
+_all_labels:43> return __ret
+_wanted:9> _tags
+_tags:3> local prev
+_tags:10> [[ '' == -- ]]
+_tags:15> ((  0  ))
+_tags:66> comptags -N
+_wanted:13> return 1
+_tilde:24> _requested named-directories expl 'named directory' compadd  
-qS/ -k nameddirs
+_requested:3> local __gopt
+_requested:5> __gopt=( )
+_requested:6> zparseopts -D -a __gopt 1 2 V J x
+_requested:8> comptags -R named-directories
+_requested:9> [[ 7 -gt 3 ]]
+_requested:10> _all_labels - named-directories expl 'named directory'  
compadd -qS/ -k nameddirs
+_all_labels:3> local __gopt __len __tmp __pre __suf '__ret=1' __descr  
__spec __prev
+_all_labels:5> [[ - == - ]]
+_all_labels:6> __prev=-
+_all_labels:7> shift
+_all_labels:10> __gopt=( )
+_all_labels:11> zparseopts -D -a __gopt 1 2 V J x
+_all_labels:13> __tmp=8
+_all_labels:14> __len=7
+_all_labels:15> [[ __tmp -lt __len ]]
+_all_labels:18> [[ __tmp -eq 7 ]]
+_all_labels:22> __pre=4
+_all_labels:23> __suf=5
+_all_labels:26> comptags -A- named-directories curtag __spec
+_all_labels:27> ((  9 > _tags_level  ))
+_all_labels:28> _tags_level=9
+_all_labels:29> _comp_tags=' local-directories  users  users   
named-directories '
+_all_labels:30> [[ named-directories == '*[^\]:*' ]]
+_all_labels:37> _description named-directories expl 'named directory'

--
Chris

On Jun 7, 2005, at 9:27 AM, Chris Wong wrote:

> I'll try that today.  Unfortunately, it doesn't always freeze up at  
> the first completion attempt.  I though it did for the first time, but  
> found out otherwise.  It may have to do with the caching happening at  
> the DirectoryService on Mac. It seems to always happen at the worst  
> possible time such as doing a demo/presentation. :-)
>
> Thanks
> Chris
>
> On Jun 7, 2005, at 2:26 AM, Peter Stephenson wrote:
>
>> Chris Wong wrote:
>>
>>> For some reason, whenever I start a zsh shell, the first completion
>>> will trigger many many LDAP lookup.
>>>
>>> % cd foo<TAB>
>>>
>>> As soon as I hit TAB, the shell appears as frozen, but at the
>>> background, it's hitting the LDAP server of my company network.  This
>>> will go one for a couple minutes, at least.  tcpdump and top revealed
>>> that mystery to me.
>>>
>>
>> Are you using the new completion system (with compinit)?  The best way
>> of tracking this down would be to see what the completion scripts are
>> doing.  You can arrange this with
>>
>> _comp_setup="$_comp_setup
>> setopt xtrace"
>>
>> This produces a huge amount of output so is useless for normal
>> completion but you should be able to see where the shell is pausing  
>> for
>> a long time.
>>
>> -- 
>> Peter Stephenson <pws@csr.com>                  Software Engineer
>> CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
>> Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223  
>> 692070
>>
>>
>> **********************************************************************
>> This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity to whom they
>> are addressed. If you have received this email in error please notify
>> the system manager.
>>
>> **********************************************************************
>>
>>
>

Re: Avoid LDAP hit during completion

[lists]

There's userdirs as per Bart's prediction (although I'm not sure what  
part it plays here).  Weird that this hasn't bitten me before.   
However, given the fact that your problem doesn't always happen upon  
the first completion, I too wonder if DirectorySerivce, the network,  
or solar flares are also contributing.

-Ryan

On Jun 7, 2005, at 6:19 PM, Chris wong wrote:

> ********************************************************************** 
> ****
> * +_all_labels:39> compadd -F _comp_ignore -J -default- -qS/ -k  
> userdirs *
> ********************************************************************** 
> ****

Re: Avoid LDAP hit during completion

[William Scott]

I think this might be happening if you have configured OSX to consult
/etc/passwd by default and have those remote directories listed in there.
If they were in NetInfo or somewhere else non-canonical that zsh would not
see, then the problem would not arise on a network.

Maybe it would be easiest to reset the $userdirs array to something
tractable.



On Tue, 7 Jun 2005, lists wrote:

> There's userdirs as per Bart's prediction (although I'm not sure what
> part it plays here).  Weird that this hasn't bitten me before.
> However, given the fact that your problem doesn't always happen upon
> the first completion, I too wonder if DirectorySerivce, the network,
> or solar flares are also contributing.
>
> -Ryan
>
> On Jun 7, 2005, at 6:19 PM, Chris wong wrote:
>
> > **********************************************************************
> > ****
> > * +_all_labels:39> compadd -F _comp_ignore -J -default- -qS/ -k
> > userdirs *
> > **********************************************************************
> > ****
>

Re: Avoid LDAP hit during completion

[Bart Schaefer]

On Jun 7,  6:32pm, William Scott wrote:
} 
} Maybe it would be easiest to reset the $userdirs array to something
} tractable.

userdirs is read-only, and automatically populated by the zsh/parameter
module.  As I said, there's no easy workaround, particularly if zsh has
been compiled to use NIS; it's worse if the default system getpwent()
is implemented to do more than just read /etc/passwd, though I don't
know for of any specific case where this occurs.

Re: Avoid LDAP hit during completion

[Peter Stephenson]

Chris wong wrote:
> Peter, thanks for the xtrace trick.  Here is the partial output. The  
> completion slow down (or appeared to be frozen) at _all_labels:39,  
> (boxed up in asterisks below).  Any help would be much appreciated.

OK, so it was userdirs... You can relatively easily stop this happening
for most contexts by copying the _users function to somewhere earlier in
your $fpath and commenting out the last line.  You can still have
a list of specific users you want to complete, but it won't query the
system.  (If you have permission you can doctor _users in place, of course.)

There are still other places that use userdirs.  One is the _expand
completer.  If you use that you can do a similar trick by commenting
out the complete expression in braces:

  { [[ ( "$word" = \~* && ${#userdirs[(I)${word[2,-1]}*]}+${#nameddirs[(I)${word
[2,-1]}*]} -gt 1 ) ||
       ( "$word" = *\$[a-zA-Z0-9_]## && 
         ${#parameters[(I)${word##*\$}*]} -ne 1 ) ]] && continue=1 }

You can decide if you need to worry about _tilde_files.

I can see uses for a feature to override special parameters like
userdirs selectively.  Unfortunately the current state of the parameter
code makes implementing this very hairy.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070



**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: Avoid LDAP hit during completion

[Chris Wong]

Peter,

My apology for the late reply.  I was out of town for a few days.

I tried your suggested solution and now it doesn't seem to hit LDAP  
any more.

Thank you so much.

Chris

On Jun 8, 2005, at 2:35 AM, Peter Stephenson wrote:

> Chris wong wrote:
>
>> Peter, thanks for the xtrace trick.  Here is the partial output. The
>> completion slow down (or appeared to be frozen) at _all_labels:39,
>> (boxed up in asterisks below).  Any help would be much appreciated.
>>
>
> OK, so it was userdirs... You can relatively easily stop this  
> happening
> for most contexts by copying the _users function to somewhere  
> earlier in
> your $fpath and commenting out the last line.  You can still have
> a list of specific users you want to complete, but it won't query the
> system.  (If you have permission you can doctor _users in place, of  
> course.)
>
> There are still other places that use userdirs.  One is the _expand
> completer.  If you use that you can do a similar trick by commenting
> out the complete expression in braces:
>
>   { [[ ( "$word" = \~* && ${#userdirs[(I)${word[2,-1]}*]}+$ 
> {#nameddirs[(I)${word
> [2,-1]}*]} -gt 1 ) ||
>        ( "$word" = *\$[a-zA-Z0-9_]## &&
>          ${#parameters[(I)${word##*\$}*]} -ne 1 ) ]] && continue=1 }
>
> You can decide if you need to worry about _tilde_files.
>
> I can see uses for a feature to override special parameters like
> userdirs selectively.  Unfortunately the current state of the  
> parameter
> code makes implementing this very hairy.
>
> -- 
> Peter Stephenson <pws@csr.com>                  Software Engineer
> CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
> Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223  
> 692070
>
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
>
> **********************************************************************
>
>