expansion of environment variables

expansion of environment variables

[ruud grosmann]

hi list,

this is my first post. Hopefully this is not the kind of question that
is asked a dozen times before.... :)

I work on different machines and use zsh as my login shell. On my own
machine I can update software, on the others not. That is why most of
the machines contain zsh version 3.0 (according to the man page),
whereas my own machine has zsh 4.3.2 (i686-pc-linux-gnu).

The new version has a feature that is not in the 3.0 version. I would
like to disable the feature, but I am not sure if an option exist for
that one. Maybe you can help me.


The feature is about expansion of variables during file completion.
Assume my directory contains

1234.log   1234.xml  2345.xml

when I set variable key to '2345' and I do

$ /opt/archives/scripts/bin/convert $key.<TAB>

the OLD version expands this to

$ /opt/archives/scripts/bin/convert $key.xml

whereas the new version expands it to

$ /opt/archives/scripts/bin/convert 2345.xml

I would like to have the old behaviour in my new zsh, so that I can
reuse the command line for different values of key. Since it is a new
feature, I suspect there must be a way to prevent the shell from
substituting the variable with its value. Am I overlooking it in the
docs?

thanks in advance, Ruud

Re: expansion of environment variables

[Bart Schaefer]

On Mar 5, 11:35am, ruud grosmann wrote:
}
} The new version has a feature that is not in the 3.0 version. I would
} like to disable the feature, but I am not sure if an option exist for
} that one. Maybe you can help me.

I don't know why this would have changed, but it does not appear that
there is any way to turn it off.  Both the expand-or-complete and the
complete-word widgets expand the variable; I would have expected the
latter not to.

Worse, if I enable the new completion system with 'compinit' then the
variable doesn't expand but the completion doesn't work.  I mention
this because with the new completion system enabled, the 'suffix'
zstyle is supposed to control exactly what Ruud is asking about, and
the default is to have the "old" behavior that he wants.
 
For benefit of zsh-workers, with compinit this is yet another problem
in _path_files.  At line 479 (latest CVS) it correctly assigns the
expanded list of files to the tmp2 array (having originally gotten
them from "compadd -D tmp1" at line 466), but then when building the
exppaths array at lines 484 or 486, it uses ${tpre}${tsuf} which have
never been expanded.

I'm also not sure how line 484 can ever be reached, since at 479
"${(@)tmp2:h}" is always going to strip off any trailing slash.  It
seems a bit strange to compare an entire array to a single pattern
at line 483 as well, but I guess it works since the last element of
the array should be representative of any single element.

Re: expansion of environment variables

[Benjamin R. Haskell]

On Thu, 5 Mar 2009, ruud grosmann wrote:

> [...]
>
> The new version has a feature that is not in the 3.0 version. I would
> like to disable the feature, but I am not sure if an option exist for
> that one. Maybe you can help me.
>
>
> The feature is about expansion of variables during file completion.
> Assume my directory contains
>
> 1234.log   1234.xml  2345.xml
>
> when I set variable key to '2345' and I do
>
> $ /opt/archives/scripts/bin/convert $key.<TAB>
>
> the OLD version expands this to
>
> $ /opt/archives/scripts/bin/convert $key.xml
>
> whereas the new version expands it to
>
> $ /opt/archives/scripts/bin/convert 2345.xml
>
> I would like to have the old behaviour in my new zsh, so that I can
> reuse the command line for different values of key. Since it is a new
> feature, I suspect there must be a way to prevent the shell from
> substituting the variable with its value. Am I overlooking it in the
> docs?

I don't get the "new" behavior in zsh-4.3.6 for a similar example with the 
following options set:

$ setopt
autocd
noautomenu
nocheckjobs
extendedglob
extendedhistory
globdots
histignoredups
histignorespace
histnostore
histsubstpattern
nohup
incappendhistory
interactive
monitor
nonotify
promptsubst
pushdsilent
shinstdin
zle
# output of allopt at: http://benizi.com/zsh-opt-2009-03-05.txt

$ key=tmp
$ l $key/zzz.<TAB>
zzz.php  zzz.txt


But, I also get the following behaviors, which seem odd (though not 
inexplicable) to me:

$ key=tmp/zzz
$ l $key.<TAB>
# (doesn't offer any completions)
$ l ${key}.<TAB>
# no completions


$ key=tm
$ l ${key}p/z<TAB>
$ l ${key}p/zzz.

I'm sure this is an uncommon problem to have [wanting to tab-complete a 
variable that contains a partial *filename*, rather than just a partial 
path].  It just strikes me as weird that it's not consistently doing:

1. expand variable, but don't insert that expansion into the line
2. attempt completion


Another example:

$ mkdir -p aaa/bbb/ccc/ddd
$ touch aaa/bbb/ccc/ddd/eee.one
$ touch aaa/bbb/ccc/ddd/eee.two

$ key=aaa/bbb/ccc/ddd/eee
$ l ${key}.<TAB>
# nothing

$ key=aaa/bbb/ccc/d
$ l ${key}d<TAB>
# nothing

$ l ${key}dd<TAB>
# nothing

$ l ${key}dd/<TAB>
# completes to ambiguity point:
$ l ${key}dd/eee.<TAB>
eee.one  eee.two

Best,
Ben

Re: expansion of environment variables

[Mikael Magnusson]

2009/3/5 Bart Schaefer <schaefer@brasslantern.com>:
> On Mar 5, 11:35am, ruud grosmann wrote:
> }
> } The new version has a feature that is not in the 3.0 version. I would
> } like to disable the feature, but I am not sure if an option exist for
> } that one. Maybe you can help me.
>
> I don't know why this would have changed, but it does not appear that
> there is any way to turn it off.  Both the expand-or-complete and the
> complete-word widgets expand the variable; I would have expected the
> latter not to.
>
> Worse, if I enable the new completion system with 'compinit' then the
> variable doesn't expand but the completion doesn't work.  I mention
> this because with the new completion system enabled, the 'suffix'
> zstyle is supposed to control exactly what Ruud is asking about, and
> the default is to have the "old" behavior that he wants.
>
> For benefit of zsh-workers, with compinit this is yet another problem
> in _path_files.  At line 479 (latest CVS) it correctly assigns the
> expanded list of files to the tmp2 array (having originally gotten
> them from "compadd -D tmp1" at line 466), but then when building the
> exppaths array at lines 484 or 486, it uses ${tpre}${tsuf} which have
> never been expanded.
>
> I'm also not sure how line 484 can ever be reached, since at 479
> "${(@)tmp2:h}" is always going to strip off any trailing slash.  It
> seems a bit strange to compare an entire array to a single pattern
> at line 483 as well, but I guess it works since the last element of
> the array should be representative of any single element.

fwiw it seems to work if $key is a directory, i correctly get
$key/<files here> without $key being expanded. But with a . instead of
a /, i get the same result, that nothing happens.

-- 
Mikael Magnusson

Re: expansion of environment variables

[Peter Stephenson]

On Thu, 05 Mar 2009 08:15:48 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:

(re

% touch stuff.txt
% foo=stuff
% echo $foo.<TAB>

etc.)

> Worse, if I enable the new completion system with 'compinit' then the
> variable doesn't expand but the completion doesn't work.  I mention
> this because with the new completion system enabled, the 'suffix'
> zstyle is supposed to control exactly what Ruud is asking about, and
> the default is to have the "old" behavior that he wants.
>  
> For benefit of zsh-workers, with compinit this is yet another problem
> in _path_files.  At line 479 (latest CVS) it correctly assigns the
> expanded list of files to the tmp2 array (having originally gotten
> them from "compadd -D tmp1" at line 466), but then when building the
> exppaths array at lines 484 or 486, it uses ${tpre}${tsuf} which have
> never been expanded.

That's too late---it's failed at that point.  See the comment above that
section.  We're in the code that triggers where compfiles (the builtin that
handles the globbing) didn't produce any matches so $#tmp1 is empty.  I
think it's just attempting to expand what it thinks is a plain string chunk
of path, so that things like $foo/<TAB> work.

> I'm also not sure how line 484 can ever be reached, since at 479
> "${(@)tmp2:h}" is always going to strip off any trailing slash.

I don't know about this...

> It
> seems a bit strange to compare an entire array to a single pattern
> at line 483 as well, but I guess it works since the last element of
> the array should be representative of any single element.

...but I think this is related to the fact that we don't have any glob
matches, just a chunk of path that may want something doing to it.

It looks to me like this is a basic problem that's hard to fix: you'd need
an eval'd version of the prefix and suffix to do the globbing on, and then
you'd need to be able to put the expanded bits of it back the way they were
at the end.  In general the only way of doing this would be to expand and
save each variable reference, command expansion, numeric expansion, etc.,
separately.

I suspect, by the way, that the original query was about the old builtin
completion, not the function system.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: expansion of environment variables

[Bart Schaefer]

On Mar 5,  4:54pm, Peter Stephenson wrote:
}
} > For benefit of zsh-workers, with compinit this is yet another problem
} > in _path_files.  At line 479 (latest CVS) it correctly assigns the
} > expanded list of files to the tmp2 array (having originally gotten
} > them from "compadd -D tmp1" at line 466), but then when building the
} > exppaths array at lines 484 or 486, it uses ${tpre}${tsuf} which have
} > never been expanded.
} 
} That's too late---it's failed at that point.  See the comment above that
} section.  We're in the code that triggers where compfiles (the builtin that
} handles the globbing) didn't produce any matches so $#tmp1 is empty.

Hmm, I misread something, but I think you did as well.

The call to "compfiles -p" at line 418 is what fills in tmp1.  It DOES
find matching files, because at line 466 we save tmp1 in tmp2 and at
that point tmp1 is the expanded list.  Then at 467 we call compadd -D
passing only the tails of what was expanded in commpfiles, and compadd
empties tmp1 again.

So I suspect this is more of the same foolishness that forced us to add
$Uopt a bit ago, but manifesting in a different way.  Or maybe we need
to pass some prefix options to compadd at that point?

} I suspect, by the way, that the original query was about the old builtin
} completion, not the function system.

Yes, I know.  I meant to say "the expand-word and complete-word builtin
widgets" in the first paragraph of my first reply, but thinko'd.

Re: expansion of environment variables

[Peter Stephenson]

Bart Schaefer wrote:
> The call to "compfiles -p" at line 418 is what fills in tmp1.  It DOES
> find matching files, because at line 466 we save tmp1 in tmp2 and at
> that point tmp1 is the expanded list.  Then at 467 we call compadd -D
> passing only the tails of what was expanded in commpfiles, and compadd
> empties tmp1 again.

You're right, I missed that last step, there is indeed stuff in tmp1
including the correct completion.

> So I suspect this is more of the same foolishness that forced us to add
> $Uopt a bit ago, but manifesting in a different way.  Or maybe we need
> to pass some prefix options to compadd at that point?

It may even have briefly worked when -U was passed to all relevant
compadds in _path_files, but we've now restricted it to approximation.
I can't remember why, but I seem to remember it was something which
should in principle have worked but no one understood what the internals
were doing.

Something clever with prefixes might fix it, only I don't know
what.  There are two problems:  first check that something needs
expanding, next do it safely.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070