Re: bug with chpwd/allexport (plus expansion/substitution remarks)

Re: bug with chpwd/allexport (plus expansion/substitution remarks)

[Bart Schaefer]

On Aug 15,  8:25am, TGAPE! wrote:
} Subject: Re: bug with chpwd/allexport (plus expansion/substitution remarks
}
} I know this was a while ago, but I frequently lack time...

I know what you mean.

} Bart Schaefer wrote:
} > On Jul 17,  2:42pm, Zefram wrote:
} >} Dominik Vogt wrote:
} >}>                        BTW if the second workaround you suggest works,
} >}>then it is a bug. The manpage states clearly, that a local variable is
} >}>still exported if the ALLEXPORT option is set and the variable was
} >}>prviously unset.
} >} 
} >} But with `local', the variable disappears at the end of the function,
} > 
} > That also doesn't affect whether the name is added to the named directory
} > hash table.  Unsetting the variable doesn't unhash the name; making it a
} > local has no effect on this problem whatsoever.
} 
} It doesn't?  It seems to in 3.1.1

Indeed, to say "no effect whatsoever" is an overstatement; I was misled by
performing my tests with a variable that already had a value.  Here's the
actual situation:

IF a variable is already set to a name that begins with a slash AND THEN
a local of the same name is added to the named directory hash table, the
result is that the original variable remains in the directory hash table
after the function exits.  Otherwise, unset of a variable including of a
local on exit from the function, removes the corresponding hash entry.

There's a slight difference between 3.0.5 and 3.1.4 in the handling of an
unset local, illustrated below.

I originally tried only the equivalent of namedir4 in test 2, which caused
me to make the overstatement excerpted above.

Test functions:

    namedir1 () {
	setopt localoptions autonamedirs
	local $1=$2
	echo namedir1: $(hash -d | grep \^$1)
    }
    namedir2 () {
    	local $1=$2
	namedir1 $1 $2
	echo namedir2: $(hash -d | grep \^$1)
    }
    namedir3 () {
        local $1
    	unset $1
	namedir1 $1 $2
	echo namedir3: $(hash -d | grep \^$1)
    }
    namedir4 () {
	namedir1 $1 $2
	echo namedir4: $(hash -d | grep \^$1)
    }
    namedir_test () {
	echo $ZSH_NAME $ZSH_VERSION
	unset test1
	test2=/usr/bin
	echo TEST 1
	namedir1 test1 /etc
	namedir2 test1 /etc
	namedir3 test1 /etc
	namedir4 test1 /etc
	echo TEST 2
	namedir1 test2 /etc
	namedir2 test2 /etc
	namedir3 test2 /etc
	namedir4 test2 /etc
    }

Results with zsh -f:

zagzig% namedir_test
zsh 3.0.5-extended
TEST 1
namedir1: test1=/etc
namedir1: test1=/etc
namedir2: test1=/etc
namedir1: test1=/etc
namedir3:
namedir1: test1=/etc
namedir4:
TEST 2
namedir1: test2=/etc
namedir1: test2=/etc
namedir2: test2=/etc
namedir1: test2=/etc
namedir3:
namedir1: test2=/etc
namedir4: test2=/usr/bin

zagzig% namedir_test
zsh 3.1.4
TEST 1
namedir1: test1=/etc
namedir1: test1=/etc
namedir2: test1=/etc
namedir1: test1=/etc
namedir3:
namedir1: test1=/etc
namedir4:
TEST 2
namedir1: test2=/etc
namedir1: test2=/etc
namedir2: test2=/etc
namedir1: test2=/etc
namedir3: test2=/usr/bin
namedir1: test2=/etc
namedir4: test2=/usr/bin

Note the difference in the handling of "local" in the namedir3 test from
3.0.5 to 3.1.4.  I tend to doubt that this discrepancy is intentional.

} Besides, it certainly seems to me that it should; if a variable doesn't
} exist, how can a reference to that variable be valid?  (Yeah, I know
} hashes aren't references to variables, even with autonamedirs, but I'm
} trying to look at this from the user's perspective.)

I agree with you.


-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: bug with chpwd/allexport

[Dominik Vogt]

> >         The problem is, that is you have the ALLEXPORT and the
> >AUTONAMEDIRS option set and use a variable in your chpwd function
> >(e.g. to build a string to print it on the windows title bar),
> >the variable is immediately put into the hashtable for named
> >directories.
> 
> That's correct behaviour.  Maybe there should be something in the man
> page to warn about the effect of this combination of options, but this
> behaviour is not going to be changed.

That is all I suggest that should be done. I have spent days chasing
this problem and I see no reason why others should do the same.I
understand perfectly well that this is the defined behaviour and I
don't mind if you have to read the manpage to circumvent the problem.
I just want to spare others the pains and efforts I have taken to
solve it.

Bye

Dominik
-- 
------------------------------------------------------------------------
Dominik Vogt                                 phone:        07031/14-4596
Hewlett-Packard GmbH                           fax:        07031/14-3883
Boeblingen Verification Solutions                                       
Herrenberger Str. 130
71034 Boeblingen                             email:  dominik_vogt@hp.com
------------------------------------------------------------------------

Re: bug with chpwd/allexport

[Zefram]

Dominik Vogt wrote:
>                        BTW if the second workaround you suggest works,
>then it is a bug. The manpage states clearly, that a local variable is
>still exported if the ALLEXPORT option is set and the variable was
>prviously unset.

But with `local', the variable disappears at the end of the function,
and so can't affect the display of the prompt.  Try adding "print -P '%~'"
in different parts of the function to see where the variable takes effect.

-zefram

Re: bug with chpwd/allexport

[Dominik Vogt]

> >                        BTW if the second workaround you suggest works,
> >then it is a bug. The manpage states clearly, that a local variable is
> >still exported if the ALLEXPORT option is set and the variable was
> >prviously unset.
> 
> But with `local', the variable disappears at the end of the function,
> and so can't affect the display of the prompt.  Try adding "print -P '%~'"
> in different parts of the function to see where the variable takes effect.

OK, you're right, you can read the manpage this way, but it's just
implicitly in the description and not stated clearly. And I do not
understand why the ALLEXPORT option is handled differently in shell
functions if the auto-exported variables are not visible in the calling
environment anyway. I think it just creates confusion about which
variables are exported to subshells of the function and which are not?
Or what is the reason for this?

Bye

Dominik
-- 
------------------------------------------------------------------------
Dominik Vogt                                 phone:        07031/14-4596
Hewlett-Packard GmbH                           fax:        07031/14-3883
Boeblingen Verification Solutions                                       
Herrenberger Str. 130
71034 Boeblingen                             email:  dominik_vogt@hp.com
------------------------------------------------------------------------

Re: bug with chpwd/allexport

[Zefram]

Dominik Vogt wrote:
>OK, you're right, you can read the manpage this way, but it's just
>implicitly in the description and not stated clearly. And I do not
>understand why the ALLEXPORT option is handled differently in shell
>functions if the auto-exported variables are not visible in the calling
>environment anyway.

You seem to be confusing the concepts of `existing outside the shell
function' and `exported' (visible to programs invoked from the shell).
The two are mostly, though not entirely, independent of each other.
In particular, exporting a variable does nothing to extend its lifetime.

>                    I think it just creates confusion about which
>variables are exported to subshells of the function and which are not?

All variables, exported or not, exist in subshells.

-zefram

Re: bug with chpwd/allexport

[Dominik Vogt]

> You seem to be confusing the concepts of `existing outside the shell
> function' and `exported' (visible to programs invoked from the shell).
> The two are mostly, though not entirely, independent of each other.

No, I just wasn't reading the manpage carefully. Perhaps the manpage
should be written in a way that a 'normal' user can understand
implications like this after reading in once. There are some other
'mysteries' of zsh. For example the exact order in which quoting
and substitution are done. For example the manpage doesn't state
which substitutions/expansions recognize which kinds of quoting.

> In particular, exporting a variable does nothing to extend its
> lifetime.

OK, that is the point the manpage might create confusion about.

...

> All variables, exported or not, exist in subshells.

Sorry, my fault, I didn't mean subshells but the commands being
started from the shell. This is basically the same in the NT port
so I mixed up the terms.

Bye

Dominik
-- 
------------------------------------------------------------------------
Dominik Vogt                                 phone:        07031/14-4596
Hewlett-Packard GmbH                           fax:        07031/14-3883
Boeblingen Verification Solutions                                       
Herrenberger Str. 130
71034 Boeblingen                             email:  dominik_vogt@hp.com
------------------------------------------------------------------------

Re: bug with chpwd/allexport (plus expansion/substitution remarks)

[Bart Schaefer]

On Jul 17,  3:27pm, Dominik Vogt wrote:
} Subject: Re: bug with chpwd/allexport
}
} > >         The problem is, that is you have the ALLEXPORT and the
} > >AUTONAMEDIRS option set and use a variable in your chpwd function
} > >(e.g. to build a string to print it on the windows title bar),
} > >the variable is immediately put into the hashtable for named
} > >directories.
} > 
} > That's correct behaviour.

This doesn't actually have anything to do with ALLEXPORT, does it?  As
soon as you set AUTONAMEDIRS, it behaves like this.

On Jul 17,  2:42pm, Zefram wrote:
} Subject: Re: bug with chpwd/allexport
}
} Dominik Vogt wrote:
} >                        BTW if the second workaround you suggest works,
} >then it is a bug. The manpage states clearly, that a local variable is
} >still exported if the ALLEXPORT option is set and the variable was
} >prviously unset.
} 
} But with `local', the variable disappears at the end of the function,

That also doesn't affect whether the name is added to the named directory
hash table.  Unsetting the variable doesn't unhash the name; making it a
local has no effect on this problem whatsoever.

On Jul 17,  5:42pm, Dominik Vogt wrote:
} Subject: Re: bug with chpwd/allexport
}
} > You seem to be confusing the concepts of `existing outside the shell
} > function' and `exported' (visible to programs invoked from the shell).
} > The two are mostly, though not entirely, independent of each other.
} 
} No, I just wasn't reading the manpage carefully. Perhaps the manpage
} should be written in a way that a 'normal' user can understand
} implications like this after reading in once.

This would be ideal, but is nearly impossible to achieve.  We can try to
catch obvious cases, and sometimes add doc for particular instances we
discover (like this one); but there are too many options and too many ways
they interact to describe all the possible combinations in the manual.

} There are some other
} 'mysteries' of zsh. For example the exact order in which quoting
} and substitution are done. For example the manpage doesn't state
} which substitutions/expansions recognize which kinds of quoting.

The info files do explain this, though you are again correct that there's
no single place that enumerates all the steps.  The section on "Quoting"
should probably be moved to appear earlier in the "Shell Grammar" section,
or even outside it (perhaps a new section on "Lexical Conventions" should
be added, to explain how "words" and "tokens" are identified).

The short answer to "which substitutions/expansions recognize which kinds
of quoting" is that -none- of them recognize quoting.  Quoting is a lexical
convention applied at the time the input is read; by the time expansions
occur, everything is already broken into words/tokens.  Did you mean "what
quotings permit or prevent which substitutions/expansions?"  The "Quoting"
section is deficient in failing to mention that history expansion occurs
inside double quotes but not inside single quotes, but is otherwise OK.

The "Expansion" section is potentially misleading, because it discusses
removal of the unquoted quotes from the input; in fact I can't think of a
case where it makes a difference when those quotes are removed, because
the quoting itself has already happened.  E.g.:

	print -l x{"a,b,c"}x

The quotes aren't removed until after brace expansion (according to the
info doc) but the commas are already quoted by the time brace expansion is
attempted.  (I have no idea what effect SH_FILE_EXPANSION has on when the
quotes are removed, or whether it matters.)

Or perhaps you mean "what quotings are recognized in the replacement text
that results from which substitutions/expansions?"  Again, the answer is
that none are.  Expansions (except alias expansion, but including history,
now that the old lexical history options are gone) are already tokenized
when replaced [*] and substitutions are subject to word splitting at IFS
unless double-quoted.

[*] Is that the only thing that distinguishes expansion from substitution?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: bug with chpwd/allexport (plus expansion/substitution remarks)

[Bart Schaefer]

On Jul 23,  1:30pm, Bart Schaefer wrote:
} Subject: Re: bug with chpwd/allexport (plus expansion/substitution remarks
}
} Or perhaps you mean "what quotings are recognized in the replacement text
} that results from which substitutions/expansions?"  Again, the answer is
} that none are.  Expansions (except alias expansion, but including history,
} now that the old lexical history options are gone) are already tokenized

I apologize; this is not entirely accurate.  History is expanded whenever
the lexer requests the next character of input; aliases are expanded after
the lexer has recognized an unquoted word, but before that word is checked
against the reserved words.  History is "tokenized" in the sense that it
is stored as an array of words, but those words are rescanned by the lexer
when expanded.

I'm surprised Zefram and PWS didn't jump on me for this one.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com