* Creating A Clean Environment For Autoloaded Functions @ 2012-12-30 9:44 Russell Harmon 2012-12-30 19:20 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Russell Harmon @ 2012-12-30 9:44 UTC (permalink / raw) To: zsh-users If I want to write a function which can be autoloaded, how can I prevent functions defined outside of my function from being accessible from within my function? For example, I have a function ls { gls "$@" }, and I know of no way to prevent that function from leaking into the definition of all the functions I've autoloaded which use ls. Additionally, is it possible to zmodload a module which is automatically unloaded from the environment after my function completes _only if_ that module was not already loaded? -- Russell Harmon ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Creating A Clean Environment For Autoloaded Functions 2012-12-30 9:44 Creating A Clean Environment For Autoloaded Functions Russell Harmon @ 2012-12-30 19:20 ` Bart Schaefer 2012-12-30 21:02 ` Russell Harmon 2012-12-31 23:30 ` Han Pingtian 0 siblings, 2 replies; 11+ messages in thread From: Bart Schaefer @ 2012-12-30 19:20 UTC (permalink / raw) To: zsh-users On Dec 30, 4:44am, Russell Harmon wrote: > > If I want to write a function which can be autoloaded, how can I > prevent functions defined outside of my function from being accessible > from within my function? For example, I have a function ls { gls "$@" > }, and I know of no way to prevent that function from leaking into the > definition of all the functions I've autoloaded which use ls. This amounts to asking how to violate a basic tenet of shell function behavior. They're supposed to act like commands that are always at the front of the search path, and the search path doesn't change based on the order in which every other command was added to it. The way you avoid this is to write every function that explicitly does not want this behavior so as to explicitly override it; for examply by using "command ls" instead of "ls". However, you could do something like this, assuming zsh/parameter is loaded: ls() { if (( $#funcstack > 1 )) then command ls "$@" else gls "$@" fi } This will prevent gls from being run inside any other function, although it won't bypass the ls wrapper function entirely. > Additionally, is it possible to zmodload a module which is > automatically unloaded from the environment after my function > completes _only if_ that module was not already loaded? There isn't a property of modules that supports this, but you can do it yourself explicitly with something like this: { zmodload -e zsh/mathfunc # for example local unload_mathfunc=$? zmodload zsh/mathfunc : do what you need to with math functions } always { (( unload_mathfunc )) && zmodload -u zsh/mathfunc } If the zsh/parameter module is not itself one of the ones you care about unloading in this way, you can be more generic: { zmodload zsh/parameter local -a existing_modules existing_modules=( ${(k)modules[(R)loaded]} ) : do whatever zmodloads you want ... } always { # Requires zsh 5.0, otherwise you need a loop zmodload -u ${(k)modules[(R)loaded]:|existing_modules} } I'm not sure that zmodload is clever enough to notice that inter-module dependencies have been satisfied by the list provided to -u and thus unload the modules in the correct order to be sure it succeeds, so a bit of tweaking to the above may be required for full generality. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Creating A Clean Environment For Autoloaded Functions 2012-12-30 19:20 ` Bart Schaefer @ 2012-12-30 21:02 ` Russell Harmon 2012-12-30 22:12 ` Bart Schaefer 2012-12-31 23:30 ` Han Pingtian 1 sibling, 1 reply; 11+ messages in thread From: Russell Harmon @ 2012-12-30 21:02 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-users On Sun, Dec 30, 2012 at 2:20 PM, Bart Schaefer <schaefer@brasslantern.com> wrote: > On Dec 30, 4:44am, Russell Harmon wrote: >> >> If I want to write a function which can be autoloaded, how can I >> prevent functions defined outside of my function from being accessible >> from within my function? For example, I have a function ls { gls "$@" >> }, and I know of no way to prevent that function from leaking into the >> definition of all the functions I've autoloaded which use ls. > > This amounts to asking how to violate a basic tenet of shell function > behavior. They're supposed to act like commands that are always at the > front of the search path, and the search path doesn't change based on > the order in which every other command was added to it. > > The way you avoid this is to write every function that explicitly does > not want this behavior so as to explicitly override it; for examply by > using "command ls" instead of "ls". > > However, you could do something like this, assuming zsh/parameter is > loaded: > > ls() { > if (( $#funcstack > 1 )) > then command ls "$@" > else gls "$@" > fi > } > > This will prevent gls from being run inside any other function, although > it won't bypass the ls wrapper function entirely. I guess that breaking with this particular shell idiom is what I want to do. Otherwise, how is one supposed to be able to create autoloadable functions with reliable and reproducible behavior? I'm thinking to add an option to autoload to undefine all functions much like how -U works with aliases. Basically, I'm trying to follow what the best practices are when writing autoloadable functions so that they will be able to run reliably in the largest number of different environments possible. > >> Additionally, is it possible to zmodload a module which is >> automatically unloaded from the environment after my function >> completes _only if_ that module was not already loaded? > > There isn't a property of modules that supports this, but you can do it > yourself explicitly with something like this: > > { > zmodload -e zsh/mathfunc # for example > local unload_mathfunc=$? > zmodload zsh/mathfunc > > : do what you need to with math functions > > } always { > (( unload_mathfunc )) && zmodload -u zsh/mathfunc > } > > If the zsh/parameter module is not itself one of the ones you care about > unloading in this way, you can be more generic: > > { > zmodload zsh/parameter > local -a existing_modules > existing_modules=( ${(k)modules[(R)loaded]} ) > > : do whatever zmodloads you want ... > > } always { > # Requires zsh 5.0, otherwise you need a loop > zmodload -u ${(k)modules[(R)loaded]:|existing_modules} > } > > I'm not sure that zmodload is clever enough to notice that inter-module > dependencies have been satisfied by the list provided to -u and thus > unload the modules in the correct order to be sure it succeeds, so a > bit of tweaking to the above may be required for full generality. Hmm, I think I'm realizing that loading & unloading of dependent modules every time a function is called might not be exactly what I want. I think what I'm looking for is more like zsh modules where you have hook functions for when the function is loaded & unloaded so that you can unload relevant modules when no longer needed. That has it's own difficulties however because it'd be hard to tell if some other function loaded after you were loaded needs a module which you are about to unload. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Creating A Clean Environment For Autoloaded Functions 2012-12-30 21:02 ` Russell Harmon @ 2012-12-30 22:12 ` Bart Schaefer 0 siblings, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2012-12-30 22:12 UTC (permalink / raw) To: zsh-users On Dec 30, 4:02pm, Russell Harmon wrote: } } On Sun, Dec 30, 2012 at 2:20 PM, Bart Schaefer } <schaefer@brasslantern.com> wrote: } > This amounts to asking how to violate a basic tenet of shell function } > behavior. They're supposed to act like commands that are always at the } > front of the search path, and the search path doesn't change based on } > the order in which every other command was added to it. } } I guess that breaking with this particular shell idiom is what I want } to do. Otherwise, how is one supposed to be able to create } autoloadable functions with reliable and reproducible behavior? As I said, you do it like this: } > The way you avoid this is to write every function that explicitly does } > not want this behavior so as to explicitly override it; for examply by } > using "command ls" instead of "ls". To be completely "reliable and reproducible" you also have to set up the emulation mode very carefully, force a fixed value of $path, etc. In practice is this is often more than either you or the caller of your function actually want. The alternative is to write scripts rather than autoloaded functions, which run in their own process and have more complete control of their context. Part of the reason for using functions rather than scripts is so that they operate in the context of the running shell, which implies that the caller is permitted to screw around with that context. } I'm thinking to add an option to autoload to undefine all functions } much like how -U works with aliases. The problem with that is that -U doesn't actually undefine the aliases, it just prevents them from being expanded at parse time. There is no equivalent parse-time mechanism for distinguishing an external command from a function. In zsh (and probably only in zsh of all shells) there actually is a way to remove all functions if you really want to. the_lonely_function() { builtin zmodload zsh/parameter builtin typeset +h functions builtin unfunction -m \* # # There are now no functions but this function, # and even this function cannot call itself. # On return, the local parameter scope will end # and other function definitions will reappear. # } There are a few obscure gotchas with this; for example, functions that were previously autoload-able will instead become real functions that use the "autoload -X" command to bootstrap. That may not always have the equivalent effect to the original autoload. } Basically, I'm trying to follow what the best practices are when } writing autoloadable functions so that they will be able to run } reliably in the largest number of different environments possible. I'd have to say that the *usual* practices are to worry about the emulation mode and setopts, use globbing and parameter expansion tricks to avoid reliance on external commands as much as possible, and to presume "autoload -U" has eliminated parse-time surprises. Beyond that and the judicious application of "builtin" or "command" prefixes where necessary, you're putting in too much effort for the incremental value. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Creating A Clean Environment For Autoloaded Functions 2012-12-30 19:20 ` Bart Schaefer 2012-12-30 21:02 ` Russell Harmon @ 2012-12-31 23:30 ` Han Pingtian 2013-01-02 5:15 ` PATCH and more remarks on parameter expansion docs Bart Schaefer 1 sibling, 1 reply; 11+ messages in thread From: Han Pingtian @ 2012-12-31 23:30 UTC (permalink / raw) To: zsh-users On Sun, Dec 30, 2012 at 11:20:44AM -0800, Bart Schaefer wrote: > { > zmodload zsh/parameter > local -a existing_modules > existing_modules=( ${(k)modules[(R)loaded]} ) > > : do whatever zmodloads you want ... > > } always { > # Requires zsh 5.0, otherwise you need a loop > zmodload -u ${(k)modules[(R)loaded]:|existing_modules} How could we know that, the "(k)" will be run before ":|" here, please? Because "k" will be applied with the subscripting, so it will be before the ":|" ? Thanks. > } ^ permalink raw reply [flat|nested] 11+ messages in thread
* PATCH and more remarks on parameter expansion docs 2012-12-31 23:30 ` Han Pingtian @ 2013-01-02 5:15 ` Bart Schaefer 2013-01-02 8:32 ` Han Pingtian 0 siblings, 1 reply; 11+ messages in thread From: Bart Schaefer @ 2013-01-02 5:15 UTC (permalink / raw) To: zsh-users Collecting a few threads about parameter expansion here in order to make some other remarks. On Jan 1, 7:30am, Han Pingtian wrote: } Subject: Re: Creating A Clean Environment For Autoloaded Functions } } > # Requires zsh 5.0, otherwise you need a loop } > zmodload -u ${(k)modules[(R)loaded]:|existing_modules} } } How could we know that, the "(k)" will be run before ":|" here, please? } Because "k" will be applied with the subscripting, so it will be before } the ":|" ? That's correct. I've added some mention of this in the patch below. On Dec 27, 3:08pm, Han Pingtian wrote: } Subject: Re: splitting in assignment forms of expansion } } I think we should say something like this about this feature in the man } page: } } Note during assignments with the A flag, that splitting is applied } to word in the assignment forms of spec before the assignment to } name is performed. This will affects the result. I added something similar in the patch. In looking through the parameter flags to see what else may have been overlooked in the "rules", I found no mention of the (c) and (t) flags. The (c) flag does what Han Pingtian was asking about in the thread Re: priority problem of ":|" and ":*" -- % x=(a b c) % print "${#x}" ${(c)#x} % 3 5 That is, (c) joins up the array for length purposes even in places where it would not normally be joined. I've expanded the doc for this a little as well, but I didn't explicitly mention it in the rules section because it only matters in combination with "#". The (t) flag is an oddball. If combined with (P) [rule 4], it takes effect after that name replacement. However, if *not* combined with (P), then it takes effect *before* subscripting [rule 3]. Thus: % x=path % print ${(t)x} scalar % print ${x[2,4]} ath % print ${(t)x[2,4]} cal % print ${(tP)x} array-unique-special I might have expected ${(t)x[2,4]} to print the type of the variable named "ath", but that isn't what happens. So I haven't attempted to work it into the rules, but it's a perfect example of how parameter expansion syntax lacks well-defined precedence. Index: Doc/Zsh/expn.yo =================================================================== RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v retrieving revision 1.148 diff -u -r1.148 expn.yo --- Doc/Zsh/expn.yo 19 Oct 2012 09:19:08 -0000 1.148 +++ Doc/Zsh/expn.yo 2 Jan 2013 04:57:04 -0000 @@ -748,6 +748,8 @@ the length in characters of the result instead of the result itself. If var(spec) is an array expression, substitute the number of elements of the result. +This has the side-effect that joining is skipped even in quoted +forms, which may affect other sub-expressions in var(spec). Note that `tt(^)', `tt(=)', and `tt(~)', below, must appear to the left of `tt(#)' when these forms are combined. ) @@ -870,11 +872,12 @@ Create an array parameter with `tt(${)...tt(=)...tt(})', `tt(${)...tt(:=)...tt(})' or `tt(${)...tt(::=)...tt(})'. If this flag is repeated (as in `tt(AA)'), create an associative -array parameter. Assignment is made before sorting or padding. -The var(name) part may be a subscripted range for ordinary -arrays; the var(word) part em(must) be converted to an array, for -example by using `tt(${(AA)=)var(name)tt(=)...tt(})' to activate -field splitting, when creating an associative array. +array parameter. Assignment is made before sorting or padding; +if field splitting is active, the var(word) part is split before +assignment. The var(name) part may be a subscripted range for +ordinary arrays; the var(word) part em(must) be converted to +an array, for example by using `tt(${(AA)=)var(name)tt(=)...tt(})' +to activate field splitting, when creating an associative array. ) item(tt(a))( Sort in array index order; when combined with `tt(O)' sort in reverse @@ -884,7 +887,9 @@ ) item(tt(c))( With tt(${#)var(name)tt(}), count the total number of characters in an array, -as if the elements were concatenated with spaces between them. +as if the elements were concatenated with spaces between them. This is not +a true join of the array, so other expressions used with this flag may have +an effect on the elements of the array before it is counted. ) item(tt(C))( Capitalize the resulting words. `Words' in this case refers to sequences @@ -1308,7 +1313,8 @@ subscript. Thus if tt(var) is an array, tt(${var[1][2]}) is the second character of the first word, but tt(${var[2,4][2]}) is the entire third word (the second word of the range of words two through four of the -original array). Any number of subscripts may appear. +original array). Any number of subscripts may appear. Flags such as +tt((k)) and tt((v)) which alter the result of subscripting are applied. ) item(tt(4.) em(Parameter name replacement))( The effect of any tt((P)) flag, which treats the value so far as a @@ -1316,11 +1322,12 @@ ) item(tt(5.) em(Double-quoted joining))( If the value after this process is an array, and the substitution -appears in double quotes, and no tt((@)) flag is present at the current -level, the words of the value are joined with the first character of the -parameter tt($IFS), by default a space, between each word (single word -arrays are not modified). If the tt((j)) flag is present, that is used for -joining instead of tt($IFS). +appears in double quotes, and neither an tt((@)) flag nor a tt(#) +length operator is present at the current level, then words of the +value are joined with the first character of the parameter tt($IFS), +by default a space, between each word (single word arrays are not +modified). If the tt((j)) flag is present, that is used for joining +instead of tt($IFS). ) item(tt(6.) em(Nested subscripting))( Any remaining subscripts (i.e. of a nested substitution) are evaluated at ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: PATCH and more remarks on parameter expansion docs 2013-01-02 5:15 ` PATCH and more remarks on parameter expansion docs Bart Schaefer @ 2013-01-02 8:32 ` Han Pingtian 2013-01-02 16:46 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Han Pingtian @ 2013-01-02 8:32 UTC (permalink / raw) To: zsh-users On Tue, Jan 01, 2013 at 09:15:01PM -0800, Bart Schaefer wrote: > On Dec 27, 3:08pm, Han Pingtian wrote: > } Subject: Re: splitting in assignment forms of expansion > } > } I think we should say something like this about this feature in the man > } page: > } > } Note during assignments with the A flag, that splitting is applied > } to word in the assignment forms of spec before the assignment to > } name is performed. This will affects the result. > > I added something similar in the patch. Thanks your nice patch. But I'm still wondering why not change the statements after "${=spec}" in man page. Or I'm misunderstanding the original statements which has been stating this "splitting before assignment" behavior is only related to "(A)" flag? > > > The (t) flag is an oddball. If combined with (P) [rule 4], it takes > effect after that name replacement. However, if *not* combined with > (P), then it takes effect *before* subscripting [rule 3]. Thus: > > % x=path > % print ${(t)x} > scalar > % print ${x[2,4]} > ath > % print ${(t)x[2,4]} > cal > % print ${(tP)x} > array-unique-special > > I might have expected ${(t)x[2,4]} to print the type of the variable > named "ath", but that isn't what happens. So I haven't attempted to > work it into the rules, but it's a perfect example of how parameter > expansion syntax lacks well-defined precedence. Yep, it looks like there are quite some special cases. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: PATCH and more remarks on parameter expansion docs 2013-01-02 8:32 ` Han Pingtian @ 2013-01-02 16:46 ` Bart Schaefer 2013-01-02 23:28 ` Han Pingtian 0 siblings, 1 reply; 11+ messages in thread From: Bart Schaefer @ 2013-01-02 16:46 UTC (permalink / raw) To: zsh-users On Jan 2, 4:32pm, Han Pingtian wrote: } } Thanks your nice patch. But I'm still wondering why not change the } statements after "${=spec}" in man page. Or I'm misunderstanding the } original statements which has been stating this "splitting before } assignment" behavior is only related to "(A)" flag? You mean this? Note that splitting is applied to WORD in the assignment forms of SPEC _before_ the assignment to NAME is performed. This affects the result of array assignments with the A flag. I must not be understanding what else you think it should say. Is it just that it isn't "... the A flag, among other things"? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: PATCH and more remarks on parameter expansion docs 2013-01-02 16:46 ` Bart Schaefer @ 2013-01-02 23:28 ` Han Pingtian 2013-01-03 19:42 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Han Pingtian @ 2013-01-02 23:28 UTC (permalink / raw) To: zsh-users On Wed, Jan 02, 2013 at 08:46:44AM -0800, Bart Schaefer wrote: > On Jan 2, 4:32pm, Han Pingtian wrote: > } > } Thanks your nice patch. But I'm still wondering why not change the > } statements after "${=spec}" in man page. Or I'm misunderstanding the > } original statements which has been stating this "splitting before > } assignment" behavior is only related to "(A)" flag? > > You mean this? > > Note that splitting is applied to WORD in the assignment forms of > SPEC _before_ the assignment to NAME is performed. This affects > the result of array assignments with the A flag. Yes, I mean this. I think it doesn't say this splitting _before_ assignments is only true when (A) is used. Looks like if (A) wasn't used, splitting will _after_ the assignments: % print -l ${=xx::=foo bar} . $xx . ${(A)=xx::=foo bar} . $xx foo bar . foo bar . foo bar . foo bar So I think maybe it should say this splitting before assignments is only true when (A) is used. The original statements give me the feeling that this splitting before assignments is peformed in any conditions. Or I am misunderstanding the statements? Thanks. > > I must not be understanding what else you think it should say. Is it > just that it isn't "... the A flag, among other things"? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: PATCH and more remarks on parameter expansion docs 2013-01-02 23:28 ` Han Pingtian @ 2013-01-03 19:42 ` Bart Schaefer 2013-01-03 22:18 ` Han Pingtian 0 siblings, 1 reply; 11+ messages in thread From: Bart Schaefer @ 2013-01-03 19:42 UTC (permalink / raw) To: zsh-users On Jan 3, 7:28am, Han Pingtian wrote: } Subject: Re: PATCH and more remarks on parameter expansion docs } } On Wed, Jan 02, 2013 at 08:46:44AM -0800, Bart Schaefer wrote: } > } > Note that splitting is applied to WORD in the assignment forms of } > SPEC _before_ the assignment to NAME is performed. This affects } > the result of array assignments with the A flag. } } I think it doesn't say this splitting _before_ assignments is only true } when (A) is used. Looks like if (A) wasn't used, splitting will _after_ } the assignments Ah, I see. OK, I agree that it's imprecise about this. It's imprecise but not in the way that you think: The splitting is in fact done before the assignment in either case, but at that point the internals have both the expanded string and the split result available. The (A) flag then determines which of those two values is used in the assignment. I'm not sure whether in practice this can ever have a visible effect. In theory if there were some oddity with the syntax of WORD that caused the splitting step to fail, then the NAME parameter would not be assigned, but I can't trivially produce an example. On the other hand, when does this "mistake" in the documentation matter? Given what it says, what did you expect ${=xx::=foo bar} to assign? The doc needs to strike a balance between giving so much detail that it's confusing and too little to tell the user what to expect. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: PATCH and more remarks on parameter expansion docs 2013-01-03 19:42 ` Bart Schaefer @ 2013-01-03 22:18 ` Han Pingtian 0 siblings, 0 replies; 11+ messages in thread From: Han Pingtian @ 2013-01-03 22:18 UTC (permalink / raw) To: zsh-users On Thu, Jan 03, 2013 at 11:42:17AM -0800, Bart Schaefer wrote: > On Jan 3, 7:28am, Han Pingtian wrote: > } Subject: Re: PATCH and more remarks on parameter expansion docs > } > } On Wed, Jan 02, 2013 at 08:46:44AM -0800, Bart Schaefer wrote: > } > > } > Note that splitting is applied to WORD in the assignment forms of > } > SPEC _before_ the assignment to NAME is performed. This affects > } > the result of array assignments with the A flag. > } > } I think it doesn't say this splitting _before_ assignments is only true > } when (A) is used. Looks like if (A) wasn't used, splitting will _after_ > } the assignments > > Ah, I see. OK, I agree that it's imprecise about this. It's imprecise > but not in the way that you think: The splitting is in fact done before > the assignment in either case, but at that point the internals have both > the expanded string and the split result available. The (A) flag then > determines which of those two values is used in the assignment. > > I'm not sure whether in practice this can ever have a visible effect. In > theory if there were some oddity with the syntax of WORD that caused the > splitting step to fail, then the NAME parameter would not be assigned, > but I can't trivially produce an example. > > On the other hand, when does this "mistake" in the documentation matter? > Given what it says, what did you expect ${=xx::=foo bar} to assign? > Since ${=xx::=foo bar} would result in two items foo and bar, but xx's value is "foo bar", so it looks like xx is assigned value "foo bar", then xx's value is splitted. But xx is a scalar here, so we cannot assign two value foo and bar to it at the same time. So I think the value would be joined again before the assignment? So I expect xx's value would be "foo bar"? Thanks. ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2013-01-03 22:28 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-12-30 9:44 Creating A Clean Environment For Autoloaded Functions Russell Harmon 2012-12-30 19:20 ` Bart Schaefer 2012-12-30 21:02 ` Russell Harmon 2012-12-30 22:12 ` Bart Schaefer 2012-12-31 23:30 ` Han Pingtian 2013-01-02 5:15 ` PATCH and more remarks on parameter expansion docs Bart Schaefer 2013-01-02 8:32 ` Han Pingtian 2013-01-02 16:46 ` Bart Schaefer 2013-01-02 23:28 ` Han Pingtian 2013-01-03 19:42 ` Bart Schaefer 2013-01-03 22:18 ` Han Pingtian
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).