* uninvited members of associative array @ 2022-12-15 23:48 Ray Andrews 2022-12-16 0:05 ` Lawrence Velázquez ` (2 more replies) 0 siblings, 3 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-15 23:48 UTC (permalink / raw) To: Zsh Users [-- Attachment #1: Type: text/plain, Size: 1156 bytes --] I just noticed something. My big array has ten named elements but if I print it: printf "%-20s %s\n" ${(kv)main} > /dev/pts/2 width 90 window main offset 0 topE 1 active 1 90 main bottomE 48 lastE 196 hight 48 196 48 List 1 0 1 list List 1 48 currentE 1 ... it all works, however 'elements' 90, 196, 0 and 1 seem to have created themselves, I didn't make them. They seem to be ghosts who's names are other element's values, eg: '90' is the value proper value of 'width' and that element holds the value 'main' which is the proper value of 'window' and so on. It doesn't seem to break anything, however it's a bit strange when you print it out. Something unintended in the printf? [-- Attachment #2: Type: text/html, Size: 1599 bytes --] ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-15 23:48 uninvited members of associative array Ray Andrews @ 2022-12-16 0:05 ` Lawrence Velázquez 2022-12-16 0:48 ` Ray Andrews 2022-12-16 3:20 ` Bart Schaefer 2022-12-16 8:12 ` Roman Perepelitsa 2 siblings, 1 reply; 45+ messages in thread From: Lawrence Velázquez @ 2022-12-16 0:05 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Thu, Dec 15, 2022, at 6:48 PM, Ray Andrews wrote: > I just noticed something. My big array has ten named elements but if I > print it: > > printf "%-20s %s\n" ${(kv)main} > /dev/pts/2 > > width 90 > window main > offset 0 > topE 1 > active 1 > 90 main > bottomE 48 > lastE 196 > hight 48 > 196 48 > List 1 > 0 1 > list List > 1 48 > currentE 1 > > ... it all works, however 'elements' 90, 196, 0 and 1 seem to have > created themselves, I didn't make them. How are you creating and populating "main"? Can you provide a test case that the rest of us can actually use? If I just guess the contents of "main" then I don't see a problem. % cat foo.zsh typeset -A main main=( active 1 bottomE 48 currentE 1 height 48 lastE 196 list 1 offset 0 topE 1 width 90 window main ) typeset -p main printf '%-20s %s\n' ${(kv)main} % zsh -f foo.zsh typeset -A main=( [active]=1 [bottomE]=48 [currentE]=1 [height]=48 [lastE]=196 [list]=1 [offset]=0 [topE]=1 [width]=90 [window]=main ) window main width 90 offset 0 topE 1 height 48 active 1 lastE 196 bottomE 48 list 1 currentE 1 -- vq ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 0:05 ` Lawrence Velázquez @ 2022-12-16 0:48 ` Ray Andrews 2022-12-16 1:29 ` Lawrence Velázquez 2022-12-16 2:09 ` Dominik Vogt 0 siblings, 2 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-16 0:48 UTC (permalink / raw) To: zsh-users On 2022-12-15 16:05, Lawrence Velázquez wrote: > How are you creating and populating "main"? Can you provide a test > case that the rest of us can actually use? There's always the chance it doesn't even come to that, some artifact in the way the array prints or something. But if you need to see an initialization: # nlist(): Initialize 'main' array. 10 elements: ----------------------------- main[window]="main" main[list]='List' main[lastE]="$#List" main[hight]=$(( mainWH - 2 )) main[width]=$(( mainWW - 2 )) main[bottomE]="$main[hight]" main[topE]=1 main[currentE]=1 main[offset]=0 main[active]=1 ... all values assign properly, so where do the uninvited keywords come from? As Bart explained, the order of the printout has no regularity, so there's no pattern to look for. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 0:48 ` Ray Andrews @ 2022-12-16 1:29 ` Lawrence Velázquez 2022-12-16 2:09 ` Dominik Vogt 1 sibling, 0 replies; 45+ messages in thread From: Lawrence Velázquez @ 2022-12-16 1:29 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Thu, Dec 15, 2022, at 7:48 PM, Ray Andrews wrote: > On 2022-12-15 16:05, Lawrence Velázquez wrote: >> How are you creating and populating "main"? Can you provide a test >> case that the rest of us can actually use? > > There's always the chance it doesn't even come to that, some artifact in > the way the array prints or something. It makes no sense to question the code before verifying that your data isn't garbage. > But if you need to see an > initialization: > > > # nlist(): Initialize 'main' array. 10 elements: > ----------------------------- > main[window]="main" > main[list]='List' > main[lastE]="$#List" > main[hight]=$(( mainWH - 2 )) > main[width]=$(( mainWW - 2 )) > main[bottomE]="$main[hight]" > main[topE]=1 > main[currentE]=1 > main[offset]=0 > main[active]=1 Still a bad test case because it depends on a bunch of other variables that we have to guess about. I don't understand why you can't provide something self-contained. Is that really all that's in "main"? What does "typeset -p main" say? > ... all values assign properly, so where do the uninvited keywords come > from? Still not seeing a problem. % cat foo.zsh typeset -A main main[window]=main main[list]=List main[lastE]=196 main[height]=48 main[width]=90 main[bottomE]="$main[height]" main[topE]=1 main[currentE]=1 main[offset]=0 main[active]=1 typeset -p main printf '%-20s %s\n' ${(kv)main} % zsh foo.zsh typeset -A main=( [active]=1 [bottomE]=48 [currentE]=1 [height]=48 [lastE]=196 [list]=List [offset]=0 [topE]=1 [width]=90 [window]=main ) width 90 window main offset 0 topE 1 height 48 active 1 bottomE 48 lastE 196 list List currentE 1 -- vq ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 0:48 ` Ray Andrews 2022-12-16 1:29 ` Lawrence Velázquez @ 2022-12-16 2:09 ` Dominik Vogt 2022-12-16 2:56 ` Ray Andrews 1 sibling, 1 reply; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 2:09 UTC (permalink / raw) To: zsh-users On Thu, Dec 15, 2022 at 04:48:51PM -0800, Ray Andrews wrote: > > On 2022-12-15 16:05, Lawrence Velázquez wrote: > > How are you creating and populating "main"? Can you provide a test > > case that the rest of us can actually use? > > There's always the chance it doesn't even come to that, some artifact in the > way the array prints or something. But if you need to see an > initialization: > > > # nlist(): Initialize 'main' array. 10 elements: > ----------------------------- > main[window]="main" > main[list]='List' > main[lastE]="$#List" > main[hight]=$(( mainWH - 2 )) > main[width]=$(( mainWW - 2 )) > main[bottomE]="$main[hight]" > main[topE]=1 > main[currentE]=1 > main[offset]=0 > main[active]=1 > > ... all values assign properly, so where do the uninvited keywords come > from? As Bart explained, the order of the printout has no regularity, so > there's no pattern to look for. Probability of 95% or more that the array gets modified after the initialisation by some stray code. Put prints all over the place to find the place where the array gets modified. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 2:09 ` Dominik Vogt @ 2022-12-16 2:56 ` Ray Andrews 2022-12-16 3:21 ` Lawrence Velázquez 0 siblings, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-16 2:56 UTC (permalink / raw) To: zsh-users On 2022-12-15 18:09, Dominik Vogt wrote: > Probability of 95% or more that the array gets modified after the > initialisation by some stray code. Put prints all over the place > to find the place where the array gets modified. Nope, I showed the output directly after the init. If something goes wrong, it goes wrong during the init itself. Lawrence: > It makes no sense to question the code before verifying that your data isn't garbage. All the named keywords have exactly correct data, there's nothing not to like as far as that. Dunno, it wouldn't be the first time that there was some silly issue in the printout, no real problem at all, so my first question assumed nothing. My ignorance remains a deep well tho slowly getting better. > Still not seeing a problem. I think I'm on the scent, the array is global. If I zero it before all the assignments (they're just numbers, no need to show the full genealogy of them), then the interlopers disappear. So now the question is where the bogus elements come from. They aren't actually assigned anywhere, that's for sure. Variable crosstalk? How does an associative array get corrupted? Let me root around a bit ... hey ... you don't suppose a zcurses crash might do it? ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 2:56 ` Ray Andrews @ 2022-12-16 3:21 ` Lawrence Velázquez 2022-12-16 4:16 ` Ray Andrews 0 siblings, 1 reply; 45+ messages in thread From: Lawrence Velázquez @ 2022-12-16 3:21 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Thu, Dec 15, 2022, at 9:56 PM, Ray Andrews wrote: > On 2022-12-15 18:09, Dominik Vogt wrote: >> Probability of 95% or more that the array gets modified after the >> initialisation by some stray code. Put prints all over the place >> to find the place where the array gets modified. > > Nope, I showed the output directly after the init. If something goes > wrong, it goes wrong during the init itself. Or before. > Lawrence: > >> It makes no sense to question the code before verifying that your > data isn't garbage. > > All the named keywords have exactly correct data, there's nothing not > to like as far as that. Dunno, it wouldn't be the first time that > there was some silly issue in the printout, no real problem at all, so > my first question assumed nothing. Wrong. It assumed that "main" contained the expected data and nothing extra. I've already asked you to use "typeset -p" to verify its actual contents, but you still haven't. > I think I'm on the scent, the array is global. If I zero it before all > the assignments (they're just numbers, no need to show the full > genealogy of them), then the interlopers disappear. If you're implementing this in one of your sourced scripts, then you should clear "main" anyway, or you risk reusing a preexisting array. > So now the > question is where the bogus elements come from. They aren't actually > assigned anywhere, that's for sure. I don't trust your judgment on this. > Variable crosstalk? How does an > associative array get corrupted? Let me root around a bit ... > hey ... you don't suppose a zcurses crash might do it? It's far more likely that you've been using a preexisting array or are inadvertently assigning to "main" from a function or your dotfiles or another location you haven't considered than that you've encountered a bug in zsh or zcurses. -- vq ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 3:21 ` Lawrence Velázquez @ 2022-12-16 4:16 ` Ray Andrews 2022-12-16 10:05 ` Dominik Vogt 0 siblings, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-16 4:16 UTC (permalink / raw) To: Lawrence Velázquez; +Cc: zsh-users On 2022-12-15 19:21, Lawrence Velázquez wrote: > Or before. Exactly the case here. This code has many global variables and ... well, no need to say more. > Wrong. It assumed that "main" contained the expected data and > nothing extra. I've already asked you to use "typeset -p" to verify > its actual contents, but you still haven't. No need, zeroing the array solves the issue. Mind, I suppose I could do a forensic on that, and figure out when and where the corruption occurs, and then, of course I'll use typeset -p and let you know. Right now I have more pressing issues. >> Variable crosstalk? How does an >> associative array get corrupted? Let me root around a bit ... >> hey ... you don't suppose a zcurses crash might do it? > It's far more likely that you've been using a preexisting array or > are inadvertently assigning to "main" from a function or your > dotfiles or another location you haven't considered than that you've > encountered a bug in zsh or zcurses. Sure, it was just a thought. Yes, probably some crosstalk. I think I'll rename the array in any case. 'main' is not likely to be unique and besides one of the zcurses windows in named 'main' too. Very unsanitary. Esp. given zsh's indiscipline with variables. I know, all the shells are the same, but my C training makes me cringe at it to this day. > > ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 4:16 ` Ray Andrews @ 2022-12-16 10:05 ` Dominik Vogt 2022-12-16 14:13 ` Ray Andrews 0 siblings, 1 reply; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 10:05 UTC (permalink / raw) To: zsh-users [-- Attachment #1: Type: text/plain, Size: 1154 bytes --] On Thu, Dec 15, 2022 at 08:16:46PM -0800, Ray Andrews wrote: > > On 2022-12-15 19:21, Lawrence Velázquez wrote: > > Or before. > Exactly the case here. This code has many global variables and ... well, no > need to say more. > > Wrong. It assumed that "main" contained the expected data and > > nothing extra. I've already asked you to use "typeset -p" to verify > > its actual contents, but you still haven't. > > No need, zeroing the array solves the issue. Mind, I suppose I could do a > forensic on that, and figure out when and where the corruption occurs The issue is not solved. You still have something that adds the bogus values before the reinitialisation. Whether this has some adverse effects depends on the scripts. (Sample script attached). Basically you probably do ~ this: init_main # ... main=($main) <-- Creates extra values # ... init_main printf "%-20s %s\n" ${(kv)main} This is a bug in your scripts and not in zsh. > and then, of course I'll use typeset -p Forget about the typeset, look for "main=$(main)" or similar. Ciao Dominik ^_^ ^_^ -- Dominik Vogt [-- Attachment #2: test --] [-- Type: text/plain, Size: 594 bytes --] #!/usr/bin/zsh emulate zsh List="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" mainWH="50" mainWW="92" typeset -A main init_main () { main[window]="main" main[list]='List' main[lastE]="$#List" main[hight]=$(( mainWH - 2 )) main[width]=$(( mainWW - 2 )) main[bottomE]="$main[hight]" main[topE]=1 main[currentE]=1 main[offset]=0 main[active]=1; echo ${(kv)main} } init_main main=($main) init_main printf "%-20s %s\n" ${(kv)main} ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 10:05 ` Dominik Vogt @ 2022-12-16 14:13 ` Ray Andrews 2022-12-16 15:19 ` Dominik Vogt 2022-12-16 16:30 ` Daniel Shahaf 0 siblings, 2 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-16 14:13 UTC (permalink / raw) To: zsh-users On 2022-12-16 02:05, Dominik Vogt wrote: > > The issue is not solved. You still have something that adds the > bogus values before the reinitialisation. Whether this has some > adverse effects depends on the scripts. (Sample script attached). > Basically you probably do ~ this: True Dominik, it's not solved, I should rather say I can put it on ice for a while while I attend to another issue more pressing. I will get back to it and I'll let you all know what I found. It is probably exactly as yourself and Bart suspect. I will catch it. Roman: I just want to point out that in general you need to quote the array expansion here or you'll get surprises if the array has empty keys and/or values. Consider this: typeset -A main=( foo '1' bar '2' baz '' qux '' ) printf '%-4s = %s\n' ${(kv)main} And the output: foo = 1 baz = bar 2 = qux ... this is exactly the sort of thing I was talking to Lawrence about -- not really a hard error, just some printout issue. I'll pursue that. I (think I) understand that AAs are very fragile creatures and as you demonstrate, easily confused. I'll bag it. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 14:13 ` Ray Andrews @ 2022-12-16 15:19 ` Dominik Vogt 2022-12-16 19:14 ` Ray Andrews 2022-12-16 16:30 ` Daniel Shahaf 1 sibling, 1 reply; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 15:19 UTC (permalink / raw) To: zsh-users On Fri, Dec 16, 2022 at 06:13:45AM -0800, Ray Andrews wrote: > not really a hard error, just some printout issue. I'll pursue > that. It's only "just a printout issue" because the quotes are missing is a harmless debug print statement. In different contexts this can be disastrous. Just make a habit to quote variables. For arrays, "$@" "${array[@]}" "${(kv)array[@]}" "${(@kv)array}" are safe. The shell automatically quotes each word of the array separately with "@" (but not with "*"). If you do not quote, you have to keep track of all possible valus of a variable and remember whether quoting is necessary or not every time you expand variables. In the wild, almost all scripts have bugs because of missing quoting, e.g. script snippets inside Jenkins pipeline definitions. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 15:19 ` Dominik Vogt @ 2022-12-16 19:14 ` Ray Andrews 0 siblings, 0 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-16 19:14 UTC (permalink / raw) To: zsh-users On 2022-12-16 07:19, Dominik Vogt wrote: > The shell automatically quotes each word of the array > separately with "@" (but not with "*"). That' s one of those little gotchas. These things lurk quietly and then bite you when you think you're safe. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 14:13 ` Ray Andrews 2022-12-16 15:19 ` Dominik Vogt @ 2022-12-16 16:30 ` Daniel Shahaf 2022-12-16 18:21 ` Ray Andrews 1 sibling, 1 reply; 45+ messages in thread From: Daniel Shahaf @ 2022-12-16 16:30 UTC (permalink / raw) To: zsh-users Ray Andrews wrote on Fri, 16 Dec 2022 14:13 +00:00: > ... this is exactly the sort of thing I was talking to Lawrence about -- > not really a hard error, just some printout issue. I'll pursue that. I > (think I) understand that AAs are very fragile creatures and as you > demonstrate, easily confused. I'll bag it. I think the better takeaway here is that the "default" (barest) syntax isn't always the right one. There's any number of examples of this: - Variable expansions, «${foo}». Scalars need to be quoted; arrays need to be quoted and @'d; assocs need to be quoted, @'d, and (kv)'d. The actual meaning of unquoted «${foo}» — expand to all non-null values — isn't commonly needed. - «ls *.txt» without a «--» guard. - «print -- $foo» without «-r». ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 16:30 ` Daniel Shahaf @ 2022-12-16 18:21 ` Ray Andrews 2022-12-16 19:04 ` Dominik Vogt 0 siblings, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-16 18:21 UTC (permalink / raw) To: zsh-users On 2022-12-16 08:30, Daniel Shahaf wrote: > I think the better takeaway here is that the "default" (barest) syntax > isn't always the right one. There's any number of examples of this: > > - Variable expansions, «${foo}». Scalars need to be quoted; arrays need > to be quoted and @'d; assocs need to be quoted, @'d, and (kv)'d. The > actual meaning of unquoted «${foo}» — expand to all non-null values — > isn't commonly needed. > > - «ls *.txt» without a «--» guard. > > - «print -- $foo» without «-r». > Yeah. I'm sure many of my own travails have been due to that kind of ... well, one could say 'shortcutting', but more often it's simply not understanding the vulnerability of unquoted vars. As a few weeks ago, when what was it ... something unquoted with '[]' in it tried to expand to a list of files. It's far to late to do anything about it design wise, but if it were up to me the shells would have evolved from their warm little ponds with a 'positive option' mentality vs. the 'negative option' that we currently have. Basically: If I want something I'll ask for it. If I want a var to expand to filenames I'll request that. If I want empty elements removed from an array I'll say so. As it is zsh loves to do helpful things whether you want them or not and you have to ask for the exemption. Mostly by quoting but the world would have been simpler if it had been the other way. Dunno, maybe not. When one is dealing on the command line, probably file-centrism makes sense. But when it's variable contents the opposite is true. Anyway the thing is to learn the lesson early: quote. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 18:21 ` Ray Andrews @ 2022-12-16 19:04 ` Dominik Vogt 2022-12-16 20:10 ` Ray Andrews 2022-12-16 20:25 ` Daniel Shahaf 0 siblings, 2 replies; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 19:04 UTC (permalink / raw) To: zsh-users On Fri, Dec 16, 2022 at 10:21:28AM -0800, Ray Andrews wrote: > Yeah. I'm sure many of my own travails have been due to that kind of ... > well, one could say 'shortcutting', but more often it's simply not > understanding the vulnerability of unquoted vars. As a few weeks ago, when > what was it ... something unquoted with '[]' in it tried to expand to a list > of files. It's far to late to do anything about it design wise, but if it > were up to me the shells would have evolved from their warm little ponds > with a 'positive option' mentality vs. the 'negative option' that we > currently have. Basically: If I want something I'll ask for it. If I want a > var to expand to filenames I'll request that. If I want empty elements > removed from an array I'll say so. As it is zsh loves to do helpful things > whether you want them or not and you have to ask for the exemption. Mostly > by quoting but the world would have been simpler if it had been the other > way. Dunno, maybe not. When one is dealing on the command line, probably > file-centrism makes sense. But when it's variable contents the opposite is > true. Anyway the thing is to learn the lesson early: quote. I think you have a grave misconception of shells. Shells are basically command line interpreters with some text processing. A command line is a string with shell specific instructions to generate a final list of constant strings. These are then interpreted as a command name and its arguments. There's no conception of symbolic values; all input is just text. Shells are are not "programming languages". The shell does not know about file names on the command line, it just expands strings to filenames if they contain globbing patterns. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 19:04 ` Dominik Vogt @ 2022-12-16 20:10 ` Ray Andrews 2022-12-16 21:15 ` Dominik Vogt 2022-12-16 20:25 ` Daniel Shahaf 1 sibling, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-16 20:10 UTC (permalink / raw) To: zsh-users On 2022-12-16 11:04, Dominik Vogt wrote: > I think you have a grave misconception of shells. Shells are > basically command line interpreters with some text processing. I know it. All the extra functionality accreted over the years. It's an evolutionary cul-de-sac in that the latter additions are somewhat clumsy in as much as they have to contend with the earlier design. Can't be helped. > There's no > conception of symbolic values; all input is just text. Except for those times when I wish it is was 'just text' but instead things are expanded. Anyway how is a variable name not a symbol for the contents of the variable? > Shells are > are not "programming languages". That's a strange thing to hear. A program is a list of things on asks a computer to accomplish and zsh accomplishes them. But you know more about all this than I do. > The shell does not know about > file names on the command line, it just expands strings to > filenames if they contain globbing patterns. > > Ciao > > Dominik ^_^ ^_^ > > -- > > Dominik Vogt > ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 20:10 ` Ray Andrews @ 2022-12-16 21:15 ` Dominik Vogt 2022-12-16 21:33 ` Bart Schaefer 2022-12-16 21:33 ` Ray Andrews 0 siblings, 2 replies; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 21:15 UTC (permalink / raw) To: zsh-users On Fri, Dec 16, 2022 at 12:10:22PM -0800, Ray Andrews wrote: > > I think you have a grave misconception of shells. Shells are > > basically command line interpreters with some text processing. > I know it. All the extra functionality accreted over the years. It's an > evolutionary cul-de-sac in that the latter additions are somewhat clumsy in > as much as they have to contend with the earlier design. Can't be helped. > > There's no > > conception of symbolic values; all input is just text. > Except for those times when I wish it is was 'just text' but instead things > are expanded. Anyway how is a variable name not a symbol for the contents > of the variable? But variables are not even a real part of the shell, the shell script or its data. Every process has an associated environment that consists of "key=value" pairs. The parameter syntax in the shell is just a way to manipulate the environment that is then passed to the started processes. The environment exists independently of the shell. > > Shells are > > are not "programming languages". > That's a strange thing to hear. A program is a list of things on asks a > computer to accomplish and zsh accomplishes them. But you know more about > all this than I do. To begin with, the shell does not even have a fixed syntax that could be written down in a rule book: You can change the meaning of the "program" from inside the program, for example by changing options at run time or by overwriting the IFS variable, e.g. $ A="a,b-c" $ echo "$A" a,b-c # fiddle with IFS $ IFS="," echo "$A" a b-c # or even read value from user input $ read IFS -<return> $ echo "$A" a,b c # fiddling with options $ X=(a b c d e) $ setopt ksh_arrays; echo "${#X}" 2 $ unsetopt ksh_arrays; echo "${#X}" 5 # user input defines type of parameter $ unset A $ A[foo]=bar zsh: A: assignment to invalid subscript range $ read X -A<return> $ typeset "$X" A $ A[foo]="bar" $ echo "$(kv)A}" foo bar # redefine commands, aliases or even builtins $ read X ls<return> $ alias echo="$X" $ echo "Hello, world!" ls: cannot access 'Hello, world!': No such file or directory I.e. the semantics of the script may depend on user input. Then, the commands that the shell runs are mostly not defined by the shell but externally. You'd probably describe the script ls . rm -f foobarbaz as "a program that lists non-hidden entries of the current directory and then removes the entry foobarbaz if it exists and is not a directory". But that is not true. What the script really does is "execute a program 'ls' with argument '.', then execute a program 'rm' with arguments '-f' 'foobarbaz'". What the called programs do is not part of the shell. So, unless you want to define the program as a sequence of "execute X with arguments Y1 ... Yn" the shell really doesn't do anything close to what the "program" is supposed to do. -- Of course modern shells have lots of self contained builtin commands and control structures that resemble programming languages. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 21:15 ` Dominik Vogt @ 2022-12-16 21:33 ` Bart Schaefer 2022-12-16 21:59 ` Dominik Vogt 2022-12-16 21:33 ` Ray Andrews 1 sibling, 1 reply; 45+ messages in thread From: Bart Schaefer @ 2022-12-16 21:33 UTC (permalink / raw) To: dominik.vogt, zsh-users On Fri, Dec 16, 2022 at 1:15 PM Dominik Vogt <dominik.vogt@gmx.de> wrote: > > But variables are not even a real part of the shell, the shell > script or its data. Every process has an associated environment > that consists of "key=value" pairs. This is entirely untrue. The shell has a separate parameter space that is unrelated to the process environment. That's why the "export" ("typeset +x") command exists: to move values from the shell parameter space into the environment space. You're just making things worse by trying to assert that shells are not programming languages. Stop. > Of course modern shells have lots of self contained builtin > commands and control structures that resemble programming > languages. Shells definitely are a form of programming language. That they combine a number of features not usually found all in the same language doesn't change that. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 21:33 ` Bart Schaefer @ 2022-12-16 21:59 ` Dominik Vogt 2022-12-16 22:15 ` Bart Schaefer 0 siblings, 1 reply; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 21:59 UTC (permalink / raw) To: zsh-users On Fri, Dec 16, 2022 at 01:33:22PM -0800, Bart Schaefer wrote: > You're just making things worse by trying to assert that shells are > not programming languages. Stop. They are not programming languages in the sense of the definition that is used in information theory. Still some people think they _are_ and thus completely misunderstand what's going on inside the shell when they type commands. One might call everything allows to turn pre-written source code into actions (input + side effects + output) a programming language (or interpreter for a language). I don't. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 21:59 ` Dominik Vogt @ 2022-12-16 22:15 ` Bart Schaefer 2022-12-16 23:33 ` Ray Andrews 2022-12-17 18:10 ` Ray Andrews 0 siblings, 2 replies; 45+ messages in thread From: Bart Schaefer @ 2022-12-16 22:15 UTC (permalink / raw) To: dominik.vogt, zsh-users On Fri, Dec 16, 2022 at 1:59 PM Dominik Vogt <dominik.vogt@gmx.de> wrote: > > They are not programming languages in the sense of the definition > that is used in information theory. I'd appreciate a pointer to that definition, since apparently my PhD in Computer Science is from too long ago to have covered it. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 22:15 ` Bart Schaefer @ 2022-12-16 23:33 ` Ray Andrews 2022-12-17 12:47 ` Pier Paolo Grassi 2022-12-17 18:10 ` Ray Andrews 1 sibling, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-16 23:33 UTC (permalink / raw) To: zsh-users On 2022-12-16 14:15, Bart Schaefer wrote: > > I'd appreciate a pointer to that definition, since apparently my PhD > in Computer Science is from too long ago to have covered it. Ha! Sometimes hairs need to be split, and Dominik is surely correct that zsh and say C are very different, but to say that zsh isn't a programing language will be a chore to sustain. Daniel: > You mean that you wrote foo[bar] and didn't realize that was > a globbing syntax too? zsh can't disable features because you don't > know them. This stuff is all so fundamental that it couldn't change even if everyone wanted it to. Disable isn't the word I chose. But there is something very unfriendly about situations where you're doing something with an array index -- and it obvious at least to oneself that this is a string thing, and then zsh zealously applies filename expansions to it when it has nothing whatsoever to do with files. I don't mind being clobbered, that's part of learning the shell, still it isn't very intuitive. Anyway I've learned the protections. But one can want to do some string globbing and still hope not to have anything to do with filenames. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 23:33 ` Ray Andrews @ 2022-12-17 12:47 ` Pier Paolo Grassi 2022-12-17 17:32 ` Ray Andrews 0 siblings, 1 reply; 45+ messages in thread From: Pier Paolo Grassi @ 2022-12-17 12:47 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users [-- Attachment #1: Type: text/plain, Size: 2499 bytes --] it certainly is to be said that, even if the shell implements all the basic programming constructs, it is meant for different objectives then those that are targeted by the things one most commonly calls "programming languages", and even among those there are many distinctions, so probably it wouldn't be so easy to establish an exact definition that includes only the "ideals" (whatever that means) programming languages and leaves everything else out. That being said, I think that one should accept the language for what it is and what is meant to be and do, and even if something would be awesome if it was engineered with hindsight (like array expansion without empty string elision with a more same syntax, or by default) it should be wise to understand what the language is or is not before dipping too much one's arms and feets in it, reading the manual at least a couple of times front to bottom, since someone has put the effort to write it. At least I try to do it this way, I experiment a bit but then I try to read everything I can before resorting to ask for help, it seems only fair given that someone has to put aside whatever he is doing to answer me. Pier Paolo Grassi Il giorno sab 17 dic 2022 alle ore 00:33 Ray Andrews <rayandrews@eastlink.ca> ha scritto: > > On 2022-12-16 14:15, Bart Schaefer wrote: > > > > I'd appreciate a pointer to that definition, since apparently my PhD > > in Computer Science is from too long ago to have covered it. > > Ha! Sometimes hairs need to be split, and Dominik is surely correct > that zsh and say C are very different, but to say that zsh isn't a > programing language will be a chore to sustain. > > Daniel: > > > You mean that you wrote foo[bar] and didn't realize that was > > a globbing syntax too? zsh can't disable features because you don't > > know them. > > This stuff is all so fundamental that it couldn't change even if > everyone wanted it to. Disable isn't the word I chose. But there is > something very unfriendly about situations where you're doing something > with an array index -- and it obvious at least to oneself that this is a > string thing, and then zsh zealously applies filename expansions to it > when it has nothing whatsoever to do with files. I don't mind being > clobbered, that's part of learning the shell, still it isn't very > intuitive. Anyway I've learned the protections. But one can want to > do some string globbing and still hope not to have anything to do with > filenames. > > > > [-- Attachment #2: Type: text/html, Size: 3214 bytes --] ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 12:47 ` Pier Paolo Grassi @ 2022-12-17 17:32 ` Ray Andrews 0 siblings, 0 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-17 17:32 UTC (permalink / raw) To: zsh-users [-- Attachment #1: Type: text/plain, Size: 3128 bytes --] On 2022-12-17 04:47, Pier Paolo Grassi wrote: > it certainly is to be said that, even if the shell implements all the > basic programming constructs, it is meant for different > objectives then those that are targeted by the things one most > commonly calls "programming languages", and even among those there are > many distinctions, so probably it wouldn't be so easy to establish an > exact definition that includes only the "ideals" (whatever that means) > programming languages and leaves everything else out. That being said, > I think that one should accept the language for what it is and what is > meant to be and do,and even if something would be awesome if it was > engineered with hindsight (like array expansion without empty string > elision with a more same syntax, or by default) it should be wise to > understand what the language is or is not before dipping too much > one's arms and feets in it, I agree. One should know what to expect and what is possible. But one's understanding of these things comes from banging up against the limitations. My criticisms are not meant to be in a negative frame of mind, I'm just given to speculation on what might be improved, and often I'm not even correct. > reading the manual at least a couple of times front to bottom, since > someone has put the effort to write it. At least I try to do it this > way, I experiment a bit but then I try to read everything I can before > resorting to ask for help, it seems only fair given that someone has > to put aside whatever he is doing to answer me. I will experiment and read and Google for many hours before asking a question. > > Pier Paolo Grassi > > > Il giorno sab 17 dic 2022 alle ore 00:33 Ray Andrews > <rayandrews@eastlink.ca> ha scritto: > > > On 2022-12-16 14:15, Bart Schaefer wrote: > > > > I'd appreciate a pointer to that definition, since apparently my PhD > > in Computer Science is from too long ago to have covered it. > > Ha! Sometimes hairs need to be split, and Dominik is surely correct > that zsh and say C are very different, but to say that zsh isn't a > programing language will be a chore to sustain. > > Daniel: > > > You mean that you wrote foo[bar] and didn't realize that was > > a globbing syntax too? zsh can't disable features because you don't > > know them. > > This stuff is all so fundamental that it couldn't change even if > everyone wanted it to. Disable isn't the word I chose. But there is > something very unfriendly about situations where you're doing > something > with an array index -- and it obvious at least to oneself that > this is a > string thing, and then zsh zealously applies filename expansions > to it > when it has nothing whatsoever to do with files. I don't mind being > clobbered, that's part of learning the shell, still it isn't very > intuitive. Anyway I've learned the protections. But one can > want to > do some string globbing and still hope not to have anything to do > with > filenames. > > > [-- Attachment #2: Type: text/html, Size: 5399 bytes --] ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 22:15 ` Bart Schaefer 2022-12-16 23:33 ` Ray Andrews @ 2022-12-17 18:10 ` Ray Andrews 2022-12-17 18:19 ` Roman Perepelitsa 1 sibling, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-17 18:10 UTC (permalink / raw) To: zsh-users Getting some scent on my troubles with the AA: I import one AA into another like this: n_input () { set -A IN ${(Pkv)${1}} # Copy the array to IN, the working copy. typeset -p $1 > /dev/pts/2 typeset -p IN > /dev/pts/2 .... ... and Roman has said this is correct. For all the functions that call n_input() everything is fine. Given the array 'main' as '$1' to the function, typesets show: typeset -g -A main=( [active]=1 [bottomE]=48 [currentE]=1 [hight]=48 [lastE]=45 [list]=List [offset]=0 [topE]=1 [width]=90 [window]=Main ) typeset -g -A IN =( [active]=1 [bottomE]=48 [currentE]=1 [hight]=48 [lastE]=45 [list]=List [offset]=0 [topE]=1 [width]=90 [window]=Main ) ... perfect. But there is one exception. The input array is the same but the output turns into a normal array! Input array is 'files': typeset -g -A files=( [active]=1 [bottomE]=48 [currentE]=1 [hight]=48 [lastE]=332 [list]=files_list [offset]=0 [topE]=1 [width]=16 [window]=files ) typeset -g -a IN =( width 16 window files offset 0 topE 1 active 1 bottomE 48 lastE 332 hight 48 list files_list currentE 1 ) ... subsequent assignments are obviously going to be haywire and they are as we've discussed. Now why on earth does that happen? All the keywords are there, and all the values are correct, but needless to say I want it to be an AA, not a normal array. When 'IN' is written back to the the input array ... sproingggg .... BTW the order in which the elements are initialized at first go seems unrelated, it doesn't match either typeset. I thought there might be some artifact in the input values but can see anything. The array consists of two strings and 8 numbers. Quoting makes no difference. Options are the same. I haven't a clue where to go from here. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 18:10 ` Ray Andrews @ 2022-12-17 18:19 ` Roman Perepelitsa 2022-12-17 20:31 ` Ray Andrews 0 siblings, 1 reply; 45+ messages in thread From: Roman Perepelitsa @ 2022-12-17 18:19 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Sat, Dec 17, 2022 at 7:11 PM Ray Andrews <rayandrews@eastlink.ca> wrote: > > > Getting some scent on my troubles with the AA: > > I import one AA into another like this: > > n_input () > > { > > set -A IN ${(Pkv)${1}} # Copy the array to IN, the working copy. > > typeset -p $1 > /dev/pts/2 > typeset -p IN > /dev/pts/2 Try this: n_input() { local -A IN set -A IN "${(@Pkv)1}" } Note: 1. IN needs to be declared. 2. Quotes are necessary to handle empty keys and/or values. 3. The extra $ expansion is not necessary. Roman. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 18:19 ` Roman Perepelitsa @ 2022-12-17 20:31 ` Ray Andrews 2022-12-17 20:49 ` Bart Schaefer 2022-12-17 21:07 ` Lawrence Velázquez 0 siblings, 2 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-17 20:31 UTC (permalink / raw) To: zsh-users On 2022-12-17 10:19, Roman Perepelitsa wrote: > n_input() { > local -A IN > set -A IN "${(@Pkv)1}" > } > > Note: > > 1. IN needs to be declared. > 2. Quotes are necessary to handle empty keys and/or values. > 3. The extra $ expansion is not necessary. > > Roman. > Right as usual Roman. The next puzzle is why it was working without that previously. I had 'IN' declared globally and it was fine. I'll restore a backup and probe that. Mind, it's better to have IN local anyway, so maybe I should let sleeping dogs lie. I don't understand the difference between '-A' and '+A' maybe that's part of it -- the global array might not be overwritten cleanly or something. But local is best anyway. Thank you Sensei. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 20:31 ` Ray Andrews @ 2022-12-17 20:49 ` Bart Schaefer 2022-12-17 21:07 ` Lawrence Velázquez 1 sibling, 0 replies; 45+ messages in thread From: Bart Schaefer @ 2022-12-17 20:49 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Sat, Dec 17, 2022 at 12:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote: > > I don't understand > the difference between '-A' and '+A' maybe that's part of it The difficulty here (unfortunately) is that "-A" doesn't always mean the same thing. "typeset -a" means a normal array and "typeset -A" (or "local" in either case) means an associative array. "set -a" was taken (short for "setopt ALL_EXPORT" from sh compatibility) so "set -A" means "assign to array" where "array" may be either normal or associative. But if the array doesn't already exist, it is implicitly created as a normal array. So you must pre-declare names with "local -A" or similar if you want them treated as associative. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 20:31 ` Ray Andrews 2022-12-17 20:49 ` Bart Schaefer @ 2022-12-17 21:07 ` Lawrence Velázquez 2022-12-17 21:52 ` Ray Andrews 2022-12-17 23:25 ` Daniel Shahaf 1 sibling, 2 replies; 45+ messages in thread From: Lawrence Velázquez @ 2022-12-17 21:07 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users Bart already addressed the actual issue, so I'll just touch on this: On Sat, Dec 17, 2022, at 3:31 PM, Ray Andrews wrote: > I don't understand > the difference between '-A' and '+A' maybe that's part of it -- the > global array might not be overwritten cleanly or something. If the array is already populated, "set +A" only replaces its leading elements, which means existing elements can get left behind. % arr=({a..j}); typeset -p arr typeset -a arr=( a b c d e f g h i j ) % set -A arr {1..5}; typeset -p arr typeset -a arr=( 1 2 3 4 5 ) % set +A arr x y z; typeset -p arr typeset -a arr=( x y z 4 5 ) I'm not sure why you brought this up. Are you using "set +A" somewhere? -- vq ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 21:07 ` Lawrence Velázquez @ 2022-12-17 21:52 ` Ray Andrews 2022-12-17 22:10 ` Bart Schaefer 2022-12-17 23:25 ` Daniel Shahaf 1 sibling, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-17 21:52 UTC (permalink / raw) To: zsh-users On 2022-12-17 13:07, Lawrence Velázquez wrote: > > I'm not sure why you brought this up. Are you using "set +A" > somewhere? No, but I'm still curious as to how my array got fouled. As just discussed with Roman, I'd been using a global 'IN' and thus it will contain whatever was in it at the next assignment, and it seemed to me that a possibility was that the overwrite was not clean. Reading up on it, there's that +A and -- tho I don't really understand --, it seemed possible that it might force a complete overwrite. But I'm absolutely happy with my local IN, that's got to be better anyway. And, per your advice, I have 'typeset -p's locked and loaded all over the place. Commented, but ready to spring into action at the next sign of trouble. This is another lesson in how one can't take arrays for granted, especially, it seems, AAs. BTW tx for example of what +A does, now I understand. Easy if it's explained clearly. Bart: > ... assign to array" where "array" may be either normal or associative. But if the array doesn't already exist, it is implicitly created as a normal array. So you must pre-declare names with "local -A" or similar if you want them treated as associative. Too bad it's not created as the same type as the input. Indeed, aren't there cases where even declared types are changed? Like, don't scalars convert to integers when you perform arithmetic on them? By the same logic one might expect IN to become associative by virtue of the assignment even if it was a plain vanilla '-a' previously. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 21:52 ` Ray Andrews @ 2022-12-17 22:10 ` Bart Schaefer 2022-12-17 23:41 ` Daniel Shahaf 2022-12-18 0:15 ` Ray Andrews 0 siblings, 2 replies; 45+ messages in thread From: Bart Schaefer @ 2022-12-17 22:10 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Sat, Dec 17, 2022 at 1:52 PM Ray Andrews <rayandrews@eastlink.ca> wrote: > > Too bad it's not created as the same type as the input. The input doesn't have a type. It's the result of a substitution, so it's just a series of separate words. Where the words came from is not remembered. Consider: set -A thing new1 new2 "${(@kv)whatever}" new3 new4 What "type" is that? > Indeed, aren't > there cases where even declared types are changed? Like, don't scalars > convert to integers when you perform arithmetic on them? That happens in a limited number of circumstances where the type being assigned-to is known and the conversion is one-to-one. Note the importance even then of knowing the type being assigned-to, which is exactly why you have to declare it before calling "set -A". ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 22:10 ` Bart Schaefer @ 2022-12-17 23:41 ` Daniel Shahaf 2022-12-18 0:15 ` Ray Andrews 1 sibling, 0 replies; 45+ messages in thread From: Daniel Shahaf @ 2022-12-17 23:41 UTC (permalink / raw) To: zsh-users; +Cc: Ray Andrews Bart Schaefer wrote on Sat, Dec 17, 2022 at 14:10:32 -0800: > On Sat, Dec 17, 2022 at 1:52 PM Ray Andrews <rayandrews@eastlink.ca> wrote: > > > > Too bad it's not created as the same type as the input. > > The input doesn't have a type. It's the result of a substitution, so > it's just a series of separate words. Where the words came from is > not remembered. Consider: > > set -A thing new1 new2 "${(@kv)whatever}" new3 new4 > > What "type" is that? > Also consider: [[[ cd "$(mktemp -d)" # create a new, empty directory and cd into it touch -- -l foo wc * ]]] What will that print? ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 22:10 ` Bart Schaefer 2022-12-17 23:41 ` Daniel Shahaf @ 2022-12-18 0:15 ` Ray Andrews 2022-12-18 0:25 ` Daniel Shahaf 2022-12-18 3:44 ` Lawrence Velázquez 1 sibling, 2 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-18 0:15 UTC (permalink / raw) To: zsh-users On 2022-12-17 14:10, Bart Schaefer wrote: > The input doesn't have a type. It's the result of a substitution, so > it's just a series of separate words. Where the words came from is > not remembered. Consider: I suppose it could be looked at that way, OTOH it could be that the type is referenced. After all the type *is* recorded. typeset -p doesn't just cough up the contents, it knows that an AA should display differently than a normal array. Since the type information is there, there's no reason why it might not be referred to. > > set -A thing new1 new2 "${(@kv)whatever}" new3 new4 > > What "type" is that? set -A IN "${(@Pkv)${1}}" # Copy the array to IN, the working copy. > Not fair. In that case 'thing' is not assigned any type, so it's defaulting to a normal array is unavoidable. But if I do: set -A IN "${(@Pkv)${1}}" .. and $1 names an array that typedef knows is an AA, then it wouldn't be very much work for IN to be matched with that type. It would avoid the gotcha that lead to my question. Mind, as usual, once one is forewarned of the issue it's not hard to deal with. As I mentioned, my case would rely on the precedent that that's the way it is with integer promotion. Good thing too, because doing arithmetic on a scalar is slightly absurd. Note the importance even then of knowing the type being assigned-to, which is > exactly why you have to declare it before calling "set -A". Well it's simple and hopefully failsafe, but I still can't help but thinking that if typedef knows the type of the input then the type of the output could be assigned as well as the values. Daniel: > Could someone please clarify this in the manual? Thanks. It makes me feel less stupid when someone agrees with me that the manual isn't as clear as it could be. The manual was written by someone too intelligent for the job -- no empathy for the less gifted. > ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 0:15 ` Ray Andrews @ 2022-12-18 0:25 ` Daniel Shahaf 2022-12-18 2:13 ` Ray Andrews 2022-12-18 3:44 ` Lawrence Velázquez 1 sibling, 1 reply; 45+ messages in thread From: Daniel Shahaf @ 2022-12-18 0:25 UTC (permalink / raw) To: zsh-users Ray Andrews wrote on Sun, 18 Dec 2022 00:15 +00:00: > On 2022-12-17 14:10, Bart Schaefer wrote: >> Could someone please clarify this in the manual? > > Thanks. It makes me feel less stupid when someone agrees with me that > the manual isn't as clear as it could be. The manual was written by > someone too intelligent for the job -- no empathy for the less gifted. No /ad homines/ arguments. If you want to propose new text for that part of the manual, please go ahead. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 0:25 ` Daniel Shahaf @ 2022-12-18 2:13 ` Ray Andrews 2022-12-18 20:12 ` Daniel Shahaf 0 siblings, 1 reply; 45+ messages in thread From: Ray Andrews @ 2022-12-18 2:13 UTC (permalink / raw) To: zsh-users On 2022-12-17 16:25, Daniel Shahaf wrote: > > No /ad homines/ arguments. > > If you want to propose new text for that part of the manual, please go ahead. I have the opposite problem of not being competent to improve it. It's not really an ad hom. When one becomes expert, one 'forgets' what it was like to not know. Over a beer I could tell some interesting stories about teaching people to use tools. What is so obvious to me that I don't even know I know it can be a blank to a novice. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 2:13 ` Ray Andrews @ 2022-12-18 20:12 ` Daniel Shahaf 2022-12-18 20:26 ` Bart Schaefer 0 siblings, 1 reply; 45+ messages in thread From: Daniel Shahaf @ 2022-12-18 20:12 UTC (permalink / raw) To: zsh-users Here is the incumbent text: [[[ If tt(PLUS()A) is used and var(name) is an array, the given arguments will replace the initial elements of that array; if no var(name) is specified, all arrays are printed without their values. ]]] What does it get right? What doesn't it get right? What should be gotten right that isn't being gotten right? How might that be phrased? ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 20:12 ` Daniel Shahaf @ 2022-12-18 20:26 ` Bart Schaefer 2022-12-18 20:41 ` Ray Andrews 0 siblings, 1 reply; 45+ messages in thread From: Bart Schaefer @ 2022-12-18 20:26 UTC (permalink / raw) To: Daniel Shahaf; +Cc: zsh-users On Sun, Dec 18, 2022 at 12:12 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > [[[ > If tt(PLUS()A) is used and var(name) is an array, the > given arguments will replace the initial elements of that array; if no > var(name) is specified, all arrays are printed without their values. > ]]] > > What does it get right? What doesn't it get right? What should be > gotten right that isn't being gotten right? How might that be phrased? So, there are a couple of problems with it. (1) It's not explicit about what "initial elements" are; that could still refer to the entire array, or to the subset of current elements with which the array was first initialized. (2) It doesn't explain what effect this has on associative arrays, which are unordered. For (1) we need to state that, when there are N arguments to "set +A", normal array positions 1 - N (or 0 - N-1 for ksharrays) are replaced, and arguments N+1 (ibid.) are unchanged. For (2) the entire associative array is replaced for +A just as it is for -A. I suppose that could be altered so that it works like aa+=(...) which might be useful. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 20:26 ` Bart Schaefer @ 2022-12-18 20:41 ` Ray Andrews 0 siblings, 0 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-18 20:41 UTC (permalink / raw) To: zsh-users On 2022-12-18 12:26, Bart Schaefer wrote: > So, there are a couple of problems with it. > I'd suggest that an example or two would be better than an involved theoretical exposition. Daniel's examples just recently made it instantly and completely clear. Glad to see you guys thinking about it tho. It's such a labor to write a really good doc and so easy to let that slip. Mind, the manual as we have it is a masterpiece of a certain kind of writing -- terse but exact, unfriendly but complete. Nice to have it a little more approachable however. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-18 0:15 ` Ray Andrews 2022-12-18 0:25 ` Daniel Shahaf @ 2022-12-18 3:44 ` Lawrence Velázquez 1 sibling, 0 replies; 45+ messages in thread From: Lawrence Velázquez @ 2022-12-18 3:44 UTC (permalink / raw) To: Ray Andrews; +Cc: zsh-users On Sat, Dec 17, 2022, at 7:15 PM, Ray Andrews wrote: > On 2022-12-17 14:10, Bart Schaefer wrote: >> The input doesn't have a type. It's the result of a substitution, so >> it's just a series of separate words. Where the words came from is >> not remembered. Consider: > I suppose it could be looked at that way It could be looked at that way because that is the way it is. > OTOH it could be that the type > is referenced. After all the type *is* recorded. typeset -p doesn't > just cough up the contents, it knows that an AA should display > differently than a normal array. Since the type information is there, > there's no reason why it might not be referred to. Parameter substitutions do not communicate variable attributes, nor should they. A core design tenet of Bourne-adjacent shells is that, once the dust settles, using a substitution is equivalent to using the substituted values directly. Thus the following sets of commands work identically in the end: % typeset -A arr1 % arr1[a]=1 % arr1[b]= % arr1[c]=3 % set -A arr2 "${(@kv)arr1}" % typeset -p arr2 typeset -a arr2=( a 1 b '' c 3 ) % unset arr2 % set -A arr2 a 1 b '' c 3 % typeset -p arr2 typeset -a arr2=( a 1 b '' c 3 ) Your proposal would make a hash of this, introducing special cases that are incoherent and work differently for no good reason. It would also likely require "set" to be reimplemented as a keyword, which would be ridiculous. >> set -A thing new1 new2 "${(@kv)whatever}" new3 new4 >> >> What "type" is that? set -A IN "${(@Pkv)${1}}" # Copy the array to IN, the working copy. >> > Not fair. In that case 'thing' is not assigned any type, so it's > defaulting to a normal array is unavoidable. But if I do: > > set -A IN "${(@Pkv)${1}}" > > .. and $1 names an array that typedef knows is an AA, then it wouldn't > be very much work for IN to be matched with that type. So the variable should only be created as an associative array if all the "set" arguments are produced by a single parameter expansion of another associative array? That's a very limited use case. > As I mentioned, my > case would rely on the precedent that that's the way it is with integer > promotion. I assume you're thinking about what happens here: % var=(a b c) % typeset -p var typeset -a var=( a b c ) % ((var = 1)) % typeset -p var typeset -i var=1 Arithmetic contexts are their own little universe. You can't really use them as a comparison point here. > Well it's simple and hopefully failsafe, but I still can't help but > thinking that if typedef knows the type of the input then the type of > the output could be assigned as well as the values. Should the following "set" command automatically create "out" as an associative array? After all, all its arguments come from a single parameter expansion of an existing associative array, just like your other examples. % typeset -A in=(a 121 b 121 c 23) % set -A out "$in" zsh: bad set of key/value pairs for associative array Hm. Okay, only if the expansion produces separate words. % set -A out "${(@)in}" zsh: bad set of key/value pairs for associative array Okay, only if it asks for both keys and values. % set -A out "${(@kvu)in}" zsh: bad set of key/value pairs for associative array Okay, only if it actually makes use of all the keys and values. % set -A out "${(@kvs:2:)in}" zsh: bad set of key/value pairs for associative array Okay, only if it doesn't modify the keys and values. % set -A out "${(@kvo)in}" % typeset -p out typeset -A out=( [121]=121 [23]=a [b]=c ) Yay! It ... works? I'll stop here. Your desired behavior would have be applied so narrowly as to be essentially useless -- and certainly not worth the added complexity. > Daniel: > >> Could someone please clarify this in the manual? > > Thanks. It makes me feel less stupid when someone agrees with me that > the manual isn't as clear as it could be. No one actually thinks that the manual is perfectly clear. The disagreement is with your notion that the manual should be more verbose and have more of a tutorial vibe, when it is a reference and should be treated as such. > The manual was written by > someone too intelligent for the job -- no empathy for the less gifted. Your learned helplessness is becoming tiresome. -- vq ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-17 21:07 ` Lawrence Velázquez 2022-12-17 21:52 ` Ray Andrews @ 2022-12-17 23:25 ` Daniel Shahaf 1 sibling, 0 replies; 45+ messages in thread From: Daniel Shahaf @ 2022-12-17 23:25 UTC (permalink / raw) To: Lawrence Velázquez; +Cc: Ray Andrews, zsh-users Lawrence Velázquez wrote on Sat, Dec 17, 2022 at 16:07:50 -0500: > Bart already addressed the actual issue, so I'll just touch on this: > > On Sat, Dec 17, 2022, at 3:31 PM, Ray Andrews wrote: > > I don't understand > > the difference between '-A' and '+A' maybe that's part of it -- the > > global array might not be overwritten cleanly or something. > > If the array is already populated, "set +A" only replaces its leading > elements, which means existing elements can get left behind. > > % arr=({a..j}); typeset -p arr > typeset -a arr=( a b c d e f g h i j ) > % set -A arr {1..5}; typeset -p arr > typeset -a arr=( 1 2 3 4 5 ) > % set +A arr x y z; typeset -p arr > typeset -a arr=( x y z 4 5 ) > > I'm not sure why you brought this up. Are you using "set +A" > somewhere? Could someone please clarify this in the manual? zshbuiltins(1) says "will replace the initial elements of the array" and it's not clear that "initial elements" doesn't mean "all elements" but only the prefix ones. Thanks, Daniel ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 21:15 ` Dominik Vogt 2022-12-16 21:33 ` Bart Schaefer @ 2022-12-16 21:33 ` Ray Andrews 1 sibling, 0 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-16 21:33 UTC (permalink / raw) To: zsh-users On 2022-12-16 13:15, Dominik Vogt wrote: > > But variables are not even a real part of the shell, the shell > script or its data. Every process has an associated environment > that consists of "key=value" pairs. The parameter syntax in the > shell is just a way to manipulate the environment that is then > passed to the started processes. The environment exists > independently of the shell. Sure, but the shell manipulates that environment, which is what we want it to do. And the various expansions are surely entirely zsh's own doing. I don't think there's anything to disagree about really. I do take your point tho. zsh is not comparable to, say, C. At first I thought it would be and it took a while to beat that out of me. To begin with, the shell does not even have a fixed syntax that > could be written down in a rule book: I've noticed! The parser code must be formidable. > Then, the commands that the shell runs are mostly not defined by > the shell but externally. You'd probably describe the script > > ls . > rm -f foobarbaz > > as "a program that lists non-hidden entries of the current > directory and then removes the entry foobarbaz if it exists and is > not a directory". But that is not true. What the script really > does is "execute a program 'ls' with argument '.', then execute a > program 'rm' with arguments '-f' 'foobarbaz'". What the called > programs do is not part of the shell. So, unless you want to > define the program as a sequence of "execute X with arguments Y1 > ... Yn" the shell really doesn't do anything close to what the > "program" is supposed to do. I understand. The shell is for interacting with command line binaries and friends. But it's internal power is such that it can get a whole lot done. I remember in C when you had to call an external program it was ... 'unnatural', but of course zsh is exactly the opposite -- it is a facilitator of external programs. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 19:04 ` Dominik Vogt 2022-12-16 20:10 ` Ray Andrews @ 2022-12-16 20:25 ` Daniel Shahaf 2022-12-16 21:43 ` Dominik Vogt 1 sibling, 1 reply; 45+ messages in thread From: Daniel Shahaf @ 2022-12-16 20:25 UTC (permalink / raw) To: zsh-users Dominik Vogt wrote on Fri, 16 Dec 2022 19:04 +00:00: > On Fri, Dec 16, 2022 at 10:21:28AM -0800, Ray Andrews wrote: >> Yeah. I'm sure many of my own travails have been due to that kind of ... >> well, one could say 'shortcutting', but more often it's simply not >> understanding the vulnerability of unquoted vars. As a few weeks ago, when >> what was it ... something unquoted with '[]' in it tried to expand to a list >> of files. It's far to late to do anything about it design wise, but if it >> were up to me the shells would have evolved from their warm little ponds >> with a 'positive option' mentality vs. the 'negative option' that we >> currently have. Basically: If I want something I'll ask for it. If I want a >> var to expand to filenames I'll request that. You mean that you wrote foo[bar] and didn't realize that was a globbing syntax too? zsh can't disable features because you don't know them. I suppose you could go ahead and disable all glob characters other than those you know, or perhaps you could propose a GLOB_VERIFY option akin to HIST_VERIFY. >> If I want empty elements >> removed from an array I'll say so. As it is zsh loves to do helpful things >> whether you want them or not and you have to ask for the exemption. Mostly >> by quoting but the world would have been simpler if it had been the other >> way. Dunno, maybe not. When one is dealing on the command line, probably >> file-centrism makes sense. But when it's variable contents the opposite is >> true. Anyway the thing is to learn the lesson early: quote. > > I think you have a grave misconception of shells. Shells are > basically command line interpreters with some text processing. A > command line is a string with shell specific instructions to > generate a final list of constant strings. These are then > interpreted as a command name and its arguments. There's no > conception of symbolic values; all input is just text. Shells are > are not "programming languages". The shell does not know about > file names on the command line, it just expands strings to > filenames if they contain globbing patterns. Yeah, the syntax rules are well-defined, but they aren't always intuitive. I don't know of any other language that does null elision on arrays, for instance: in most languages, if you loop over an array that contains ('foo' '' 'bar'), you get three iterations. In zsh you get two unless you specifically request three. In other words, the "default" syntax (the one with the fewest tokens) doesn't give the should-be-default behaviour. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 20:25 ` Daniel Shahaf @ 2022-12-16 21:43 ` Dominik Vogt 0 siblings, 0 replies; 45+ messages in thread From: Dominik Vogt @ 2022-12-16 21:43 UTC (permalink / raw) To: zsh-users On Fri, Dec 16, 2022 at 08:25:54PM +0000, Daniel Shahaf wrote: > Yeah, the syntax rules are well-defined, but they aren't always > intuitive. I don't know of any other language that does null elision on > arrays, for instance: in most languages, if you loop over an array that > contains ('foo' '' 'bar'), you get three iterations. I'd not exactly call it a "language" but it's mostly the same in fvwm scripts. If the last level of quoting around an empty string goes away, the argument evaporates. That's because the fvwm syntax was inspired by shells. Ciao Dominik ^_^ ^_^ -- Dominik Vogt ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-15 23:48 uninvited members of associative array Ray Andrews 2022-12-16 0:05 ` Lawrence Velázquez @ 2022-12-16 3:20 ` Bart Schaefer 2022-12-16 4:15 ` Ray Andrews 2022-12-16 8:12 ` Roman Perepelitsa 2 siblings, 1 reply; 45+ messages in thread From: Bart Schaefer @ 2022-12-16 3:20 UTC (permalink / raw) To: Ray Andrews; +Cc: Zsh Users On Thu, Dec 15, 2022 at 3:48 PM Ray Andrews <rayandrews@eastlink.ca> wrote: > > I just noticed something. My big array has ten named elements but if I print it: > ... it all works, however 'elements' 90, 196, 0 and 1 seem to have created themselves Let's look at the "mystery" elements: > 90 main > 196 48 > 0 1 > 1 48 > List 1 Now let's pair off the values of the "expected" elements: > width 90 > window main > offset 0 > topE 1 > active 1 > bottomE 48 > lastE 196 > hight 48 > list List > currentE 1 Notice anything? I strongly suspect that somewhere you've tried to copy or save/restore the array by doing something like list=( $main ) main=( $list ) but have forgotten that for an associative array $main means only the values, not the keys as well. Given that you later say "main" is global, you may even have done this accidentally while working on what you thought was another parameter named "main", and now you're just finding the leftovers. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-16 3:20 ` Bart Schaefer @ 2022-12-16 4:15 ` Ray Andrews 0 siblings, 0 replies; 45+ messages in thread From: Ray Andrews @ 2022-12-16 4:15 UTC (permalink / raw) To: zsh-users On 2022-12-15 19:20, Bart Schaefer wrote: > > I strongly suspect that somewhere you've tried to copy or save/restore > the array by doing something like > > list=( $main ) > main=( $list ) > > but have forgotten that for an associative array $main means only the > values, not the keys as well. > > Given that you later say "main" is global, you may even have done this > accidentally while working on what you thought was another parameter > named "main", and now you're just finding the leftovers. > No time right now, but I'm going to bet that's exactly what's happened, I'd naively do exactly as you suggest. I've only just started playing with AAs. It's begging for trouble that I have " zcurses addwin main " and many similar. And: " main[window]=main ". But this goes back to Sebastian's original code. I've been meaning to do something about it for a long time. ^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: uninvited members of associative array 2022-12-15 23:48 uninvited members of associative array Ray Andrews 2022-12-16 0:05 ` Lawrence Velázquez 2022-12-16 3:20 ` Bart Schaefer @ 2022-12-16 8:12 ` Roman Perepelitsa 2 siblings, 0 replies; 45+ messages in thread From: Roman Perepelitsa @ 2022-12-16 8:12 UTC (permalink / raw) To: Ray Andrews; +Cc: Zsh Users On Fri, Dec 16, 2022 at 12:48 AM Ray Andrews <rayandrews@eastlink.ca> wrote: > > I just noticed something. My big array has ten named elements but if I print it: > > printf "%-20s %s\n" ${(kv)main} > /dev/pts/2 I just want to point out that in general you need to quote the array expansion here or you'll get surprises if the array has empty keys and/or values. Consider this: typeset -A main=( foo '1' bar '2' baz '' qux '' ) printf '%-4s = %s\n' ${(kv)main} And the output: foo = 1 baz = bar 2 = qux Notice that the output is wrong. Quotes to the rescue: printf '%-4s = %s\n' "${(@kv)main}" Now it works: foo = 1 baz = bar = 2 qux = Roman. ^ permalink raw reply [flat|nested] 45+ messages in thread
end of thread, other threads:[~2022-12-18 20:45 UTC | newest] Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-12-15 23:48 uninvited members of associative array Ray Andrews 2022-12-16 0:05 ` Lawrence Velázquez 2022-12-16 0:48 ` Ray Andrews 2022-12-16 1:29 ` Lawrence Velázquez 2022-12-16 2:09 ` Dominik Vogt 2022-12-16 2:56 ` Ray Andrews 2022-12-16 3:21 ` Lawrence Velázquez 2022-12-16 4:16 ` Ray Andrews 2022-12-16 10:05 ` Dominik Vogt 2022-12-16 14:13 ` Ray Andrews 2022-12-16 15:19 ` Dominik Vogt 2022-12-16 19:14 ` Ray Andrews 2022-12-16 16:30 ` Daniel Shahaf 2022-12-16 18:21 ` Ray Andrews 2022-12-16 19:04 ` Dominik Vogt 2022-12-16 20:10 ` Ray Andrews 2022-12-16 21:15 ` Dominik Vogt 2022-12-16 21:33 ` Bart Schaefer 2022-12-16 21:59 ` Dominik Vogt 2022-12-16 22:15 ` Bart Schaefer 2022-12-16 23:33 ` Ray Andrews 2022-12-17 12:47 ` Pier Paolo Grassi 2022-12-17 17:32 ` Ray Andrews 2022-12-17 18:10 ` Ray Andrews 2022-12-17 18:19 ` Roman Perepelitsa 2022-12-17 20:31 ` Ray Andrews 2022-12-17 20:49 ` Bart Schaefer 2022-12-17 21:07 ` Lawrence Velázquez 2022-12-17 21:52 ` Ray Andrews 2022-12-17 22:10 ` Bart Schaefer 2022-12-17 23:41 ` Daniel Shahaf 2022-12-18 0:15 ` Ray Andrews 2022-12-18 0:25 ` Daniel Shahaf 2022-12-18 2:13 ` Ray Andrews 2022-12-18 20:12 ` Daniel Shahaf 2022-12-18 20:26 ` Bart Schaefer 2022-12-18 20:41 ` Ray Andrews 2022-12-18 3:44 ` Lawrence Velázquez 2022-12-17 23:25 ` Daniel Shahaf 2022-12-16 21:33 ` Ray Andrews 2022-12-16 20:25 ` Daniel Shahaf 2022-12-16 21:43 ` Dominik Vogt 2022-12-16 3:20 ` Bart Schaefer 2022-12-16 4:15 ` Ray Andrews 2022-12-16 8:12 ` Roman Perepelitsa
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).