tilde syntax

tilde syntax

[Ray Andrews]

     local zsh_case='(#i)'    # insensitive
     # Depending on options: zsh_case=''

     # Where input is some directory name, possibly incomplete and 
matches are looked for:

     subdirs=( $1*(/N) )

     subdirs=( (#i)$1*(/N) )

     subdirs=( $~zsh_case$1*(/N) ) # This works: but tilde must follow 
first dollar sign


The first two seem intuitive,  but the 3d, what does the tilde really 
do?  It has to be there, but one might expect '$zsh_case' to just 
substitute and all's well.  I used to understand this, I remember Roman 
explaining it, but I can't remember and my notes are sketchy.  The tilde 
turns globing into pattern matching, yes? But '(#i)' works if literal, 
it's not mistaken for a glob pattern so why should the use of a variable 
change anything?  Again, pardon, I used to know this.

Re: tilde syntax

[Bart Schaefer]

On Thu, Jan 4, 2024 at 2:32 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>      subdirs=( $~zsh_case$1*(/N) ) # This works: but tilde must follow
> first dollar sign
>
> [...] what does the tilde really do?

Results of parameter expansion by default (in native zsh emulation)
are not subject to further interpretation.  The tilde says that it's
OK to interpret the result as part of a pattern (globbing or string
comparison, depending on context).

> The tilde turns globing into pattern matching, yes?

No, it turns "do nothing else" into "also act as a glob or pattern".
In this example, it's a glob.

> But '(#i)' works if literal, it's not mistaken for a glob pattern

What do you mean by "works"?  With extendeglob set, it IS a (glob)
pattern, it's the pattern than means "the rest of this pattern is case
insensitive".

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]


On 2024-01-04 14:49, Bart Schaefer wrote:
> Results of parameter expansion by default (in native zsh emulation)
> are not subject to further interpretation.
Hmmm ... I don't think I've known that.  I've tended to think that once 
a variable is substituted out, there is no difference.
>
> No, it turns "do nothing else" into "also act as a glob or pattern".
> In this example, it's a glob.
I tried to find something about that in the manual but it's one of those 
things where you need to know what to call it in words. Searching for " 
$~ " hardly works.
> What do you mean by "works"?

I mean the code does what I'm expecting.


[-- Attachment #2: Type: text/html, Size: 1515 bytes --]

Re: tilde syntax

[Lawrence Velázquez]

On Thu, Jan 4, 2024, at 6:04 PM, Ray Andrews wrote:
> On 2024-01-04 14:49, Bart Schaefer wrote:
>> No, it turns "do nothing else" into "also act as a glob or pattern".
>> In this example, it's a glob. I tried to find something about that in the manual but it's one of those things where you need to know what to call it in words.  Searching for " $~ " hardly works.

Search for "${~spec}", or go here:

https://zsh.sourceforge.io/Doc/Release/Expansion.html#index-GLOB_005fSUBST_002c-toggle

>> What do you mean by "works"?  
> I mean the code does what I'm expecting.

This is thoroughly unhelpful.  Different people can expect different
behaviors, and no one here is psychic.  You need to explicitly state
what your expectations are.

-- 
vq

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]


On 2024-01-04 15:40, Lawrence Velázquez wrote:
> https://zsh.sourceforge.io/Doc/Release/Expansion.html#index-GLOB_005fSUBST_002c-toggle
Thanks. 'Parameter Expansion' is what I need to remember.
> I mean the code does what I'm expecting.
> This is thoroughly unhelpful.  Different people can expect different
> behaviors, and no one here is psychic.  You need to explicitly state
> what your expectations are.

But no further help is needed since I'm happy with what I have. My 
expectations are met.

I need to understand this tilde better but that's enough for now.  I 
think 'further interpretation'

is going to be over my head for some time, I'm stuck thinking that " 
(#i) " is just what it is

and 'further interpretation' is a mystery.  I'll understand eventually.




>

[-- Attachment #2: Type: text/html, Size: 1869 bytes --]

Re: tilde syntax

[Lawrence Velázquez]

On Thu, Jan 4, 2024, at 7:12 PM, Ray Andrews wrote:
> On 2024-01-04 15:40, Lawrence Velázquez wrote:
>> https://zsh.sourceforge.io/Doc/Release/Expansion.html#index-GLOB_005fSUBST_002c-toggle
> Thanks. 'Parameter Expansion' is what I need to remember.

Another important part occurs earlier in "Parameter Expansion" and
can admittedly be hard to find:

        ${name}

		[...]  In addition, more complicated forms of
		substitution usually require the braces to be
		present; exceptions, which only apply if the option
		KSH_ARRAYS is not set, are a single subscript or
		any colon modifiers appearing after the name, or
		any of the characters `^', `=', `~', `#' or `+'
		appearing before the name, all of which work with
		or without braces.

This is why you can use $~foo instead of ${~foo} sometimes.


> I need to understand this tilde better but that's enough for  now.  I 
> think 'further interpretation' 
>
> is going to be over my head for some time

That just refers to the result of the parameter expansion being
subsequently used as something other than a literal value, if the
context calls for it.


-- 
vq

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]


On 2024-01-04 17:41, Lawrence Velázquez wrote:

> Another important part occurs earlier in "Parameter Expansion" and
> can admittedly be hard to find:

Yes.  Seems almost inevitable that with such terse syntax normal search

culture is going to be frustrating.  One must master the shell's jargon.

I wonder if a syntax index could be contemplated, it would look rather 
strange:

$(

$((

$~

${~

if[

if[[

... every last construction.

> This is why you can use $~foo instead of ${~foo} sometimes.

I've noticed that but I don't even try to understand it.  I am in awe of 
the

geniuses who wrote the parser.  The list of ifs, ands, buts and maybes is

not for a mere mortal to understand.  That's what accretion gets you -- 
couldn't

be otherwise.

> That just refers to the result of the parameter expansion being
> subsequently used as something other than a literal value, if the
> context calls for it.

God knows.  It's a steep hill to climb if you're not familiar with it.  
Anyway I'm

at least alert to the issue.

Tx.


>
>

[-- Attachment #2: Type: text/html, Size: 2342 bytes --]

Re: tilde syntax

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 2545 bytes --]

When you include the value of a variable (parameter) in a command line, the
replacement usually happens too late for the resulting value to participate
in other sorts of expansion.

 For example, if you do `echo *`, you'll get a list of the files in the
directory. But if you put the `*` into a variable and echo it:

    $ asterisk='*'
    $ echo $asterisk
    *

Likewise, if you have a variable with a space in the value, it's not
normally subject to word splitting - if you pass it to a command, that
command will get a single argument containing a space, not multiple
arguments:

    $ words='one two'
    $ printf '%s\n' $words
    one two

(It's worth mentioning that in both of the above cases the behavior of zsh
is different from the behavior of bash or ksh, in which the parameter value
_is_ subject to further expansion.)

You can selectively enable further processing with the `~` and `=`
modifiers.  The `~` means that the result of the variable substitution will
be subject to wildcard expansion:

    $ echo $~asterisk
    Applications Desktop Documents Downloads Library bin etc git lib src
tmp

While the `=` means it will be subject to word-splitting:

    $ printf '%s\n' $=words
    one
    two





On Thu, Jan 4, 2024 at 9:04 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

>
> On 2024-01-04 17:41, Lawrence Velázquez wrote:
>
> Another important part occurs earlier in "Parameter Expansion" and
>
> can admittedly be hard to find:
>
> Yes.  Seems almost inevitable that with such terse syntax normal search
>
> culture is going to be frustrating.  One must master the shell's jargon.
>
> I wonder if a syntax index could be contemplated, it would look rather
> strange:
>
> $(
>
> $((
>
> $~
>
> ${~
>
> if[
>
> if[[
>
> ... every last construction.
>
> This is why you can use $~foo instead of ${~foo} sometimes.
>
> I've noticed that but I don't even try to understand it.  I am in awe of
> the
>
> geniuses who wrote the parser.  The list of ifs, ands, buts and maybes is
>
> not for a mere mortal to understand.  That's what accretion gets you --
> couldn't
>
> be otherwise.
>
> That just refers to the result of the parameter expansion being
>
> subsequently used as something other than a literal value, if the
> context calls for it.
>
> God knows.  It's a steep hill to climb if you're not familiar with it.
> Anyway I'm
>
> at least alert to the issue.
>
> Tx.
>
>
>

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 4124 bytes --]

Re: tilde syntax

[Ray Andrews]

On 2024-01-05 06:41, Mark J. Reed wrote:

Running this:


asterisk='*'

echo $asterisk

echo $~asterisk

echo '*'

echo *

... gives me:

*
1 2 3 4 5
*
1 2 3 4 5

So one might say that by default a variable expands with implicit single 
quotes around it?  But the tilde reverts it to 'unquoted' (for lack of a 
better word)?  So in my original, " zsh_case='(#i)' " what I was doing, 
without the tilde, is sending " (#i) " not as an expansion directive, 
but as a literal string?  Yeah, where I crashed and burned was not 
seeing that " (#i) " and " * " are subject to the same rule -- both are 
presumed to be directives but both can also be literal, and when used 
via a variable, they are literal.  Yes?  And this was a design choice at 
variance with shell tradition, so obviously very deliberately chosen?  
Well then, counter intuitive as it is, I'll take it on faith that this 
is the better design.    Subject to further correction:  by George, I 
think I've got it.

Besides: " echo '$asterisk' "  obviously won't do what I might have been 
aiming at.  So the only alternative would be if the tilde worked in 
reverse -- presume expandable but force literal -- and that's merely 
another design choice -- six of one half a dozen of the other -- so we 
have what the devs considered the most convenient :-)  This will fit 
several more pieces into the puzzle, not just one.  zsh is full of these 
invisible differences: is that a directive or is it text?  Sometimes the 
presumption is one, sometimes the other.  This helps me see the 
invisible difference.

Thank you Sensei.

Re: tilde syntax

[Bart Schaefer]

On Fri, Jan 5, 2024 at 8:42 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> So one might say that by default a variable expands with implicit single
> quotes around it?

Approximately so, yes.  Not precisely so in the case of nested expansions.

> But the tilde reverts it to 'unquoted' (for lack of a
> better word)?

Only with respect to pattern characters, but yes.

> And this was a design choice at
> variance with shell tradition, so obviously very deliberately chosen?

It was in fact one of the primary reasons for the creation of zsh in
the first place, to make the behavior of parameter expansions more
consistent without always having to use double quotes around them.

> So the only alternative would be if the tilde worked in
> reverse -- presume expandable but force literal

  setopt glob_subst
  print -- ${~~asterisk}
does exactly that.

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]


On 2024-01-05 11:58, Bart Schaefer wrote:
> Approximately so, yes.  Not precisely so in the case of nested expansions.
Close will count for now.
> Only with respect to pattern characters, but yes.

Ah, so this behavior is special.  Is there some way to watch the parsing 
unfold?

If I could see what happened, step by step it could sure be useful.  
Seems to me

I remember someone showed how.  'set +x' isn't detailed enough.

> It was in fact one of the primary reasons for the creation of zsh in
> the first place, to make the behavior of parameter expansions more
> consistent without always having to use double quotes around them.
Yes,   I know that historically zsh was a design fix.
>
>    setopt glob_subst
>    print -- ${~~asterisk}
> does exactly that.

Cool.  Manual mentions that, but now it makes sense.

Tx.


[-- Attachment #2: Type: text/html, Size: 1914 bytes --]

Re: tilde syntax

[Bart Schaefer]

On Fri, Jan 5, 2024 at 2:50 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Ah, so this behavior is special.  Is there some way to watch the parsing unfold?

There really isn't any "parsing" per se -- internally, an "active"
pattern character is represented differently than its literal
counterpart.  All that $~x (or glob_subst) does is make the pattern
characters in $x be active instead of literal.  The rest is just
ordinary globbing (or pattern matching, in [[ ... ]] etc.).

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]


On 2024-01-05 15:27, Bart Schaefer wrote:
> On Fri, Jan 5, 2024 at 2:50 PM Ray Andrews<rayandrews@eastlink.ca>  wrote:
> There really isn't any "parsing" per se -- internally, an "active"
> pattern character is represented differently than its literal
> counterpart.

No way to see that unfold?  Anyway your word 'active' is the thing: Is 
that literal or is it active?  I know it's several decades too late to 
do anything about it, but if we had to do it all over again, I'd 
advocate for clear and *visible* ways of coding one or the other.  Mind 
... when expanding a variable ... back to tildes!  A variable is by 
definition a symbol but what it contains could be another symbol or 
something literal.  Logical completion would demand arbitrary nesting of 
any combination. That's further than I need to go right now.

But you know, it would save much grief if someone told you this up front 
but nobody does: "One of the endless problems with zsh -- or any other 
shell -- is that there are places where entered characters are presumed 
to be literal -- but can be forced to indicate some action or command -- 
and other places where entered characters are presumed to be actions -- 
but can be forced to become literal characters."  ... such a sentence 
would have saved endless grief if I'd read it on my first day using 
zsh.  As always I'm ruined by my knowledge of C, where there is never 
the slightest doubt.

[-- Attachment #2: Type: text/html, Size: 2124 bytes --]

Re: tilde syntax

[Bart Schaefer]

On Fri, Jan 5, 2024 at 3:47 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2024-01-05 15:27, Bart Schaefer wrote:
>
> On Fri, Jan 5, 2024 at 2:50 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> There really isn't any "parsing" per se -- internally, an "active"
> pattern character is represented differently than its literal
> counterpart.
>
> No way to see that unfold?

I don't even know what that would mean.  It's like asking to see
strcpy() "unfold."  The only difference is that when say a "*" is in
in the input, a different byte is put into the "active" output.
Instrumenting that for casual observability would be stupidly slow in
the normal use case.

> I'd advocate for clear and *visible* ways of coding one or the other.

But that's just it:  The base case in zsh is that if you CAN see it
(and you don't see quotes around it) then it IS active, and if you
CAN'T see it (or you do see quotes) then it's literal.  The only time
you need to get fancy is when you've deliberately hidden something
("inside" a parameter) and you need to expose it.  In most shells
other than zsh the patterns are active whether you can see them or
not, unless you see quotes, which means you see a lot of quotes.  The
corollary to the base case is that parameter references expand in
double quotes but not in single quotes, but expanding does not mean
making the contents active.

> "One of the endless problems with zsh -- or any other shell -- is that there are places where entered characters are presumed to be literal -- but can be forced to indicate some action or command -- and other places where entered characters are presumed to be actions -- but can be forced to become literal characters."

That's close to a complete mischaracterization.  Or conversely, it's
true of every programming language, just with differences of syntax.

Re: tilde syntax

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2536 bytes --]


On 2024-01-05 16:23, Bart Schaefer wrote:
> But that's just it: The base case in zsh is that if you CAN see it
> (and you don't see quotes around it) then it IS active, and if you
> CAN'T see it (or you do see quotes) then it's literal.  The only time
> you need to get fancy is when you've deliberately hidden something
> ("inside" a parameter) and you need to expose it.  In most shells
> other than zsh the patterns are active whether you can see them or
> not, unless you see quotes, which means you see a lot of quotes.  The
> corollary to the base case is that parameter references expand in
> double quotes but not in single quotes, but expanding does not mean
> making the contents active.
I'm going to save that and reread it a few times.  Much has been made 
clear, more will come.
>> "One of the endless problems with zsh -- or any other shell -- is that there are places where entered characters are presumed to be literal -- but can be forced to indicate some action or command -- and other places where entered characters are presumed to be actions -- but can be forced to become literal characters."
> That's close to a complete mischaracterization.  Or conversely, it's
> true of every programming language, just with differences of syntax.

I never had a problem in C -- what's literal is always transparent.  
Maybe at your level of understanding it's true of zsh as well but at my 
level what I said is exactly how it seems. Mind, C constructions are 
trivially simple compared to shell. Actually given the awesome power I'm 
quite astonished that parsing works as effectively as it does.

...

Na ... I see it.  What's quoted is literal and params are literal unless 
the tilde is used.  And quotes seem to 'cancel' the tilde, which is logical:

asterisk='*'
echo $asterisk      # Params are literal unless tilde is used
echo "$asterisk"    # Quotes here have no work to do, it's already literal
echo $~asterisk    # Param with tilde is expanded.
echo "$~asterisk"    # Quotes here seem to neutralize the tilde, which 
strikes me as just fine. Outermost control has the final say.

Output:

1 /aWorking/Zsh/Source/Wk/9 1 % . ../test1
*
*
1 2 3 4 5
*

Maybe, finally, it really is simple :-)

Oh, but then there's substitutions where ${ .../.../}  and so on involve 
literals but that kind of thing is special to the 'internal grammar' of 
the particular substitution.

Thanks Bart, you've been patient as always.  I understand it well 
enough, let's let it drop I've wasted enough of your time.


[-- Attachment #2: Type: text/html, Size: 3592 bytes --]