triviality with prompts

triviality with prompts

[Ray Andrews]

Reading in the scrolls, I find this:

     setopt prompt_subst

... which permits this:

     _red=$'%{\e[1;31m%}'

    _grn=$'%{\e[1;32m%}'
    _yel=$'%{\e[1;33m%}'
    _blu=$'%{\e[1;34m%}'
    _mag=$'%{\e[1;35m%}'

    PS3=$'\n $_red MAKE A SELECTION ... '
    PS1=$'\n$_yel%l $_red%m $_grn%n $_blu%d $_mag%(3L.%L.)$ %{\e[0m%}'

... which sure makes things more readable.  However I note that
PS3  will permit that variable substitution even without
" setopt prompt_subst ", whereas PS1 requires it.  Is that an
anomaly or necessary?  Could that setopt be on by default?
Interesting that the variable expands even tho inside single
quotes, not that I'm complaining.

Re: triviality with prompts

[Bart Schaefer]

On Dec 2,  4:17pm, Ray Andrews wrote:
}
} PS3  will permit that variable substitution even without
} " setopt prompt_subst ", whereas PS1 requires it.

I think you've deluded yourself somehow.  PS3 behaves just like all the
other prompts.

} Interesting that the variable expands even tho inside single
} quotes, not that I'm complaining.

Once again you're confused about how quoting works.  The variable isn't
inside single quotes at the time the prompt is expanded.  In fact, in your
example $'...' is NOT the same as single quotes.

If you'd read a little further through the scrolls, you'd have seen you
need neither _red=$'%{\e[1;31m%}' nor prompt_subst:

PS3=$'\n %F{red} MAKE A SELECTION ... %f'
PS1=$'\n%F{yellow}%l %F{red}%m %F{green}%n %F{blue}%d %F{magenta}%(3L.%L.)$ %f'

Re: triviality with prompts

[Ray Andrews]

On 12/02/2014 07:02 PM, Bart Schaefer wrote:
> On Dec 2,  4:17pm, Ray Andrews wrote:
> }
> } PS3  will permit that variable substitution even without
> } " setopt prompt_subst ", whereas PS1 requires it.
>
> I think you've deluded yourself somehow.  PS3 behaves just like all the
> other prompts.
A red herring there.
I had this form, which seems not to need the option:

PS3=$'\n '"$_red MAKE A SELECTION ... "

... and I changed it to this form, which does need it:
PS3=$'\n $_red MAKE A SELECTION ... '

... whereas in PS1 I had the (what look very much like) single quotes
only version, which does need it, so one seemed to need it, and the
other not.

... so the issue was quote voodoo, as usual.
>
> Once again you're confused about how quoting works.  The variable isn't
> inside single quotes at the time the prompt is expanded.  In fact, in your
> example $'...' is NOT the same as single quotes.
God knows.  If ' ... ' isn't single quotes then is sure has me fooled.  
One day
it will all make sense.  I know everything is different with prompt 
strings tho,
it's a world unto itself where single quotes are not single quotes.
> If you'd read a little further through the scrolls, you'd have seen you
> need neither _red=$'%{\e[1;31m%}' nor prompt_subst:
>
> PS3=$'\n %F{red} MAKE A SELECTION ... %f'
> PS1=$'\n%F{yellow}%l %F{red}%m %F{green}%n %F{blue}%d %F{magenta}%(3L.%L.)$ %f'
There are so many scrolls!  I'll get there.  Anyway, I like my way better.
Even knowing only a tiny bit about the prompt, it is astonishing how much
love went into that.

Re: triviality with prompts

[Lawrence Velázquez]

On Dec 2, 2014, at 11:28 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> On 12/02/2014 07:02 PM, Bart Schaefer wrote:
> 
>> Once again you're confused about how quoting works.  The variable isn't
>> inside single quotes at the time the prompt is expanded.  In fact, in your
>> example $'...' is NOT the same as single quotes.
> 
> God knows.  If ' ... ' isn't single quotes then is sure has me fooled.  One day
> it will all make sense.  I know everything is different with prompt strings tho,
> it's a world unto itself where single quotes are not single quotes.

Peruse the "QUOTING" section in zshmisc(1). Surrounding a string with single quotes ('foo') is not the same as surrounding a string with single quotes preceded by a dollar sign ($'foo').

vq

Re: triviality with prompts

[Ray Andrews]

On 12/02/2014 08:38 PM, Lawrence Velázquez wrote:

> Peruse the "QUOTING" section in zshmisc(1). Surrounding a string with 
> single quotes ('foo') is not the same as surrounding a string with 
> single quotes preceded by a dollar sign ($'foo'). vq 

Thanks, that's a nice summary.  It's especially nice of it to mention 
the   $'foo'    gotcha,
which is ignored in other docs.  Just when I thought I knew what a 
single quote was :(
But the zshbuiltins page, under 'print' doesn't go into any detail about 
this    $'foo'
form.  Never mind, the prompt string is the only place I've seen it so 
far and I'm
forewarned now for the next time I see it.

Re: triviality with prompts

[Roman Neuhauser]

# rayandrews@eastlink.ca / 2014-12-03 08:20:43 -0800:
> On 12/02/2014 08:38 PM, Lawrence Velázquez wrote:
> > Peruse the "QUOTING" section in zshmisc(1). Surrounding a string with 
> > single quotes ('foo') is not the same as surrounding a string with 
> > single quotes preceded by a dollar sign ($'foo'). vq 
> 
> Thanks, that's a nice summary.  It's especially nice of it to mention
> the $'foo' gotcha, which is ignored in other docs.  Just when I
> thought I knew what a single quote was :( But the zshbuiltins page,
> under 'print' doesn't go into any detail about this $'foo' form.
> Never mind, the prompt string is the only place I've seen it so far
> and I'm forewarned now for the next time I see it.
 
well, $'' is apparently different syntax from '', and it's dangerous to
assume that two different syntaxes have the same semantics.

YMMV, but when i'm working on a script, the biggest slice of the time
spent with the various zsh* man pages is in zshmisc(1) and zshexpn(1).
you might also get some new insight from giving them a read or two.

-- 
roman

Re: triviality with prompts

[Ray Andrews]

On 12/03/2014 08:43 AM, Roman Neuhauser wrote:

> well, $'' is apparently different syntax from '', and it's dangerous 
> to assume that two different syntaxes have the same semantics. 
*Another* gotcha yet again?  Yes, making assumptions is dangerous, but 
not doing anything for fear of the next gotcha is not an option.  I 
guess one just walks through the mine-field and when things blow up, you 
know you've found a mine.  It's only software, so no one dies.  Trial 
and error.

> YMMV, but when i'm working on a script, the biggest slice of the time 
> spent with the various zsh* man pages is in zshmisc(1) and zshexpn(1). 
> you might also get some new insight from giving them a read or two. 
Thanks.  So many scrolls!

Re: triviality with prompts

[Roman Neuhauser]

# rayandrews@eastlink.ca / 2014-12-03 09:10:54 -0800:
> So many scrolls!

the full listing is at the top of zshall(1). :)

-- 
roman

Re: triviality with prompts

[Bart Schaefer]

On Wed, Dec 3, 2014 at 9:10 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> On 12/03/2014 08:43 AM, Roman Neuhauser wrote:
>
>> well, $'' is apparently different syntax from '', and it's dangerous to
>> assume that two different syntaxes have the same semantics.
>
> *Another* gotcha yet again?

No, this is the same gotcha as before, so to speak.  That's
dollar-single-single, not dollar-double.  There is no special syntax
for a double-quote preceded by a dollar.  Roman just didn't put
anything in between his quotes.

Re: triviality with prompts

[Ray Andrews]

On 12/03/2014 01:14 PM, Bart Schaefer wrote:
> No, this is the same gotcha as before, so to speak.  That's
> dollar-single-single, not dollar-double.  There is no special syntax
> for a double-quote preceded by a dollar.  Roman just didn't put
> anything in between his quotes.
>
Ok, I'll 'stand down' on that.  I'm thinking that might be the end of
the line for quote complexities? Or at least for all the gross
forms?

\
' '
" "
$' '
$( )

... all cross-pollinating each other in various ways, but is that it for
the main species?  It's really not that bad if only one could get
the complete overview somewhere before diving in.

Oh, and of course there's the various flavorings like 'rc_quotes' and
I guess a few others like that.  Tractable.

Re: triviality with prompts

[Bart Schaefer]

On Dec 3,  3:14pm, Ray Andrews wrote:
} Subject: Re: triviality with prompts
}
} \
} ' '
} " "
} $' '
} $( )
} 
} ... all cross-pollinating each other in various ways, but is that it for
} the main species?

$( ) is substitution, not quoting.  It is true that if you have both
outer double quotes and a $( ) inside them, then the rules for parsing
command substitution take over until the matching close-paren is found;
but otherwise nothing is quoted differently just because it is inside
a command substitution.

} Oh, and of course there's the various flavorings like 'rc_quotes' and
} I guess a few others like that.  Tractable.

The other major one is aritmetic evaluation in (( )) where everything
behaves as if double quoted, and then there are the special cases in
parameter subscript expressions, which can get a bit strange but in
practice you'll almost never run into.

Re: triviality with prompts

[Ray Andrews]

On 12/03/2014 09:06 PM, Bart Schaefer wrote:
> On Dec 3,  3:14pm, Ray Andrews wrote:
> } Subject: Re: triviality with prompts
> }
> } \
> } ' '
> } " "
> } $' '
> } $( )
> }
> } ... all cross-pollinating each other in various ways, but is that it for
> } the main species?
>
> $( ) is substitution, not quoting.  It is true that if you have both
> outer double quotes and a $( ) inside them, then the rules for parsing
> command substitution take over until the matching close-paren is found;
> but otherwise nothing is quoted differently just because it is inside
> a command substitution.
>
> } Oh, and of course there's the various flavorings like 'rc_quotes' and
> } I guess a few others like that.  Tractable.
>
> The other major one is aritmetic evaluation in (( )) where everything
> behaves as if double quoted, and then there are the special cases in
> parameter subscript expressions, which can get a bit strange but in
> practice you'll almost never run into.
>
<ignorable-philosophical-musing>

Thinking about all that, I find my self still fuzzy.  What do we really mean
by 'quote'?  It's a delineated block of characters in which some prescribed
semantics will apply. So "   $()   " is also a delineated block of 
characters
in which some prescribed semantics will apply,  ditto "  (())   " and
"   [[]]  ", yes?  So, what's so special about what we call 'quoting'?  
Maybe
what I'm reaching for with the list above is a complete list of
'delineators'.  So, if one had such a list, one could learn the syntactic
rules inside each one in a tractable way.  Maybe the whole idea of 'quote'
is not so special, it's just two or three of the many more ways that zsh
will process a delineated block of characters.  The only thing 'special'
about it is that the delineators are the keyboard characters that we call
quotation marks, thus we are 'quoting'.  Or is quoting really,
syntactically, a different 'kind'  of thing than "  $()  "  or " []  "?

Re: triviality with prompts

[Bart Schaefer]

On Dec 4, 10:42am, Ray Andrews wrote:
} Subject: Re: triviality with prompts
}
} On 12/03/2014 09:06 PM, Bart Schaefer wrote:
} >
} > $( ) is substitution, not quoting.
} 
} Thinking about all that, I find my self still fuzzy. What do we really
} mean by 'quote'? It's a delineated block of characters in which some
} prescribed semantics will apply.

Not exactly.  It's a delineated block of characters in which a defined
syntax will apply.  Of course in the colloquial sense the meaning of an
individual grapheme in a syntax is its "semantics", but in the computer
science sense the syntax and semantics of a language are distinct.

Quoting defines the meaning of individual characters, e.g., transforming
the digraph \n into something else -- either a literal "n" or a newline,
depending on the interpretation of the backslash.  Quoting also creates
a syntactic unit, affecting how a collection of graphemes is assembled
into tokens (most often, resulting in a single token instead of many,
and usually resulting in tokens that are treated as plain strings and
not as having any particular higher-level meaning).

Constructs like $( ) define the meaning and higher-level structure of
the language tokens that came from interpretation of the syntax.  The
shell language is unusual among programming languages in that it allows
certain semantic constructs to be interpreted even when inside certain
syntactic constructs (double-quoting, mostly), so that it's necessary
to interleave syntactic and semantic analysis (and even execution of
the code resulting from the latter) to arrive at a final tokenization.

Most programming languages force you to make that explicit, but the
shell is all about building up or cutting up strings by implicit con-
catenation and delineation, so it takes all kinds of shortcuts that
make the language ugly to formally define but faster to interact with
in the common cases.

E.g. in
	'$(echo foo bar)'
the definition of single quotes means that none of $ ( or ) is special,
and results in one plain-string token made up of the characters between
the single quotes.  Conversely in
	"$(echo foo bar)"
the definition of double quotes means that $ is special, which in turn
means that the digraph $( introduces a subexpression that has to be
interpreted using the higher-level rules of command substitution, which
consumes everything up through the closing paren.  The final token can
only be generated after the command inside the parens is executed and
its output captured, but it still results in only one plain string.

Compare to $(echo foo bar) without the quotes, which normally results
in two string tokens foo and bar, or to
	"\$(echo foo bar)"
where the definition of backslash means that the digraph \$ transforms
into a non-special $ which therefore is NOT part of a $( digraph and
therefore does not introduce command substitution.

} Maybe what I'm reaching for with the list above is a complete list of
} 'delineators'.  So, if one had such a list, one could learn the syntactic
} rules inside each one in a tractable way.

In zsh's info doc, sections 6.9 (Quoting) and 14 (Expansion) together
make up that list.

Re: triviality with prompts

[Ray Andrews]

On 12/06/2014 01:28 PM, Bart Schaefer wrote:
> Not exactly.  It's a delineated block of characters in which a defined
> syntax will apply.  Of course in the colloquial sense the meaning of an
> individual grapheme in a syntax is its "semantics", but in the computer
> science sense the syntax and semantics of a language are distinct.
Yeah, may as well be strict.  I don't want to split hairs too finely tho,
and I only know the rules for this in natural language.
> Quoting defines the meaning of individual characters, e.g., transforming
> the digraph \n into something else -- either a literal "n" or a newline,
> depending on the interpretation of the backslash.  Quoting also creates
> a syntactic unit, affecting how a collection of graphemes is assembled
> into tokens (most often, resulting in a single token instead of many,
> and usually resulting in tokens that are treated as plain strings and
> not as having any particular higher-level meaning).
>
> Constructs like $( ) define the meaning and higher-level structure of
> the language tokens that came from interpretation of the syntax.  The
> shell language is unusual among programming languages in that it allows
> certain semantic constructs to be interpreted even when inside certain
> syntactic constructs (double-quoting, mostly), so that it's necessary
> to interleave syntactic and semantic analysis (and even execution of
> the code resulting from the latter) to arrive at a final tokenization.
>
> Most programming languages force you to make that explicit, but the
> shell is all about building up or cutting up strings by implicit con-
> catenation and delineation, so it takes all kinds of shortcuts that
> make the language ugly to formally define but faster to interact with
> in the common cases.
You know, once you 'just say it' half the problem goes away.  I'm still
looking for rigor.  Once I just stop looking for it I won't get into a 
lather
nearly so  often.

> E.g. in
> 	'$(echo foo bar)'
> the definition of single quotes means that none of $ ( or ) is special,
> and results in one plain-string token made up of the characters between
> the single quotes.  Conversely in
> 	"$(echo foo bar)"
> the definition of double quotes means that $ is special, which in turn
> means that the digraph $( introduces a subexpression that has to be
> interpreted using the higher-level rules of command substitution, which
> consumes everything up through the closing paren.  The final token can
> only be generated after the command inside the parens is executed and
> its output captured, but it still results in only one plain string.
>
> Compare to $(echo foo bar) without the quotes, which normally results
> in two string tokens foo and bar, or to
> 	"\$(echo foo bar)"
> where the definition of backslash means that the digraph \$ transforms
> into a non-special $ which therefore is NOT part of a $( digraph and
> therefore does not introduce command substitution.
>
> } Maybe what I'm reaching for with the list above is a complete list of
> } 'delineators'.  So, if one had such a list, one could learn the syntactic
> } rules inside each one in a tractable way.
>
> In zsh's info doc, sections 6.9 (Quoting) and 14 (Expansion) together
> make up that list.
>
That, sir, was beautifully written.

Re: triviality with prompts

[Lawrence Velázquez]

On Dec 3, 2014, at 11:20 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> Thanks, that's a nice summary.  It's especially nice of it to mention
> the $'foo' gotcha, which is ignored in other docs.  Just when I thought
> I knew what a single quote was :(

It's not really a gotcha; the dollar sign frequently has special meaning
in shell scripts. As a rule of thumb, you shouldn't ever assume that
a "$" is a literal.

> But the zshbuiltins page, under 'print' doesn't go into any detail about
> this $'foo' form.

zshmisc(1) says:

    A string enclosed between `$'' and `'' is processed the same way as
    the string arguments of the print builtin, and the resulting string
    is considered to be entirely quoted.

Which means you want to figure out how the print builtin treats its
string arguments, from zshbuiltins(1):

    With no flags or with the flag `-', the arguments are printed on the
    standard output as described by echo, with the following
    differences...

Then you probably also want to look up what echo does:

    Write each arg on the standard output, with a space separating each
    one. If the -n flag is not present, print a newline at the end. echo
    recognizes the following escape sequences...

vq

Re: triviality with prompts

[Bart Schaefer]

(Oops, sorry for Cc'ing everybody on that last message.)

On Wed, Dec 3, 2014 at 8:20 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> But the zshbuiltins page, under 'print' doesn't go into any detail about
> this    $'foo'
> form.

It doesn't go into detail about ANY form of quoting ... because
quoting is evaluated the same way no matter what command is involved.

(The special meaning of backslash to certain commands like print and
echo is entirely separate from quoting, though often similar in
effect.)

Re: triviality with prompts

[Ray Andrews]

On 12/03/2014 01:22 PM, Bart Schaefer wrote:
> (The special meaning of backslash to certain commands like print and 
> echo is entirely separate from quoting, though often similar in effect.) 
Yeah, that's a gotcha that I now get, but it sure caused me some grief.  
I wish there was some string modifier that distinguished between 
backslash as quote, and backslash as escape.  Maybe that's not doable, 
but I run into situations where I need to protect a ' \n  ' but don't 
care about stripping the backslash off what is a 'real' quoted 
character.  I now know that the escape situation is, technically, not 
the shell's business, OTOH shell builtins might perhaps not be treated 
like total strangers when it comes to escapes since they are obviously 
in the family.