Avoiding the zshells intelligence...in one case

Avoiding the zshells intelligence...in one case

[Meino.Cramer]

Hi,

I want to wriet a zscript, which first argument is a URL like
this: http://www.domain.com/file/scrap?t=5O2%20&30
(this URL is fiction and probably nonsense...)

Or in other words: Those URLs may contain a lot of "active"
stuff, which zsh will transform,replace,expand,glob and 
what else (and normally I am quite happy with that :)

Previously I used commands like

    zscript 'http://<URL>'

but over the time I get annoyed by the '' since the
stuff in between was cute'n'pasted from somewhere else.

Is there any way to write a script/alias which takes
its first/nth argument verbatim and character by character
without any intelligent intervention of the zshell?

Thank you very much in advance for any help!
Cheers
Meino

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Sun, 22 Jan 2017, Meino.Cramer@gmx.de wrote:

> Previously I used commands like
>
>     zscript 'http://<URL>'
>
> but over the time I get annoyed by the '' since the
> stuff in between was cute'n'pasted from somewhere else.
>
> Is there any way to write a script/alias which takes
> its first/nth argument verbatim and character by character
> without any intelligent intervention of the zshell?

You have a few choices:

One is to use url-quote-magic (and possibly one of bracketed-paste-magic
or bracketed-paste-url-magic) so that the quoting is done for you by the
editor when you enter the URL.

Another is to use the "noglob" modifier, although that won't quote things
that look like other expansions, so if your URLs contain "$" or "{" "}"
all the following will fall short.

Simplest:

alias zscript='noglob zscript'

However, this will apply to all arguments, not just the first.  So one
workaround is this convoluted looking thing:

# Quote the first N-1 arguments [of the alias] and glob the Nth+
globN+ () { ${(b)@[2,$1+1]} ${~@[$1+2,-1]} }
alias zscript='noglob globN+ 2 zscript'

You can perhaps see how to extend that to quoting specific positions.

(Aside to zsh-workers, it'd be nice if "noglob function { body } args"
worked to call an anonymous function without globbing.  Hard, I bet.)

The most radical choice is to "setopt nonomatch", but that's probably more
than what you want.  However, you can combine that with noglob:

nonomatch() { setopt localoptions nonomatch; ${~@} }
alias zscript='noglob nonomatch zscript'

Note this doesn't recursively expand the alias because alias expansion
occurs before parameter expansion, so $1 will always be looked up as a
function, builtin, or command within the nonomatch function.

Re: Avoiding the zshells intelligence...in one case

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> nonomatch() { setopt localoptions nonomatch; ${~@} }
> alias zscript='noglob nonomatch zscript'

This does not seem to work: Setting of nonomatch comes to late

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Sun, 22 Jan 2017, Martin Vaeth wrote:

> Bart Schaefer <schaefer@brasslantern.com> wrote:
> >
> > nonomatch() { setopt localoptions nonomatch; ${~@} }
> > alias zscript='noglob nonomatch zscript'
>
> This does not seem to work: Setting of nonomatch comes to late

In exactly what way does it not work?

torch% zscript() { print -l -- $@ }
torch% nonomatch() { setopt localoptions nonomatch; ${~@} }
torch% alias zscript='noglob nonomatch zscript'
torch% zscript *x *c
*x
Doc
Etc
Src
torch%

Maybe you need a more thorough version:

nonomatch() {
  setopt localoptions nonomatch nonullglob nocshnullglob
  ${~@}
}

Re: Avoiding the zshells intelligence...in one case

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>> >
>> > nonomatch() { setopt localoptions nonomatch; ${~@} }
>> > alias zscript='noglob nonomatch zscript'
>>
>> This does not seem to work: Setting of nonomatch comes to late
>
> In exactly what way does it not work?

% nonomatch() { setopt localoptions nonomatch; ${~@} }
% alias t='noglob nonomatch echo'
% setopt nomatch
% t ~dummy
zsh: no such user or named directory: dummy
% setopt nonomatch
% t ~dummy
~dummy

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Mon, 23 Jan 2017, Martin Vaeth wrote:

> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > In exactly what way does it not work?
>
> % t ~dummy
> zsh: no such user or named directory: dummy

Sorry, I should have mentioned "~" along with variables and braces in my
original reply.  That's because "noglob" doesn't consider tilde-expansion
to be globbing, even though "nomatch" does.  There's no way to temporarily
disable tilde.

If Meino has URLs where the schema starts with a tilde, though, he's
in worse trouble than this.

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 23/01/17 02:26 PM, Bart Schaefer wrote:
>
> If Meino has URLs where the schema starts with a tilde, though, he's
> in worse trouble than this.
>
Too bad there was no option to just turn *everything* off for the one 
command.  So often we end up fighting the shell's expansions with 
various incantations whereas:


setopt everyexpansionoff

command

unsetopt everyexpansionoff

... would be so much more understandable.  I know it's not the way 
shells work, but why not?  I now understand why this sort of thing can't 
be done with the 'whence -m' situation but so long as the option was 
unset before a command is reached, it should be at least possible in 
theory.  Can't the whole expansion engine just be bypassed in one very 
long jump?

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Mon, 23 Jan 2017, Martin Vaeth wrote:

> % nonomatch() { setopt localoptions nonomatch; ${~@} }
> % alias t='noglob nonomatch echo'
> % setopt nomatch
> % t ~dummy
> zsh: no such user or named directory: dummy

You could try this, but it's potentially risky:

re_nomatch() { { ${~@} } always { setopt nomatch } }
alias t='setopt nonomatch && noglob re_nomatch echo'

Re: Avoiding the zshells intelligence...in one case

[Eric Cook]

On 01/23/2017 05:40 PM, Ray Andrews wrote:
> On 23/01/17 02:26 PM, Bart Schaefer wrote:
>>
>> If Meino has URLs where the schema starts with a tilde, though, he's
>> in worse trouble than this.
>>
> Too bad there was no option to just turn *everything* off for the one command.  So often we end up fighting the shell's expansions with various incantations whereas:
> 
> 
> setopt everyexpansionoff
> 
> command
> 
> unsetopt everyexpansionoff
> 
> ... would be so much more understandable.  I know it's not the way shells work, but why not?  I now understand why this sort of thing can't be done with the 'whence -m' situation but so long as the option was unset before a command is reached, it should
> be at least possible in theory.  Can't the whole expansion engine just be bypassed in one very long jump?
> 

That the point of having single quotes, so nothing within them would be treated syntactically.
This thread is just another plead at being even more lazy, to not have remember to use an
option/syntax to treat a string literally.

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 23/01/17 06:48 PM, Eric Cook wrote:
>
> That the point of having single quotes, so nothing within them would be treated syntactically.
> This thread is just another plead at being even more lazy, to not have remember to use an
> option/syntax to treat a string literally.
>
Except that that doesn't work with command line arguments nor with 
things like:

$ alias junk='echo $path'

$ alias wense='whence -m "$1"'


and it seems to me that there's lots of other little exceptions here and 
there where the quotes end up getting stripped off. Thus the need for 
'noglob', and for exactly the same sorts of reasons it would be good to 
be able to assure literal strings in any situation and do it in such a 
way that it is spelled out what is going on.  As Bart pointed out, the 
tilde can't be protected even by 'noglob', but 'noexpansionsatall' would 
give perfect results even on Tuesday and be absolutely frank as to what 
has been done in exotic situations where simple quoting doesn't work.  
Why does it have to be difficult?  Aren't options like 'noglob' the 
correct answer for this sort of thing?  So why not use the same logic to 
extend complete protection for strings when needed, as in Meino's 
problem?  Even when there is better syntax, sometimes you just want 
something to work and the option would not only work but be 
self-commenting.  Or not.  Maybe this really is sloppy thinking.

Re: Avoiding the zshells intelligence...in one case

[Eric Cook]

On 01/24/2017 12:42 AM, Ray Andrews wrote:
> On 23/01/17 06:48 PM, Eric Cook wrote:
>>
>> That the point of having single quotes, so nothing within them would be treated syntactically.
>> This thread is just another plead at being even more lazy, to not have remember to use an
>> option/syntax to treat a string literally.
>>
> Except that that doesn't work with command line arguments nor with things like:
> 
Except that it does.
> $ alias junk='echo $path'
> 
> $ alias wense='whence -m "$1"'

And does so during both of those alias definitions, you may be confusing what happens
during the definition of the alias and what happens during the execution of the alias.

Running:
% alias -L

Will show you that both alias were defined exactly as you inputted them, because the single
quotes prevented the expansions

And ignoring the expansions of the commandline, the original poster used an url that contains
an ampersand; & isn't an expansion, just other shell syntax that a noglob-like option or bart's
current attempts will not be able to disable.

% curl -s http://site?param&disown
Can be a legal site url, but could also be meant as a curl command to be ran in the background and
disown that process. short of reading the user's mind how would you make it easier?
prepending noglob is already two more characters than '', then you would need an `nosyntax' like
command to deactivate &.

I get that shells, unlike other programming languages are used far more interactively and that
less typing is normally always good. But in nearly every other programming language if you don't
want the syntax/metacharacters to be treated specially, you escape/quote them correctly.

I just don't see how remembering to use an alias/option is less inconvenient than remembering to use
single quotes on strings with questionable characters.

hell, with recent versions of zsh with bracketed-paste you could copy and paste this entire email
into your shell, invoke the quote-line widget, then prefix print -r -- and have a command that will
work anywhere regardless of the options set (rcquotes being the sole exception) or alias/functions
that the user may have.

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 24/01/17 07:58 AM, Eric Cook wrote:
> And does so during both of those alias definitions, you may be 
> confusing what happens
> during the definition of the alias and what happens during the execution of the alias.
>
> Running:
> % alias -L
>
> Will show you that both alias were defined exactly as you inputted them, because the single
> quotes prevented the expansions

Sure, but if we have any chains -- one command calling another -- we end 
up loosing the quotes as things are passed along.

> And ignoring the expansions of the commandline, the original poster used an url that contains
> an ampersand; & isn't an expansion, just other shell syntax that a noglob-like option or bart's
> current attempts will not be able to disable.

And that's just what I'm saying -- it would be nice to have some sort of 
bomb-proof zero expansion ability.
>
> % curl -s http://site?param&disown
> Can be a legal site url, but could also be meant as a curl command to be ran in the background and
> disown that process. short of reading the user's mind how would you make it easier?
> prepending noglob is already two more characters than '', then you would need an `nosyntax' like
> command to deactivate &.

My suggestions are really questions in disguise, it's interesting to 
consider how any of this could really be done.  All I know for sure is 
that half or more of my own troubles with zsh have been efforts to stop 
expansion of one thing or another, or to protect some character from 
being interpreted or stripped out.
>
> I just don't see how remembering to use an alias/option is less inconvenient than remembering to use
> single quotes on strings with questionable characters.

Not convenient, that's hardly the point.  More like predictability and 
robustness and transparency.  FWIW I've always thought that terseness in 
code is a false virtue since it isn't typing that slows us down, but 
design and debugging and -- where terseness is the rule -- just studying 
up on the correct syntax can take a thousand times longer than the 
actual keystrokes.
>
> hell, with recent versions of zsh with bracketed-paste you could copy and paste this entire email
> into your shell, invoke the quote-line widget, then prefix print -r -- and have a command that will
> work anywhere regardless of the options set (rcquotes being the sole exception) or alias/functions
> that the user may have.
True, there's almost always a way.  Seems the only thing that's 
impossible is passing a command's literal arguments to the command, tho 
noglob is at least a partial cure for that.  But, as time goes by, I 
confess that more and more I see the virtues of zsh's way of looking at 
things, but I do like pondering alternatives.
>
>

Re: Avoiding the zshells intelligence...in one case

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> You could try this, but it's potentially risky:
>
> re_nomatch() { { ${~@} } always { setopt nomatch } }
> alias t='setopt nonomatch && noglob re_nomatch echo'

Why is this risky? Because it can leave nomatch in the wrong state?
The following does not have this problem and also takes the other
settings you mentioned into account:

unsetopt_nomatch() {
	if unsetopt | grep -q nonomatch
	then	restore_nomatch() setopt nomatch
	else	restore_nomatch() :
	fi
	unsetopt nomatch
}
noexpand_restore_nomatch() { {
() {
	setopt local_options no_nullglob n_ocshnullglob
	${~@}
} $@ } always {
	restore_nonomatch
	unfunction restore_nomatch
} }
alias noexpand='unsetopt_nomatch; noglob noexpand_restore_nomatch '

Example usage: alias wget='noexpand wget'

It is clumsy, because the "setopt nomatch" must possibly be excluded from
the "localoptions" range; except for using an anonymous function
and passing the arguments to it, I did not find another solution...
To get correct completion for the above alias, I guess you also have to

compdef noexpand_restore_nomatch=noglob

after the initialization of the completion system.

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Tue, 24 Jan 2017, Martin Vaeth wrote:

> Bart Schaefer <schaefer@brasslantern.com> wrote:
> >
> > re_nomatch() { { ${~@} } always { setopt nomatch } }
> > alias t='setopt nonomatch && noglob re_nomatch echo'
>
> Why is this risky? Because it can leave nomatch in the wrong state?

Because if you accidentally use it in a pipeline it'll throw away its
standard input, and if you use it in a complex && || series it may not
produce the expected result.

Both of which are even worse if instead of "&&" you use a ";" as you did
in your more complicated formulation.

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Tue, 24 Jan 2017, Ray Andrews wrote:

> Sure, but if we have any chains -- one command calling another -- we end
> up loosing the quotes as things are passed along.

That's not true UNLESS you've either used a poorly constructed alias (which
is not really "one command calling another") or you have explicitly removed
quoting with "eval" or a parameter expansion flag.  Once a word is properly
quoted inside a parameter value, even a positional parameter, it remains
properly quoted until you mess with it somehow (which might include having
SH_WORD_SPLIT set, but that's why that's not set by default).

> > an ampersand; & isn't an expansion, just other shell syntax
>
> And that's just what I'm saying -- it would be nice to have some sort of
> bomb-proof zero expansion ability.

You're missing the point.  You can't "bomb-proof" syntactic tokens like
"&" without fundamentally changing the language.  It's like asking a C
compiler not to recognize semicolons.

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 24/01/17 02:31 PM, Bart Schaefer wrote:
> On Tue, 24 Jan 2017, Ray Andrews wrote:
>
>> Sure, but if we have any chains -- one command calling another -- we end
>> up loosing the quotes as things are passed along.
> That's not true UNLESS you've either used a poorly constructed alias (which
> is not really "one command calling another") or you have explicitly removed
> quoting with "eval" or a parameter expansion flag.  Once a word is properly
> quoted inside a parameter value, even a positional parameter, it remains
> properly quoted until you mess with it somehow (which might include having
> SH_WORD_SPLIT set, but that's why that's not set by default).

God knows.  I've probably zigged to fix my zags  on occasion, which is 
to say that I've had to do funny stuff to fix other funny stuff and the 
right way is not nearly as bad as I think.  I'd almost pay someone to 
look over my functions and proof read them, probably lots of bad code.
> And that's just what I'm saying -- it would be nice to have some sort of
> bomb-proof zero expansion ability.
> You're missing the point.  You can't "bomb-proof" syntactic tokens like
> "&" without fundamentally changing the language.  It's like asking a C
> compiler not to recognize semicolons.

But you can single quote, no?

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Tue, 24 Jan 2017, Ray Andrews wrote:

> But you can single quote, no?

Of course, but I thought this whole discussion was about turning other
things off so that you didn't need quotes.

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Tue, 24 Jan 2017, Ray Andrews wrote:

> On 24/01/17 02:31 PM, Bart Schaefer wrote:
> > On Tue, 24 Jan 2017, Ray Andrews wrote:
> >
> > > Sure, but if we have any chains -- one command calling another -- we
> > > end up loosing the quotes as things are passed along.
> > That's not true UNLESS ...
>
> God knows.  I've probably zigged to fix my zags on occasion, which is to
> say that I've had to do funny stuff to fix other funny stuff

Probably we're back to the "different levels of quoting" thing here; i.e.,
even if you've successfully protected everything from interpretation by
the shell language parser, you can still encounter another interpretation
done by the command itself, such as backslashes within "print" and "echo".

In nearly all cases those have a control of their own to force literal
use of arguments ("echo -E", "print -r", etc.).

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 24/01/17 07:46 PM, Bart Schaefer wrote:
> On Tue, 24 Jan 2017, Ray Andrews wrote:
>
>> But you can single quote, no?
> Of course, but I thought this whole discussion was about turning other
> things off so that you didn't need quotes.
>
Sorta, but if one method is possible then an option would work too, it's 
simply another method of transmitting an instruction.  And ... you said 
that 'eval' strips quotes?  I use eval to re-expand globs and just a 
while back you showed me another way so ...maybe all my grief comes from 
'eval' and that's where I keep losing quotation??  Nuts.

Re: Avoiding the zshells intelligence...in one case

[Ray Andrews]

On 24/01/17 08:56 PM, Bart Schaefer wrote:
> Probably we're back to the "different levels of quoting" thing here; 
> i.e.,
> even if you've successfully protected everything from interpretation by
> the shell language parser, you can still encounter another interpretation
> done by the command itself, such as backslashes within "print" and "echo".
>
> In nearly all cases those have a control of their own to force literal
> use of arguments ("echo -E", "print -r", etc.).
>
Sure.  If only one couldlearn this stuff other than the hard way.  I'd 
sure like to have a conversation with myself of three years ago and tell 
me what I know now.  I think back to expecting zsh to be something like 
C, now I understand that they couldn't be more different and why that is.

Re: Avoiding the zshells intelligence...in one case

[Bart Schaefer]

On Tue, 24 Jan 2017, Ray Andrews wrote:

> 'eval' strips quotes?  I use eval to re-expand globs and just a while
> back you showed me another way so ...maybe all my grief comes from
> 'eval' and that's where I keep losing quotation??  Nuts.

"eval" behaves for practical purposes as if you had re-typed the command
line and hit Enter again.  So if typing out and running the command would
change the quoting, eval will change the quoting.