suprise with -=

suprise with -=

[Ray Andrews]

    test1 ()
    {
    integer first=1
    integer second=2

    ((first+=second))
    echo $first

    ((first-=second))
    echo $first

    first+=second
    echo $first

    first-=second
    echo $first

    }

    3
    1
    3
    test1:15: command not found: first-=second
    3


That's a bit of a surprise, why is zsh fussier with '-=' than with '+='?

Re: suprise with -=

[Bart Schaefer]

On Oct 18, 10:46pm, Ray Andrews wrote:
}
} That's a bit of a surprise, why is zsh fussier with '-=' than with '+='?

Outside of the (( )), -= is not an assignment operator at all, because
the default is to do either array or string assignment, and there is
no sensible way to "subtract" one array or string from another.

Conversely += is defined to mean "append", so it is a valid operator.

I'd almost call it a bug that [outside of (( )) context] first+=second
does arithmetic when $first is an integer.  It ought to be either an
error or forcibly convert $first to string and append "second".

Re: suprise with -=

[Ray Andrews]

On 10/19/2015 11:35 AM, Bart Schaefer wrote:
> On Oct 18, 10:46pm, Ray Andrews wrote:
> }
> } That's a bit of a surprise, why is zsh fussier with '-=' than with '+='?
>
> Outside of the (( )), -= is not an assignment operator at all, because
> the default is to do either array or string assignment, and there is
> no sensible way to "subtract" one array or string from another.
>
> Conversely += is defined to mean "append", so it is a valid operator.
>
> I'd almost call it a bug that [outside of (( )) context] first+=second
> does arithmetic when $first is an integer.  It ought to be either an
> error or forcibly convert $first to string and append "second".
I've managed to suppress my gag reflex with all that arithmetic 
stuff--goes back to
the need to declare variable types, which is what I'm trying to do with 
'integer' and
after that I'd expect both increment and decrement to behave the same 
way.  I see
what you mean tho--we can append strings but we can't un-append them 
(tho one
might consider that as  " first=${first%$second} " (not sure if I wrote 
that right, but you
get me) ).  For now I feel comforted to always use the (( )) because 
it's explicit and you
can mercifully space things:  (( first -= second )).  Just to comment, 
I'd consider it
very rude for any forcible conversions to occur.  Better an error, tho 
again once one
has declared an integer one might expect one's operators to behave 
consistently.
We don't have warnings do we?  In C, of course, if one is doing 
something strange
the compiler doesn't stop you, but it will warn you.  I once had great 
fun writing an
encrypt function that performed arithmetic on strings--the compiler 
didn't like it,
but it did what it was told. Na ... can't have warnings in interpreted 
stuff, tho
there could be a proofreader ...

Re: suprise with -=

[Bart Schaefer]

On Oct 19, 12:34pm, Ray Andrews wrote:
}
} [...] declare variable types, which is what I'm trying to do with
} 'integer' and after that I'd expect both increment and decrement to
} behave the same way.

The parser doesn't know that "first" is an integer, or even that it's
a variable that was previously declared.  This is perfectly legal:

    if (( RANDOM % 7 ))
    then integer first
    else declare first
    fi
    first+=second

(Which is why I'm inclined to say it ought to always be string context
when not explicitly math context, but it's probably way to late for that.)

} I'd consider it very rude for any forcible conversions to occur.
} Better an error, tho again once one has declared an integer one might
} expect one's operators to behave consistently.

Note:

torch% integer first=1 second=2
torch% first+=(second)
torch% echo $first
1 second

This has silently forced $first to change into an array because of
explicit array context; it did not interpret "(second)" as arithmetic
parens.

Also just to mess things up a bit more:

torch% integer first=1 second=2
torch% third=first+second
torch% print $third
first+second
torch% integer third             
torch% print $third
3

If something has a string value and you re-declare it integer, it does
math on its string value.  Doesn't happen for array to integer.

Re: suprise with -=

[Ray Andrews]

On 10/19/2015 05:27 PM, Bart Schaefer wrote:
> On Oct 19, 12:34pm, Ray Andrews wrote:
> }
> } [...] declare variable types, which is what I'm trying to do with
> } 'integer' and after that I'd expect both increment and decrement to
> } behave the same way.
>
> The parser doesn't know that "first" is an integer, or even that it's
> a variable that was previously declared.  This is perfectly legal:
>
>      if (( RANDOM % 7 ))
>      then integer first
>      else declare first
>      fi
>      first+=second
Then what does 'integer' or 'typset -i' do?
> (Which is why I'm inclined to say it ought to always be string context
> when not explicitly math context, but it's probably way to late for that.)

Given all these little gotchas, I think you are right.  (()) is math, 
otherwise you're on
thin ice.  Still tho, surely 'integer' should be  a typecast and not 
changeable, tho of
course it's too late to do anything about that.  I can't help but wonder 
what was going
on in the minds of the guys who coded the first shells and decided to 
permit this sort
of anarchy, I'm sure they had their reasons but since the shells are 
written in C, I'd have
expected normal C discipline to set the standard.
> } I'd consider it very rude for any forcible conversions to occur.
> } Better an error, tho again once one has declared an integer one might
> } expect one's operators to behave consistently.
>
> Note:
>
> torch% integer first=1 second=2
> torch% first+=(second)
> torch% echo $first
> 1 second
>
> This has silently forced $first to change into an array because of
> explicit array context; it did not interpret "(second)" as arithmetic
> parens.

Well I guess that's the universal reality of shells, so I just hafta get 
used to it.  Actually I am already more or less used to it.
>
> Also just to mess things up a bit more:
>
> torch% integer first=1 second=2
> torch% third=first+second
> torch% print $third
> first+second
> torch% integer third
> torch% print $third
> 3
>
> If something has a string value and you re-declare it integer, it does
> math on its string value.  Doesn't happen for array to integer.
>
Thanks for the examples, these are so educational.  I'll play with them 
for several days.

Re: suprise with -=

[Bart Schaefer]

On Oct 20,  7:55pm, Ray Andrews wrote:
} Subject: Re: suprise with -=
}
} On 10/19/2015 05:27 PM, Bart Schaefer wrote:
} > The parser doesn't know that "first" is an integer, or even that it's
} > a variable that was previously declared.  This is perfectly legal:
} >
} >      if (( RANDOM % 7 ))
} >      then integer first
} >      else declare first
} >      fi
} >      first+=second
} 
} Then what does 'integer' or 'typset -i' do?

They declare that the internal representation of the variable should be
an integer, so that integer formatting may be applied when the variable
is converted back to a value with $first.

} I can't help but wonder what was going on in the minds of the guys who
} coded the first shells

The first shells didn't have integers or arrays at all.  They had only
strings, and a few (external) programs like "expr" that could interpret
strings of digits as numbers.

Arrays came next and numbers/math last of all, so numbers must conform
to the rules established when the shell was strings-only.  Shells were
never intended to look or behave anything like C (well, except for csh,
but it was only supposed to look like C, not act like it, and it got a
lot of other things wrong).

Re: suprise with -=

[Ray Andrews]

On 10/19/2015 05:27 PM, Bart Schaefer wrote:
> The parser doesn't know that "first" is an integer, or even that it's
> a variable that was previously declared.
That puzzles me.  Have not all your examples demonstrated that the 
parser is aware of the type of the variable and that it will perform 
it's operations accordingly?

    test2 ()
    {

        integer first=1 second=2

        third=first+second
        print A $third
        integer third
        print B $third

        integer fourth=first+second
        print C $fourth

    }
    A first+second    # 'third' is scalar so does the 'wrong' thing.
    B 0               # 'third' is now known to be integer by the 'print'.
    C 3               # 'fourth' is integer up front and remembered to
    be so.


I see very clearly that the type can change silently or sometimes not 
change at all and just sorta 'do nothing':

    test1 ()

    {

        integer first=1
        string1=foo

        first+=string1
        echo ${(t)first}
        echo "$first"

    }
    integer-local
    1            # The addition does nothing at all, but no error is thrown.


... and "${(t) ...}" is surely the demonstration that types are 
remembered?  I'm makinga deep error here, probably.


> The first shells didn't have integers or arrays at all.  They had only
> strings, and a few (external) programs like "expr" that could interpret
> strings of digits as numbers.
Thanks.  These 'history lessons' are invaluable (to me, anyway).  If 
anything besides strings were never anticipated in the original design 
of shells,then integers would bea 'tack on' and one could see that the 
whole issue of declarations/typing would behandled poorly.  The lesson 
is that one must be bloody careful. OTOH, whereas in Cif one wants to 
force a typecast it's a
labor, whereas in zsh one can do it not onlyeffortlessly, but even 
invisibly.
Powerful but dangerous.Caveat emptor.

Re: suprise with -=

[ZyX]

21.10.2015, 21:02, "Ray Andrews" <rayandrews@eastlink.ca>:
> On 10/19/2015 05:27 PM, Bart Schaefer wrote:
>>  The parser doesn't know that "first" is an integer, or even that it's
>>  a variable that was previously declared.
>
> That puzzles me. Have not all your examples demonstrated that the
> parser is aware of the type of the variable and that it will perform
> it's operations accordingly?
>
>     test2 ()
>     {
>
>         integer first=1 second=2
>
>         third=first+second
>         print A $third
>         integer third
>         print B $third
>
>         integer fourth=first+second
>         print C $fourth
>
>     }
>     A first+second # 'third' is scalar so does the 'wrong' thing.
>     B 0 # 'third' is now known to be integer by the 'print'.
>     C 3 # 'fourth' is integer up front and remembered to
>     be so.
>
> I see very clearly that the type can change silently or sometimes not
> change at all and just sorta 'do nothing':

How is *parser* related? To make

    integer fourth=first+second

set `fourth` to `3` it is *absolutely* not needed to parse this as

    integer(assign("fourth", plus("first", "second")))

. Just at the time of parsing you parse it as

    integer(assign("fourth", "first+second"))

and at the time of *execution* you evaluate `first+second` as an expression.

>
>     test1 ()
>
>     {
>
>         integer first=1
>         string1=foo
>
>         first+=string1
>         echo ${(t)first}
>         echo "$first"
>
>     }
>     integer-local
>     1 # The addition does nothing at all, but no error is thrown.
>
> ... and "${(t) ...}" is surely the demonstration that types are
> remembered? I'm makinga deep error here, probably.

Remembering types has *absolutely* nothing to do with parsing.
${(t)V} is executed at runtime, after parsing. Neither parser needs to know
what is `V` and whether it is defined.

Basically this happens in every dynamically typed language. (Though most
statically typed language developers do not think that parsing
two constructs differently depending on variable types is not a good idea;
this does not prevent e.g. `foo.a` to be an error due to incorrect `foo` type,
but this is checked after parsing stage and is always read as
something like `getattr("foo", "a")`.)

>
>>  The first shells didn't have integers or arrays at all. They had only
>>  strings, and a few (external) programs like "expr" that could interpret
>>  strings of digits as numbers.
>
> Thanks. These 'history lessons' are invaluable (to me, anyway). If
> anything besides strings were never anticipated in the original design
> of shells,then integers would bea 'tack on' and one could see that the
> whole issue of declarations/typing would behandled poorly. The lesson
> is that one must be bloody careful. OTOH, whereas in Cif one wants to
> force a typecast it's a
> labor, whereas in zsh one can do it not onlyeffortlessly, but even
> invisibly.
> Powerful but dangerous.Caveat emptor.

Re: suprise with -=

[Bart Schaefer]

On Oct 21, 11:01am, Ray Andrews wrote:
} Subject: Re: suprise with -=
}
} On 10/19/2015 05:27 PM, Bart Schaefer wrote:
} > The parser doesn't know that "first" is an integer, or even that it's
} > a variable that was previously declared.
} That puzzles me.  Have not all your examples demonstrated that the 
} parser is aware of the type of the variable and that it will perform 
} it's operations accordingly?

No.  Parser is syntax, not semantics.  It knows "x=y" has a valid shape,
and to create code to invoke an assignment at run time, but not what the
assignment will do with x or y when run.

You can think of "third=first+second" as syntactic sugar for the command
"typeset third=first+second" -- it's the command that knows the type of
the variable, not the parser.

Re: suprise with -=

[Ray Andrews]

On 10/21/2015 11:43 AM, ZyX wrote:

> and at the time of *execution* you evaluate `first+second` as an 
> expression. 

Ok, I think I begin to see where I'm going wrong.  I'm extending 
'parsing' in my head into what only happens at execution. Another 
difference between a compiled language vs. an interpreted one.  I guess 
parsing, in an interpreted language, is strictly speaking a much simple 
matter.  At run time more work is done.  Thanks.

Re: suprise with -=

[ZyX]

22.10.2015, 18:31, "Ray Andrews" <rayandrews@eastlink.ca>:
> On 10/21/2015 11:43 AM, ZyX wrote:
>
>>  and at the time of *execution* you evaluate `first+second` as an
>>  expression.
>
> Ok, I think I begin to see where I'm going wrong. I'm extending
> 'parsing' in my head into what only happens at execution. Another
> difference between a compiled language vs. an interpreted one. I guess
> parsing, in an interpreted language, is strictly speaking a much simple
> matter. At run time more work is done. Thanks.

It is not always wrong, BTW. E.g. VimL has no “parsing” stage, it always directly *executes* the input string, doing any parsing in process. This is why e.g. when calling

    :let var=[system("echo bar>baz"),

file `baz` will appear, but `var` will not get assigned due to parsing error: VimL executor does not see absense of `]` at the time it is executing `system()` call. Also meaning of

    :echo var.val

depends on `var` variable type (it may either be `var . val` (string concat) or `var['val']` (dictionary index)).

Though I know no language implementations other then VimL (and, maybe, tcsh: have not actually seen its source code, but it looks like it is also executed in-place) that do not have the parsing stage.

Re: suprise with -=

[Ray Andrews]

On 10/22/2015 08:43 AM, ZyX wrote:
> 22.10.2015, 18:31, "Ray Andrews" <rayandrews@eastlink.ca>:
>
> It is not always wrong, BTW. E.g. VimL has no “parsing” stage, it always directly *executes* the input string, doing any parsing in process. This is why e.g. when calling
>
>      :let var=[system("echo bar>baz"),
>
> file `baz` will appear, but `var` will not get assigned due to parsing error: VimL executor does not see absense of `]` at the time it is executing `system()` call. Also meaning of
Well then I should be grateful for whatever parsing zsh does--at least 
it picks up clear errors in syntax.  But as Bart says, semantics is not 
the same thing.  One can think one understands these things but still 
have a brain full of deep errors.  If I'd been involved in this 20 years 
ago I'd have written: "The Tao of shells--why everything a C coder 
thinks he knows about programming ain't necessarily so."

Re: suprise with -=

[Bart Schaefer]

On Oct 22,  6:43pm, ZyX wrote:
}
} Though I know no language implementations other then VimL (and, maybe,
} tcsh: have not actually seen its source code, but it looks like it is
} also executed in-place) that do not have the parsing stage.

Yes, (t)csh also executes in place, or close to it.  Csh parses one line
at a time, executes it, then parses the next line.  It's not quite as
bad as VimL, but it means that e.g. loops within scripts are implemented
by using fseek to rewind the file pointer to the beginning of the loop.

You can create some really entertaining self-modifying scripts in csh by
having the script rewrite parts of its own source file before they are
read (or re-read, if looping).

Csh is like Gromit in "The Wrong Trousers", laying down his tracks the
instant before the train runs across them.  VimL apparently lays down
the individual railroad ties.

Re: suprise with -=

[Daniel Shahaf]

Bart Schaefer wrote on Thu, Oct 22, 2015 at 16:56:26 -0700:
> On Oct 22,  6:43pm, ZyX wrote:
> }
> } Though I know no language implementations other then VimL (and, maybe,
> } tcsh: have not actually seen its source code, but it looks like it is
> } also executed in-place) that do not have the parsing stage.
> 
> Yes, (t)csh also executes in place, or close to it.  Csh parses one line
> at a time, executes it, then parses the next line.  It's not quite as
> bad as VimL, but it means that e.g. loops within scripts are implemented
> by using fseek to rewind the file pointer to the beginning of the loop.
> 
> You can create some really entertaining self-modifying scripts in csh by
> having the script rewrite parts of its own source file before they are
> read (or re-read, if looping).
> 
> Csh is like Gromit in "The Wrong Trousers", laying down his tracks the
> instant before the train runs across them.  VimL apparently lays down
> the individual railroad ties.

TeX is also a lot of fun:

% cat texput.tex
                   xmessage{this is file contents}
\catcode`x=0\relax xmessage{this is a compile-time message}
xbye
% tex texput.tex
This is TeX, Version 3.14159265 (TeX Live 2015/dev/Debian) (preloaded format=tex)
(./texput.tex this is a compile-time message [1] )
Output written on texput.dvi (1 page, 244 bytes).
Transcript written on texput.log.
% dvi2tty texput.dvi | grep .
xmessagethis is file contents
                                    1
% 

I suppose you might say that TeX not only lays down the individual
railroad ties like VimL, but also allows each tie to specify the
dimensions and color of the next tie...

Daniel
(tokenization is changeable at runtime; I made x equivalent to
backslash, so, for instance, the last line invokes \bye)