zsh at perl conference and few questions

zsh at perl conference and few questions

[Marc Chantreux]

hello people,

i'm happy to annonce i'll talk about zsh during the european edition of
the perl conference (glasgow)

http://act.perlconference.org/tpc-2018-glasgow/talk/7338

as i prepare my slides those days, some questions came to me. some of
these are (detailed below):

a) can someone tell me why isn't the "alternative" syntax more used ?
b) why the while loop can't take (( )) or single instruction as do list ?
c) it seems the (+) syntax can't be used outside file expansions
d) is there a plan to have something like namespaces ?

any comment/feedback/thought would be really appreciated.

regards
marc


a) can someone tell me why isn't the "alternative" syntax more used ?

i felt in love with zsh about 20 years now and one of the reason is
the alternative syntax. so can someone explain to me why the "old"
one seems to be prefered even nowdays ?

    for x in {1..20}; do
        print "$x * 2 = $[x * 2]"
    done

seems terrible to me compared to

    for x ({1..20}) print "$x * 2 = $[x * 2]"

as time passed, i added those alias in my .zshenv

    alias @='for it'
    alias @-='while {read it}'

so i daily (hourly ...) write those kind of thing

    @ (*.txt) gzip $it

b) why the while loop can't take (( )) or single instruction as do list ?

those works fine

    if {true} print ok
    if (( 2 == 2 )) print ok
    for ((count=3; count; count--)) print $count
    while ((count)) {print $[count--]}

so why not those ?

    while ((count)) print $[count--]
    while {read it} ((sum+=it))

c) it seems the (+) syntax can't be used outside file expansions
   (or did i miss something?)

as you can write

   by_word_count () REPLY=$( wc -w < $REPLY )
   print -l *.html(.o+by_word_count)

why shouldn't i write

   by_word_count () REPLY=$( wc -w < $REPLY )
   files=( *.html(.) )
   print -l ${(o+by_word_count)files}

or filter with something like

   large () (( $( wc -w < $REPLY ) > 200 ))
   files=( *.html(.) )
   print -l ${(+large)files}

d) is there a plan to have something like namespaces ?


using setopt pathdirs, you can put functions into files
in your path and reuse the name of it into the function
names. i use it to create a poor man namespace.

so if i have net/utils in my path i can fill it with

    net/utils/running\? ()
        ping -W 1 -c 1 ${1?ip or hostname to ping} \
        &> /dev/null

    net/utils/report () {
        local status="missing"
        net/utils/running\? ${1?ip or hostname to ping} &&
            status="online"
        print -u2 "$1 is $status"
    }

so now i can write

    $ . net/utils
    $ net/utils/report prometheus
    prometheus is online

in uze.zsh, i created something to export a function into the main
namespace (just copy the values of $functions )

so i can write

    $ . net/utils report
    $ prometheus
    prometheus is online

so for the user of the net/utils lib, that's fine ...
*but* when i write functions, i have to be careful about
the current namespace. the thing i dream about is the ability
to rewrite

    net/utils/report () {
        local status="missing"
        net/utils/running\? ${1?ip or hostname to ping} &&
            status="online"
        print -u2 "$1 is $status"
    }

as "the report function of this namespace"... if a // prefix
could exist, i could write something like

    //report () {
        local status="missing"
        //running\? ${1?ip or hostname to ping} &&
            status="online"
        print -u2 "$1 is $status"
    }

this should be awesome but i don't know if realistic as zsh
don't have syntax closure as well.

Re: zsh at perl conference and few questions

[Peter Stephenson]

On Mon, 23 Apr 2018 11:24:12 +0200
Marc Chantreux <eiro@phear.org> wrote:
> a) can someone tell me why isn't the "alternative" syntax more used ?
> 
> i felt in love with zsh about 20 years now and one of the reason is
> the alternative syntax. so can someone explain to me why the "old"
> one seems to be prefered even nowdays ?
> 
>     for x in {1..20}; do
>         print "$x * 2 = $[x * 2]"
>     done
> 
> seems terrible to me compared to
> 
>     for x ({1..20}) print "$x * 2 = $[x * 2]"

I don't really know how widely it's used, but you get a certain amount
using short loops without having to remember novel syntax.

for x  in {1..20}; print "$x * 2 = $[x * 2]"

All you need to note here is there's no "do" / "done", which I can
manage.  Parentheses are already rather overloaded so I don't do
anything myself that adds yet more.

It's quite hard to ensure alternative syntax gets parsed consistently
--- I'm sure there are lots of inconsistencies --- but a lot of that is
hidden.

> b) why the while loop can't take (( )) or single instruction as do
> list ?

Maybe because it's missing the code at the bottom?  As this makes
something which was a parse error into something which isn't I don't
think this can break anything.  It's certainly not a compatibility
problem because this is what SHORT_LOOPS takes care of.  So I suppose
it's just an oversight --- perhaps it didn't seem so obviously
useful because the simple command at the end would need some exit
condition for the while loop as well as doing it's basic function.

> c) it seems the (+) syntax can't be used outside file expansions
>    (or did i miss something?)

Yes, there's no general "execute a function that does something" in
other cases because there's no obvious definition of whit it would do
--- unlike globbing qualifiers which are there as a simple filter.

> d) is there a plan to have something like namespaces ?

It was first discussed a long time ago, but no one has bitten the
bullet.  Simple minded namespaces --- allow dots in the variable works
--- are trivial, but the variable code is very complicated and working
out how to do it properly is a big task that no one has been prepared to
look at (saying "someone else ought to do this" does not count as
looking at it" :-)).

pws

diff --git a/Src/parse.c b/Src/parse.c
index 47e5a24..83383f1 100644
--- a/Src/parse.c
+++ b/Src/parse.c
@@ -1510,8 +1510,10 @@ par_while(int *cmplx)
 	if (tok != ZEND)
 	    YYERRORV(oecused);
 	zshlex();
-    } else
+    } else if (unset(SHORTLOOPS)) {
 	YYERRORV(oecused);
+    } else
+	par_save_list1(cmplx);
 
     ecbuf[p] = WCB_WHILE(type, ecused - 1 - p);
 }

Re: zsh at perl conference and few questions

[Marc Chantreux]

hello,

thanks for your reply

> It's quite hard to ensure alternative syntax gets parsed consistently
> --- I'm sure there are lots of inconsistencies --- but a lot of that is
> hidden.

well... i used it at least 15 years long without having problems so
we can imagine it's consistent enought :)

> > b) why the while loop can't take (( )) or single instruction as do
> > list ?
> 
> Maybe because it's missing the code at the bottom?

should this code be merged in the zsh codebase ? :)

> > c) it seems the (+) syntax can't be used outside file expansions
> >    (or did i miss something?)
> 
> Yes, there's no general "execute a function that does something" in
> other cases because there's no obvious definition of whit it would do
> --- unlike globbing qualifiers which are there as a simple filter.

is it a possible open discussion with zsh-workers ? it seems to me i
have a clear idea of how things could work but i have now idea of how to
implement it. maybe i should document it ? (or maybe there are some
archives outthere about why it cannot be obvious?)

> > d) is there a plan to have something like namespaces ?
> It was first discussed a long time ago, but no one has bitten the
> bullet.  Simple minded namespaces --- allow dots in the variable works
> --- are trivial, but the variable code is very complicated and working
> out how to do it properly is a big task that no one has been prepared to
> look at (saying "someone else ought to do this" does not count as
> looking at it" :-)).

in this case i wasn't talking about variables but only function names but
i imagine it goes the same.

once again thanks for your reply.

regards
marc

Re: zsh at perl conference and few questions

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

On 23 April 2018 at 12:06, Peter Stephenson <p.stephenson@samsung.com>
wrote:

> >     for x in {1..20}; do
> >         print "$x * 2 = $[x * 2]"
> >     done
> >
> > seems terrible to me compared to
> >
> >     for x ({1..20}) print "$x * 2 = $[x * 2]"
>
> I don't really know how widely it's used, but you get a certain amount
> using short loops without having to remember novel syntax.
>
> for x  in {1..20}; print "$x * 2 = $[x * 2]"
>
>
Mikael always reminds that having short_loops enabled weakens the parser
ability to detect errors, so maybe it's worth mentioning here.

> d) is there a plan to have something like namespaces ?
>
> It was first discussed a long time ago, but no one has bitten the
> bullet.  Simple minded namespaces --- allow dots in the variable works
> --- are trivial, but the variable code is very complicated and working
> out how to do it properly is a big task that no one has been prepared to
> look at (saying "someone else ought to do this" does not count as
> looking at it" :-)).
>

It's an occasion to me to tell the observation that function namespaces
would be rather more needed than variable ones. The point is that hook
plugins like z-sy-h and zsh-autosuggestions copy each widget to hook into
it. So it's ${#widgets}**2 functions out there with those plugins loaded,
and times of "lets see my functions print -rl -- ${(k)functions}" are over.

-- 
Best regards,
Sebastian Gniazdowski

Re: zsh at perl conference and few questions

[Marc Chantreux]

hello,

> It's an occasion to me to tell the observation that function namespaces
> would be rather more needed than variable ones.

yes, i really asked for namespaces in functions.

> plugins like z-sy-h and zsh-autosuggestions copy each widget to hook into
> it. So it's ${#widgets}**2 functions out there with those plugins loaded,
> and times of "lets see my functions print -rl -- ${(k)functions}" are over.

this is the same situation in the uze exporter mechanism and as i said:
from the "user" counterpart, everything is ok:

    uze TAP :all

    boolean_works () {
        true    ; ok "true  is right"
        :       ; ok ":     is right"
        ! false ; ok "false is right"
    }

    prove boolean_works

renders

    ok 1 - true  is right
    ok 2 - :     is right
    not ok 3 - false is right
    1..3

works because there is a TAP.zsh somewhere in $path that

    * implements TAP/{ok,prove}
    * implements uze/export/TAP populating a local variable
      used by the exporter

        EXPORT_TAGS=(
            :all
            'prove ok not_ok plan is isnt note note- expected unexpected'
        )

so when you do

    uze TAP :all
    which prove TAP/prove

you get

    prove () TAP/prove "$@"
    TAP/prove () {
        typeset -A TAPCTX=(index 0 start 1 plan 0) 
        TAP/start
        "$@"
        TAP/done
    }

the boring part is for the implementor: to be consistent,
the file this/very/long/and/boring/ns.zsh will contain

    this/very/long/and/boring/ns/foo () {
        this/very/long/and/boring/ns/bar
        echo bye
    }

    this/very/long/and/boring/ns/bar () {
        echo hello
    }

with namespaces, it will contain

    //foo () {
        //bar
        echo bye
    }

    //bar () {
        echo hello
    }

and this should be renamed simply. note that if zsh had closures,
we could use the same hack the javascript world use.

() {
    my _P=this/very/long/and/boring/ns
    $_P/foo () { $_P/bar ; echo bye }
    $_P/bar () echo hello
}

which leads me to the real motivation of this mail: zsh is realllly
powerfull and simple and in many circonstances, it could be used
instead of interpreted langages like python, perl, ruby. some claim
that shell miss nested data structures which is right but there
are workaround (or alternate way of thinking ?)

    We have persistant(sic) objects, they’re called files.
        — Ken Thompson

i put a demo here http://eiro.github.io/unix/zsh-oo-demo.txt.

when it comes to variables, what i miss the most are
lexical variables and closures.

see, this code

    () {
        local x=12
        plus  () l $[x++]
        minus () l $[x--]
    }
    plus

raise an error message because

    plus:6: numeric parameter x created globally in function plus

and the output is

    0

the perl behavior (and recent versions of javascript)
is much more superior on this part:

* local localize a variable (the way zsh does)
* my create a lexical variable
* whenever a lexical variable of a scope is used
  inside a function definition, this create a
  state preserved through function calls

in zsh, it could be

    counter () {
        my x=${2-0}
        $1/+  () { let x + $1; echo $x }
        $1/-  () { let x + $1; echo $x }
    }

    counter c 2
    c/+ 2

to return 4. this case is obviously useless because it's about
one variable but it could be a way to capture a complex context.

for the moment, the only way to do this is to create subshells
and fifo to have coroutines. but it can be painful to write.

i don't know how much this topic is relevent to zsh
implementation but i really think closures and namespaces are
the 2 missing features to make zsh reasonable
on larger codebases.

plus: we need something like cpan (https://metacpan.org/) or
or pypi, crates, epm, ctan, npm,  ...

regards
marc

Re: zsh at perl conference and few questions

[Bart Schaefer]

On Mon, Apr 23, 2018 at 3:06 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> Simple minded namespaces --- allow dots in the variable works
> --- are trivial,

It's actually not that trivial because the same code for parsing an
"identifier" is used in a lot of places.  It's not possible to just
add "." to the set of identifier characters, because we don't want
$this.that to parse as ${this.that}, among other things.

Nevertheless this part --

> but the variable code is very complicated and working
> out how to do it properly is a big task that no one has been prepared to
> look at

-- is definitely the bigger part of the picture.  For one thing, it
would be ideal if true namerefs were working before namespaces were.

Re: zsh at perl conference and few questions

[Marc Chantreux]

hello,

> -- is definitely the bigger part of the picture.  For one thing, it
> would be ideal if true namerefs were working before namespaces were.

by true namerefs, you talk about the ability yo create nested
structures? something like

    nicknames=( joe bob henry )
    local -A anonymous=(
        realname "John Doe"
        nicknames ${&nicknames}
    )

so
    $anymous[nicknames][2] == "bob"

if it so, this would be awesome indeed ...

regards
marc

Re: zsh at perl conference and few questions

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

On 23 April 2018 at 11:24, Marc Chantreux <eiro@phear.org> wrote:

> hello people,
>
> i'm happy to annonce i'll talk about zsh during the european edition of
> the perl conference (glasgow)
>
> http://act.perlconference.org/tpc-2018-glasgow/talk/7338
>
> as i prepare my slides those days, some questions came to me. some of
> these are (detailed below):
>

You probably have reduced many truths and facts about Zsh for this talk,
but maybe you're looking for some more content, then I have reduced some
stuff too like a) matching in AND style or b) what's going on with "@"
splitting, etc. or c) executing code when substituting. It's here:

http://zdharma.org/Zsh-100-Commits-Club/Zsh-Native-Scripting-Handbook.html

Best regards,
Sebastian Gniazdowski