Feature request (@M):# with context matches

Feature request (@M):# with context matches

[Sebastian Gniazdowski]

Hello,
doing:

list=( "${(@M)list:#(#i)*$~search_pattern*}" )

can be used to replace call to grep. Array can be filtered with a
pattern, no fork is required. However, grep supports -A, -B, -C
options, that return also "A"-elements after or "B"-elements before,
or "C"-elements before and after a match. I thought that this is maybe
not difficult to add to Zsh? It's very useful thing. For example, when
using "less" to search for a keyword (like when reading zshall), user
finds surrounding text and it's basically actually what he searches
for.

Best regards,
Sebastian Gniazdowski

Re: Feature request (@M):# with context matches

[Sebastian Gniazdowski]

The idea is to confront Bash Advanced Scripting Guide coders, BASG
coders for short. Every sed, awk, grep can be replaced by Zsh code. No
need for `` / $() what Bash coders love (not all, could name at least
one that doesn't, is present at #zsh IRC channel). However, grep -C is
not very possible to replace with Zsh code. I plan to write
Markdown/PDF  document where every GNU utility will be coded via
native Zsh code, without forks, and the grep -C is one problem that I
can perceive now.

Best regards,
Sebastian Gniazdowski

Re: Feature request (@M):# with context matches

[Christian Heinrich]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

Hi Sebastian,

I'm not sure what you want to do, but I my guts tells me that you want
to re-invent the wheel as you stated that you want to replace several
tools with ZSH. If you can achieve your goal by calling a GNU tool, you
should do that. These are mature tools and there is nothing wrong about
using them. (And it's certainly a benefit that you don't have to
maintain them.)

Calling "grep" is also more understandable by everyone than the pattern
you gave in your first email.

Best regards
Christian

> The idea is to confront Bash Advanced Scripting Guide coders, BASG
> coders for short. Every sed, awk, grep can be replaced by Zsh code.
> No
> need for `` / $() what Bash coders love (not all, could name at least
> one that doesn't, is present at #zsh IRC channel). However, grep -C
> is
> not very possible to replace with Zsh code. I plan to write
> Markdown/PDF  document where every GNU utility will be coded via
> native Zsh code, without forks, and the grep -C is one problem that I
> can perceive now.
> 
> Best regards,
> Sebastian Gniazdowski

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Feature request (@M):# with context matches

[Sebastian Gniazdowski]

On 30 January 2016 at 15:38, Christian Heinrich
<christian@gladbachcity.de> wrote:
> Hi Sebastian,
>
> I'm not sure what you want to do, but I my guts tells me that you want
> to re-invent the wheel as you stated that you want to replace several
> tools with ZSH. If you can achieve your goal by calling a GNU tool, you
> should do that. These are mature tools and there is nothing wrong about
> using them. (And it's certainly a benefit that you don't have to
> maintain them.)
>
> Calling "grep" is also more understandable by everyone than the pattern
> you gave in your first email.

I'll try to contradict the "more understandable" with an example –
antigen, a probably very well written BSG code. Take this function for
example:

https://github.com/zsh-users/antigen/blob/master/antigen.zsh#L100-L112

antigen-bundles () {
    # Bulk add many bundles at one go. Empty lines and lines starting with a `#`
    # are ignored. Everything else is given to `antigen-bundle` as is, no
    # quoting rules applied.

    local line

    grep '^[[:space:]]*[^[:space:]#]' | while read line; do
        # Using `eval` so that we can use the shell-style quoting in each line
        # piped to `antigen-bundles`.
        eval "antigen-bundle $line"
    done
}

For me it's hard to understand this function. Grep, forks, pipes,
evals. My point is that this is what "$( )" leads to in general.
That's what ABSG advises to do, and such code is spread everywhere,
across companies, open source projects. Other example:

https://github.com/zsh-users/antigen/blob/master/antigen.zsh#L347-L355

    # Find directores in ADOTDIR/repos, that are not in the bundles record.
    local unused_clones="$(comm -13 \
        <(-antigen-echo-record |
            awk '$4 == "true" {print $1}' |
            while read line; do
                -antigen-get-clone-dir "$line"
            done |
            sort -u) \
        <(\ls -d "$ADOTDIR/repos/"* | sort -u))"

This is one candidate to most outstanding BSG snippet in the project.

On the other hand, such programming is maybe something distinct from
typical imperative programming. With my "GNU -> zsh" document, I would
maybe advocate how to do normal programming in Zsh. But I believe I
wouldn't, Zsh has it's own unique traits, own ways of compressing much
into little of code, without use of Awk.

Best regards,
Sebastian Gniazdowski

Re: Feature request (@M):# with context matches

[Bart Schaefer]

On Jan 30,  1:13pm, Sebastian Gniazdowski wrote:
}
} However, grep -C is not very possible to replace with Zsh code.

I'm a little puzzled about how you think this would work in a
parameter substitution context.  Grep operates on a stream of
input lines, parameter substitution works on values in memory.  Do
you propose to capture the entire input in an array before even
performing the search?

In any case the way you want to do this kind of thing is with array
subscript flags.

 r
     Reverse subscripting: if this flag is given, the EXP is taken as a
     pattern and the result is the first matching array element,
     substring or word (if the parameter is an array, if it is a
     scalar, or if it is a scalar and the `w' flag is given,
     respectively).  The subscript used is the number of the matching
     element, so that pairs of subscripts such as `$foo[(r)??,3]' and
     `$foo[(r)??,(r)f*]' are possible if the parameter is not an
     associative array.

 R
     Like `r', but gives the last match.  For associative arrays, gives
     all possible matches.

 i
     Like `r', but gives the index of the match instead; this may not be
     combined with a second argument.

 I
     Like `i', but gives the index of the last match, or all possible
     matching keys in an associative array.

(The docs have a lot more, I just copied the critical bits.)

So you want something like

    while (( ( hit = $list[(i)*$search_pattern*] ) < $#list ))
    do
	print -r -- $list[hit-3,hit+3]
	shift $hit list
    done

with some appropriate checking that (hit-3) doesn't become negative and
wrap around to the other end of the array, and other foo to manage the
possiblity of overlapping contexts.  Note you don't even need the tilde
in $search_pattern, [(i)...] forces everything into pattern context; if
you do NOT want patterns, you need ${(b)search_pattern} (or in older
zsh ${(q)search_pattern} will usually work).

Re: Feature request (@M):# with context matches

[Bart Schaefer]

On Jan 30,  8:54am, Bart Schaefer wrote:
}
}     while (( ( hit = $list[(i)*$search_pattern*] ) < $#list ))
}     do
} 	print -r -- $list[hit-3,hit+3]
} 	shift $hit list
}     done
} 
} with some appropriate checking that (hit-3) doesn't become negative and
} wrap around to the other end of the array, and other foo to manage the
} possiblity of overlapping contexts.

If you don't care about multiple hits or merging overlapping contexts, you
can even mash this down to a single statement.

E.g.

torch% declare S=path		# Searching in $path
torch% integer hit=0 C=2	# Context is 2 elements
torch% print -rl ${${(P)S}[$((
mathsubst> (hit = ${${(P)S}[(i)*local*]}) > C
mathsubst> ? hit < $#path
mathsubst>   ? hit-C
mathsubst>   : hit
mathsubst> : 1
mathsubst> )),$((
mathsubst> hit < $#path
mathsubst> ? hit+C
mathsubst> : hit
mathsubst> ))]}
/home/schaefer/bin
/usr/local/bin
/bin
/usr/bin

(Math parsing permits embedded newlines, subscript parsing does not, thus
the placement of "[$((" and ")),$((" and "))]".)

Note the ?: ternary tests would be different if using (I) to search from
right to left.

min() max() math functions (was: Re: Feature request (@M):# with context matches)

[Daniel Shahaf]

Bart Schaefer wrote on Sat, Jan 30, 2016 at 08:54:56 -0800:
> So you want something like
> 
>     while (( ( hit = $list[(i)*$search_pattern*] ) < $#list ))
>     do
> 	print -r -- $list[hit-3,hit+3]
> 	shift $hit list
>     done
> 
> with some appropriate checking that (hit-3) doesn't become negative and
> wrap around to the other end of the array, [...]

I've wished a number of times for a built-in max() math function, which
would allow:

    $list[max(hit-3, 0), hit+3]

There are a number of places in the distribution that could use this:

    % git grep '[(][(].*[?].*[:]'
    Completion/Unix/Command/_git:    (( n = $#file > $#prefix ? $#file : $#prefix ))
    Completion/Zsh/Type/_ps1234:    (( cols = cols > 255 ? 255 : cols ))
    Functions/Misc/zargs:   shift $((end > ARGC ? ARGC : end))
    Functions/Zle/modify-current-argument:  (( CURSOR = wordoff + (poschar > repmax ? repmax : poschar) - 1 ))

I also needed this a number of times in my zshrc, too, such as for
.
    MANWIDTH=$(( min(COLUMNS, 80) ))

I imagine that function would be variadic and take parameters of any
numeric type...  Something along these lines:

    _mathfunc_min _mathfunc_max() {
      local result=$1 ; shift
      while (( $# )) ; do
        case $0 in
          (max) (( $1 > result )) && result=$1;;
          (min) (( $1 < result )) && result=$1;;
        esac
        shift
      done
      (( result )) # return
    }
    functions -M max 1 -1 _mathfunc_max # at least one argument
    functions -M min 1 -1 _mathfunc_min # at least one argument

And while at it:

    _mathfunc_sum() {
      local sum
      while (( $# )) ; do
        (( sum += $1 ))
        shift
      done
      (( sum ))
    }
    functions -M sum 0 -1 _mathfunc_sum

Cheers,

Daniel

Re: min() max() math functions (was: Re: Feature request (@M):# with context matches)

[Bart Schaefer]

On Feb 7, 12:22am, Daniel Shahaf wrote:
} Subject: min() max() math functions (was: Re: Feature request (@M):# with 
}
} I've wished a number of times for a built-in max() math function

You provided such nice implementations with "functions -M".  Why is it
important that they be built-ins?

Generally speaking we've done builtins only where (a) there is a library
implementation easily linked with or (b) the results of C type coercion
are at odds with the desired behavior of shell numeric-typed parameters.

Re: min() max() math functions (was: Re: Feature request (@M):# with context matches)

[Daniel Shahaf]

Bart Schaefer wrote on Sat, Feb 06, 2016 at 17:00:40 -0800:
> On Feb 7, 12:22am, Daniel Shahaf wrote:
> } Subject: min() max() math functions (was: Re: Feature request (@M):# with 
> }
> } I've wished a number of times for a built-in max() math function
> 
> You provided such nice implementations with "functions -M".  Why is it
> important that they be built-ins?
> 
> Generally speaking we've done builtins only where (a) there is a library
> implementation easily linked with or (b) the results of C type coercion
> are at odds with the desired behavior of shell numeric-typed parameters.

I said "built-in" but I meant in zsh/mathfunc, with all other math
functions, for consistency.

I suppose I could package them as an autoload.  Patch below. 

I changed 'while' to 'for' to avoid unnecessary shifts.
(The functions would otherwise be O(N²) in the number of arguments)

How do I shield zmathfunc against callers that unsetopt functionargzero?
.
    % autoload zmathfunc && zmathfunc
    % echo $((max(42,43)))
    43
    % unsetopt functionargzero
    % echo $((max(42,43)))      # now $0 is /bin/zsh, no 'case' branch matches, so $argv[1] is returned
    42
.
I guess I'll have to drop the the multifuncdef usage altogether?

Cheers,

Daniel

diff --git a/Doc/Zsh/contrib.yo b/Doc/Zsh/contrib.yo
index 07a5eb0..11678e5 100644
--- a/Doc/Zsh/contrib.yo
+++ b/Doc/Zsh/contrib.yo
@@ -3796,6 +3796,23 @@ enditem()
 
 See the comments in the function for a few extra tips.
 )
+findex(max)
+findex(min)
+findex(sum)
+findex(zmathfunc)
+xitem(tt(min+LPAR())var(arg)tt(, ...+RPAR()))
+xitem(tt(max+LPAR())var(arg)tt(, ...+RPAR()))
+xitem(tt(sum+LPAR())var(arg)tt(, ...+RPAR()))
+item(tt(zmathfunc))(
+Autoloading tt(zmathfunc) defines the three mathematical functions
+tt(min), tt(max), and tt(sum).  The functions tt(min) and tt(max) take
+one or more arguments.  The function tt(sum) takes zero or more arguments.
+Arguments can be of different types (ints and floats).
+
+Not to be confused with the tt(zsh/mathfunc) module, described in
+ifzman(the section `The zsh/mathfunc Module' in zmanref(zshmodules))\
+ifnzman(noderef(The zsh/mathfunc Module)).
+)
 findex(zmathfuncdef)
 item(tt(zmathfuncdef) [ var(mathfunc) [ var(body) ] ])(
 A convenient front end to tt(functions -M).
diff --git a/Doc/Zsh/mod_mathfunc.yo b/Doc/Zsh/mod_mathfunc.yo
index 5239da5..af1abcc 100644
--- a/Doc/Zsh/mod_mathfunc.yo
+++ b/Doc/Zsh/mod_mathfunc.yo
@@ -36,6 +36,11 @@ that it is therefore only useful immediately after a call to tt(gamma) or
 tt(lgamma).  Note also that `tt(signgam+LPAR()RPAR())' and `tt(signgam)' are
 distinct expressions.
 
+The functions tt(min), tt(max), and tt(sum) are defined not in this module
+but in the tt(zmathfunc) autoloadable function, described in
+ifzman(the section `Mathematical Functions' in zmanref(zshcontrib))\
+ifnzman(noderef(Mathematical Functions)).
+
 The following functions take two floating point arguments: tt(copysign),
 tt(fmod), tt(hypot), tt(nextafter).
 
diff --git a/Functions/Math/.distfiles b/Functions/Math/.distfiles
new file mode 100644
index 0000000..f03668b
--- /dev/null
+++ b/Functions/Math/.distfiles
@@ -0,0 +1,2 @@
+DISTFILES_SRC='
+'
diff --git a/Functions/Math/zmathfunc b/Functions/Math/zmathfunc
new file mode 100644
index 0000000..198454d
--- /dev/null
+++ b/Functions/Math/zmathfunc
@@ -0,0 +1,29 @@
+#autoload
+
+setopt localoptions multifuncdef
+
+zsh_math_func_min zsh_math_func_max() {
+  local result=$1
+  shift
+  local arg
+  for arg ; do
+    case $0 in
+      (*max) (( $arg > result )) && result=$arg;;
+      (*min) (( $arg < result )) && result=$arg;;
+    esac
+  done
+  (( result )) # return
+}
+functions -M max 1 -1 zsh_math_func_max # at least one argument
+functions -M min 1 -1 zsh_math_func_min # at least one argument
+
+zsh_math_func_sum() {
+  local sum
+  local arg
+  for arg ; do
+    (( sum += $arg ))
+  done
+  (( sum ))
+}
+functions -M sum 0 -1 zsh_math_func_sum
+
diff --git a/Src/zsh.mdd b/Src/zsh.mdd
index 86dd588..324435d 100644
--- a/Src/zsh.mdd
+++ b/Src/zsh.mdd
@@ -2,7 +2,7 @@ name=zsh/main
 link=static
 load=yes
 # load=static should replace use of alwayslink
-functions='Functions/Chpwd/* Functions/Exceptions/* Functions/Misc/* Functions/MIME/* Functions/Prompts/* Functions/VCS_Info/* Functions/VCS_Info/Backends/*'
+functions='Functions/Chpwd/* Functions/Exceptions/* Functions/Math/* Functions/Misc/* Functions/MIME/* Functions/Prompts/* Functions/VCS_Info/* Functions/VCS_Info/Backends/*'
 
 nozshdep=1
 alwayslink=1