Filtering argument lists (e.g. for grep)

Filtering argument lists (e.g. for grep)

[Dominik Vogt]

For some commands, there are some file patterns that I never want
to pass to the command (unless explicitly stated otherwise).  For
example, grep'ing should normally ignore backup and ChangeLog files

 *ChangeLog*
 *~
 \#*

Maybe grep is a bad example because this can be done with the
--exclude= option.  But could zsh help filtering the names
generated by globbing in a more general way so that I could write

  $ <foo> *

and have zsh automagically filter the results of the * (not
everywhere; only for commands that have this feature enabled) so
that the non-matching names are not passed to the command in the
first place?

The only way I can think of is to write some function for each
command to be preprocessed, parse the arguments to figure out
which ones are file names and then use some utility function to
filter them.

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt
IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Dominik Vogt]

On Mon, Dec 07, 2015 at 11:56:22AM +0100, Dominik Vogt wrote:
> But could zsh help filtering the names
> generated by globbing in a more general way so that I could write
> 
>   $ <foo> *
> 
> and have zsh automagically filter the results of the * (not
> everywhere; only for commands that have this feature enabled) so
> that the non-matching names are not passed to the command in the
> first place?

Or in other words:  A context sensitive interpretation (or post
processing) of globbing patterns.  Is that possible?

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt
IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Peter Stephenson]

On Mon, 7 Dec 2015 11:56:22 +0100
Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
> For some commands, there are some file patterns that I never want
> to pass to the command (unless explicitly stated otherwise).  For
> example, grep'ing should normally ignore backup and ChangeLog files
> 
>  *ChangeLog*
>  *~
>  \#*
> 
> Maybe grep is a bad example because this can be done with the
> --exclude= option.  But could zsh help filtering the names
> generated by globbing in a more general way so that I could write
> 
>   $ <foo> *
> 
> and have zsh automagically filter the results of the * (not
> everywhere; only for commands that have this feature enabled) so
> that the non-matching names are not passed to the command in the
> first place?

You could use a global alias, e.g.

alias -g '@*'='*~(*\~|\#*|ChangeLog)'

Ig you want that first * to be something more flexible you can use a
glob qualifier.

  gi () {
    [[ $REPLY != (*\~|\#*|ChangeLog) ]]
  }

and use

  <foo> *(+gi)

pws

Re: Filtering argument lists (e.g. for grep)

[Dominik Vogt]

On Mon, Dec 07, 2015 at 11:23:54AM +0000, Peter Stephenson wrote:
> On Mon, 7 Dec 2015 11:56:22 +0100
> Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
> > Maybe grep is a bad example because this can be done with the
> > --exclude= option.  But could zsh help filtering the names
> > generated by globbing in a more general way so that I could write
> > 
> >   $ <foo> *
> > 
> > and have zsh automagically filter the results of the * (not
> > everywhere; only for commands that have this feature enabled) so
> > that the non-matching names are not passed to the command in the
> > first place?
 
> You could use a global alias, e.g.
> 
> alias -g '@*'='*~(*\~|\#*|ChangeLog)'

Yes, but then I'd need an alias for every potential pattern, e.g.
@*.s*, @**/*, @*.c.* etc.

> Ig you want that first * to be something more flexible you can use a
> glob qualifier.
> 
>   gi () {
>     [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>   }
> 
> and use
> 
>   <foo> *(+gi)

That sounds good, but is there a way to make that qualifier a
default for certain commands?  As an alternative, is it possible
to access the command name from inside the qualifier function?

  function gi () {
    if <command should be filtered>; then
      [[ $REPLY != (*\~|\#*|ChangeLog) ]]
    fi
  }

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt
IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Nikolay Aleksandrovich Pavlov (ZyX)]

07.12.2015, 14:51, "Dominik Vogt" <vogt@linux.vnet.ibm.com>:
> On Mon, Dec 07, 2015 at 11:23:54AM +0000, Peter Stephenson wrote:
>>  On Mon, 7 Dec 2015 11:56:22 +0100
>>  Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
>>  > Maybe grep is a bad example because this can be done with the
>>  > --exclude= option. But could zsh help filtering the names
>>  > generated by globbing in a more general way so that I could write
>>  >
>>  > $ <foo> *
>>  >
>>  > and have zsh automagically filter the results of the * (not
>>  > everywhere; only for commands that have this feature enabled) so
>>  > that the non-matching names are not passed to the command in the
>>  > first place?
>
>>  You could use a global alias, e.g.
>>
>>  alias -g '@*'='*~(*\~|\#*|ChangeLog)'
>
> Yes, but then I'd need an alias for every potential pattern, e.g.
> @*.s*, @**/*, @*.c.* etc.
>
>>  Ig you want that first * to be something more flexible you can use a
>>  glob qualifier.
>>
>>    gi () {
>>      [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>    }
>>
>>  and use
>>
>>    <foo> *(+gi)
>
> That sounds good, but is there a way to make that qualifier a
> default for certain commands? As an alternative, is it possible
> to access the command name from inside the qualifier function?
>
>   function gi () {
>     if <command should be filtered>; then
>       [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>     fi
>   }

I have pseudo-alias commands like zmw and zpy which do automagic escaping of their arguments (e.g. `zpy import zsh; print(zsh.getvalue("PATH"))` transforms into `zpython "import zsh; print(zsh.getvalue(\"PATH\"))"`) by hooking accept-line zle widget:

    _-accept-line () {
    	emulate -L zsh
    	local -r autopushd=${options[autopushd]} 
    	options[autopushd]=off 
    	cd $PWD || cd
    	options[autopushd]=$autopushd 
    	if [[ ${BUFFER[1,3]} = ":h " ]]
    	then
    		_HISTLINE=$BUFFER 
    		BUFFER=":h ${(q)BUFFER[4,-1]}" 
    	elif [[ ${BUFFER[1,4]} = "zmw " ]]
    	then
    		_HISTLINE=$BUFFER 
    		BUFFER="zmw "${(j. .)${(q)${(z)BUFFER[5,-1]}}} 
    	elif [[ ${BUFFER[1,4]} = "zpy " ]]
    	then
    		_HISTLINE=$BUFFER 
    		BUFFER="zpython ${(qqq)BUFFER[5,-1]}" 
    	fi
    	zle .accept-line
    }
    zle -N accept-line                       _-accept-line

. You may use the same technique to do anything you like with the line you typed, though this “anything” will sometimes be rather tricky to implement. `(z)` parameter expansion flag will be very useful on this path.

>
> Ciao
>
> Dominik ^_^ ^_^
>
> --
>
> Dominik Vogt
> IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Nikolay Aleksandrovich Pavlov (ZyX)]

07.12.2015, 14:51, "Dominik Vogt" <vogt@linux.vnet.ibm.com>:
> On Mon, Dec 07, 2015 at 11:23:54AM +0000, Peter Stephenson wrote:
>>  On Mon, 7 Dec 2015 11:56:22 +0100
>>  Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
>>  > Maybe grep is a bad example because this can be done with the
>>  > --exclude= option. But could zsh help filtering the names
>>  > generated by globbing in a more general way so that I could write
>>  >
>>  > $ <foo> *
>>  >
>>  > and have zsh automagically filter the results of the * (not
>>  > everywhere; only for commands that have this feature enabled) so
>>  > that the non-matching names are not passed to the command in the
>>  > first place?
>
>>  You could use a global alias, e.g.
>>
>>  alias -g '@*'='*~(*\~|\#*|ChangeLog)'
>
> Yes, but then I'd need an alias for every potential pattern, e.g.
> @*.s*, @**/*, @*.c.* etc.
>
>>  Ig you want that first * to be something more flexible you can use a
>>  glob qualifier.
>>
>>    gi () {
>>      [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>    }
>>
>>  and use
>>
>>    <foo> *(+gi)
>
> That sounds good, but is there a way to make that qualifier a
> default for certain commands? As an alternative, is it possible
> to access the command name from inside the qualifier function?
>
>   function gi () {
>     if <command should be filtered>; then
>       [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>     fi
>   }

And there is another possibility: considering you want to do this thing with command `foo` you need to do the following:

1. Create an alias `foo='noglob foo'`.
2. Create a function `foo` like this:

        function foo()
        {
            local -a args=( "${@[@]}" )
            local -a new_args
            for (( I=2; I<= $#args; I++ )) ; do
                if [[ $args[I] != ${${args[I]}//[*?]} ]] ; then # If argument contains glob pattern
                    args[I]+="(+gi)"
                    new_args=( $~args[I] )
                    args[I,I]=( $new_args )
                    (( I += #new_args - 1 ))
                fi
            done
            command foo "${args[@]}"
        }

    . I.e. in place of leaving zsh to expand globs, expand it in your function “manually”, with necessary additions.

>
> Ciao
>
> Dominik ^_^ ^_^
>
> --
>
> Dominik Vogt
> IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Nikolay Aleksandrovich Pavlov (ZyX)]

07.12.2015, 15:03, "Nikolay Aleksandrovich Pavlov (ZyX)" <kp-pav@yandex.ru>:
> 07.12.2015, 14:51, "Dominik Vogt" <vogt@linux.vnet.ibm.com>:
>>  On Mon, Dec 07, 2015 at 11:23:54AM +0000, Peter Stephenson wrote:
>>>   On Mon, 7 Dec 2015 11:56:22 +0100
>>>   Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
>>>   > Maybe grep is a bad example because this can be done with the
>>>   > --exclude= option. But could zsh help filtering the names
>>>   > generated by globbing in a more general way so that I could write
>>>   >
>>>   > $ <foo> *
>>>   >
>>>   > and have zsh automagically filter the results of the * (not
>>>   > everywhere; only for commands that have this feature enabled) so
>>>   > that the non-matching names are not passed to the command in the
>>>   > first place?
>>
>>>   You could use a global alias, e.g.
>>>
>>>   alias -g '@*'='*~(*\~|\#*|ChangeLog)'
>>
>>  Yes, but then I'd need an alias for every potential pattern, e.g.
>>  @*.s*, @**/*, @*.c.* etc.
>>
>>>   Ig you want that first * to be something more flexible you can use a
>>>   glob qualifier.
>>>
>>>     gi () {
>>>       [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>>     }
>>>
>>>   and use
>>>
>>>     <foo> *(+gi)
>>
>>  That sounds good, but is there a way to make that qualifier a
>>  default for certain commands? As an alternative, is it possible
>>  to access the command name from inside the qualifier function?
>>
>>    function gi () {
>>      if <command should be filtered>; then
>>        [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>      fi
>>    }
>
> And there is another possibility: considering you want to do this thing with command `foo` you need to do the following:
>
> 1. Create an alias `foo='noglob foo'`.
> 2. Create a function `foo` like this:
>
>         function foo()
>         {
>             local -a args=( "${@[@]}" )
>             local -a new_args
>             for (( I=2; I<= $#args; I++ )) ; do
>                 if [[ $args[I] != ${${args[I]}//[*?]} ]] ; then # If argument contains glob pattern
>                     args[I]+="(+gi)"
>                     new_args=( $~args[I] )
>                     args[I,I]=( $new_args )
>                     (( I += #new_args - 1 ))
>                 fi
>             done
>             command foo "${args[@]}"
>         }
>
>     . I.e. in place of leaving zsh to expand globs, expand it in your function “manually”, with necessary additions.

Though this variant is for one command. For multiple you need some adjustments:

1. `alias foo='noglob filterglob foo'`
2. Function is `filterglob`, starts with `local -r cmd="$1"; shift`, ends with `command "$cmd" "${args[@]}"`.

And I should not have used `I=2` (it initially meant to skip command) in any case, replace `I=2` with `I=1`.

>
>>  Ciao
>>
>>  Dominik ^_^ ^_^
>>
>>  --
>>
>>  Dominik Vogt
>>  IBM Germany

Re: Filtering argument lists (e.g. for grep)

[Nikolay Aleksandrovich Pavlov (ZyX)]

07.12.2015, 15:18, "Nikolay Aleksandrovich Pavlov (ZyX)" <kp-pav@yandex.ru>:
>  07.12.2015, 15:03, "Nikolay Aleksandrovich Pavlov (ZyX)" <kp-pav@yandex.ru>:
>>   07.12.2015, 14:51, "Dominik Vogt" <vogt@linux.vnet.ibm.com>:
>>>    On Mon, Dec 07, 2015 at 11:23:54AM +0000, Peter Stephenson wrote:
>>>>     On Mon, 7 Dec 2015 11:56:22 +0100
>>>>     Dominik Vogt <vogt@linux.vnet.ibm.com> wrote:
>>>>     > Maybe grep is a bad example because this can be done with the
>>>>     > --exclude= option. But could zsh help filtering the names
>>>>     > generated by globbing in a more general way so that I could write
>>>>     >
>>>>     > $ <foo> *
>>>>     >
>>>>     > and have zsh automagically filter the results of the * (not
>>>>     > everywhere; only for commands that have this feature enabled) so
>>>>     > that the non-matching names are not passed to the command in the
>>>>     > first place?
>>>
>>>>     You could use a global alias, e.g.
>>>>
>>>>     alias -g '@*'='*~(*\~|\#*|ChangeLog)'
>>>
>>>    Yes, but then I'd need an alias for every potential pattern, e.g.
>>>    @*.s*, @**/*, @*.c.* etc.
>>>
>>>>     Ig you want that first * to be something more flexible you can use a
>>>>     glob qualifier.
>>>>
>>>>       gi () {
>>>>         [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>>>       }
>>>>
>>>>     and use
>>>>
>>>>       <foo> *(+gi)
>>>
>>>    That sounds good, but is there a way to make that qualifier a
>>>    default for certain commands? As an alternative, is it possible
>>>    to access the command name from inside the qualifier function?
>>>
>>>      function gi () {
>>>        if <command should be filtered>; then
>>>          [[ $REPLY != (*\~|\#*|ChangeLog) ]]
>>>        fi
>>>      }
>>
>>   And there is another possibility: considering you want to do this thing with command `foo` you need to do the following:
>>
>>   1. Create an alias `foo='noglob foo'`.
>>   2. Create a function `foo` like this:
>>
>>           function foo()
>>           {
>>               local -a args=( "${@[@]}" )
>>               local -a new_args
>>               for (( I=2; I<= $#args; I++ )) ; do
>>                   if [[ $args[I] != ${${args[I]}//[*?]} ]] ; then # If argument contains glob pattern
>>                       args[I]+="(+gi)"
>>                       new_args=( $~args[I] )
>>                       args[I,I]=( $new_args )
>>                       (( I += #new_args - 1 ))
>>                   fi
>>               done
>>               command foo "${args[@]}"
>>           }
>>
>>       . I.e. in place of leaving zsh to expand globs, expand it in your function “manually”, with necessary additions.
>
>  Though this variant is for one command. For multiple you need some adjustments:
>
>  1. `alias foo='noglob filterglob foo'`
>  2. Function is `filterglob`, starts with `local -r cmd="$1"; shift`, ends with `command "$cmd" "${args[@]}"`.
>
>  And I should not have used `I=2` (it initially meant to skip command) in any case, replace `I=2` with `I=1`.

I have checked and the following filterglob function works:

    filterglob () {
    	local -r cmd="$1" 
    	shift
    	local -a args=("${@[@]}") 
    	local -a new_args
    	for ((I=1; I<=$#args; I++ )) do
    		if [[ $args[I] != ${${args[I]}/[*?]} ]]
    		then
    			args[I]+="~*.png" 
    			new_args=($~args[I]) 
    			args[I,I]=("${new_args[@]}") 
    			(( I += #new_args - 1 ))
    		fi
    	done
    	"$cmd" "${args[@]}"
    }

. But there is one downside: you need additional code to handle CSH_NULL_GLOB, NULL_GLOB and NOMATCH options. There are the following combinations:

1. nonomatch, nocshnullglob, nonullglob: leave pattern as-is. Requires 1. adding (N) at the end of the pattern 2. reverting args[I] change. (Note: *.c literally is also possible file name, so you can’t just “check if #new_args == 1 and new_args[1] == args[I] and if yes revert”, (N) is needed.)
2. cshnullglob, nonullglob: if there are no matches, leave no arguments, but if all patterns on the command-line have no matches, error out. Requires 1. adding (N) (or it will error out when processing new_args) 2. saving 1 if there were expanded globs.
3. nullglob: should work as-is, including when variant with (N) is used.
4. nomatch: should also work as-is, but when (N) is used requires additionally explicitly errorring out.

The final function works something like this:

    filterglob () {
        local -r cmd="$1"
        shift
        local -a args
        args=( "${@[@]}" )
        local -a new_args
        local -i expandedglobs=0
        local first_unexpanded_glob=
        for ((I=1; I<=$#args; I++ )) do
            if [[ $args[I] != ${${args[I]}/[*?]} ]]
            then
                local initial_arg=${args[I]}
                args[I]+="~*.png(N)"
                new_args=( $~args[I] )
                if (( $#new_args )) ; then
                    expandedglobs=1
                else
                    if [[ $options[cshnullglob] == off
                          && $options[nullglob] == off ]] ; then
                        if [[ $options[nomatch] == on ]] ; then
                            : ${~${args[I]%\(N\)}}  # Will error out.
                        else
                            new_args=( "$initial_arg" )
                        fi
                    fi
                    if [[ -z $first_unexpanded_glob ]] ; then
                        first_unexpanded_glob=${args[I]%\(N\)}
                        readonly first_unexpanded_glob
                    fi
                fi
                args[I,I]=( "${new_args[@]}" )
                (( I += $#new_args - 1 ))
            fi
        done
        if [[ $options[cshnullglob] == on && $options[nullglob] == off ]] ; then
            if (( !expandedglob )) ; then
                : $~first_unexpanded_glob  # Will error out.
            fi
        fi
        "$cmd" "${args[@]}"
    }

. Note that this function has no `emulate -L zsh` at the top, so may be broken by some options. Also AFAIK you cannot use this specific function without `setopt extendedglob` and `zmodload zsh/parameter` (for `$options`).

// Do not forget to replace `~*.png` with whatever you need, I used this for testing purposes.


>>>    Ciao
>>>
>>>    Dominik ^_^ ^_^
>>>
>>>    --
>>>
>>>    Dominik Vogt
>>>    IBM Germany