Some problems with recursive globbing

Some problems with recursive globbing

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

I am trying to write a function where I collect files within a file tree to
do some processing.

My goal is to find all files which are not in any directory called
'target', at any level.

Here's a simplified version of my function:

myfiles() {
    setopt LOCAL_OPTIONS EXTENDED_GLOB

    local dir

    [[ -n $1 ]] && dir=${1:a}/

    local filepattern="${dir}**/*~${dir}(**/|)target(/**/*|/*|)"

    print -c ${~filepattern}
}

So this prints out the relative paths of all files in the current
directory. If a directory is given as an argument, the absolute paths are
printed of files within the given directory. (I know there's no check that
the argument is actually a directory).

This works, sort of, but I have a question and a problem.

Question: I found it surprisingly difficult to to find a glob pattern that
excluded target directories and their contents at all levels. Have I
complicated this too much, is there an easier way to express this glob?

Problem: I can't get this to work for both cases of a) directories with
spaces in their names and b) directories with parenthesis in their names

Suppose I'm standing in a directory called "/tmp/test(1)", and this
directory contains a directory called "src".

% myfiles src

I get this:

myfiles:9: no matches found:
/tmp/test(1)/src/**/*~/tmp/test(1)/src/(**/|)target(/**/*|/*|)

If I change the fifth line in myfiles() to this:

  [[ -n $1 ]] && dir=${(q)1:a}/

it works for the root directory with parentheses. However, standing in a
directory called "/tmp/test 1", I now get this:

myfiles:9: no matches found: /tmp/test\ 1/src/**/*~/tmp/test\
1/src/(**/|)target(/**/*|/*|)

So with quoting, it works with parentheses. Without quoting, it works with
spaces. It feels as though I've tried everything, but I can't find a way to
quote this so that it works for both parentheses and spaces.

Re: Some problems with recursive globbing

[Peter Stephenson]

On Thu, 7 May 2015 16:35:57 +0200
Jesper Nygårds <jesper.nygards@gmail.com> wrote:
> Question: I found it surprisingly difficult to to find a glob pattern that
> excluded target directories and their contents at all levels. Have I
> complicated this too much, is there an easier way to express this glob?

Yes, you can use '(^target/)#'.

Note also that after "~" is just a pure pattern --- / isn't special.
So if you had to exclude explicitly as /target/ or target/ or /target
you'd use ~(|*/)target(|*/)

> Problem: I can't get this to work for both cases of a) directories with
> spaces in their names and b) directories with parenthesis in their names

Native zsh options are *supposed* to be set up to be insensitive to this
sort of nonsense...

Try putting

emulate -L zsh

as the first line of your function.  I may have missed something more
subtle.

pws

Re: Some problems with recursive globbing

[Peter Stephenson]

On Thu, 7 May 2015 16:52:50 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> Note also that after "~" is just a pure pattern --- / isn't special.
> So if you had to exclude explicitly as /target/ or target/ or /target
> you'd use ~(|*/)target(|*/)

...~(|*/)target(|/*)

pws

Re: Some problems with recursive globbing

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

Hmm, I still can't get this to work.

I tried this:
myfiles() {
    emulate -L zsh
    setopt LOCAL_OPTIONS EXTENDED_GLOB

    local dir="${1:a}/"

    local filepattern="${dir}**/*"

    print -c ${~filepattern}
}

% cd /private/tmp/test\(1\)
% myfiles src
myfiles:8: no matches found: /private/tmp/test(1)/src/**/*


And then this:
myfiles() {
    emulate -L zsh
    setopt LOCAL_OPTIONS EXTENDED_GLOB

    local dir="${(q)1:a}/"

    local filepattern="${dir}**/*"

    print -c ${~filepattern}
}

% cd /private/tmp/test\ 1
% myfiles src
myfiles:8: no matches found: /private/tmp/test\ 1/src/**/*

So it still fails on either parentheses or spaces, depending on if I quote
the parameter.

Re: Some problems with recursive globbing

[Peter Stephenson]

On Thu, 7 May 2015 18:46:58 +0200
Jesper Nygårds <jesper.nygards@gmail.com> wrote:
> I tried this:
> myfiles() {
>     emulate -L zsh
>     setopt LOCAL_OPTIONS EXTENDED_GLOB
> 
>     local dir="${1:a}/"
> 
>     local filepattern="${dir}**/*"
> 
>     print -c ${~filepattern}
> }
> 
> % cd /private/tmp/test\(1\)
> % myfiles src
> myfiles:8: no matches found: /private/tmp/test(1)/src/**/*

$dir contains a straight string with unquoted parentheses.  The
~filepattern then turns those parentheses into pattern characters.

I'm not sure why you want filepattern anyway, but

print -c ${dir}**/*

or

local filepattern="**/*"
print -c ${dir}${~filepattern}

ought to work.  Otherwise you'll need to quote metacharacters in dir,
which is possible but should be unnecessary.

pws

Re: Some problems with recursive globbing

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On Thu, May 7, 2015 at 7:00 PM, Peter Stephenson <p.stephenson@samsung.com>
wrote:

> $dir contains a straight string with unquoted parentheses.  The
> ~filepattern then turns those parentheses into pattern characters.
>
> Yes, I understand.


> I'm not sure why you want filepattern anyway, but
>
> I tried to simplify my function to make my problem obvious. In my "real"
function, I am collecting several function arguments into a combined
pattern, which is why I need to use this indirect method.

local filepattern="**/*"

> print -c ${dir}${~filepattern}
>
> ought to work.  Otherwise you'll need to quote metacharacters in dir,
> which is possible but should be unnecessary.
>

Yes, but as my second example demonstrates: if I quote my filepattern, it
then doesn't work for files with spaces in their names. I was hoping for a
solution where it would be possible to get this to work for both situations.

Re: Some problems with recursive globbing

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]

I found a solution to my particular problem. I can quote the dir, and then
remove the quoting of the spaces afterwards.

myfiles() {
    emulate -L zsh
    setopt LOCAL_OPTIONS EXTENDED_GLOB

    local dir="${(q)1:a}/"

    dir=${dir/\\\ / }

    local filepattern="${dir}**/*"

    print -c ${~filepattern}
}

I realize this could brake on even more exotic file names, but for now I'm
happy with this solution.


On Thu, May 7, 2015 at 7:21 PM, Jesper Nygårds <jesper.nygards@gmail.com>
wrote:

> On Thu, May 7, 2015 at 7:00 PM, Peter Stephenson <p.stephenson@samsung.com
> > wrote:
>
>> $dir contains a straight string with unquoted parentheses.  The
>> ~filepattern then turns those parentheses into pattern characters.
>>
>> Yes, I understand.
>
>
>> I'm not sure why you want filepattern anyway, but
>>
>> I tried to simplify my function to make my problem obvious. In my "real"
> function, I am collecting several function arguments into a combined
> pattern, which is why I need to use this indirect method.
>
> local filepattern="**/*"
>
>> print -c ${dir}${~filepattern}
>>
>> ought to work.  Otherwise you'll need to quote metacharacters in dir,
>> which is possible but should be unnecessary.
>>
>
> Yes, but as my second example demonstrates: if I quote my filepattern, it
> then doesn't work for files with spaces in their names. I was hoping for a
> solution where it would be possible to get this to work for both situations.
>
>
>

Re: Some problems with recursive globbing

[Peter Stephenson]

On Fri, 8 May 2015 05:37:59 +0200
Jesper Nygårds <jesper.nygards@gmail.com> wrote:
> I found a solution to my particular problem. I can quote the dir, and then
> remove the quoting of the spaces afterwards.

Yes, or the other approach is quote only the metacharacters (which don't
include space) in the first place.  I think there may be some examples
lying around using ${...//.../...}.  Oh, yes, here's one from _rm:

${line//(#m)[\[\]()\\*?#<>~\^\|]/\\$MATCH}

It would be overall simpler not to pass the pattern and a fixed string
around in the same variable, involving a bit of structural change.

pws

Re: Some problems with recursive globbing

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

Thank you for all the help, Peter.


On Fri, May 8, 2015 at 10:40 AM, Peter Stephenson <p.stephenson@samsung.com>
wrote:

> On Fri, 8 May 2015 05:37:59 +0200
> Jesper Nygårds <jesper.nygards@gmail.com> wrote:
> > I found a solution to my particular problem. I can quote the dir, and
> then
> > remove the quoting of the spaces afterwards.
>
> Yes, or the other approach is quote only the metacharacters (which don't
> include space) in the first place.  I think there may be some examples
> lying around using ${...//.../...}.  Oh, yes, here's one from _rm:
>
> ${line//(#m)[\[\]()\\*?#<>~\^\|]/\\$MATCH}
>
> It would be overall simpler not to pass the pattern and a fixed string
> around in the same variable, involving a bit of structural change.
>
> pws
>