zmv exits from function

zmv exits from function

[Ray Andrews]

I'm using zmv to rename files in a directory tree recursively via a 
'for' loop which visits each subdirectory.  It works fine if there are 
files found to rename, but if not, then the entire function crashes back 
to CL.  How can I persuade zmv to just let the function cycle to the 
next subdir?  I see no option that seems relevant.

Cut down to the essentials:

function global ()
{
     f ,dB >! /tmp/global_tmp    # My function.  Gives list of subdirs, 
works fine.
     curdir=$PWD
     for aa in $(cat /tmp/global_tmp); do     # For every line:

         cd $aa

         zmv '(*).SNT' '$1.eml'
         zmv '(*).MES' '$1.eml'

         cd $curdir
     done
}

Lots of other test commands in place of zmv cycle fine in case of no 
match, but zmv insists on returning.

Re: zmv exits from function

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

On Sat, Dec 30, 2023 at 9:43 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I'm using zmv to rename files in a directory tree recursively via a
> 'for' loop which visits each subdirectory.  It works fine if there are
> files found to rename, but if not, then the entire function crashes back
> to CL.  How can I persuade zmv to just let the function cycle to the
> next subdir?

zmv forces zsh emulation which means the nomatch option is in effect,
and you can't change that without editing the zmv source.

Just run the zmv in a subshell:

  ( zmv '(*).SNT' '$1.eml' )
  ( zmv '(*).MES' '$1.eml' )

Also make sure that you don't have the errr_exit or err_return setopts
in effect or the loop will break even with the subshells.

Alternately you could test whether the pattern matches any files
before calling zmv in the first place.

Arguably zmv could use null_glob.  Thoughts from -workers?

[-- Attachment #2: zmv-nullglob.txt --]
[-- Type: text/plain, Size: 513 bytes --]

diff --git a/Functions/Misc/zmv b/Functions/Misc/zmv
index 269fe5ba5..51c8ad3ab 100644
--- a/Functions/Misc/zmv
+++ b/Functions/Misc/zmv
@@ -236,12 +236,14 @@ if [[ $pat = (#b)(*)\((\*\*##/)\)(*) ]]; then
 else
   fpat=$pat
 fi
-files=(${~fpat})
+files=(${~fpat}(#qN))
 
 [[ -n $hasglobqual ]] && pat=$opat
 
 errs=()
 
+[[ -n $files ]] || errs=( "no files matched \`$fpat'" )
+
 for f in $files; do
   if [[ $pat = (#b)(*)\(\*\*##/\)(*) ]]; then
     # This looks like a recursive glob.  This isn't good enough,

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 601 bytes --]


On 2023-12-30 12:38, Bart Schaefer wrote:
> Just run the zmv in a subshell:
>
>    ( zmv '(*).SNT' '$1.eml' )
>    ( zmv '(*).MES' '$1.eml' )
>
Perfect :-)
> Arguably zmv could use null_glob.  Thoughts from -workers?

That's that option that makes everything grind to a halt if nothing if 
found?  Think I recall that " (N) " setting to cope with it?  From a 
users point of view it just makes trouble where no trouble really 
exists.  BTW, I did look at the source and noticed no return values, I 
had thought that something like ' zmv ... && echo "Success" || echo 
"Nope"' ... might catch it.


[-- Attachment #2: Type: text/html, Size: 1272 bytes --]

Re: zmv exits from function

[Bart Schaefer]

On Sat, Dec 30, 2023 at 1:02 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2023-12-30 12:38, Bart Schaefer wrote:
> >
> > Arguably zmv could use null_glob.  Thoughts from -workers?
>
> That's that option that makes everything grind to a halt if nothing if found?

No, you're thinking of nomatch, which as I said is already in effect
and causing the result that you see.  The null_glob option supersedes
no_match and makes globs that don't match anything disappear without
an error.  This is in contrast to the no_nomatch option which leaves
the glob un-expanded but does not remove it.  Setting null_glob
globally can be dangerous because it can leave a command with no
arguments at all or with its arguments in a different order than
expected.

> Think I recall that " (N) " setting to cope with it?

The (N) glob qualifier turns ON null_glob to override no_match, yes.
That's the suggested amendment to zmv for which I'm asking -workers
opinion.

Re: zmv exits from function

[Mikael Magnusson]

On 12/30/23, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sat, Dec 30, 2023 at 9:43 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>> I'm using zmv to rename files in a directory tree recursively via a
>> 'for' loop which visits each subdirectory.  It works fine if there are
>> files found to rename, but if not, then the entire function crashes back
>> to CL.  How can I persuade zmv to just let the function cycle to the
>> next subdir?
>
> zmv forces zsh emulation which means the nomatch option is in effect,
> and you can't change that without editing the zmv source.
>
> Just run the zmv in a subshell:
>
>   ( zmv '(*).SNT' '$1.eml' )
>   ( zmv '(*).MES' '$1.eml' )

You can also do this:
  { zmv '(*).SNT' '$1.eml' } always { TRY_BLOCK_ERROR=0 }

I have this aliased as
  alias always_continue='always { TRY_BLOCK_ERROR=0 }'
which lets you do the slightly neater
  { zmv '(*).SNT' '$1.eml' } always_continue

-- 
Mikael Magnusson

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2083 bytes --]


On 2023-12-30 13:15, Mikael Magnusson wrote:
> You can also do this:
>    { zmv '(*).SNT' '$1.eml' } always { TRY_BLOCK_ERROR=0 }

Sheesh, of course I'm no expert, still I've never seen any such 
construction before.  Always new domains of syntax to study.  So when 
zmv quits, crashing out of the function is not immediate -- there is 
further parsing, thus this 'always' word is handled. Interesting.

On 2023-12-30 12:38, Bart Schaefer wrote:

> Arguably zmv could use null_glob.  Thoughts from -workers?

That's that option that makes everything grind to a halt if nothing if found?

> No, you're thinking of nomatch, which as I said is already in effect
and causing the result that you see.  The null_glob option supersedes
no_match and makes globs that don't match anything disappear without
an error.

Yeah, for some unknown reason I said it backwards -- I know that (N) smooths continuation of flow, it doesn't break it.

> This is in contrast to the no_nomatch option which leaves
the glob un-expanded but does not remove it.

I've run into that, and it can be ugly -- literal asterisks being fed to some command that doesn't want them.  It's good to understand what's going down there.

   Setting null_glob
globally can be dangerous because it can leave a command with no
arguments at all or with its arguments in a different order than
expected.

Yes, another potential landmine.  What might be intuitive is some way of simply making a command 'stop', but that might not be so simple -- something that holds it's place, but has no value -- it 'counts' as an argument but has no content.  It's so dumb when: "ls no-such-files.*" ends up listing everything in the directory rather than nothing.  Dunno who thought that was a good idea.


>  zmv -n '(*/)#(*).(SNT|MES)' '$1$2.eml'

If that looks like it would do the right thing, remove the "-n".

I'll give it a go.  So much can be done with all that globbing functionality.  Such power.  But I can never remember that tersest of all terse syntaxes -- gotta make myself a cheat sheet including a page full of examples.


[-- Attachment #2: Type: text/html, Size: 2860 bytes --]

Re: zmv exits from function

[Bart Schaefer]

On Sat, Dec 30, 2023 at 7:43 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>   { zmv '(*).SNT' '$1.eml' } always { TRY_BLOCK_ERROR=0 }
>
> So when zmv quits, crashing out of the function is not immediate -- there is further parsing

No, the parsing is all done first: "{ ... } always { ... }" is a full
expression.  Zsh "knows" the always-part is coming before it even
starts executing the first block.

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]


On 2023-12-30 19:58, Bart Schaefer wrote:
> No, the parsing is all done first: "{ ... } always { ... }" is a full
> expression.
Zsh "knows" the always-part is coming before it even
> starts executing the first block.
>
So that's multi-pass parsing.  Anyway, first I've seen of anything like 
it.  I'm reading up on it right now.  'always' is a reserved word then.  
Bottom line is that I now have a tool for handling 'no files' 
situations.  Good to know because there could be any number of 
housekeeping functions that might want to clean up or rename some group 
of files if they exist but it's not a problem if they don't.



[-- Attachment #2: Type: text/html, Size: 1368 bytes --]

Re: zmv exits from function

[Bart Schaefer]

On Sun, Dec 31, 2023 at 7:53 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> So that's multi-pass parsing.

Again, no.  It's only parsed once, into an internal format
(essentially, the same format used for "zcompile" files) and then
executed if syntactically correct.  The only way to get another parse
is to use "eval".

Expansion and word splitting on can occur during execution, but that
doesn't change the syntax tree.  Parsing does happen inside command
substitution but is limited to that syntactic scope.

Re: zmv exits from function

[Ray Andrews]

On 2023-12-31 13:44, Bart Schaefer wrote:
> On Sun, Dec 31, 2023 at 7:53 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> So that's multi-pass parsing.
> Again, no.  It's only parsed once

I don't have the background to understand it and it's probably not worth 
your time to make it clear.  It 'look's like' a multiple parsing -- 
before zmv executes, it knows that errors will be handled differently 
than default due to following code, and then it 'goes back' and actually 
executes.  Or ... zmv returns innocently and then the next code rescues 
the function from aborting?  Nevermind, I don't need to know.  It's just 
sorta fuzzy trying to understand at exactly what point the function 
returns if zmv isn't happy.  I'm thinking in C naturally.

Re: zmv exits from function

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

On Sun, Dec 31, 2023 at 5:07 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

>
> On 2023-12-31 13:44, Bart Schaefer wrote:
> > On Sun, Dec 31, 2023 at 7:53 AM Ray Andrews <rayandrews@eastlink.ca>
> wrote:
> >> So that's multi-pass parsing.
> > Again, no.  It's only parsed once
>
> I don't have the background to understand it and it's probably not worth
> your time to make it clear.
>

It's not parsed multiple times, but it _is parsed fully before anything
runs.  _Parsing_ is just reading the code and determining what it means;
actually _running_ it comes later. Zsh parses the whole expression before
it executes any part of it – it doesn't just stop and execute the first
thing that looks like a command. So it has already read the `always` block
and knows it's there before it executes the code to the left of it.

And the presence of the `always` doesn't affect parsing; the code on both
sides works the same way it would without the `always`. The only thing it
does is tell Zsh that, as long as the code on the left doesn't exit the
shell entirely, it doesn't matter what happens there; the code on the right
needs to get executed next. Even if the left code interrupts the normal
control flow with a `return` or `break` or `continue`, the code on the
right will get executed before the next thing, wherever that next thing is.

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 2012 bytes --]

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]


On 2024-01-02 06:51, Mark J. Reed wrote:
>
> It's not parsed multiple times, but it _is parsed fully before 
> anything runs.  _Parsing_ is just reading the code and determining 
> what it means; actually _running_ it comes later. Zsh parses the whole 
> expression before it executes any part of it – it doesn't just stop 
> and execute the first thing that looks like a command. So it has 
> already read the `always` block and knows it's there before it 
> executes the code to the left of it.
That's what I meant.  Right tho -- it's handled in two passes but one is 
called  parsing, the second is called running.   But the parse sets a 
memo that zmv will not be permitted to crash the function at run time.
>
> And the presence of the `always` doesn't affect parsing; the code on 
> both sides works the same way it would without the `always`. The only 
> thing it does is tell Zsh that, as long as the code on the left 
> doesn't exit the shell entirely, it doesn't matter what happens there; 
> the code on the right needs to get executed next. Even if the left 
> code interrupts the normal control flow with a `return` or `break` or 
> `continue`, the code on the right will get executed before the next 
> thing, wherever that next thing is.

This appears to be established practice, but I must say, seeing it for 
the first time, it sure looks bizarre.  It looks like some hideous hack 
to handle a code flow discontinuity that might have had a more 
straightforward handling:

     zmv ...   ...

     if [ $? -eq 0 ]; then echo "Files found, files moved boss."; fi

     if [ $? -eq 1 ]; then echo "Ooops, no files found, but no problemo, 
keep calm and carry on.";  fi

     or:

     if [ $? -eq 1 ]; then echo "Ooops, no files found so this function 
won't have any more work to do so just return."; return; fi

     or:

     if [ $? -eq 1 ]; then echo "WARNING!, no files found!!  This is a 
major problem so crash the entire shell."; exit;  fi

... You guys know best however.  But everything I've learned prior to 
this issue would have me expect that if zmv can't do anything it simply 
returns and code flow continues.  I wonder why it can't be just that 
simple.  It could very well be that other zsh functions  have this 
'crash the calling function' property but it's the first time I've seen 
it.  (and zmv is soooo wonderful!)





[-- Attachment #2: Type: text/html, Size: 3659 bytes --]

Re: zmv exits from function

[Peter Stephenson]

> On 30/12/2023 20:38 GMT Bart Schaefer <schaefer@brasslantern.com> wrote:
> Arguably zmv could use null_glob.  Thoughts from -workers?

I guess the right thing to do regardless of option is that it should
fail gracefully just by returning status 1 --- so not propagating the
error to the caller.  If the effect of NULL_GLOB is we loop over nothing,
that presumably gives us status 0?  I'd say we should probably detect that
as a (return status) failure for minimum surprises.

pws

Re: zmv exits from function

[Ray Andrews]

On 2024-01-02 03:50, Peter Stephenson wrote:
>> On 30/12/2023 20:38 GMT Bart Schaefer <schaefer@brasslantern.com> wrote:
>> Arguably zmv could use null_glob.  Thoughts from -workers?
> I guess the right thing to do regardless of option is that it should
> fail gracefully just by returning status 1 --- so not propagating the
> error to the caller.

That's just was I was trying to say :-)

There's a lot to be said for graceful failure.  If one wishes to crash 
the calling function one should  of course have that option, but why 
make it the default?  There's a sorta 'invisible return' there and the 
'always' thing counteracts it, but it all seems so unnecessary.  No 
invisible code, please.

BTW, just to whine, 'always' is sure hard to find in the help. One 
obviously can't just search for that word, and it doesn't show up in the 
list of reserved words neither.

Re: zmv exits from function

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 2690 bytes --]

On Tue, Jan 2, 2024 at 12:09 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

> BTW, just to whine, 'always' is sure hard to find in the help. One
> obviously can't just search for that word, and it doesn't show up in the
> list of reserved words neither.
>

Shell Grammar, under Complex Commands. (A bit of jargon, really; a command
line to execute a program once is a "simple" command; anything that
combines simple commands together – a loop or conditional or whatever –is a
"complex" command):

https://zsh.sourceforge.io/Doc/Release/Shell-Grammar.html#Complex-Commands

> That's what I meant.  Right tho -- it's handled in two passes but one is
called  parsing, the second is called running.   But the parse sets a memo
that zmv will not be permitted to crash the function at run time.

Well, no.  It runs the code on the left, and then when it finishes it runs
the code on the right. That's the extent of the `always` functionality.
What makes it special is that the code on the right runs even if the code
on the left blows up somehow.

It's similar to a feature called the "finally block" in other languages,
where it's usually attached to the exception-handling mechanism; something
like `try { start here } catch (ErrorType) { do this if the previous block
failed this particular way } finally { run this no matter what }`.  Because
of that similarity, the code on the left of the `always` is called the "try
block" or "try list", even though zsh doesn't use the keyword "try". The
code on the right is called the "always block" or "always list".

So, the point of `always` is to run the always block no matter what, even
if the try block blows up. But now we have an information problem: the main
point of the exercise was to run the code in the try block, and presumably
the caller would like to know whether it blew up or not. But how does that
information get back to them? You can't just go by exit code
(`$?`/`$status`), because by the time control gets back to the caller, that
will reflect the result of the always block, not the try block.

Zsh uses a variable called TRY_BLOCK_ERROR (which should be read as "an
error that occurred in the 'try' block" rather than some version of "try to
block the error" :)) to remember whether the try block errored out or not.
If it did, that variable will be 1, and after the always block executes,
zsh will go back to handling the error (i.e. potentially blowing up the
world). If you want the world not to be blown up, you can set
TRY_BLOCK_ERROR to 0, and then zsh will forget that anything went wrong and
continue about its business.

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 3543 bytes --]

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1874 bytes --]


On 2024-01-02 09:47, Mark J. Reed wrote:
>
>
> https://zsh.sourceforge.io/Doc/Release/Shell-Grammar.html#Complex-Commands
>
Thanks.  Man, I wish there were more examples :(


> Well, no.  It runs the code on the left, and then when it finishes it 
> runs the code on the right. That's the extent of the `always` 
> functionality. What makes it special is that the code on the right 
> runs even if the code on the left blows up somehow.
Yeah, I think I get it.  But the 'memo' has to be there -- zmv must ... 
or maybe not ... when zmv returns, if the 'always' keyword (sorta) it 
there, then it knows to not crash but to execute the following code.  
So, yeah, it could be seen as 'linear'.
>
> It's similar to a feature called the "finally block" in other 
> languages, where it's usually attached to the exception-handling 
> mechanism; something like `try { start here } catch (ErrorType) { do 
> this if the previous block failed this particular way } finally { run 
> this no matter what }`.  Because of that similarity, the code on the 
> left of the `always` is called the "try block" or "try list", even 
> though zsh doesn't use the keyword "try". The code on the right is 
> called the "always block" or "always list".
That's interesting.  So this isn't some desperate hack -- that kind of 
functionality is standard to some degree.  Ok, good to know. But as I 
said, it sure looks strange the first time you see it.  Reading a bit 
more, I see it's got other uses than preventing meltdowns. Cool.
>
> If you want the world not to be blown up, you can set TRY_BLOCK_ERROR 
> to 0, and then zsh will forget that anything went wrong and continue 
> about its business.

I myself hardly ever want Ragnarok, so next time I run into this sort of 
issue, I've got that to defuse it.  Thanks Mark.  Hey, I wonder if that 
could be an option? NO_MELTDOWNS=on


[-- Attachment #2: Type: text/html, Size: 3607 bytes --]

Re: zmv exits from function

[Bart Schaefer]

On Tue, Jan 2, 2024 at 10:16 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Yeah, I think I get it.  But the 'memo' has to be there -- zmv must ... or maybe not ... when zmv returns, if the 'always' keyword (sorta) it there, then it knows to not crash but to execute the following code.  So, yeah, it could be seen as 'linear'.

To avoid giving other readers a wrong conclusion ...

There are a number of failure cases defined by e.g. the POSIX
specification, or in some cases by longstanding zsh practice, that are
considered fatal errors.  When one of those errors is encountered,
whatever it is that the shell is doing, simply stops.  In a script,
this exits the entire script; in an interactive shell, it returns to
the top-level prompt.  In this specific example, there is no "zmv
returns":  it ends, full stop, no return value, nothing back to the
caller.  This is not a "crash", it's a well-defined exit condition.
(Whether it SHOULD be that condition specifically in zmv, is the topic
elsewhere in this thread.)  For a C equivalence, it's similar to
abort() having been called.

The "always" construct does nothing to the behavior of e.g. zmv, and
does not "memo" anything.  It allows calling code to intercept the
well-defined exit state and clean up.  (A "crash" would be an
unrecoverable condition in which not even "always" would be possible.)
 If the calling code does not intercept the exit state, then the
calling code also stops, and so on up the chain until (in an
interactive shell) the top-level interpreter implicitly sets a
non-zero status and prompts again.

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1133 bytes --]


On 2024-01-02 12:24, Bart Schaefer wrote:
> When one of those errors is encountered,
> whatever it is that the shell is doing, simply stops.  In a script,
> this exits the entire script; in an interactive shell, it returns to
> the top-level prompt.  In this specific example, there is no "zmv
> returns":  it ends, full stop, no return value, nothing back to the
> caller.

That's clear, but I had no idea that was the case.  Nothing back?  
Sounds inadvisable  but ...

>    This is not a "crash", it's a well-defined exit condition.
Right, I'm corrected.  Yes, 'crash' is a sloppy choice of words. I don't 
doubt zsh 'knows what it's doing' but it has the look of what Elon calls 
a rapid unscheduled disassembly.
> The "always" construct does nothing to the behavior of e.g. zmv, and
> does not "memo" anything.  It allows calling code to intercept the
> well-defined exit state and clean up.

I think I'd have to see the actual nuts and bolts of it to really 
understand.  It's not important.  I learned quite a bit.

Oh, when the devs have a final cut of the new zmv, if it's posted, I'll 
paste it over my copy here.



[-- Attachment #2: Type: text/html, Size: 2052 bytes --]

Re: zmv exits from function

[Peter Stephenson]

> On 02/01/2024 17:08 GMT Ray Andrews <rayandrews@eastlink.ca> wrote:
> 
>  
> On 2024-01-02 03:50, Peter Stephenson wrote:
> >> On 30/12/2023 20:38 GMT Bart Schaefer <schaefer@brasslantern.com> wrote:
> >> Arguably zmv could use null_glob.  Thoughts from -workers?
> > I guess the right thing to do regardless of option is that it should
> > fail gracefully just by returning status 1 --- so not propagating the
> > error to the caller.
> 
> That's just was I was trying to say :-)

So how about this...

pws

diff --git a/Doc/Zsh/contrib.yo b/Doc/Zsh/contrib.yo
index f43ac2257..182fc5f0a 100644
--- a/Doc/Zsh/contrib.yo
+++ b/Doc/Zsh/contrib.yo
@@ -4667,11 +4667,12 @@ renames `tt(foo.lis)' to `tt(foo.txt)', `tt(my.old.stuff.lis)' to
 `tt(my.old.stuff.txt)', and so on.
 
 The pattern is always treated as an tt(EXTENDED_GLOB) pattern.  Any file
-whose name is not changed by the substitution is simply ignored.  Any
-error (a substitution resulted in an empty string, two substitutions gave
-the same result, the destination was an existing regular file and tt(-f)
-was not given) causes the entire function to abort without doing
-anything.
+whose name is not changed by the substitution is simply ignored; if no
+files are matched by the pattern, the function silently returns status
+1.  Any error (a substitution resulted in an empty string, two
+substitutions gave the same result, the destination was an existing
+regular file and tt(-f) was not given) causes the entire function to
+abort without doing anything.
 
 In addition to pattern replacement, the variable tt($f) can be referred
 to in the second (replacement) argument.  This makes it possible to
diff --git a/Functions/Misc/zmv b/Functions/Misc/zmv
index 269fe5ba5..177428f08 100644
--- a/Functions/Misc/zmv
+++ b/Functions/Misc/zmv
@@ -236,7 +236,11 @@ if [[ $pat = (#b)(*)\((\*\*##/)\)(*) ]]; then
 else
   fpat=$pat
 fi
-files=(${~fpat})
+files=(${~fpat}(#qN))
+
+if (( ${#files} == 0 )); then
+  return 1
+fi
 
 [[ -n $hasglobqual ]] && pat=$opat

Re: zmv exits from function

[Bart Schaefer]

On Tue, Jan 2, 2024 at 9:48 AM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> So how about this...

That's what my patch in users/29387 does, except mine emits an error
message similar to all the other zmv error messages for pattern
failures, rather than silently returning 1.

Re: zmv exits from function

[Ray Andrews]

On 2024-01-02 11:07, Bart Schaefer wrote:
> On Tue, Jan 2, 2024 at 9:48 AM Peter Stephenson
> <p.w.stephenson@ntlworld.com> wrote:
>> So how about this...
> That's what my patch in users/29387 does, except mine emits an error
> message similar to all the other zmv error messages for pattern
> failures, rather than silently returning 1.
Messages are always friendly especially since one can redirect them to 
null if one doesn't want to see them.  Or just have a 'silent' 
option/switch.
>

Re: zmv exits from function

[Peter Stephenson]

> On 02/01/2024 19:07 GMT Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Tue, Jan 2, 2024 at 9:48 AM Peter Stephenson
> <p.w.stephenson@ntlworld.com> wrote:
> >
> > So how about this...
> 
> That's what my patch in users/29387 does, except mine emits an error
> message similar to all the other zmv error messages for pattern
> failures, rather than silently returning 1.

Sounds like that ought to be fine.

pws

Re: zmv exits from function

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 111 bytes --]


On 2024-01-03 01:55, Peter Stephenson wrote:
> Sounds like that ought to be fine.
May I have a copy gentlemen?

[-- Attachment #2: Type: text/html, Size: 551 bytes --]

Re: zmv exits from function

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 97 bytes --]

Updated patch including doc edit will be sent to zsh-workers.  Here's
a copy of the edited file.

[-- Attachment #2: zmv --]
[-- Type: application/octet-stream, Size: 11545 bytes --]

# function zmv {
# zmv, zcp, zln:
#
# This is a multiple move based on zsh pattern matching.  To get the full
# power of it, you need a postgraduate degree in zsh.  However, simple
# tasks work OK, so if that's all you need, here are some basic examples:
#   zmv '(*).txt' '$1.lis'
# Rename foo.txt to foo.lis, etc.  The parenthesis is the thing that
# gets replaced by the $1 (not the `*', as happens in mmv, and note the
# `$', not `=', so that you need to quote both words).
#   zmv '(**/)(*).txt '$1$2.lis'
# The same, but scanning through subdirectories.  The $1 becomes the full
# path.  Note that you need to write it like this; you can't get away with
# '(**/*).txt'.
#   zmv -w '**/*.txt' '$1$2.lis'
#   noglob zmv -W **/*.txt **/*.lis
# These are the lazy version of the one above; with -w, zsh inserts the
# parentheses for you in the search pattern, and with -W it also inserts
# the numbered variables for you in the replacement pattern.  The catch
# in the first version is that you don't need the / in the replacement
# pattern.  (It's not really a catch, since $1 can be empty.)  Note that
# -W actually inserts ${1}, ${2}, etc., so it works even if you put a
# number after a wildcard (such as zmv -W '*1.txt' '*2.txt').
#   zmv -C '**/(*).txt' ~/save/'$1'.lis
# Copy, instead of move, all .txt files in subdirectories to .lis files
# in the single directory `~/save'.  Note that the ~ was not quoted.
# You can test things safely by using the `-n' (no, not now) option.
# Clashes, where multiple files are renamed or copied to the same one, are
# picked up.
#
# Here's a more detailed description.
#
# Use zsh pattern matching to move, copy or link files, depending on
# the last two characters of the function name.  The general syntax is
#   zmv '<inpat>' '<outstring>'
# <inpat> is a globbing pattern, so it should be quoted to prevent it from
# immediate expansion, while <outstring> is a string that will be
# re-evaluated and hence may contain parameter substitutions, which should
# also be quoted.  Each set of parentheses in <inpat> (apart from those
# around glob qualifiers, if you use the -Q option, and globbing flags) may
# be referred to by a positional parameter in <outstring>, i.e. the first
# (...) matched is given by $1, and so on.  For example,
#   zmv '([a-z])(*).txt' '${(C)1}$2.txt'
# renames algernon.txt to Algernon.txt, boris.txt to Boris.txt and so on.
# The original file matched can be referred to as $f in the second
# argument; accidental or deliberate use of other parameters is at owner's
# risk and is not covered by the (non-existent) guarantee.
#
# As usual in zsh, /'s don't work inside parentheses.  There is a special
# case for (**/) and (***/):  these have the expected effect that the
# entire relevant path will be substituted by the appropriate positional
# parameter.
#
# There is a shortcut avoiding the use of parenthesis with the option -w
# (with wildcards), which picks out any expressions `*', `?', `<range>'
# (<->, <1-10>, etc.), `[...]', possibly followed by `#'s, `**/', `***/', and
# automatically parenthesises them. (You should quote any ['s or ]'s which
# appear inside [...] and which do not come from ranges of the form
# `[:alpha:]'.)  So for example, in
#    zmv -w '[[:upper:]]*' '${(L)1}$2'
# the $1 refers to the expression `[[:upper:]]' and the $2 refers to
# `*'. Thus this finds any file with an upper case first character and
# renames it to one with a lowercase first character.  Note that any
# existing parentheses are active, too, so you must count accordingly.
# Furthermore, an expression like '(?)' will be rewritten as '((?))' --- in
# other words, parenthesising of wildcards is independent of any existing
# parentheses.
#
# Any file whose name is not changed by the substitution is simply ignored.
# Any error --- a substitution resulted in an empty string, two
# substitutions gave the same result, the destination was an existing
# regular file and -f was not given --- causes the entire function to abort
# without doing anything.
#
# Options:
#  -f  force overwriting of destination files.  Not currently passed
#      down to the mv/cp/ln command due to vagaries of implementations
#      (but you can use -o-f to do that).
#  -i  interactive: show each line to be executed and ask the user whether
#      to execute it.  Y or y will execute it, anything else will skip it.
#      Note that you just need to type one character.
#  -n  no execution: print what would happen, but don't do it.
#  -q  Turn bare glob qualifiers off:  now assumed by default, so this
#      has no effect.
#  -Q  Force bare glob qualifiers on.  Don't turn this on unless you are
#      actually using glob qualifiers in a pattern (see below).
#  -s  symbolic, passed down to ln; only works with zln or z?? -L.
#  -v  verbose: print line as it's being executed.
#  -o <optstring>
#      <optstring> will be split into words and passed down verbatim
#      to the cp, ln or mv called to perform the work.  It will probably
#      begin with a `-'.
#  -p <program>
#      Call <program> instead of cp, ln or mv.  Whatever it does, it should
#      at least understand the form '<program> -- <oldname> <newname>',
#      where <oldname> and <newname> are filenames generated. <program>
#      will be split into words.
#  -P <program>
#      As -p, but the program doesn't understand the "--" convention.
#      In this case the file names must already be sane.
#  -w  Pick out wildcard parts of the pattern, as described above, and
#      implicitly add parentheses for referring to them.
#  -W  Just like -w, with the addition of turning wildcards in the
#      replacement pattern into sequential ${1} .. ${N} references.
#  -C
#  -L
#  -M  Force cp, ln or mv, respectively, regardless of the name of the
#      function.
#
# Bugs:
#   Parenthesised expressions can be confused with glob qualifiers, for
#   example a trailing '(*)' would be treated as a glob qualifier in
#   ordinary globbing.  This has proved so annoying that glob qualifiers
#   are now turned off by default.  To force the use of glob qualifiers,
#   give the flag -Q.
#
#   The pattern is always treated as an extendedglob pattern.  This
#   can also be interpreted as a feature.
#
# Unbugs:
#   You don't need braces around the 1 in expressions like '$1t' as
#   non-positional parameters may not start with a number, although
#   paranoiacs like the author will probably put them there anyway.

emulate -RL zsh
setopt extendedglob

local f g args match mbegin mend files action myname tmpf opt exec
local opt_f opt_i opt_n opt_q opt_Q opt_s opt_M opt_C opt_L 
local opt_o opt_p opt_P opt_v opt_w opt_W MATCH MBEGIN MEND
local pat repl errstr fpat hasglobqual opat
typeset -A from to
integer stat

local dashes=--

myname=${(%):-%N}

while getopts ":o:p:P:MCLfinqQsvwW" opt; do
  if [[ $opt = "?" ]]; then
    print -r -- "$myname: unrecognized option: -$OPTARG" >&2
    return 1
  fi
  eval "opt_$opt=\${OPTARG:--\$opt}"
done
(( OPTIND > 1 )) && shift $(( OPTIND - 1 ))

[[ -z $opt_Q ]] && setopt nobareglobqual
[[ -n $opt_M ]] && action=mv
[[ -n $opt_C ]] && action=cp
[[ -n $opt_L ]] && action=ln
[[ -n $opt_p ]] && action=$opt_p
[[ -n $opt_P ]] && action=$opt_P dashes=

if [[ -z $action ]]; then
  action=$myname[-2,-1]

  if [[ $action != (cp|mv|ln) ]]; then
    print -r "$myname: action $action not recognised: must be cp, mv or ln." >&2
    return 1
  fi
fi

if (( $# != 2 )); then
  print -P "Usage:
  %N [OPTIONS] oldpattern newpattern
where oldpattern contains parenthesis surrounding patterns which will
be replaced in turn by \$1, \$2, ... in newpattern.  For example,
  %N '(*).lis' '\$1.txt'
renames 'foo.lis' to 'foo.txt', 'my.old.stuff.lis' to 'my.old.stuff.txt',
and so on.  Something simpler (for basic commands) is the -W option:
  %N -W '*.lis' '*.txt'
This does the same thing as the first command, but with automatic conversion
of the wildcards into the appropriate syntax.  If you combine this with
noglob, you don't even need to quote the arguments.  For example,
  alias mmv='noglob zmv -W'
  mmv *.c.orig orig/*.c" >&2
  return 1
fi

pat=$1
repl=$2
shift 2

if [[ -n $opt_s && $action != ln ]]; then
  print -r -- "$myname: invalid option: -s" >&2
  return 1
fi

if [[ -n $opt_w || -n $opt_W ]]; then
  # Parenthesise all wildcards.
  local tmp find
  integer cnt=0
  # Well, this seems to work.
  # The tricky bit is getting all forms of [...] correct, but as long
  # as we require inactive bits to be backslashed its not so bad.
  find='(#m)((\*\*##/|[*?]|<[0-9]#-[0-9]#>|\[(^|)(\]|)(\[:[a-z]##:\]|\\?|[^\]])##\])\##|?\###)'
  tmp="${pat//${~find}/$[++cnt]}"
  if [[ $cnt = 0 ]]; then
    print -r -- "$myname: warning: no wildcards were found in search pattern" >&2
  else
    pat="${pat//${~find}/($MATCH)}"
  fi
  if [[ -n $opt_W ]]; then
    # Turn wildcards into ${1} .. ${N} references.
    local open='${' close='}'
    integer N=0
    repl="${repl//${~find}/$open$[++N]$close}"
    if [[ $N != $cnt ]]; then
      print -P "%N: error: number of wildcards in each pattern must match" >&2
      return 1
    fi
    if [[ $N = 0 ]]; then
      print -P "%N: warning: no wildcards were found in replacement pattern" >&2
    fi
  fi
fi

if [[ -n $opt_Q && $pat = (#b)(*)\([^\)\|\~]##\) ]]; then
  hasglobqual=q
  # strip off qualifiers for use as ordinary pattern
  opat=$match[1]
fi

if [[ $pat = (#b)(*)\((\*\*##/)\)(*) ]]; then
  fpat="$match[1]$match[2]$match[3]"
  # Now make sure we do depth-first searching.
  # This is so that the names of any files are altered before the
  # names of the directories they are in.
  if [[ -n $opt_Q && -n $hasglobqual ]]; then
    fpat[-1]="odon)"
  else
    setopt bareglobqual
    fpat="${fpat}(odon)"
  fi
else
  fpat=$pat
fi

[[ -n $hasglobqual ]] && pat=$opat

errs=()

() {
  # (#qN) breaks bareglobqual -Q option, so:
  setopt localoptions nullglob
  files=(${~fpat})
}
(( ${#files} )) || errs=( "no files matched \`$fpat'" )

for f in $files; do
  if [[ $pat = (#b)(*)\(\*\*##/\)(*) ]]; then
    # This looks like a recursive glob.  This isn't good enough,
    # because we should really enforce that $match[1] and $match[2]
    # don't match slashes unless they were explicitly given.  But
    # it's a start.  It's fine for the classic case where (**/) is
    # at the start of the pattern.
    pat="$match[1](*/|)$match[2]"
  fi
  [[ -e $f && $f = (#b)${~pat} ]] || continue
  set -- "$match[@]"
  { {
    g=${(Xe)repl}
  } 2> /dev/null } always {
    if (( TRY_BLOCK_ERROR )); then
      print -r -- "$myname: syntax error in replacement" >&2
      return 1
    fi
  }
  if [[ -z $g ]]; then
    errs+=("\`$f' expanded to an empty string")
  elif [[ $f = $g ]]; then
    # don't cause error: more useful just to skip
    #   errs=($errs "$f not altered by substitution")
    [[ -n $opt_v ]] && print -r -- "$f not altered, ignored"
    continue
  elif [[ -n $from[$g] && ! -d $g ]]; then
    errs+=("$f and $from[$g] both map to $g")
  elif [[ -f $g && -z $opt_f && ! ($f -ef $g && $action = mv) ]]; then
    errs+=("file exists: $g")
  fi
  from[$g]=$f
  to[$f]=$g
done

if (( $#errs )); then
  print -r -- "$myname: error(s) in substitution:" >&2
  print -lr -- $errs >&2
  return 1
fi

for f in $files; do
  [[ -z $to[$f] ]] && continue
  exec=(${=action} ${=opt_o} $opt_s $dashes $f $to[$f])
  [[ -n $opt_i$opt_n$opt_v ]] && print -r -- ${(q-)exec}
  if [[ -n $opt_i ]]; then
    read -q 'opt?Execute? ' || continue
  fi
  if [[ -z $opt_n ]]; then
    $exec || stat=1
  fi
done

return $stat
# }

Re: zmv exits from function

[Bart Schaefer]

On Sat, Dec 30, 2023 at 9:43 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I'm using zmv to rename files in a directory tree recursively via a
> 'for' loop which visits each subdirectory.

Incidentally, unless your "f" function is doing something non-obvious,
zmv can recursively search directories all by itself.

Try (without your "for" wrapper and all the "cd"-ing):

  zmv -n '(*/)#(*).(SNT|MES)' '$1$2.eml'

If that looks like it would do the right thing, remove the "-n".