* zargs parallelization support blows up with |&
@ 2015-07-27 9:36 Dima Kogan
2015-07-27 16:15 ` Bart Schaefer
0 siblings, 1 reply; 4+ messages in thread
From: Dima Kogan @ 2015-07-27 9:36 UTC (permalink / raw)
To: zsh-workers
Hi. I can use this to invoke a perl script 10 times, 2 processes at a
time:
zargs -P2 -i -l1 `seq 10` -- perl -e 'sleep 20' {}
This works ok. However, when I do the same thing, but filter stdout and
stderr for some output, then ALL the processes are invoked at the same
time, not just 2 at a time like I asked:
zargs -P2 -i -l1 `seq 10` -- perl -e 'sleep 20' {} |& grep x
I don't know if this is a bug or an expected consequence of some
internal implementation details, but this is a really nasty failure
mode.
Thanks for looking.
dima
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: zargs parallelization support blows up with |&
2015-07-27 9:36 zargs parallelization support blows up with |& Dima Kogan
@ 2015-07-27 16:15 ` Bart Schaefer
2015-07-27 16:37 ` Bart Schaefer
0 siblings, 1 reply; 4+ messages in thread
From: Bart Schaefer @ 2015-07-27 16:15 UTC (permalink / raw)
To: Dima Kogan, zsh-workers
On Jul 27, 2:36am, Dima Kogan wrote:
}
} [... when piping] stdout and
} stderr for some output, then ALL the processes are invoked at the same
} time, not just 2 at a time like I asked:
}
} zargs -P2 -i -l1 `seq 10` -- perl -e 'sleep 20' {} |& grep x
}
} I don't know if this is a bug or an expected consequence of some
} internal implementation details, but this is a really nasty failure
} mode.
The problem is that when you append a pipe to the end, zargs ends up
as a subshell, and you can't do job control in a subshell (which you
would have seen if you weren't grepping stderr):
(eval):wait:1: can't manipulate jobs in subshell
The workaround for now is as follows:
Copy the zargs script file to somewhere in your $path, e.g. ~/bin/.
Edit the copy to add a #!/bin/zsh (or your local path) at the top,
and "chmod +x" the file.
Then either always run zargs as a script instead of an autoload, or
when you need to pipeline it, use "command zargs". This is a little
less convenient because it loses access to interactively-defined
functions, but if you're zargs-ing another external command such as
perl it should work fine.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: zargs parallelization support blows up with |&
2015-07-27 16:15 ` Bart Schaefer
@ 2015-07-27 16:37 ` Bart Schaefer
2015-07-27 18:19 ` Dima Kogan
0 siblings, 1 reply; 4+ messages in thread
From: Bart Schaefer @ 2015-07-27 16:37 UTC (permalink / raw)
To: zsh-workers
On Jul 27, 9:15am, Bart Schaefer wrote:
}
} The problem is that when you append a pipe to the end, zargs ends up
} as a subshell, and you can't do job control in a subshell (which you
} would have seen if you weren't grepping stderr):
}
} (eval):wait:1: can't manipulate jobs in subshell
This turned out to be easier to fix than I expected:
diff --git a/Functions/Misc/zargs b/Functions/Misc/zargs
index 71fd428..28ebca7 100644
--- a/Functions/Misc/zargs
+++ b/Functions/Misc/zargs
@@ -73,7 +73,7 @@
emulate -L zsh || return 1
local -a opts eof n s l P i
-local ZARGS_VERSION="1.4"
+local ZARGS_VERSION="1.5"
if zparseopts -a opts -D -- \
-eof::=eof e::=eof \
@@ -254,7 +254,7 @@ then
bg='&'
if zmodload -i zsh/parameter 2>/dev/null
then
- wait='wait %${(k)^jobstates[(R)running:*]}'
+ wait='wait ${${jobstates[(R)running:*]/#*:/}/%=*/}'
else
wait='wait'
fi
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: zargs parallelization support blows up with |&
2015-07-27 16:37 ` Bart Schaefer
@ 2015-07-27 18:19 ` Dima Kogan
0 siblings, 0 replies; 4+ messages in thread
From: Dima Kogan @ 2015-07-27 18:19 UTC (permalink / raw)
To: Bart Schaefer; +Cc: zsh-workers
Bart Schaefer <schaefer@brasslantern.com> writes:
> On Jul 27, 9:15am, Bart Schaefer wrote:
> }
> } The problem is that when you append a pipe to the end, zargs ends up
> } as a subshell, and you can't do job control in a subshell (which you
> } would have seen if you weren't grepping stderr):
> }
> } (eval):wait:1: can't manipulate jobs in subshell
>
> This turned out to be easier to fix than I expected:
>
> <patch>
The patch works well for me. Thanks!
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-07-27 18:18 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-27 9:36 zargs parallelization support blows up with |& Dima Kogan
2015-07-27 16:15 ` Bart Schaefer
2015-07-27 16:37 ` Bart Schaefer
2015-07-27 18:19 ` Dima Kogan
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).