[-- Attachment #1: Type: text/plain, Size: 1694 bytes --] Let's assume I have a series of large commands which need to be piped to eachother. Additionally, the pipeline itself must behave differently based on some global flags or arguments. There's a lot of way to make logic for this (e.g. functions), but one that I've become a fan of is something like this below. cmd1=( some long command $FLAG bar baz ) cmd2=( grep -E "$@" ) cmd3=( grep -v # Don't match these -e 'fizz|buzz' ) "${cmd1[@]}" | "${cmd2[@]}" | "${cmd3[@]}" However, conditional logic makes this annoying and a bit opaque -- functions definitely excel here. Another issue is that of adding comments to the arguments and the pipeline itself -- lines that end with '\' cannot contain comments. Functions obviously can contain comments within themselves. What I've discovered is that something like this works out pretty well... { printf "%s\n" a b foo c bar d fizz XfooX XbuzzX } | { grep -E 'foo|bar' } | { # If the user specified '--no-fizz-buzz', remove those entries if (( NO_FIZZ_BUZZ )); then grep -vE 'fizz|buzz' else # Pass through everything cat fi } I have a few questions about this construct. 1. Am I insane for doing this? 2. In what ways is this a terrible idea? 3. Is the use of {} better than ()? 4. How much of a performance hit does this make, versus hand-writing a different pipeline? 5. Are there any ways to improve this? For example, replacing 'cat' in the default case. Thanks for the attention, just curious what everybody thinks about this abuse of pipelines and conditional logic. *Zach Riggle* [-- Attachment #2: Type: text/html, Size: 2224 bytes --]
> On May 11, 2022, at 2:37 AM, Zach Riggle <zachriggle@gmail.com> wrote: > What I've discovered is that something like this works out pretty well... > > { printf "%s\n" a b foo c bar d fizz XfooX XbuzzX } | > { grep -E 'foo|bar' } | > { > # If the user specified '--no-fizz-buzz', remove those entries > if (( NO_FIZZ_BUZZ )); then > grep -vE 'fizz|buzz' > else > # Pass through everything > cat > fi > } > > I have a few questions about this construct. > > 1. Am I insane for doing this? > 2. In what ways is this a terrible idea? There isn’t anything wrong with (or novel about) using grouped commands as pipeline stages. (However, in your example the braces in the first two stages are unnecessary and distracting. They make me wonder whether there’s some obscure reason you added them.) > 3. Is the use of {} better than ()? I can’t think of a compelling reason to use () other than to force the final stage to run in a subshell. (I believe the other stages run in subshells regardless.) > Thanks for the attention, just curious what everybody thinks about this abuse of pipelines and conditional logic. It’s not abuse. It’s entirely valid to use complex commands as pipeline stages. Now, whether or not that is more legible or maintainable than the alternatives is a different, context-dependent question. -- vq Sent from my iPhone
In addition to what Lawrence said ... On Tue, May 10, 2022 at 11:35 PM Zach Riggle <zachriggle@gmail.com> wrote: > > 3. Is the use of {} better than ()? In most cases it won't matter. Putting something on the left side of a pipe in zsh already starts it in a subshell, and most such subshells are subject to an optimization such that zsh will exec the last command to replace the shell process. On the right side of a pipe, however commands in { } stay in the current shell whereas those in ( ) run in a forked subshell. > 4. How much of a performance hit does this make, versus hand-writing a different pipeline? It's definitely going to perform better than a pipeline built by assigning and expanding array parameters. Otherwise the performance difference should be minimal. > 5. Are there any ways to improve this? For example, replacing 'cat' in the default case. You're not going to do better than "cat" in that context.