zsh-users
 help / color / mirror / code / Atom feed
* Commenting and large pipelines
@ 2022-05-11  6:34 Zach Riggle
  2022-05-11  7:13 ` Lawrence Velázquez
  2022-05-12  6:15 ` Bart Schaefer
  0 siblings, 2 replies; 3+ messages in thread
From: Zach Riggle @ 2022-05-11  6:34 UTC (permalink / raw)
  To: Zsh Users

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

Let's assume I have a series of large commands which need to be piped to
eachother.

Additionally, the pipeline itself must behave differently based on some
global flags or arguments.

There's a lot of way to make logic for this (e.g. functions), but one that
I've become a fan of is something like this below.

    cmd1=( some long command $FLAG bar baz )
    cmd2=( grep -E "$@" )
    cmd3=(
        grep
            -v # Don't match these
            -e 'fizz|buzz'
    )

    "${cmd1[@]}" | "${cmd2[@]}" | "${cmd3[@]}"

However, conditional logic makes this annoying and a bit opaque --
functions definitely excel here.

Another issue is that of adding comments to the arguments and the pipeline
itself -- lines that end with '\' cannot contain comments.  Functions
obviously can contain comments within themselves.

What I've discovered is that something like this works out pretty well...

    { printf "%s\n" a b foo c bar d fizz XfooX XbuzzX } |
    { grep -E 'foo|bar' } |
    {
        # If the user specified '--no-fizz-buzz', remove those entries
        if (( NO_FIZZ_BUZZ )); then
            grep -vE 'fizz|buzz'
        else
        # Pass through everything
            cat
        fi
    }

I have a few questions about this construct.

    1. Am I insane for doing this?
    2. In what ways is this a terrible idea?
    3. Is the use of {} better than ()?
    4. How much of a performance hit does this make, versus hand-writing a
different pipeline?
    5. Are there any ways to improve this?  For example, replacing 'cat' in
the default case.

Thanks for the attention, just curious what everybody thinks about this
abuse of pipelines and conditional logic.

*Zach Riggle*

[-- Attachment #2: Type: text/html, Size: 2224 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Commenting and large pipelines
  2022-05-11  6:34 Commenting and large pipelines Zach Riggle
@ 2022-05-11  7:13 ` Lawrence Velázquez
  2022-05-12  6:15 ` Bart Schaefer
  1 sibling, 0 replies; 3+ messages in thread
From: Lawrence Velázquez @ 2022-05-11  7:13 UTC (permalink / raw)
  To: Zach Riggle; +Cc: Zsh Users

> On May 11, 2022, at 2:37 AM, Zach Riggle <zachriggle@gmail.com> wrote:
> What I've discovered is that something like this works out pretty well...
> 
>     { printf "%s\n" a b foo c bar d fizz XfooX XbuzzX } |
>     { grep -E 'foo|bar' } |
>     {
>         # If the user specified '--no-fizz-buzz', remove those entries
>         if (( NO_FIZZ_BUZZ )); then
>             grep -vE 'fizz|buzz'
>         else
>         # Pass through everything
>             cat
>         fi
>     }
> 
> I have a few questions about this construct.
> 
>     1. Am I insane for doing this?
>     2. In what ways is this a terrible idea?

There isn’t anything wrong with (or novel about) using grouped commands as pipeline stages.

(However, in your example the braces in the first two stages are unnecessary and distracting. They make me wonder whether there’s some obscure reason you added them.)

>     3. Is the use of {} better than ()?

I can’t think of a compelling reason to use () other than to force the final stage to run in a subshell. (I believe the other stages run in subshells regardless.)

> Thanks for the attention, just curious what everybody thinks about this abuse of pipelines and conditional logic.

It’s not abuse. It’s entirely valid to use complex commands as pipeline stages.

Now, whether or not that is more legible or maintainable than the alternatives is a different, context-dependent question.

-- 
vq
Sent from my iPhone


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Commenting and large pipelines
  2022-05-11  6:34 Commenting and large pipelines Zach Riggle
  2022-05-11  7:13 ` Lawrence Velázquez
@ 2022-05-12  6:15 ` Bart Schaefer
  1 sibling, 0 replies; 3+ messages in thread
From: Bart Schaefer @ 2022-05-12  6:15 UTC (permalink / raw)
  To: Zach Riggle; +Cc: Zsh Users

In addition to what Lawrence said ...

On Tue, May 10, 2022 at 11:35 PM Zach Riggle <zachriggle@gmail.com> wrote:
>
>     3. Is the use of {} better than ()?

In most cases it won't matter.  Putting something on the left side of
a pipe in zsh already starts it in a subshell, and most such subshells
are subject to an optimization such that zsh will exec the last
command to replace the shell process.

On the right side of a pipe, however commands in { } stay in the
current shell whereas those in ( ) run in a forked subshell.

>     4. How much of a performance hit does this make, versus hand-writing a different pipeline?

It's definitely going to perform better than a pipeline built by
assigning and expanding array parameters.  Otherwise the performance
difference should be minimal.

>     5. Are there any ways to improve this?  For example, replacing 'cat' in the default case.

You're not going to do better than "cat" in that context.


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-05-12  6:16 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-11  6:34 Commenting and large pipelines Zach Riggle
2022-05-11  7:13 ` Lawrence Velázquez
2022-05-12  6:15 ` Bart Schaefer

Code repositories for project(s) associated with this inbox:

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).