From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 27540 invoked from network); 6 Nov 2021 21:59:35 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 6 Nov 2021 21:59:35 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1636235975; b=MbkPQ4F8iRRqm3P8zOldeoH6zNky1K1lIsk3fNoFCuISUVEvHmImNjF5IjpafY1q7yULbJMqYX Caa/VEOGuG/BHo3iETdoexjzkSf0+zL8bNQhipzjqtteGo1c/2Oc/Id+JvPtn0exSGe3jf9YhU XY5tm0EtSfT2Cfsuf7TQhS6dBhs5stlclPmXaIT+tfkrO0WWumxAuOrQXiclScY4W91/0jZgUu E2SAQcx4BeGFf5l5jcbAzoTCaMbVpJEPtgtiG8w7QcU2tMqxWOkno/C3hQZZ1/hQXeWEELdfJP p+qFPXiwtdPH3toONdlezJyNg0sz5XiY2i376SSTe1fFvA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-wm1-f53.google.com) smtp.remote-ip=209.85.128.53; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1636235975; bh=jQS7SioO1sIU/RxBJ2odHOxA0XNe4qyb1EvFlDI2CMA=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:DKIM-Signature:DKIM-Signature; b=Ys/fb/+feP02AA2HPvwzxvi59lj30d0eQw4Tslw/dCUnn1KUB9KNUgY+rQIOHJLSFwFXiqNf6T tx+aXbMYyvjbk2wRl1jtVr0QfvKr3SOqCTul4Jd0EMS5XsE5tdm/K4FXN7KG3vQbC/Ld7MFC9J rvW3YhmdQfpRz3j3HjYU7ygmOpiL8seDRdU1M8pneAJIVAy9xpyoRgJ/gEvdTqA/j8O6t9b14t B/7KhNTz275wywWM0/q3u4S+xZQ4hJzENUy3POOMRJ1BWQyC2lD65xHPIODWd3VuLv+xCcppRe gHZfjW/p1T1CgD5uVZDfTe7koNNNXAqGUxG2kTBZOW9NhA==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=Q7Bn1Q8jHwtZ4vVppwGU4mX0c4O0ITKmQtCBL0nT1l0=; b=bcpePCX78jpIzRjDmMAKno8Pvu Z5DGhBz6RO98+cHUSKOB76FJ0S1Vd5t87nxYl14D2cwV4R/8totzx5DUZILtcQbga60zAdubfBVIa 0M+3rf5U8qRUFwu09N+6eKypS2C4miubNO8ytfj63+MxwcAsUc/3uFmSeuqBjJYxIr42/GPz1qHAm Fxb+rVmI6iKdg89lgv7p+kqBM4Qcq2sUdKS/fGrgNmWYEoub9NWX9Hd/9/BAIgMlOg8dlbk2UtU/p d2zZPLpxvmL8OicUsa/ONMKdfgkvSdu294d1BtWgdO/JRi5flAHYPn6BnwUdJob572/6CoSAmdm23 5XtFKlVw==; Received: from authenticated user by zero.zsh.org with local id 1mjTio-000Hfz-20; Sat, 06 Nov 2021 21:59:34 +0000 Authentication-Results: zsh.org; iprev=pass (mail-wm1-f53.google.com) smtp.remote-ip=209.85.128.53; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-wm1-f53.google.com ([209.85.128.53]:50895) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1mjTiA-000GzW-BF; Sat, 06 Nov 2021 21:58:55 +0000 Received: by mail-wm1-f53.google.com with SMTP id 133so10003114wme.0 for ; Sat, 06 Nov 2021 14:58:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Q7Bn1Q8jHwtZ4vVppwGU4mX0c4O0ITKmQtCBL0nT1l0=; b=d0J1BjJOtf5+4XJQkKSVUfaAkhlmfYtmS4RgM+N2z89oz+Fiv38as4QjyjDyDdH9dw lMr1ARbUiyfx7SatkiYfH3PXmsFcD2Hj96aqN1r49ysgTfvrtBraBi18Pz1vEiZveXJD HQVUPgxMJPUmsijD2VkuKkt3uY+HmqTNudMvmp7TFzrt0xMwwM1sny5gVTI78yRRs6eO X2frdgHV0UGpnqAGyY0M2/MO1FjFmv90ZQPtVScCJkEcPClkPDhuq6gptl8ruBpfCDvI HAd/idoG11qI0J2Wb7vaby+aznfpw0M47C2EFnOamHXp0uF/V7NlUR69qeBeV5pOEuWu syCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Q7Bn1Q8jHwtZ4vVppwGU4mX0c4O0ITKmQtCBL0nT1l0=; b=yr4rLf4yJ3vbyxR0fNWnCqoEvJq8qQe0BxxSx8qAI1p7mE7k1+U/iMH56NtrCd1YwB /uPdRm7bSRG9Gj2xPxdRrfzBxI3ceoyuQJdb/RwfsngAxGPeeJ92VtvpTEUMhNcuC34m 5m5TM+rb7uNZ63XBHUhVIt1CrI6y8lyOmAN+5mwT0TzsaI99H+h8XWBRe6g0wuiZkbxn qtHKDbUbRjbuWY4eplFF6g6wNPdNgQ65I5JOdxehPID2hgpHCfmCPEmHHJ7AtUc6ZZdk sLjuA6KkwS8aZf59sAXAUnBqNBnmkoMhU8bnPufH+uzeJtGnLFBErrdZhay0i0blWqsL djRQ== X-Gm-Message-State: AOAM533oybJj+3xnJ8X+nAkBldorBAtgsuqJdTfCqYVhqSRSpjv+MzhF Az4asbHwIIiygnmnpN4GXUIryK8I6rpNuCovFLn8pjWOHxq7FQ== X-Google-Smtp-Source: ABdhPJxxpRqcbZILE3X4vMY5hHcm9OyQXz6lbCT9hLOXlAtra0VApIeSxyzzoinfSrQ8+tabFMFETNj0x8fiF4Xc2So= X-Received: by 2002:a7b:ca4c:: with SMTP id m12mr38942009wml.119.1636235933828; Sat, 06 Nov 2021 14:58:53 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Zach Riggle Date: Sat, 6 Nov 2021 16:58:42 -0500 Message-ID: Subject: Re: Efficient way to map a list of values to multiple processes, then accumulate their output To: Bart Schaefer Cc: Zsh Users Content-Type: multipart/alternative; boundary="0000000000009acbb805d025dd4d" X-Seq: 27320 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: --0000000000009acbb805d025dd4d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Wow. I'm genuinely impressed, it even supports all of the flags that I need! And has all of the fun non-determinism of parallel execution (with -P)! This satisfies about 60% of what I need, and is a great tool to have handy! The limitation appears to be that the command is run in a sub-shell / separate process (as one might expect), so there's no way to correlate input74 =E2=86=92 output74. I suppose with "zargs -n1" I can redirect the output of each to an individual file (in an e.g. temporary file), and then just read them back and stick everything into an associative array. A very simple test works exactly correct, but a slight variation gives me "zargs: argument list too long": Uploaded the source here for ease of viewing and syntax highlighting: https://gist.github.com/zachriggle/b53b35faa5a60b674575e1dc6cae1d2e *Zach Riggle* On Sat, Nov 6, 2021 at 11:18 AM Bart Schaefer wrote: > On Sat, Nov 6, 2021 at 6:41 AM Zach Riggle wrote: > > > > Ultimately, I'd like to have some quasi-implementation of xargs -P in > pure zsh -- which maps a __function__ to a list of arguments > > Look at "zargs" in "man zshcontrib" > --0000000000009acbb805d025dd4d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Wow.=C2=A0 I'm genuinely impressed, it even supports a= ll of the flags that I need!=C2=A0 And has all of the fun non-determinism o= f parallel execution (with -P)!=C2=A0 This satisfies about 60% of what I ne= ed, and is a great tool to have handy!

The limitation ap= pears to be that the command is run in a sub-shell=C2=A0/ separate process = (as one might expect), so there's no way to correlate input74 =E2=86=92= output74.

I suppose with "zargs -n1" I = can redirect the output of each to an individual file (in an e.g. temporary= file), and then just read them back and stick everything into an associati= ve array.

A very simple test works exactly correct= , but a slight variation gives me "zargs: argument list too long"= :

Uploaded the source here for ease of viewing and= syntax highlighting:=C2=A0https://gist.github.com/zachriggle/b53b35fa= a5a60b674575e1dc6cae1d2e

Zach Riggle

<= /div>
O= n Sat, Nov 6, 2021 at 11:18 AM Bart Schaefer <schaefer@brasslantern.com> wrot= e:
On Sat, Nov 6= , 2021 at 6:41 AM Zach Riggle <zachriggle@gmail.com> wrote:
>
> Ultimately, I'd like to have some quasi-implementation of xargs -P= in pure zsh -- which maps a __function__ to a list of arguments

Look at "zargs" in "man zshcontrib"
--0000000000009acbb805d025dd4d--