From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HTML_MESSAGE,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 11444 invoked from network); 28 Mar 2022 13:33:18 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 28 Mar 2022 13:33:18 -0000 ARC-Seal: i=2; cv=pass; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1648474398; b=kurT5sWrVu3MH23GTy+Zc9hcFfouhH6C+JwHmmif302z2wJlIUWzsIs3wpLfgnA7hAZetAGMbU Ka31u4McxSlsouxZyWHMaAtNDdiSUUjZlKQKTx7z0H2iN92COfdlpbLWgEPdISICq9doDEGr9p iaiPedHrnQ5dB6nuqZrujj4tca94f8yDrT1vI2VDaBnRiTExt9oyqfnT7HxL9vNdMqPn6r0VHB axO+ZTJUMTaszZHYrjTyK+9haPTc1BFvwKNpaoWuHjl9qX9+Rg3DWLgazc5RsTHC69dG51/Asl DP7oLD3MZmcauD0mIaS6SuR8KtXX5Oi3Ciai3S5oEXZPhw==; ARC-Authentication-Results: i=2; zsh.org; iprev=pass (mcgt-001.nmsrv.com) smtp.remote-ip=199.10.31.235; dkim=pass header.d=easesoftware.com header.s=default header.a=rsa-sha256; dmarc=none header.from=easesoftware.com; arc=pass (i=1) header.s=arc-2022 arc.oldest-pass=1 smtp.remote-ip=199.10.31.235 ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1648474398; bh=Rv39BgPh6MZbn5+r/WvJUc6g02NwRjiz5I10ptxpfFQ=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Message-ID:In-Reply-To:To:References:Date:Subject: MIME-Version:Content-Type:From:DKIM-Signature:DKIM-Signature; b=jQ/wth7hpnocj7EDr1oYHX0W9ruObjGyGt7Fp8cvWqcVSSyehmhVhRrxVEYDKZoSAN4c3E4Jaw HIvjPXKZrZS8MYawdQ6m0Bp4v5CUqYjUz0+mPXU8OBBW/xvYQDfK7zz+c1nz1ldJlM99WHMdE8 BZ4+x3TszWyiHS1lzC+N5Hpwey8eGQWTjHhg2mf80ahbFYw/Yd9fXlt5PQPFZAip4cHvhajlx9 a57ob4m68tmTF46Ld6T9iAmujpmlVtJlaezZeSp+dxcANDKk7ZIUNewiSxCWhcME7VqNBgKMIl mzPVfuuCDFRhiM2LkVQsa6hi6s2HuFFvxldNXoWV3m4G3Q==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Message-Id:In-Reply-To:To:References: Date:Subject:Mime-Version:Content-Type:From:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=lZnaM8tEa62v7OTMANBpwBQv0xXQ5xfXOM6IT4G7ums=; b=cIBO6LcaO//hiJcbKFSYHeTz0H O+OlkqPHxcKSm6k77DgTZYNionj5/hHERezrD5xivOO0Sc4Q8oEQRErU9huetRpCbRWw6LSMNm0Kq x5Zb+NOQ/pErUzhzWXeveFA+j9UP8eXGn7PO5HGEt7qA3Jpu1Erb6V/ysiWufIFMMT8qE18ATOSC5 jel4pYCtAZbc32wS5eyv0NTXFoDslR9JzLjjhJRbtze6h7CwKvFKa+azwSngSYxY8VDbC8DgTa7rl Vgx1tWyiLLjhDBo2DMC6v8L9itQnmXtwW8TeBI3WmyHqMpko/N/1l80Gfe5gp5bqtzOkiXnPIjlRW ZOkHsaEA==; Received: from authenticated user by zero.zsh.org with local id 1nYpUi-000FPf-Mt; Mon, 28 Mar 2022 13:33:16 +0000 Authentication-Results: zsh.org; iprev=pass (mcgt-001.nmsrv.com) smtp.remote-ip=199.10.31.235; dkim=pass header.d=easesoftware.com header.s=default header.a=rsa-sha256; dmarc=none header.from=easesoftware.com; arc=pass (i=1) header.s=arc-2022 arc.oldest-pass=1 smtp.remote-ip=199.10.31.235 Received: from mcgt-001.nmsrv.com ([199.10.31.235]:25634 helo=relay.mailchannels.net) by zero.zsh.org with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) id 1nYpTm-000EgZ-Tm; Mon, 28 Mar 2022 13:32:20 +0000 X-Sender-Id: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id A6AB26C1D58 for ; Mon, 28 Mar 2022 13:31:59 +0000 (UTC) Received: from mi3-ss4.a2hosting.com (unknown [127.0.0.6]) (Authenticated sender: a2hosting) by relay.mailchannels.net (Postfix) with ESMTPA id EDC496C16C1 for ; Mon, 28 Mar 2022 13:31:58 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1648474319; a=rsa-sha256; cv=none; b=5LckN2duJqAb5yiY12O1aWy0Vu/JD2Ef+37Sh4H+87/xFwOnbad/itRbn4otQGDdBvBsLX LMuPIHfBkz5HEtSfhB3rDE2cJHsfptSOirm+Tb8DWGTixf5B1oGB9i+p1Gf9eJO1Sju2c+ WoRGUwp9S77D64R40QZIpur6vCBr9QpFFrU6OugJS1iQEeWuwQhJKxMLvestSVnUNnbRvj /e9hd5smTYG9Kcz+QzdQHEOVvWCy6LvhsvoGblBIGHsGK0ijW7xik+wkAUCaGkQqW1v5FA 9L2tNK1f3rZHo5pUAbxtwMVDfyWz9lwK49gf2HedvdzMta74ZsMVKYVKj5UJbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1648474319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lZnaM8tEa62v7OTMANBpwBQv0xXQ5xfXOM6IT4G7ums=; b=MAiKXScs+TQA6NI7PYRXJw6CpA3cnfsiTly/i7dQlLd2DUB1vq507qasK8kskCr1oxZ/wT SR3MvTAYykK6GPQOXvkNtv7Je9oV+ocihUOFdk1HEPY2F1fohb25oYv0w38Bo/Bom3bGCX U1ysrd+Wl5jliAqemAfQQJ6AY6DFChb/UNjt3YnbMLmNcU5iyYU2IAMLgRT/AtYlzRGWoI XgKb1jIlzJSmvHO3l0OeCXVV/ETTTXG60kw7/A65hkoHXurI9L1bmgLHOVE1oeDxVjTPUH 2G8KBIVJlVogMc07Res1H02JkB9qci55aXe2FkXK9hIXAtApw7Rrsk51TBWYPw== ARC-Authentication-Results: i=1; rspamd-cf7f5c5bf-zjq4v; auth=pass smtp.auth=a2hosting smtp.mailfrom=pedz@easesoftware.com X-Sender-Id: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com Received: from mi3-ss4.a2hosting.com (mi3-ss4.a2hosting.com [68.66.200.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.106.158.142 (trex/6.5.3); Mon, 28 Mar 2022 13:31:59 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com X-MailChannels-Auth-Id: a2hosting X-Reign-Versed: 0d6c627c0adf0db6_1648474319338_93905287 X-MC-Loop-Signature: 1648474319338:3568305759 X-MC-Ingress-Time: 1648474319338 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=easesoftware.com; s=default; h=Message-Id:In-Reply-To:To:References:Date: Subject:Mime-Version:Content-Type:From:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=lZnaM8tEa62v7OTMANBpwBQv0xXQ5xfXOM6IT4G7ums=; b=Bi5hRGQ1SphIIagUyHU9WqOD6S gUDevBxwPGE93JGkRN4QVxV/lGWt0dO+ktDVV2/3nl34+9FXfbr1V8o+ktbBiD/O8jL+KtvsrDEve i+arGbNghvdybSEoCB2H40+tpiFd0CEB94tFhBNQTXeL5/cWMRAvsKCqOK0D7MPyTJKj7jLqD1qrc fwbOZQDoXAZfKZVLuXC+zBMTW6DrjCDImEBMMBJq74CtUf2chagLe04d511sjczTAvZBGoTCZSHVT m0IsPK96EEqyomhBaXQxvLxx+FqfX5c9yELNgoPg4J4OiB5AJGctunmB4vGGg/90wcPJ4BU6lCm3l groCjwrw==; Received: from cpe-70-94-128-193.satx.res.rr.com ([70.94.128.193]:51913 helo=smtpclient.apple) by mi3-ss4.a2hosting.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nYpTR-0002uf-QZ for zsh-users@zsh.org; Mon, 28 Mar 2022 09:31:57 -0400 From: Perry Smith Content-Type: multipart/alternative; boundary="Apple-Mail=_6321A00D-BC66-4360-831D-8B114E31801C" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.80.82.1.1\)) Subject: Re: Parallel processing Date: Mon, 28 Mar 2022 08:31:57 -0500 References: <1E0E1226-E3E8-40AD-87CD-93A602B1B08B@easesoftware.com> To: zsh-users@zsh.org In-Reply-To: <1E0E1226-E3E8-40AD-87CD-93A602B1B08B@easesoftware.com> Message-Id: X-Mailer: Apple Mail (2.3696.80.82.1.1) X-AuthUser: pedz+easesoftware.com@mi3-ss4.a2hosting.com X-Seq: 27635 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: --Apple-Mail=_6321A00D-BC66-4360-831D-8B114E31801C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Perhaps I need to think about this another way and that is via = =E2=80=9Clocking=E2=80=9D. =46rom a 10,000 foot view, I could map each target to a file (path) and = do the old fashion practice of putting the PID in the file and do it = atomically and with the ability to detect that the owner of the lock = might have died, etc etc etc. Now just spin off N tasks each with a complete list of targets. The = basic flow would be for each item in the list of targets, see if it is = done. If it isn=E2=80=99t done, see if it is locked. If it isn=E2=80=99t= locked, set the lock (atomically) and then do the processing to create = the target. Now, if I want 4 processes running, I can just start four jobs in the = back ground. > On Mar 24, 2022, at 23:34, Perry Smith wrote: >=20 > Has something like prll (parallel) https://github.com/exzombie/prll = been added to zsh? >=20 > I need to do about 20 commands. Each will take several hours to = perhaps days. I=E2=80=99d like to start some fixed number like 4 jobs = that are running. The others are waiting for one of the others to get = finished. >=20 > I tried creating a Makefile and use the -j option in make but my = targets have spaces in them and make doesn=E2=80=99t like that. (This = is on a BSD system.) >=20 > Thank you, > Perry --Apple-Mail=_6321A00D-BC66-4360-831D-8B114E31801C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Perhaps I need to think about this another way and that is = via =E2=80=9Clocking=E2=80=9D.

=46rom a 10,000 foot view, I could map each target to a file = (path) and do the old fashion practice of putting the PID in the file = and do it atomically and with the ability to detect that the owner of = the lock might have died, etc etc etc.

Now just spin off N tasks each with a = complete list of targets.  The basic flow would be for each item in = the list of targets, see if it is done.  If it isn=E2=80=99t done, = see if it is locked.  If it isn=E2=80=99t locked, set the lock = (atomically) and then do the processing to create the target.

Now, if I want 4 = processes running, I can just start four jobs in the back ground.

On Mar 24, 2022, at 23:34, Perry Smith <pedz@easesoftware.com> wrote:

Has something like = prll (parallel) https://github.com/exzombie/prll been added to = zsh?

I need to do = about 20 commands.  Each will take several hours to perhaps days. =  I=E2=80=99d like to start some fixed number like 4 jobs that are = running.  The others are waiting for one of the others to get = finished.

I = tried creating a Makefile and use the -j option in make but my targets = have spaces in them and make doesn=E2=80=99t like that.  (This is = on a BSD system.)

Thank you,
Perry

= --Apple-Mail=_6321A00D-BC66-4360-831D-8B114E31801C--