From: Perry Smith <email@example.com> To: firstname.lastname@example.org Subject: Re: Parallel processing Date: Mon, 28 Mar 2022 08:31:57 -0500 [thread overview] Message-ID: <C05C228C-07DE-4B43-BE27-0FABD5B6258C@easesoftware.com> (raw) In-Reply-To: <1E0E1226-E3E8-40AD-87CD-93A602B1B08B@easesoftware.com> [-- Attachment #1: Type: text/plain, Size: 1304 bytes --] Perhaps I need to think about this another way and that is via “locking”. From a 10,000 foot view, I could map each target to a file (path) and do the old fashion practice of putting the PID in the file and do it atomically and with the ability to detect that the owner of the lock might have died, etc etc etc. Now just spin off N tasks each with a complete list of targets. The basic flow would be for each item in the list of targets, see if it is done. If it isn’t done, see if it is locked. If it isn’t locked, set the lock (atomically) and then do the processing to create the target. Now, if I want 4 processes running, I can just start four jobs in the back ground. > On Mar 24, 2022, at 23:34, Perry Smith <email@example.com> wrote: > > Has something like prll (parallel) https://github.com/exzombie/prll <https://github.com/exzombie/prll> been added to zsh? > > I need to do about 20 commands. Each will take several hours to perhaps days. I’d like to start some fixed number like 4 jobs that are running. The others are waiting for one of the others to get finished. > > I tried creating a Makefile and use the -j option in make but my targets have spaces in them and make doesn’t like that. (This is on a BSD system.) > > Thank you, > Perry [-- Attachment #2: Type: text/html, Size: 2269 bytes --]
next prev parent reply other threads:[~2022-03-28 13:33 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-25 4:34 Perry Smith 2022-03-25 18:27 ` Bart Schaefer 2022-03-26 18:10 ` Philippe Troin 2022-03-26 22:19 ` Bart Schaefer 2022-03-27 17:32 ` Philippe Troin 2022-03-27 17:42 ` Bart Schaefer 2022-03-27 19:23 ` Dominik Vogt 2022-03-28 13:26 ` Perry Smith 2022-03-28 13:31 ` Perry Smith [this message] 2022-03-26 4:34 jdh 2022-03-27 18:37 jdh 2022-03-27 19:06 ` Bart Schaefer 2022-03-27 19:24 ` Ray Andrews 2022-03-28 13:47 ` Perry Smith
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=C05C228C-07DE-4B43-BE27-0FABD5B6258C@easesoftware.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: Parallel processing' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).