From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from primenet.com.au (ns1.primenet.com.au [203.24.36.2]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 3e188d89 for ; Sun, 8 Mar 2020 18:39:58 +0000 (UTC) Received: (qmail 4180 invoked by alias); 8 Mar 2020 18:39:50 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 45519 Received: (qmail 7272 invoked by uid 1010); 8 Mar 2020 18:39:50 -0000 X-Qmail-Scanner-Diagnostics: from nef2.ens.fr by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.102.2/25744. spamassassin: 3.4.2. Clear:RC:0(129.199.96.40):SA:0(-4.2/5.0):. Processed in 1.730986 secs); 08 Mar 2020 18:39:50 -0000 X-Envelope-From: ware@phare.normalesup.org X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: pass (ns1.primenet.com.au: SPF record at ens.fr designates 129.199.96.40 as permitted sender) X-ENS-nef-client: 129.199.129.80 Date: Sun, 8 Mar 2020 19:39:07 +0100 From: Cedric Ware To: dana Cc: "zsh-workers@zsh.org" Subject: Re: [PATCH] Enable sub-second timeout in zsystem flock Message-ID: <20200308183907.mxnhqrr2uflwooax@phare.normalesup.org> References: <20190729203521.upp5ku3bsr3hsnxq@phare.normalesup.org> <20200104184734.ienw42rwq2xu6aap@phare.normalesup.org> <43775C64-0254-45AB-81AB-B04AE80C4416@dana.is> <20200106173030.eb2pg4rhhgysh35r@phare.normalesup.org> <20200111154143.fjtwgfnztqfmkyda@phare.normalesup.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20170113 (1.7.2) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nef.ens.fr [129.199.96.32]); Sun, 08 Mar 2020 19:39:08 +0100 (CET) Hello, dana (Saturday 2020-01-11): > Here's a basic, almost-does-nothing test script for it. Obv need to change the > expected error messages in the second test to whatever you decide on. You can > run it with `make check TESTNUM=V13` Thank you. Please find below a revised patch against 5.8. This is still to enable fractional-seconds timeout and retry interval for zsystem flock. One thing about the tests: I've used the (...)& syntax to launch a sub-shell in the background for concurrency checks. But does the test script wait for sub-processes to finish between tests? If not, then we may need to add a 0.1-second sleep after the 3rd test ("zsystem flock unsuccessful wait test") because it starts a sub-shell that waits for 0.2 seconds ("zselect -t 20") but the rest of the test times out on purpose after only 0.11 seconds; so the sub-shell will still be holding the lock for 0.09 seconds. Also note that the test script requires zselect in addition to zsystem, because I needed something that would reliably sleep a fractional number of seconds. The regular sleep command isn't guaranteed to support that, AFAIK. Thanks, best regards, Cedric Ware. diff -ruN zsh-5.8.orig/Doc/Zsh/mod_system.yo zsh-5.8/Doc/Zsh/mod_system.yo --- zsh-5.8.orig/Doc/Zsh/mod_system.yo 2020-01-13 06:39:15.000000000 +0100 +++ zsh-5.8/Doc/Zsh/mod_system.yo 2020-03-08 18:39:32.672683779 +0100 @@ -166,7 +166,7 @@ printed in the last case, but the parameter tt(ERRNO) will reflect the error that occurred. ) -xitem(tt(zsystem flock) [ tt(-t) var(timeout) ] [ tt(-f) var(var) ] [tt(-er)] var(file)) +xitem(tt(zsystem flock) [ tt(-t) var(timeout) ] [ tt(-i) var(interval) ] [ tt(-f) var(var) ] [tt(-er)] var(file)) item(tt(zsystem flock -u) var(fd_expr))( The builtin tt(zsystem)'s subcommand tt(flock) performs advisory file locking (via the manref(fcntl)(2) system call) over the entire contents @@ -196,9 +196,16 @@ By default the shell waits indefinitely for the lock to succeed. The option tt(-t) var(timeout) specifies a timeout for the lock in -seconds; currently this must be an integer. The shell will attempt -to lock the file once a second during this period. If the attempt -times out, status 2 is returned. +seconds; fractional seconds are allowed. During this period, the +shell will attempt to lock the file every var(interval) seconds +if the tt(-i) var(interval) option is given, otherwise once a second. +(This var(interval) is shortened before the last attempt if needed, +so that the shell waits only until the var(timeout) and not longer.) +If the attempt times out, status 2 is returned. + +(Note: var(interval) must be less than LONG_MAX microseconds. +This is many millenia on 64-bit systems, but only about 35 minutes +on 32-bit systems.) If the option tt(-e) is given, the file descriptor for the lock is preserved when the shell uses tt(exec) to start a new process; diff -ruN zsh-5.8.orig/Src/Modules/system.c zsh-5.8/Src/Modules/system.c --- zsh-5.8.orig/Src/Modules/system.c 2020-02-06 20:53:18.000000000 +0100 +++ zsh-5.8/Src/Modules/system.c 2020-03-08 18:43:02.756951315 +0100 @@ -532,6 +532,9 @@ { int cloexec = 1, unlock = 0, readlock = 0; zlong timeout = -1; + double timeout_tmp; + long timeout_retry = 1e6; + mnumber timeout_param; char *fdvar = NULL; #ifdef HAVE_FCNTL_H struct flock lck; @@ -583,7 +586,44 @@ } else { optarg = *args++; } - timeout = mathevali(optarg); + timeout_param = matheval(optarg); + if (!(timeout_param.type & MN_FLOAT)) { + timeout_param.type = MN_FLOAT; + timeout_param.u.d = (double)timeout_param.u.l; + } + timeout_tmp = timeout_param.u.d * 1e6; + if ((timeout_tmp < 1) || (timeout_tmp > ZLONG_MAX / 2)) { + zwarnnam(nam, "flock: invalid timeout value"); + return 1; + } + timeout = (zlong)timeout_tmp; + break; + + case 'i': + /* retry interval in seconds */ + if (optptr[1]) { + optarg = optptr + 1; + optptr += strlen(optarg) - 1; + } else if (!*args) { + zwarnnam(nam, + "flock: option %c requires " + "a numeric retry interval", + opt); + return 1; + } else { + optarg = *args++; + } + timeout_param = matheval(optarg); + if (!(timeout_param.type & MN_FLOAT)) { + timeout_param.type = MN_FLOAT; + timeout_param.u.d = (double)timeout_param.u.l; + } + timeout_tmp = timeout_param.u.d * 1e6; + if ((timeout_tmp < 1) || (timeout_tmp > LONG_MAX)) { + zwarnnam(nam, "flock: invalid interval value"); + return 1; + } + timeout_retry = (long)timeout_tmp; break; case 'u': @@ -647,7 +687,8 @@ lck.l_len = 0; /* lock the whole file */ if (timeout > 0) { - time_t end = time(NULL) + (time_t)timeout; + zlong now; + zlong end = time_clock_us() + timeout; while (fcntl(flock_fd, F_SETLK, &lck) < 0) { if (errflag) { zclose(flock_fd); @@ -658,11 +699,15 @@ zwarnnam(nam, "failed to lock file %s: %e", args[0], errno); return 1; } - if (time(NULL) >= end) { + now = time_clock_us(); + if (now >= end) { zclose(flock_fd); return 2; } - sleep(1); + if (now + timeout_retry > end) { + timeout_retry = end - now; + } + zsleep(timeout_retry); } } else { while (fcntl(flock_fd, timeout == 0 ? F_SETLK : F_SETLKW, &lck) < 0) { diff -ruN zsh-5.8.orig/Src/utils.c zsh-5.8/Src/utils.c --- zsh-5.8.orig/Src/utils.c 2020-01-13 06:39:15.000000000 +0100 +++ zsh-5.8/Src/utils.c 2020-03-08 18:43:02.756951315 +0100 @@ -2745,6 +2745,26 @@ } /* + * Return the current time in microseconds, using the system's + * monotonic clock if supported, the wall clock if not. + */ + +/**/ +zlong +time_clock_us(void) +{ +#if defined(HAS_CLOCK_GETTIME) && defined(CLOCK_MONOTONIC) + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return ts.tv_sec * (zlong)1e6 + ts.tv_nsec / 1000; +#else + struct timeval tv; + gettimeofday(&tv, NULL); + return tv.tv_sec * (zlong)1e6 + tv.tv_usec; +#endif +} + +/* * Sleep for the given number of microseconds --- must be within * range of a long at the moment, but this is only used for * limited internal purposes. diff -ruN zsh-5.8.orig/Test/V13system.ztst zsh-5.8/Test/V13system.ztst --- zsh-5.8.orig/Test/V13system.ztst 1970-01-01 01:00:00.000000000 +0100 +++ zsh-5.8/Test/V13system.ztst 2020-03-08 19:04:56.550666311 +0100 @@ -0,0 +1,98 @@ +# Test zsh/system module + +%prep + + if zmodload -s zsh/system && zmodload -s zsh/zselect; then + tst_dir=V13.tmp + mkdir -p -- $tst_dir + else + ZTST_unimplemented='the zsh/system and zsh/zselect modules are not available' + fi + +%test + + ( + : > $tst_dir/file + zsystem flock -t 0.1 -i 0.000001 $tst_dir/file + ) +0:zsystem flock valid time arguments + + ( + zsystem flock -t -1 $tst_dir/file || + zsystem flock -t 0.49e-6 $tst_dir/file || + zsystem flock -t 1e100 $tst_dir/file || + zsystem flock -i -1 $tst_dir/file || + zsystem flock -i 0.49e-6 $tst_dir/file || + zsystem flock -i 1e100 $tst_dir/file + ) +1:zsystem flock invalid time arguments +?(eval):zsystem:2: flock: invalid timeout value +?(eval):zsystem:3: flock: invalid timeout value +?(eval):zsystem:4: flock: invalid timeout value +?(eval):zsystem:5: flock: invalid interval value +?(eval):zsystem:6: flock: invalid interval value +?(eval):zsystem:7: flock: invalid interval value + + ( + (zsystem flock $tst_dir/file && zselect -t 20) & + zselect -t 1 + zsystem flock -t 0.1 $tst_dir/file + ) +2:zsystem flock unsuccessful wait test + + ( + typeset -F SECONDS + start=$SECONDS + (zsystem flock $tst_dir/file && zselect -t 50) & + zselect -t 1 + if zsystem flock $tst_dir/file; then + elapsed=$[ $SECONDS - $start ] + if [[ $elapsed -ge 0.3 && $elapsed -le 0.7 ]]; then + echo "elapsed time seems OK" 1>&2 + else + echo "elapsed time $elapsed should be ~ 0.5 second" 1>&2 + fi + else + false + fi + ) +0:zsystem flock successful wait test, no timeout +?elapsed time seems OK + + ( + typeset -F SECONDS + start=$SECONDS + (zsystem flock $tst_dir/file && zselect -t 50) & + zselect -t 1 + if zsystem flock -t 1 $tst_dir/file; then + elapsed=$[ $SECONDS - $start ] + if [[ $elapsed -ge 0.9 && $elapsed -le 1.1 ]]; then + echo "elapsed time seems OK" 1>&2 + else + echo "elapsed time $elapsed should be ~ 1.01 second" 1>&2 + fi + else + false + fi + ) +0:zsystem flock successful wait test, integral seconds +?elapsed time seems OK + + ( + typeset -F SECONDS + start=$SECONDS + (zsystem flock $tst_dir/file && zselect -t 25) & + zselect -t 1 + if zsystem flock -t 0.4 -i 0.1 $tst_dir/file; then + elapsed=$[ $SECONDS - $start ] + if [[ $elapsed -ge 0.2 && $elapsed -le 0.5 ]]; then + echo "elapsed time seems OK" 1>&2 + else + echo "elapsed time $elapsed should be ~ 0.3 second" 1>&2 + fi + else + false + fi + ) +0:zsystem flock successful wait test, fractional seconds +?elapsed time seems OK