From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11225 invoked by alias); 24 Feb 2010 21:27:05 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 27754 Received: (qmail 8058 invoked from network); 24 Feb 2010 21:26:57 -0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 Received-SPF: pass (ns1.primenet.com.au: SPF record at ntlworld.com designates 81.103.221.47 as permitted sender) Date: Wed, 24 Feb 2010 21:26:44 +0000 From: Peter Stephenson To: zsh-workers@zsh.org Subject: Re: locking a file in zsh? Message-ID: <20100224212644.13de30f6@pws-pc> In-Reply-To: <100223194151.ZM28002@torch.brasslantern.com> References: <19331.60495.60802.575961@fisica.ufpr.br> <20100223220550.1d934b0c@pws-pc> <100223194151.ZM28002@torch.brasslantern.com> X-Mailer: Claws Mail 3.7.4 (GTK+ 2.18.7; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Cloudmark-Analysis: v=1.1 cv=1ggfb5FlKZQUfF3vzm9UBYZ2uTfLsbs/8dSljwg5+mE= c=1 sm=0 a=q2GGsy2AAAAA:8 a=NLZqzBF-AAAA:8 a=juaOhcPklIXnf-RMYU8A:9 a=pBPu8DzgwGz1CiAuw6MA:7 a=tSewPx_xxAMl26MvSHiW5c6Sn8UA:4 a=I6wTmPyJxzYA:10 a=_dQi-Dcv4p4A:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 On Tue, 23 Feb 2010 19:41:51 -0800 Bart Schaefer wrote: > On Feb 23, 10:05pm, Peter Stephenson wrote: > } > } Here's a *very* minimal implementation that basically copies Wayne's > } code from hist.c to a new builtin, "zsystem", in the zsh/system module, as > } the subcommand "flock". > > zsystem flock /tmp/zsystem_filelock # I believe you meant? (Moved this to zsh-workers.) Sorry, my follow up correction got lost in mail queue land. Here's a less minimal implementation with documentation (but yesterday's example as corrected still works as it did). The only obvious further extension I can see is to allow the timeout to include decimals, but that needs a generic replacement for usleep / nanosleep which is another job, which might be useful in a "zsystem sleep" command. Index: Doc/Zsh/mod_system.yo =================================================================== RCS file: /cvsroot/zsh/zsh/Doc/Zsh/mod_system.yo,v retrieving revision 1.5 diff -p -u -r1.5 mod_system.yo --- Doc/Zsh/mod_system.yo 26 Jan 2009 09:49:40 -0000 1.5 +++ Doc/Zsh/mod_system.yo 24 Feb 2010 21:22:41 -0000 @@ -1,8 +1,8 @@ COMMENT(!MOD!zsh/system A builtin interface to various low-level system features. !MOD!) -The tt(zsh/system) module makes available three builtin commands and -two parameters. +The tt(zsh/system) module makes available various builtin commands and +parameters. subsect(Builtins) @@ -109,6 +109,45 @@ to the command, or 2 for an error on the printed in the last case, but the parameter tt(ERRNO) will reflect the error that occurred. ) +xitem(tt(zsystem flock [ -t) var(timeout) tt(] [ -f) var(var) tt(] [-er]) var(file)) +item(tt(zsystem flock -u) var(fd_expr))( +The builtin tt(zsystem)'s subcommand tt(flock) performs advisory file +locking (via the manref(fcntl)(2) system call) over the entire contents +of the given file. This form of locking requires the processes +accessing the file to cooperate; its most obvious use is between two +instances of the shell itself. + +In the first form the named var(file), which must already exist, is +locked by opening a file descriptor to the file and applying a lock to +the file descriptor. The lock terminates when the shell process that +created the lock exits; it is therefore often convenient to create file +locks within subshells, since the lock is automatically released when +the subshell exits. Status 0 is returned if the lock succeeds, else +status 1. + +In the second form the file descriptor given by the arithmetic +expression tt(fd_expr) is closed, releasing a lock. The file descriptor +can be queried by using the `tt(-f) var(var)' form during the lock; +on a successful lock, the shell variable var(var) is set to the file +descriptor used for locking. The lock will be released if the +file descriptor is closed by any other means, for example using +`tt(exec {)var(var)tt(}>&-)'; however, the form described here performs +a safety check that the file descriptor is in use for file locking. + +By default the shell waits indefinitely for the lock to succeed. +The option tt(-t) var(timeout) specifies a timeout for the lock in +seconds; currently this must be an integer. The shell will attempt +to lock the file once a second during this period. If the attempt +times out, status 2 is returned. + +If the option tt(-e) is given, the file descriptor for the lock is +preserved when the shell uses tt(exec) to start a new process; +otherwise it is closed at that point and the lock released. + +If the option tt(-r) is given, the lock is only for reading, otherwise +it is for reading and writing. The file descriptor is opened +accordingly. +) enditem() subsect(Parameters) Index: Src/exec.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/exec.c,v retrieving revision 1.176 diff -p -u -r1.176 exec.c --- Src/exec.c 22 Feb 2010 10:12:31 -0000 1.176 +++ Src/exec.c 24 Feb 2010 21:22:42 -0000 @@ -136,6 +136,12 @@ mod_export int coprocin; /**/ mod_export int coprocout; +/* count of file locks recorded in fdtable */ + +/**/ +int fdtable_flocks; + + /* != 0 if the line editor is active */ /**/ @@ -2716,7 +2722,8 @@ execcmd(Estate state, int input, int out if ((how & Z_ASYNC) || (!do_exec && (((is_builtin || is_shfunc) && output) || - (!is_cursh && (last1 != 1 || nsigtrapped || havefiles()))))) { + (!is_cursh && (last1 != 1 || nsigtrapped || havefiles() || + fdtable_flocks))))) { pid_t pid; int synch[2], flags; Index: Src/utils.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/utils.c,v retrieving revision 1.239 diff -p -u -r1.239 utils.c --- Src/utils.c 22 Feb 2010 11:36:42 -0000 1.239 +++ Src/utils.c 24 Feb 2010 21:22:42 -0000 @@ -1691,13 +1691,43 @@ redup(int x, int y) } else { check_fd_table(y); fdtable[y] = fdtable[x]; + if (fdtable[y] == FDT_FLOCK || fdtable[y] == FDT_FLOCK_EXEC) + fdtable[y] = FDT_INTERNAL; } + /* + * Closing any fd to the locked file releases the lock. + * This isn't expected to happen, it's here for completeness. + */ + if (fdtable[x] == FDT_FLOCK) + fdtable_flocks--; zclose(x); } return ret; } +/* + * Indicate that an fd has a file lock; if cloexec is 1 it will be closed + * on exec. + * The fd should already be known to fdtable (e.g. by movefd). + * Note the fdtable code doesn't care what sort of lock + * is used; this simply prevents the main shell exiting prematurely + * when it holds a lock. + */ + +/**/ +mod_export void +addlockfd(int fd, int cloexec) +{ + if (cloexec) { + if (fdtable[fd] != FDT_FLOCK) + fdtable_flocks++; + fdtable[fd] = FDT_FLOCK; + } else { + fdtable[fd] = FDT_FLOCK_EXEC; + } +} + /* Close the given fd, and clear it from fdtable. */ /**/ @@ -1713,6 +1743,8 @@ zclose(int fd) */ DPUTS2(fd > max_zsh_fd && fdtable[fd] != FDT_UNUSED, "BUG: fd is %d, max_zsh_fd is %d", fd, max_zsh_fd); + if (fdtable[fd] == FDT_FLOCK) + fdtable_flocks--; fdtable[fd] = FDT_UNUSED; while (max_zsh_fd > 0 && fdtable[max_zsh_fd] == FDT_UNUSED) max_zsh_fd--; @@ -1725,6 +1757,22 @@ zclose(int fd) return -1; } +/* + * Close an fd returning 0 if used for locking; return -1 if it isn't. + */ + +/**/ +mod_export int +zcloselockfd(int fd) +{ + if (fd > max_zsh_fd) + return -1; + if (fdtable[fd] != FDT_FLOCK && fdtable[fd] != FDT_FLOCK_EXEC) + return -1; + zclose(fd); + return 0; +} + #ifdef HAVE__MKTEMP extern char *_mktemp(char *); #endif Index: Src/zsh.h =================================================================== RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v retrieving revision 1.162 diff -p -u -r1.162 zsh.h --- Src/zsh.h 27 Jan 2010 19:25:34 -0000 1.162 +++ Src/zsh.h 24 Feb 2010 21:22:43 -0000 @@ -350,6 +350,15 @@ enum { * Entry used by output from the XTRACE option. */ #define FDT_XTRACE 3 +/* + * Entry used for file locking. + */ +#define FDT_FLOCK 4 +/* + * As above, but the fd is not marked for closing on exec, + * so the shell can still exec the last process. + */ +#define FDT_FLOCK_EXEC 5 #ifdef PATH_DEV_FD /* * Entry used by a process substition. @@ -357,7 +366,7 @@ enum { * decremented on exit; we don't close entries greater than * FDT_PROC_SUBST except when closing everything. */ -#define FDT_PROC_SUBST 4 +#define FDT_PROC_SUBST 6 #endif /* Flags for input stack */ Index: Src/Modules/system.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/Modules/system.c,v retrieving revision 1.9 diff -p -u -r1.9 system.c --- Src/Modules/system.c 6 Jul 2007 21:52:40 -0000 1.9 +++ Src/Modules/system.c 24 Feb 2010 21:22:43 -0000 @@ -173,7 +173,7 @@ bin_sysread(char *nam, char **args, Opti select_tv.tv_usec = 0; } - while ((ret = select(infd+1, (SELECT_ARG_2_T) &fds, + while ((ret = select(infd+1, (SELECT_ARG_2_T) &fds, NULL, NULL,&select_tv)) < 1) { if (errno != EINTR || errflag || retflag || breaks || contflag) break; @@ -340,10 +340,174 @@ bin_syserror(char *nam, char **args, Opt return 0; } +/**/ +static int +bin_zsystem_flock(char *nam, char **args, UNUSED(Options ops), UNUSED(int func)) +{ + int cloexec = 1, unlock = 0, readlock = 0; + time_t timeout = 0; + char *fdvar = NULL; +#ifdef HAVE_FCNTL_H + struct flock lck; + int flock_fd, flags; +#endif + + while (*args && **args == '-') { + int opt; + char *optptr = *args + 1, *optarg; + args++; + if (!*optptr || !strcmp(optptr, "-")) + break; + while ((opt = *optptr)) { + switch (opt) { + case 'e': + /* keep lock on "exec" */ + cloexec = 0; + break; + + case 'f': + /* variable for fd */ + if (optptr[1]) { + fdvar = optptr + 1; + optptr += strlen(fdvar) - 1; + } else if (*args) { + fdvar = *args++; + } + if (fdvar == NULL || !isident(fdvar)) { + zwarnnam(nam, "flock: option %c requires a variable name", + opt); + return 1; + } + break; + + case 'r': + /* read lock rather than read-write lock */ + readlock = 1; + break; + + case 't': + /* timeout in seconds */ + if (optptr[1]) { + optarg = optptr + 1; + optptr += strlen(optarg) - 1; + } else if (!*args) { + zwarnnam(nam, "flock: option %c requires a numeric timeout", + opt); + return 1; + } else { + optarg = *args++; + } + timeout = (time_t)mathevali(optarg); + break; + + case 'u': + /* unlock: argument is fd */ + unlock = 1; + break; + + default: + zwarnnam(nam, "flock: unknown option: %c", *optptr); + return 1; + } + optptr++; + } + } + + + if (!args[0]) { + zwarnnam(nam, "flock: not enough arguments"); + return 1; + } + if (args[1]) { + zwarnnam(nam, "flock: too many arguments"); + return 1; + } + +#ifdef HAVE_FCNTL_H + if (unlock) { + flock_fd = (int)mathevali(args[0]); + if (zcloselockfd(flock_fd) < 0) { + zwarnnam(nam, "flock: file descriptor %d not in use for locking", + flock_fd); + return 1; + } + return 0; + } + + if (readlock) + flags = O_RDONLY | O_NOCTTY; + else + flags = O_RDWR | O_NOCTTY; + if ((flock_fd = open(unmeta(args[0]), flags)) < 0) { + zwarnnam(nam, "failed to open %s for writing: %e", args[0], errno); + return 1; + } + flock_fd = movefd(flock_fd); + if (flock_fd == -1) + return 1; +#ifdef FD_CLOEXEC + if (cloexec) + { + long fdflags = fcntl(flock_fd, F_GETFD, 0); + if (fdflags != (long)-1) + fcntl(flock_fd, F_SETFD, fdflags | FD_CLOEXEC); + } +#endif + addlockfd(flock_fd, cloexec); + + lck.l_type = readlock ? F_RDLCK : F_WRLCK; + lck.l_whence = SEEK_SET; + lck.l_start = 0; + lck.l_len = 0; /* lock the whole file */ + + if (timeout > 0) { + time_t end = time(NULL) + (time_t)timeout; + while (fcntl(flock_fd, F_SETLK, &lck) < 0) { + if (errno != EINTR && errno != EACCES && errno != EAGAIN) { + zwarnnam(nam, "failed to lock file %s: %e", args[0], errno); + return 1; + } + if (time(NULL) >= end) + return 2; + sleep(1); + } + } else { + while (fcntl(flock_fd, F_SETLKW, &lck) < 0) { + if (errno == EINTR) + continue; + zwarnnam(nam, "failed to lock file %s: %e", args[0], errno); + return 1; + } + } + + if (fdvar) + setiparam(fdvar, flock_fd); + + return 0; +#else /* HAVE_FCNTL_H */ + zwarnnam(nam, "flock: not implemented on this system"); + return 255; +#endif /* HAVE_FCNTL_H */ +} + + +/**/ +static int +bin_zsystem(char *nam, char **args, Options ops, int func) +{ + /* If more commands are implemented, this can be more sophisticated */ + if (!strcmp(*args, "flock")) { + return bin_zsystem_flock(nam, args+1, ops, func); + } + zwarnnam(nam, "unknown subcommand: %s", *args); + return 1; +} + static struct builtin bintab[] = { BUILTIN("syserror", 0, bin_syserror, 0, 1, 0, "e:p:", NULL), BUILTIN("sysread", 0, bin_sysread, 0, 1, 0, "c:i:o:s:t:", NULL), BUILTIN("syswrite", 0, bin_syswrite, 1, 1, 0, "c:o:", NULL), + BUILTIN("zsystem", 0, bin_zsystem, 1, -1, 0, NULL, NULL) }; -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/