File locking within zsh?

File locking within zsh?

[Lloyd Zusman]

Do any of you know of any functions, primitives, tricks, hacks, or even
outright abominations which will allow me to do cooperative file locking
from within zsh?

I know that I can do this with a number of compiled executables, but I'm
looking for a zsh-only solution.

Assuming some sort of zsh locking operator called "lock", consider
this example (within a zsh script):

  lock -x -t 0 file  # for this example of a hypothetical operator, '-x'
                     # means to wait until I get an exclusive lock, and
                     # '-t 0' means no time out
  print foo bar baz >>file
  # do a whole lot of other stuff to "file"
  unlock file # release the lock

In this example, any other zsh script which asks for an exclusive lock
on "file" using this hypothetical "lock" operator will block until the
"unlock" operator has been invoked.

Can this be done somehow in zsh, or do I have to rely on a compiled
executable to accomplish this?

Thanks in advance.

-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: File locking within zsh?

[Tim Writer]

Lloyd Zusman <ljz@asfast.com> writes:

> Do any of you know of any functions, primitives, tricks, hacks, or even
> outright abominations which will allow me to do cooperative file locking
> from within zsh?
> 
> I know that I can do this with a number of compiled executables, but I'm
> looking for a zsh-only solution.
> 
> Assuming some sort of zsh locking operator called "lock", consider
> this example (within a zsh script):
> 
>   lock -x -t 0 file  # for this example of a hypothetical operator, '-x'
>                      # means to wait until I get an exclusive lock, and
>                      # '-t 0' means no time out
>   print foo bar baz >>file
>   # do a whole lot of other stuff to "file"
>   unlock file # release the lock
> 
> In this example, any other zsh script which asks for an exclusive lock
> on "file" using this hypothetical "lock" operator will block until the
> "unlock" operator has been invoked.
> 
> Can this be done somehow in zsh, or do I have to rely on a compiled
> executable to accomplish this?

The usual way to lock within shell scripts is to use ln. This works on all
UNIX like systems because creating a hard link (with ln) is an atomic
operation which fails if the target already exists. Your example above can be
written like this:

    while ! ln file file.lock 2>/dev/null
    do
        sleep 1
    done
    # Lock obtained

    print foo bar baz >>file
    # do a whole lot of other stuff to "file"

    rm -f file.lock
    # Lock released

Wrapping this idiom into lock/unlock functions is left as an exercise for the
reader. :-)

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Re: File locking within zsh?

[Brian K. White]

----- Original Message ----- 
From: "Tim Writer" <tim@starnix.com>
To: "Lloyd Zusman" <ljz@asfast.com>
Cc: <zsh-users@sunsite.dk>
Sent: Tuesday, May 09, 2006 11:31 PM
Subject: Re: File locking within zsh?


> Lloyd Zusman <ljz@asfast.com> writes:
>
>> Do any of you know of any functions, primitives, tricks, hacks, or even
>> outright abominations which will allow me to do cooperative file locking
>> from within zsh?
>>
>> I know that I can do this with a number of compiled executables, but I'm
>> looking for a zsh-only solution.
>>
>> Assuming some sort of zsh locking operator called "lock", consider
>> this example (within a zsh script):
>>
>>   lock -x -t 0 file  # for this example of a hypothetical operator, '-x'
>>                      # means to wait until I get an exclusive lock, and
>>                      # '-t 0' means no time out
>>   print foo bar baz >>file
>>   # do a whole lot of other stuff to "file"
>>   unlock file # release the lock
>>
>> In this example, any other zsh script which asks for an exclusive lock
>> on "file" using this hypothetical "lock" operator will block until the
>> "unlock" operator has been invoked.
>>
>> Can this be done somehow in zsh, or do I have to rely on a compiled
>> executable to accomplish this?
>
> The usual way to lock within shell scripts is to use ln. This works on all
> UNIX like systems because creating a hard link (with ln) is an atomic
> operation which fails if the target already exists. Your example above can 
> be
> written like this:
>
>    while ! ln file file.lock 2>/dev/null
>    do
>        sleep 1
>    done
>    # Lock obtained
>
>    print foo bar baz >>file
>    # do a whole lot of other stuff to "file"
>
>    rm -f file.lock
>    # Lock released
>
> Wrapping this idiom into lock/unlock functions is left as an exercise for 
> the
> reader. :-)


Is that better or worse, and why, than  umask 222 ; >file ?

http://www.unix.org.ua/orelly/unix/upt/ch45_36.htm


Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!

Re: File locking within zsh?

[Lloyd Zusman]

"Brian K. White" <brian@aljex.com> writes:

> From: "Tim Writer" <tim@starnix.com>
>
>> [ ... ]
>>
>>    while ! ln file file.lock 2>/dev/null
>>    do
>>        sleep 1
>>    done
>>    # Lock obtained
>>
>>    print foo bar baz >>file
>>    # do a whole lot of other stuff to "file"
>>
>>    rm -f file.lock
>>    # Lock released
>>
>> Wrapping this idiom into lock/unlock functions is left as an exercise
>> for the
>> reader. :-)
>
>
> Is that better or worse, and why, than  umask 222 ; >file ?
>
> http://www.unix.org.ua/orelly/unix/upt/ch45_36.htm

Thanks to both of you (Brian K. White and Tim Writer).  I personally
like the umask version better, because I think I can implement a
variation of it without spawning any extra processes (my goal), given
that umask is an internal zsh command.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: File locking within zsh?

[Tim Writer]

Lloyd Zusman <ljz@asfast.com> writes:

> "Brian K. White" <brian@aljex.com> writes:
> 
> > From: "Tim Writer" <tim@starnix.com>
> >
> >> [ ... ]
> >>
> >>    while ! ln file file.lock 2>/dev/null
> >>    do
> >>        sleep 1
> >>    done
> >>    # Lock obtained
> >>
> >>    print foo bar baz >>file
> >>    # do a whole lot of other stuff to "file"
> >>
> >>    rm -f file.lock
> >>    # Lock released
> >>
> >> Wrapping this idiom into lock/unlock functions is left as an exercise
> >> for the
> >> reader. :-)
> >
> >
> > Is that better or worse, and why, than  umask 222 ; >file ?
> >
> > http://www.unix.org.ua/orelly/unix/upt/ch45_36.htm
> 
> Thanks to both of you (Brian K. White and Tim Writer).  I personally
> like the umask version better, because I think I can implement a
> variation of it without spawning any extra processes (my goal), given
> that umask is an internal zsh command.

Keep in mind that you have to put umask in a subshell, like this:

    ( umask 222; >file )

in order to preserve it. IIRC, zsh avoids forking in subshells but don't most
shells implement subshells with fork()?

Keep in mind that ln is a tiny wrapper around the link(2) system call and so
there's only a very small amount of overhead in running it. With zsh, you
can have your cake and eat it too using the zsh/files which makes ln a
builtin (among other things).

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Re: File locking within zsh?

[Bart Schaefer]

On May 10,  6:04pm, Tim Writer wrote:
}
} IIRC, zsh avoids forking in subshells but don't most
} shells implement subshells with fork()?

Zsh forks for subshells, too.  It may avoid an *additional* fork when
the only thing the subshell does is execute an external command, but
it always forks at least once.

Re: File locking within zsh?

[Tim Writer]

Bart Schaefer <schaefer@brasslantern.com> writes:

> On May 10,  6:04pm, Tim Writer wrote:
> }
> } IIRC, zsh avoids forking in subshells but don't most
> } shells implement subshells with fork()?
> 
> Zsh forks for subshells, too.  It may avoid an *additional* fork when
> the only thing the subshell does is execute an external command, but
> it always forks at least once.

Okay. But I don't understand this:

    tim@ganesh% echo $ZSH_VERSION
    4.2.5
    tim@ganesh% echo $$; ( echo $$ )
    18095
    18095
    tim@ganesh%

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Re: File locking within zsh?

[Peter Stephenson]

"Tim Writer" <tim@starnix.com> wrote:
> Okay. But I don't understand this:
> 
>     tim@ganesh% echo $ZSH_VERSION
>     4.2.5
>     tim@ganesh% echo $$; ( echo $$ )
>     18095
>     18095
>     tim@ganesh%

That's standard shell behaviour:  $$ is supposed to be unique to the parent
shell, not to every subshell instance.  You need tricks to get the actual
pid.  This will work on Linux:

zmodload -i zsh/stat
stat +link /proc/self

getpid() is another thing which could be added to the zsh/system module.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

Re: File locking within zsh?

[Tim Writer]

Peter Stephenson <pws@csr.com> writes:

> "Tim Writer" <tim@starnix.com> wrote:
> > Okay. But I don't understand this:
> > 
> >     tim@ganesh% echo $ZSH_VERSION
> >     4.2.5
> >     tim@ganesh% echo $$; ( echo $$ )
> >     18095
> >     18095
> >     tim@ganesh%
> 
> That's standard shell behaviour:  $$ is supposed to be unique to the parent
> shell, not to every subshell instance.

Thanks for the clarification. I think I knew that at one time, it's amazing
the things I forget. :-)

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Subshells and parameters (was: File locking within zsh?)

[Vincent Lefevre]

On 2006-05-11 16:01:23 +0100, Peter Stephenson wrote:
> "Tim Writer" <tim@starnix.com> wrote:
> > Okay. But I don't understand this:
> > 
> >     tim@ganesh% echo $ZSH_VERSION
> >     4.2.5
> >     tim@ganesh% echo $$; ( echo $$ )
> >     18095
> >     18095
> >     tim@ganesh%
> 
> That's standard shell behaviour: $$ is supposed to be unique to the
> parent shell, not to every subshell instance. [...]

The zshparam man page says:

       $ <S>  The process ID of this shell.

IMHO, it should give more details, e.g.: in subshells, the value
of this parameter is not changed. Ditto for $PPID. Possibly add a
section about subshells in the man pages.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Subshells and parameters

[Lloyd Zusman]

Vincent Lefevre <vincent@vinc17.org> writes:

> On 2006-05-11 16:01:23 +0100, Peter Stephenson wrote:
>>
>> [ ... ]
>>
>> That's standard shell behaviour: $$ is supposed to be unique to the
>> parent shell, not to every subshell instance. [...]
>
> The zshparam man page says:
>
>        $ <S>  The process ID of this shell.
>
> IMHO, it should give more details, e.g.: in subshells, the value
> of this parameter is not changed. Ditto for $PPID. Possibly add a
> section about subshells in the man pages.

So how can we get the process ID of any given subshell?

-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Subshells and parameters

[Vincent Lefevre]

On 2006-05-12 08:09:26 -0400, Lloyd Zusman wrote:
> So how can we get the process ID of any given subshell?

dixsept:~> cat > getpid.c
#include <stdio.h>
#include <unistd.h>
int main(void)
{
  printf("%d\n", (int) getppid());
  return 0;
}
dixsept:~> gcc -Wall -O2 getpid.c -o getpid
dixsept:~> echo $$; ./getpid
13372
13372
dixsept:~> (echo $$; ./getpid)
13372
14153
dixsept:~> (stat +link /proc/self; ./getpid)
14168
14168

Note: as Peter said, /proc is specific to Linux.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Subshells and parameters

[Lloyd Zusman]

Vincent Lefevre <vincent@vinc17.org> writes:

> On 2006-05-12 08:09:26 -0400, Lloyd Zusman wrote:
>> So how can we get the process ID of any given subshell?
>
> dixsept:~> cat > getpid.c
> #include <stdio.h>
> #include <unistd.h>
> int main(void)
> {
>   printf("%d\n", (int) getppid());
>   return 0;
> }
> dixsept:~> gcc -Wall -O2 getpid.c -o getpid
> dixsept:~> echo $$; ./getpid
> 13372
> 13372
> dixsept:~> (echo $$; ./getpid)
> 13372
> 14153
> dixsept:~> (stat +link /proc/self; ./getpid)
> 14168
> 14168
>
> Note: as Peter said, /proc is specific to Linux.

Thanks.  However, just as in my original message in the other thread,
I'm wondering if there's a solution that doesn't require a separate
executable.  In other words, is it possible to get a subshell's pid in a
zsh-only manner, without using anything like the "stat" executable nor
the "getpid" utility described above?

Thanks.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Subshells and parameters

[Vincent Lefevre]

On 2006-05-12 22:05:23 -0400, Lloyd Zusman wrote:
> Thanks. However, just as in my original message in the other thread,
> I'm wondering if there's a solution that doesn't require a separate
> executable. In other words, is it possible to get a subshell's pid
> in a zsh-only manner, without using anything like the "stat"
> executable nor the "getpid" utility described above?

You need a module, but this isn't implemented yet. Peter said:

  getpid() is another thing which could be added to the zsh/system
  module.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Subshells and parameters

[Bart Schaefer]

On May 17,  4:24pm, Vincent Lefevre wrote:
} Subject: Re: Subshells and parameters
}
} On 2006-05-12 22:05:23 -0400, Lloyd Zusman wrote:
} > I'm wondering if there's a solution that doesn't require a separate
} > executable.
} 
} You need a module, but this isn't implemented yet.

As someone pointed out earlier, you can fork a new copy of zsh to find
this information; to split hairs, that's a separate process, but not a
separate executable.

Example:

    getpid() { typeset -g PID=$(exec $ZSH_NAME -fc 'print $PPID') }

The exec is important, otherwise you spawn two processes and get the
PID of the intermediate descendant.

Compare:

    getpid; print $$ $PID
    ( getpid; print $$ $PID )

Re: Subshells and parameters

[Vincent Lefevre]

On 2006-05-17 08:42:54 -0700, Bart Schaefer wrote:
> Example:
> 
>     getpid() { typeset -g PID=$(exec $ZSH_NAME -fc 'print $PPID') }

However, if $PATH has changed, this won't necessarily find the
executable.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Subshells and parameters

[Dan Nelson]

In the last episode (May 13), Vincent Lefevre said:
> On 2006-05-12 08:09:26 -0400, Lloyd Zusman wrote:
> > So how can we get the process ID of any given subshell?
> 
> dixsept:~> cat > getpid.c
> #include <stdio.h>
> #include <unistd.h>
> int main(void)
> {
>   printf("%d\n", (int) getppid());
>   return 0;
> }
> dixsept:~> gcc -Wall -O2 getpid.c -o getpid
> dixsept:~> echo $$; ./getpid
> 13372
> 13372
> dixsept:~> (echo $$; ./getpid)
> 13372
> 14153
> dixsept:~> (stat +link /proc/self; ./getpid)
> 14168
> 14168
> 
> Note: as Peter said, /proc is specific to Linux.

You could even use zsh itself for this:

function getppid() { $SHELL -fc 'echo $PPID' }

-- 
	Dan Nelson
	dnelson@allantgroup.com

Re: File locking within zsh?

[Brian K. White]

----- Original Message ----- 
From: "Tim Writer" <tim@starnix.com>
To: "Bart Schaefer" <schaefer@brasslantern.com>
Cc: <zsh-users@sunsite.dk>
Sent: Thursday, May 11, 2006 10:25 AM
Subject: Re: File locking within zsh?


> Bart Schaefer <schaefer@brasslantern.com> writes:
>
>> On May 10,  6:04pm, Tim Writer wrote:
>> }
>> } IIRC, zsh avoids forking in subshells but don't most
>> } shells implement subshells with fork()?
>>
>> Zsh forks for subshells, too.  It may avoid an *additional* fork when
>> the only thing the subshell does is execute an external command, but
>> it always forks at least once.
>
> Okay. But I don't understand this:
>
>    tim@ganesh% echo $ZSH_VERSION
>    4.2.5
>    tim@ganesh% echo $$; ( echo $$ )
>    18095
>    18095
>    tim@ganesh%

Without special measures, wouldn't both $$ be expanded at the same time 
before the line is executed, before the subshell is forked?

However, I avoid all possible globbing issues and still get your result:
set |(cat;set) |less

'$'=6511
PPID=6474
'$'=6511
PPID=6474

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!

Re: File locking within zsh?

[Vincent Lefevre]

On 2006-05-10 01:59:57 -0400, Brian K. White wrote:
> Is that better or worse, and why, than  umask 222 ; >file ?

Are you sure that this is guaranteed to work with any NFS server?

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: File locking within zsh?

[Brian K. White]

----- Original Message ----- 
From: "Vincent Lefevre" <vincent@vinc17.org>
To: <zsh-users@sunsite.dk>
Sent: Wednesday, May 10, 2006 1:13 PM
Subject: Re: File locking within zsh?


> On 2006-05-10 01:59:57 -0400, Brian K. White wrote:
>> Is that better or worse, and why, than  umask 222 ; >file ?
> 
> Are you sure that this is guaranteed to work with any NFS server?

Not at all sure, never having tried on any nfs mount.

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!

Re: File locking within zsh?

[Vincent Lefevre]

On 2006-05-11 11:45:21 -0400, Brian K. White wrote:
> 
> ----- Original Message ----- 
> From: "Vincent Lefevre" <vincent@vinc17.org>
> To: <zsh-users@sunsite.dk>
> Sent: Wednesday, May 10, 2006 1:13 PM
> Subject: Re: File locking within zsh?
> 
> 
> >On 2006-05-10 01:59:57 -0400, Brian K. White wrote:
> >>Is that better or worse, and why, than  umask 222 ; >file ?
> >
> >Are you sure that this is guaranteed to work with any NFS server?
> 
> Not at all sure, never having tried on any nfs mount.

So, it's probably better to use the symbolic link solution (ln -s),
which is guaranteed to be atomic, even on NFS. This is the solution
chosen by Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=76431

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: File locking within zsh?

[Tim Writer]

Vincent Lefevre <vincent@vinc17.org> writes:

> On 2006-05-11 11:45:21 -0400, Brian K. White wrote:
> > 
> > ----- Original Message ----- 
> > From: "Vincent Lefevre" <vincent@vinc17.org>
> > To: <zsh-users@sunsite.dk>
> > Sent: Wednesday, May 10, 2006 1:13 PM
> > Subject: Re: File locking within zsh?
> > 
> > 
> > >On 2006-05-10 01:59:57 -0400, Brian K. White wrote:
> > >>Is that better or worse, and why, than  umask 222 ; >file ?
> > >
> > >Are you sure that this is guaranteed to work with any NFS server?
> > 
> > Not at all sure, never having tried on any nfs mount.
> 
> So, it's probably better to use the symbolic link solution (ln -s),
> which is guaranteed to be atomic, even on NFS. This is the solution
> chosen by Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=76431

The solution was a hard link solution. I'm not sure that the symlink() system
call is guaranteed to be atomic but link() is.

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Re: File locking within zsh?

[Vincent Lefevre]

On 2006-05-12 15:40:30 -0400, Tim Writer wrote:
> I'm not sure that the symlink() system call is guaranteed to be
> atomic but link() is.

OK for link(). The Linux open(2) man page says:

  O_EXCL When used with O_CREAT, if the file  already  exists  it  is  an
         error and the open() will fail. In this context, a symbolic link
         exists, regardless of where it points to.  O_EXCL is  broken  on
         NFS file systems; programs which rely on it for performing lock-
         ing tasks will contain a race condition.  The solution for  per-
         forming  atomic  file  locking  using  a lockfile is to create a
         unique file on the same file system (e.g.,  incorporating  host-
         name  and  pid),  use link(2) to make a link to the lockfile. If
         link() returns  0,  the  lock  is  successful.   Otherwise,  use
         stat(2)  on  the  unique  file  to  check  if its link count has
         increased to 2, in which case the lock is also successful.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: File locking within zsh?

[Tim Writer]

"Brian K. White" <brian@aljex.com> writes:

> ----- Original Message ----- 
> From: "Tim Writer" <tim@starnix.com>
> To: "Lloyd Zusman" <ljz@asfast.com>
> Cc: <zsh-users@sunsite.dk>
> Sent: Tuesday, May 09, 2006 11:31 PM
> Subject: Re: File locking within zsh?
> 
> 
> > Lloyd Zusman <ljz@asfast.com> writes:
> >
> >> Do any of you know of any functions, primitives, tricks, hacks, or even
> >> outright abominations which will allow me to do cooperative file locking
> >> from within zsh?
> >>
> >> I know that I can do this with a number of compiled executables, but I'm
> >> looking for a zsh-only solution.
> >>
> >> Assuming some sort of zsh locking operator called "lock", consider
> >> this example (within a zsh script):
> >>
> >>   lock -x -t 0 file  # for this example of a hypothetical operator, '-x'
> >>                      # means to wait until I get an exclusive lock, and
> >>                      # '-t 0' means no time out
> >>   print foo bar baz >>file
> >>   # do a whole lot of other stuff to "file"
> >>   unlock file # release the lock
> >>
> >> In this example, any other zsh script which asks for an exclusive lock
> >> on "file" using this hypothetical "lock" operator will block until the
> >> "unlock" operator has been invoked.
> >>
> >> Can this be done somehow in zsh, or do I have to rely on a compiled
> >> executable to accomplish this?
> >
> > The usual way to lock within shell scripts is to use ln. This works on all
> > UNIX like systems because creating a hard link (with ln) is an atomic
> > operation which fails if the target already exists. Your example above canbe
> 
> > written like this:
> >
> >    while ! ln file file.lock 2>/dev/null
> >    do
> >        sleep 1
> >    done
> >    # Lock obtained
> >
> >    print foo bar baz >>file
> >    # do a whole lot of other stuff to "file"
> >
> >    rm -f file.lock
> >    # Lock released
> >
> > Wrapping this idiom into lock/unlock functions is left as an exercise for the
> 
> > reader. :-)
> 
> 
> Is that better or worse, and why, than  umask 222 ; >file ?
> 
> http://www.unix.org.ua/orelly/unix/upt/ch45_36.htm

I think the ln solution is better because it's guaranteed to be atomic.
Granted, a sane implementation of ">file" will use open(2) with O_CREAT what
guarantees do you have that your shell (and I'm not talking about zsh here)
is sane? I have often seen code like this pseudocode:

    if (file exists)
        open file
    else {
        creat file
        open file
    }
  
I don't know that any shells contain crap like this but I don't know they
don't either.

-- 
tim writer <tim@starnix.com>                                  starnix inc.
647.722.5301                                      toronto, ontario, canada
http://www.starnix.com              professional linux services & products

Re: File locking within zsh?

[DervishD]

    Hi Lloyd :)

 * Lloyd Zusman <ljz@asfast.com> dixit:
> Do any of you know of any functions, primitives, tricks, hacks, or even
> outright abominations which will allow me to do cooperative file locking
> from within zsh?
> 
> I know that I can do this with a number of compiled executables, but I'm
> looking for a zsh-only solution.

    Apart from the 'ln' trick, you can do this in Perl if you don't
want compiled executables. This way you can embed the locking into
your zsh script without the need of having separate scripts or
compiled executables, using "perl -e".

    Unless you need real file locking (because you have to cooperate
with another external binary), you should use the 'ln' trick instead
of using locks.

    Which solution to use depends utterly on which problem are you
trying to solve. If your problem is that you have to write
exclusively to a file, and that file may have been "lockf()"ed by
another program, then you need real lock semantics and that may be a
problem to do from zsh. I would switch to Perl or something similar
to do the task (or a zsh+Perl mix, for example). If, on the other
hand, you just need to make sure that all instances of your script
won't write to the same file at the same time, go ahead with the 'ln'
solution.

    If you could ellaborate on the problem you're trying to solve, I
can try to give a better solution.

    Good luck :)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!

Re: File locking within zsh?

[Lloyd Zusman]

DervishD <zsh@dervishd.net> writes:

>     Hi Lloyd :)

¡Hola Raúl!


>  * Lloyd Zusman <ljz@asfast.com> dixit:
>> 
>> [ ... ]
>> 
>> I know that I can do this with a number of compiled executables, but I'm
>> looking for a zsh-only solution.
>
>     Apart from the 'ln' trick, you can do this in Perl if you don't
> want compiled executables.

Well, the perl interpreter is a compiled executable, and a rather heavy
one, at that.  I'm trying to avoid spawning any extra processes.


> [ ... ]
>
>     If you could ellaborate on the problem you're trying to solve, I
> can try to give a better solution.

I'm writing a zsh shell function that might be invoked within concurrent
terminal sessions.  There's a critical section of code within this shell
function that I want to protect with an exclusive lock.  I only need to
protect this critical section from other invocations of this same shell
function.

I know that there are various file locking executables, and I can also
write my own.  Plus, I know that I can get this functionality via
scripts in perl, ruby, python, etc.  I've just been wondering if there's
a way to do this solely within zsh, without any ancillary executables
being spawned.

Perhaps I can write a loadable zsh module which can supply this
capability.


>     Good luck :)

Muchas gracias. :)


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: File locking within zsh?

[DervishD]

    Hola Lloyd :)))

 * Lloyd Zusman <ljz@asfast.com> dixit:
> >> I know that I can do this with a number of compiled executables,
> >> but I'm looking for a zsh-only solution.
> >
> >     Apart from the 'ln' trick, you can do this in Perl if you don't
> > want compiled executables.
> 
> Well, the perl interpreter is a compiled executable, and a rather
> heavy one, at that.  I'm trying to avoid spawning any extra
> processes.

    OK. I thought that your problem was that you didn't want to write
it in C, for example. If you're trying to avoid heavy binaries, avoid
Perl ;))

> >     If you could ellaborate on the problem you're trying to solve, I
> > can try to give a better solution.
> 
> I'm writing a zsh shell function that might be invoked within concurrent
> terminal sessions.  There's a critical section of code within this shell
> function that I want to protect with an exclusive lock.  I only need to
> protect this critical section from other invocations of this same shell
> function.

    OK, then probably the 'ln' solution will do.
 
> Perhaps I can write a loadable zsh module which can supply this
> capability.

    Probably, but that's a lot of work for something you can achieve
using 'ln'. You can even write a couple of shell functions using 'ln'
to do the locking.

    The only problem I see with the 'ln' solution is that it doesn't
block, you're forced to use a busy-wait loop :((

    If you don't want to use a busy-wait loop, you can try to use a
FIFO (in the filesystem, I mean) or a socket. Synchronizing more than
two tasks using those mechanism is tricky, but I think it can be
done. Think of your problem not as a "locking problem" but just as an
"IPC problem". A bit complex, anyway, because the shell doesn't
provide atomic operations that block (AFAIK).

    Of course, a zsh module for doing locking would be great ;) (or
adding the needed functions to zsh/files, for example).

> >     Good luck :)
> 
> Muchas gracias. :)

    De nada :))) I'm happy of reading you again :)

    BTW, this issue has been partially addressed in zsh-users 8632
and 8633.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!