Re: Any way to allow clobbering empty files when noclobber is set?

[Peter Stephenson <p.w.stephenson@ntlworld.com>]

On Sat, 2020-06-06 at 00:53 +0000, Daniel Shahaf wrote:
> Peter Stephenson wrote on Fri, 05 Jun 2020 12:41 +00:00:
> > > On 05 June 2020 at 03:07 Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> > > Roman Perepelitsa wrote on Thu, 04 Jun 2020 07:41 +0200:
> > > > The regular noclobber can use O_CREAT | O_EXCL to avoid races. How
> > > > would any of the proposed modifications work? That is, how to
> > > > implement unlink-if-empty without ever unlinking non-empty files?
> > > 
> > > I don't think that's possible: it'd require an atomic
> > > stat-and-conditionally-unlink operation, and I don't think there's
> > > a syscall for this.  fstatat(2)-then-unlinkat(2) gets close, but it's
> > > still racy.
> > > 
> > > Good point.
> > 
> > I think we'd just have to document that this option doesn't provide
> > guarantees against asynchronous file access.  It wasn't requested
> > in order to implement standard behaviour, and we're not changing
> > the existing behaviour without the new option (I think we're mostly
> > agreed), and zsh already has so many options to sanitise if you do
> > want standard behaviour I don't think adding another is a big deal
> > --- assuming, of course, it actually is generally useful.
> > 
> > For what it's worth, I don't think it would have occurred to me
> > that there might be a race-free way of ensuring a file was empty on
> > opening, but the discussion has been interesting anyway.
> E
> [ Was this intentionally offlist?  Quoting without trimming; feel free
> to re-Cc the list on reply. ]

This was indeed intended to go to everyone.

> I'm not sure I understand.  As said on list, CLOBBER_EMPTY _can_ easily
> be implemented without races, using open()-then-fstat(); whereas
> UNLINK_EMPTY_AFTER_FAILURE cannot be implemented without races, because
> there's no fstatat()-then-unlinkat() syscall that's atomic against
> concurrent unlink()/creat() calls affecting the directory in question.
> Thus, I don't quite follow what the rationale is for _not_ implementing
> CLOBBER_EMPTY atomically.
> 
> Would you elaborate on the rationale, please?

My understanding of the semantics (feel free to put me right, of
course), is that as any use of fstat() would be on a non-clobbering
open, you can't stop anyone else writing to the file (the same file,
rahter than a newly created one with the same one) at any point.  Worse,
you're now claiming you've clobbered that open file, and you haven't.
So now you have Roman's nightmare scneario where you're claiming you've
opened a new file because the old one was empty, but actually two
processes have written to it.  Not only have you not fixed a race,
you've created a new one.

Assuming both processes are actually doing clobbering opens (O_CREAT |
O_EXCL), you can't get into that position.  You can't guarantee that
someone else hasn't written to a file of that name, but you can
guarantee that two so-called clobbering opens actually are that.

This is without using file locking --- I trust we don't want to go down
that route.  I mention it because (this is a key point, so if I've got
it wrong the whole argument falls apart, but I think Bart is saying the
same thing) just using normal system calls without locking there's no
way of stopping multiple processes opening existing files for writing.

pws