Re: unlink on NFS volume fails silently

[Rich Felker <dalias@libc.org>]

On Sun, Jul 06, 2025 at 04:25:31PM +1000, Stephen Von Takach wrote:
> Hi,
> 
> We recently had to move a service from being built on alpine linux to
> debian linux as we were getting silent failures when deleting a directory
> with many files on an NFS volume. Basically this call to unlink was not
> raising an error if the file failed to delete
> https://github.com/crystal-lang/crystal/blob/master/src/crystal/system/unix/file.cr#L129
> 
> We replicated the issue in an alpine container with rm -rf
> /nfs_mount/git_repo_to_delete and it also failed to successfully delete all
> the files, it did raise an error though (I assume it checked the file was
> removed before continuing) not entirely sure.
> 
> Both these operations succeed with glibc when using debian.
> Looks a bit like this issue:
> https://gitlab.alpinelinux.org/alpine/aports/-/issues/10960

Assuming you actually traced and saw the unlink syscall succeed, the
root cause here is the filesystem/kernel lying about that. Standard
"NFS Considered Harmful" stuff.

But the fact that you're seeing different behavior on Alpine is almost
surely a matter of busybox rm vs GNU coreutils differences in how they
behave under faulty kernel behavior. The easy solution is probably
installing the coreutils package. Otherwise, investigate what busybox
is doing differently and if there's a way it could be made more
reliable in this situation.

Rich