The feature I want to achieve is to close all file handles after fork to avoid file handles being inherited to child processes.
We have a component which is used in a process that uses too much memory (about 50BG) and uses too many file handles(about 100,000 files).
In order to avoid file handles inheriting from child processes, we implemented a API like system,close all file handles after vfork.
We can't close all file handles by iterating over all file handles number, because the maximum number of files limit is set very large, the performance of this method is too poor.
And we can't use fork, fork will fail because of vm.overcommit_memory.
In order to optimize this, the method we used is to read the file handes in /proc/self/fd directory and close the file handles one by one.
The component is based on glibc, currently this program runs for about 10 years without deadlock.
Recently, this component is used in musl-based embedded systems, and it hangs often.
After reading the vfork manual and the guidance you gave, i understand calling readdir after vfork is problematic. and I also learned that I can call getent64 to achieve this
but what I want to say is that, based on glibc, after calling readdir after vfork, there is no deadlock problem at present.
more information: I googled and found this(
https://lwn.net/Articles/789023/), probably the best solution, but it seems that the required kernel version is too new for my program to work with.
However, I think my problem is solved anyway, thank you all.