* [musl] Feature request: posix_spawnattr_setrlimit_np() @ 2023-11-15 22:00 Tavian Barnes 2023-11-15 22:39 ` Rich Felker 0 siblings, 1 reply; 4+ messages in thread From: Tavian Barnes @ 2023-11-15 22:00 UTC (permalink / raw) To: musl I have a program that raises its soft RLIMIT_NOFILE, but wants to spawn processes with the original value (in case they use select(), for example). There seems to be no nice way to do this with posix_spawn(). I can temporarily lower the rlimit in the parent, but that interferes with other threads, and can make posix_spawn() fail with EMFILE. Corresponding glibc feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=31049 -- Tavian Barnes ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] Feature request: posix_spawnattr_setrlimit_np() 2023-11-15 22:00 [musl] Feature request: posix_spawnattr_setrlimit_np() Tavian Barnes @ 2023-11-15 22:39 ` Rich Felker 2023-11-16 0:29 ` Tavian Barnes 0 siblings, 1 reply; 4+ messages in thread From: Rich Felker @ 2023-11-15 22:39 UTC (permalink / raw) To: Tavian Barnes; +Cc: musl On Wed, Nov 15, 2023 at 05:00:51PM -0500, Tavian Barnes wrote: > I have a program that raises its soft RLIMIT_NOFILE, but wants to > spawn processes with the original value (in case they use select(), > for example). There seems to be no nice way to do this with > posix_spawn(). I can temporarily lower the rlimit in the parent, but > that interferes with other threads, and can make posix_spawn() fail > with EMFILE. > > Corresponding glibc feature request: > https://sourceware.org/bugzilla/show_bug.cgi?id=31049 I don't have any objection to this as long as it's coordinated and there's agreement from other implementors, but there *is* a way to do it already. You posix_spawnp: sh -c 'ulimit -n whatever && exec "$0" "$@"' your_program args... This is the general solution to doing all sorts of "child process state setup" things that posix_spawn doesn't have a dedicated attribute for. Note that for a proposal for setting rlimits via an attribute, one complication that needs to be specified is whether the limits take place before or after file actions, since they could change the outcome of file actions. I'm not sure what the answer is, but just YOLO'ing an implementation without thinking about that is a bad idea. Rich ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] Feature request: posix_spawnattr_setrlimit_np() 2023-11-15 22:39 ` Rich Felker @ 2023-11-16 0:29 ` Tavian Barnes 2023-11-16 16:23 ` Adhemerval Zanella Netto 0 siblings, 1 reply; 4+ messages in thread From: Tavian Barnes @ 2023-11-16 0:29 UTC (permalink / raw) To: Rich Felker; +Cc: musl On Wed, Nov 15, 2023 at 5:38 PM Rich Felker <dalias@libc.org> wrote: > > On Wed, Nov 15, 2023 at 05:00:51PM -0500, Tavian Barnes wrote: > > I have a program that raises its soft RLIMIT_NOFILE, but wants to > > spawn processes with the original value (in case they use select(), > > for example). There seems to be no nice way to do this with > > posix_spawn(). I can temporarily lower the rlimit in the parent, but > > that interferes with other threads, and can make posix_spawn() fail > > with EMFILE. > > > > Corresponding glibc feature request: > > https://sourceware.org/bugzilla/show_bug.cgi?id=31049 > > I don't have any objection to this as long as it's coordinated and > there's agreement from other implementors, but there *is* a way to do > it already. You posix_spawnp: > > sh -c 'ulimit -n whatever && exec "$0" "$@"' your_program args... True! Except that ulimit -n is not POSIX. There are ways around that, like making a dedicated shim binary that just does setrlimit(RLIMIT_NOFILE, make_rlimit(argv[1])); execv(argv[2], argv + 2); (or making it a special mode of your_program). But I think it would be better to have a convenient interface for it. And the double-exec() is not free either. I'm hoping if some libcs implement this, POSIX will reconsider https://www.austingroupbugs.net/view.php?id=603. Alternatively I suppose POSIX could standardize ulimit -n, which is already existing practice ~everywhere. > This is the general solution to doing all sorts of "child process > state setup" things that posix_spawn doesn't have a dedicated > attribute for. > > Note that for a proposal for setting rlimits via an attribute, one > complication that needs to be specified is whether the limits take > place before or after file actions, since they could change the > outcome of file actions. I'm not sure what the answer is, but just > YOLO'ing an implementation without thinking about that is a bad idea. True! Actually when I implemented my own posix_spawn()-like interface, I had setrlimit() as a file_action. Then at least the order is unambiguous. > Rich -- Tavian Barnes ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] Feature request: posix_spawnattr_setrlimit_np() 2023-11-16 0:29 ` Tavian Barnes @ 2023-11-16 16:23 ` Adhemerval Zanella Netto 0 siblings, 0 replies; 4+ messages in thread From: Adhemerval Zanella Netto @ 2023-11-16 16:23 UTC (permalink / raw) To: musl, Tavian Barnes, Rich Felker On 15/11/23 21:29, Tavian Barnes wrote: > On Wed, Nov 15, 2023 at 5:38 PM Rich Felker <dalias@libc.org> wrote: >> >> On Wed, Nov 15, 2023 at 05:00:51PM -0500, Tavian Barnes wrote: >>> I have a program that raises its soft RLIMIT_NOFILE, but wants to >>> spawn processes with the original value (in case they use select(), >>> for example). There seems to be no nice way to do this with >>> posix_spawn(). I can temporarily lower the rlimit in the parent, but >>> that interferes with other threads, and can make posix_spawn() fail >>> with EMFILE. >>> >>> Corresponding glibc feature request: >>> https://sourceware.org/bugzilla/show_bug.cgi?id=31049 >> >> I don't have any objection to this as long as it's coordinated and >> there's agreement from other implementors, but there *is* a way to do >> it already. You posix_spawnp: >> >> sh -c 'ulimit -n whatever && exec "$0" "$@"' your_program args... > > True! Except that ulimit -n is not POSIX. There are ways around > that, like making a dedicated shim binary that just does > > setrlimit(RLIMIT_NOFILE, make_rlimit(argv[1])); > execv(argv[2], argv + 2); > > (or making it a special mode of your_program). But I think it would > be better to have a convenient interface for it. And the > double-exec() is not free either. > > I'm hoping if some libcs implement this, POSIX will reconsider > https://www.austingroupbugs.net/view.php?id=603. Alternatively I > suppose POSIX could standardize ulimit -n, which is already existing > practice ~everywhere. The helper process is what OpenJDK [1] to apply some file actions, close all file descriptors (similar to close_range), and change the working directory. At least with a recent glibc, all can be accomplish with posix_spawn. And I agree that the traditional resource limits seems to be missing features from posix_spawn, even when recent Linux interfaces are leaning to a more fine-grained (and way more complex) interfaces like cgroupv2. > >> This is the general solution to doing all sorts of "child process >> state setup" things that posix_spawn doesn't have a dedicated >> attribute for. >> >> Note that for a proposal for setting rlimits via an attribute, one >> complication that needs to be specified is whether the limits take >> place before or after file actions, since they could change the >> outcome of file actions. I'm not sure what the answer is, but just >> YOLO'ing an implementation without thinking about that is a bad idea. > > True! Actually when I implemented my own posix_spawn()-like > interface, I had setrlimit() as a file_action. Then at least the > order is unambiguous. I am not sure if the resource limits really maps to a file action, although it does solve the ordering issue. For a glibc prototype [2], I added before the file actions because caller may define how many file descriptors the new process would require (since glibc provides the posix_spawn_file_actions_addclosefrom_np extension). However, since this might not be applicable to all libc implementations I am also not sure which would be best ordering. [1] https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/childproc.c [2] https://sourceware.org/bugzilla/show_bug.cgi?id=31049 ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-11-16 16:23 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-11-15 22:00 [musl] Feature request: posix_spawnattr_setrlimit_np() Tavian Barnes 2023-11-15 22:39 ` Rich Felker 2023-11-16 0:29 ` Tavian Barnes 2023-11-16 16:23 ` Adhemerval Zanella Netto
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).