Re: sysconf(_SC_NPROCESSORS_CONF) returns the wrong value

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

----- On Mar 26, 2019, at 1:45 PM, Szabolcs Nagy nsz@port70.net wrote:

> * Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com> [2019-03-26
> 12:23:34 -0400]:
>> > i think we need to know why does a process care if musl returns
>> > the wrong number? or what are the valid uses of such a number?
>> > (there are heterogeous systems like arm big-little, numa systems
>> > with many sockets, containers, virtualization,.. how deep may a
>> > user process need to go down in this rabbit hole?)
>> 
>> Does the answers from Mathieu Desnoyers [1] and Florian Weimer [2] fit the bill?
> 
> yes
> 
>> 
>> [1] https://www.openwall.com/lists/musl/2019/03/16/3
>> [2] https://www.openwall.com/lists/musl/2019/03/19/1
>> > 
>> > note that most of /sys/devices/system/cpu/* is documented under
>> > Documentation/ABI/testing in linux, not in Documentation/ABI/stable
>> > and the format is not detailed, and some apis (e.g. /proc/cpuinfo)
>> > are known to be different on android (and grsec?) kernels it may
>> > be unmounted during early boot or in chroots, so sysfs parsing is
>> > only done when really necessary.
>> 
>> For what it's worth, uclibc and uclibc-ng seem to iterate over
>> /sys/devices/system/cpu/* and fallback on online calculation if necessary.
>> 
>> https://cgit.uclibc-ng.org/cgi/cgit/uclibc-ng.git/tree/libc/unistd/sysconf.c#n102
>> 
>> In the mean time, we implemented a fallback similar to this when we do not
>> "know"
>> the libc used (since musl does not come with __musl__, I read the reasons why,
>> no need to discuss this).
>> 
>> Not sure of the direction musl should take but I strongly believe that the
>> behaviour regarding _SC_NPROCESSORS_CONF is not the appropriate one.
> 
> i agree that the current behaviour is not ideal, but
> iterating over /sys/devices/system/cpu/cpu* may not
> be correct either.. based on current linux api docs.
> 
> i don't understand why is that number different from the
> cpu set in /sys/devices/system/cpu/possible

I suspect both iteration over /sys/devices/system/cpu/cpu* and
content of /sys/devices/system/cpu/possible should provide the
same result. However, looking at Linux
Documentation/ABI/testing/sysfs-devices-system-cpu ,
it appears that /sys/devices/system/cpu/possible was introduced
in December 2008, whereas /sys/devices/system/cpu/cpu#/ was there
pre-git history.

This could explain why glibc uses the iteration method.

Thoughts ?

Thanks,

Mathieu

> 
> it seems any upper bound on the number of cpus would be
> valid but it's not clear how to provide that guarantee.





-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com