From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 7414 invoked from network); 3 Jul 2021 04:00:39 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 3 Jul 2021 04:00:39 -0000 Received: (qmail 27651 invoked by uid 550); 3 Jul 2021 04:00:36 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 26596 invoked from network); 3 Jul 2021 04:00:36 -0000 X-Virus-Scanned: Debian amavisd-new at disroot.org Content-Type: multipart/mixed; boundary=fef167f5816f3e53acf9be4f464f8f9c5b254fa84aea966c09a01b5bb870 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disroot.org; s=mail; t=1625283560; bh=eyFA+/HD2/iuLykBDXbJiCUqi77CQ5xdbM/5ZzyBJBk=; h=Subject:From:To:Date; b=T4ZJHH0jjX3ij5625f8VCy+HtwjrdhYWm04WyzchGpgEs4vgMRttAOZnp4dEaNWjn uZhkQoGPCdNCLKClFbb3kxbZphgNI9n6R2oEUewJQf9FyfP60C94z5WouBY6zRCXi2 Qu/Hkdb7YsxINeDiUY/i/vRD2rl+SJwjN5ESIY5F13dHcyLfzTRGIAUfE3au+1r99M ZG6ERYRlOlJVG5x7QB01fK6/wGlpOX8gneBZWvdFsu7QHx8WHOMsBwoYJZ2UuISkZq 4LoOSfTrbVCRxvp9xkFr2H1vgvz6DaNZxJsbDEpMiEDkYEclLwdk3pXZ+IOrr/kbN1 TdvHzyu9MPuXQ== Mime-Version: 1.0 From: =?utf-8?q?=C3=89rico_Nogueira?= To: Date: Fri, 02 Jul 2021 23:58:26 -0300 Message-Id: Subject: [musl] NGROUPS_MAX definition conflicts with kernel one --fef167f5816f3e53acf9be4f464f8f9c5b254fa84aea966c09a01b5bb870 Content-Type: multipart/alternative; boundary=45b1b910395895aca0519a5d664b49151f7919748f082e2b460fa951ec8e --45b1b910395895aca0519a5d664b49151f7919748f082e2b460fa951ec8e Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Content-Type: text/plain; charset=UTF-8 Hello all. I brought this up on IRC and we thought it best to bring it up on the ML as well. musl's NGROUPS_MAX is defined to 32, which is the kernel's definition for the value for linux < 2.6.4. Meanwhile, linux >=3D 2.6.4 has 65536. Linux 2.6.4 also introduced /proc/sys/kernel/ngroups_max, to allow for this value to be changed in the kernel without libc having to keep changing their definitions. glibc's headers define NGROUPS_MAX to 65536, the new limit, so there is already some discrepancy between the two libraries. Fortunately, the man pages for getgroups(2) suggests using sysconf(_SC_NGROUPS_MAX) instead of the NGROUPS_MAX value directly, since the sysconf function can, theoretically, query the information from the currently running kernel. glibc's does just that: it tries to read from /proc, and if that fails, it will return NGROUPS_MAX as defined in . Unfortunately, musl's implementation directly returns NGROUPS_MAX from its own definition of it, which means it returns 32. There are few scenarios where I can see this being relevant, to be honest; hardly anyone is going to have more than 32 supplementary groups. However, in the case they do, functions like getgroups(2) will fail even if the caller passes a buffer with size sysconf(_SC_NGROUPS_MAX). An application could be forgiven for not trying to increase the buffer size past it in the case that sysconf(_SC_NGROUPS_MAX) exists (doesn't return -1). There are a few ways forward from here: NGROUPS_MAX can be updated to the latest kernel maximum and/or sysconf(_SC_NGROUPS_MAX) can be fixed to try and read values from /proc; or we do nothing. Not doing nothing would also require rolling out further fixes: - musl's initgroups(3) uses a static array with 32 members for the groups, which isn't enough if we want to support cases with more groups; that would require implementing dynamic allocation, which, with naive logic, would allocate 256KB (65536 entries * sizeof(gid_t)) - any applications/libraries mainly used in the musl ecosystem might be relying on NPROCS_MAX being a small value that can be used, for example, as the size of a stack-allocated array; with a bigger size, this would no longer be true (see [1] as an example, though I am already looking into fixing that particular case) [1] https://github.com/pikhq/musl-nscd/blob/9ab730cfb5fe3dc224ded1033e1b36e= a60c60be6/src/socket_handle.c#L232 In conclusion, as mentioned on IRC, "it's a place where the old limit was a feature", since it avoids big allocations when very very few use cases need them. IMO, if we want to be more correct while keeping the advantage of a low value, sysconf(_SC_NPROCS_MAX) could read /proc, but NPROCS_MAX could be kept as 32. But I don't know how the initgroups(3) case would deal with it. If we decide to do nothing, such a limitation (even if derived from a design choice) should at least end up documented clearly. A suggestion on IRC was also that users requiring more than 32 supplementary groups could patch musl to work on their environment. For that purpose, I have attached a patch to this email to make it simpler for such patching to occur. Cheers, =C3=89rico --45b1b910395895aca0519a5d664b49151f7919748f082e2b460fa951ec8e-- --fef167f5816f3e53acf9be4f464f8f9c5b254fa84aea966c09a01b5bb870 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=0001-use-NGROUPS_MAX-instead-of-hardcoded-value-where-pos.patch Content-Type: text/plain; charset=utf-8; name=0001-use-NGROUPS_MAX-instead-of-hardcoded-value-where-pos.patch RnJvbSBmYjk4MGQ2NDc0NGUwNzZjZmNiM2U5M2Q4ZjVlNDZiM2ZmOTU0M2JiIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/PUMzPTg5cmljbz0yME5vZ3VlaXJhPz0gPGVy aWNvbnJAZGlzcm9vdC5vcmc+CkRhdGU6IFNhdCwgMyBKdWwgMjAyMSAwMDozMTo1NSAtMDMwMApT dWJqZWN0OiBbUEFUQ0hdIHVzZSBOR1JPVVBTX01BWCBpbnN0ZWFkIG9mIGhhcmRjb2RlZCB2YWx1 ZSB3aGVyZSBwb3NzaWJsZQoKaWYgc29tZW9uZSB1c2luZyBtdXNsIGZpbmRzIGl0IG5lY2Vzc2Fy eSB0byBwYXRjaCBtdXNsIHRvIHVzZSA2NTUzNiBhcwpOR1JPVVBTX01BWCwgYXMgZGVmaW5lZCBi eSBrZXJuZWxzID49IDIuNi40LCB0aGV5IGNhbiBwYXRjaCBpdCBpbiBvbmUKcGxhY2UsIGluY2x1 ZGUvbGltaXRzLmgsIGluc3RlYWQgb2YgaGF2aW5nIHRvIGZpbmQgYWxsIHRoZSBwbGFjZXMgd2hl cmUKaXQncyBoYXJkY29kZWQKLS0tCiBpbmNsdWRlL3N5cy9wYXJhbS5oIHwgMiArLQogc3JjL2Nv bmYvc3lzY29uZi5jICB8IDIgKy0KIDIgZmlsZXMgY2hhbmdlZCwgMiBpbnNlcnRpb25zKCspLCAy IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2luY2x1ZGUvc3lzL3BhcmFtLmggYi9pbmNsdWRl L3N5cy9wYXJhbS5oCmluZGV4IGNlNmI4MDE5Li5mNDc4MDU4OCAxMDA2NDQKLS0tIGEvaW5jbHVk ZS9zeXMvcGFyYW0uaAorKysgYi9pbmNsdWRlL3N5cy9wYXJhbS5oCkBAIC02LDcgKzYsNyBAQAog I2RlZmluZSBNQVhOQU1MRU4gMjU1CiAjZGVmaW5lIE1BWFBBVEhMRU4gNDA5NgogI2RlZmluZSBO QkJZIDgKLSNkZWZpbmUgTkdST1VQUyAzMgorI2RlZmluZSBOR1JPVVBTIE5HUk9VUFNfTUFYCiAj ZGVmaW5lIENBTkJTSVogMjU1CiAjZGVmaW5lIE5PRklMRSAyNTYKICNkZWZpbmUgTkNBUkdTIDEz MTA3MgpkaWZmIC0tZ2l0IGEvc3JjL2NvbmYvc3lzY29uZi5jIGIvc3JjL2NvbmYvc3lzY29uZi5j CmluZGV4IDNiYWFlZDMyLi4xYTVkMzg2MCAxMDA2NDQKLS0tIGEvc3JjL2NvbmYvc3lzY29uZi5j CisrKyBiL3NyYy9jb25mL3N5c2NvbmYuYwpAQCAtMjgsNyArMjgsNyBAQCBsb25nIHN5c2NvbmYo aW50IG5hbWUpCiAJCVtfU0NfQVJHX01BWF0gPSBKVF9BUkdfTUFYLAogCQlbX1NDX0NISUxEX01B WF0gPSBSTElNKE5QUk9DKSwKIAkJW19TQ19DTEtfVENLXSA9IDEwMCwKLQkJW19TQ19OR1JPVVBT X01BWF0gPSAzMiwKKwkJW19TQ19OR1JPVVBTX01BWF0gPSBOR1JPVVBTX01BWCwKIAkJW19TQ19P UEVOX01BWF0gPSBSTElNKE5PRklMRSksCiAJCVtfU0NfU1RSRUFNX01BWF0gPSAtMSwKIAkJW19T Q19UWk5BTUVfTUFYXSA9IFRaTkFNRV9NQVgsCi0tIAoyLjMyLjAKCg== --fef167f5816f3e53acf9be4f464f8f9c5b254fa84aea966c09a01b5bb870--