Re: [musl] Add SYSCALL_USE_SOCKETCALL for old arch

[Rich Felker <dalias@libc.org>]

On Tue, Aug 04, 2020 at 12:44:41PM +0930, David Timber wrote:
> Yes, I can confirm. Attached are the qemu machines for testing(they're
> 2.6.35.9 kernel). The BusyBox cannot set up network interfaces because
> it calls socket() to interface with Netlink. "Function not
> implemented" messages appear after init.

Indeed it looks like, at least on powerpc, the separate syscalls were
added in 2.6.36. So unless there's some other reason earlier kernel
versions were unusable on powerpc (note: on ARM 2.6.15 and earlier
are unusable with musl since they didn't yet have EABI, the ABI we
use, and EABI has always had the separate socket syscalls) I think
this is a genuine bug. Likewise for sh.

> On the 2.6.0 source tree, run:
> ```
> for f in include/asm-*/unistd*.h
> do
>    echo "$f"; cat "$f" | grep __NR_socket
> done
> ```
> 
> Look for the ones without __NR_socket. Looks like there's more than 3...
> 
> The output:
> include/asm-alpha/unistd.h
> #define __NR_socket 97
> #define __NR_socketpair 135
> include/asm-arm26/unistd.h
> #define __NR_socketcall (__NR_SYSCALL_BASE+102)
> include/asm-arm/unistd.h
> #define __NR_socketcall (__NR_SYSCALL_BASE+102)
> include/asm-cris/unistd.h
> #define __NR_socketcall 102
> include/asm-h8300/unistd.h
> #define __NR_socketcall 102
> include/asm-i386/unistd.h
> #define __NR_socketcall 102
> include/asm-ia64/unistd.h
> #define __NR_socket 1190
> #define __NR_socketpair 1197
> include/asm-m68knommu/unistd.h
> #define __NR_socketcall 102
> include/asm-m68k/unistd.h
> #define __NR_socketcall 102
> include/asm-mips/unistd.h
> #define __NR_socketcall (__NR_Linux + 102)
> #define __NR_socket (__NR_Linux + 183)
> #define __NR_socketpair (__NR_Linux + 184)
> #define __NR_socket (__NR_Linux +  40)
> #define __NR_socketpair (__NR_Linux +  52)
> #define __NR_socket (__NR_Linux +  40)
> #define __NR_socketpair (__NR_Linux +  52)
> include/asm-parisc/unistd.h
> #define __NR_socket              (__NR_Linux + 17)
> #define __NR_socketpair          (__NR_Linux + 56)
> /* #define __NR_socketcall         (__NR_Linux + 102) */
> include/asm-ppc64/unistd.h
> #define __NR_socketcall 102
> include/asm-ppc/unistd.h
> #define __NR_socketcall 102
> include/asm-s390/unistd.h
> #define __NR_socketcall         102
> include/asm-sh/unistd.h
> #define __NR_socketcall 102
> include/asm-sparc64/unistd.h
> #define __NR_socket              97 /* Common
>             */
> #define __NR_socketpair         135 /* Common
>             */
> #define __NR_socketcall         206 /* Linux Specific
>             */
> include/asm-sparc/unistd.h
> #define __NR_socket              97 /* Common
>             */
> #define __NR_socketpair         135 /* Common
>             */
> #define __NR_socketcall         206 /* Linux Specific
>             */
> include/asm-um/unistd.h
> include/asm-v850/unistd.h
> #define __NR_socketcall 102
> include/asm-x86_64/unistd.h
> #define __NR_socket                             41
> __SYSCALL(__NR_socket, sys_socket)
> #define __NR_socketpair                         53
> __SYSCALL(__NR_socketpair, sys_socketpair)

Of these, only ppc, ppc64, sh, and possibly s390 (we only support
64-bit "s390x" and I'm not sure if it ever lacked the broken-down
syscalls) are relevant. The rest are either unsupported by musl
(including pre-EABI arm) or already using SYS_socketcall.




> On Mon, 3 Aug 2020 at 23:45, Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Aug 03, 2020 at 11:40:25PM +0930, David Timber wrote:
> > > And it does seem reasonable to me, but it'll mean that using no macro
> > > at all and adding a few more bytes to the binary if you're getting rid
> > > of SYSCALL_USE_SOCKETCALL. Quite tricky to support everything without
> > > kenel headers, huh. Maybe fallback to socketcall() for those 2 arch
> > > only. But if you don't want complexity, I don't know. Your project,
> > > your rules.
> >
> > If we're going to use the new syscalls on additional archs (including
> > for example i386) which has been requested due to seccomp, then the
> > fallback is needed for all of them as well.
> >
> > > On Mon, 3 Aug 2020 at 23:31, David Timber <mieabby@gmail.com> wrote:
> > > >
> > > > I didn't know that musl does not include any kernel headers. You're
> > > > right. Checking __NR_socket wouldn't work in this case. That explains
> > > > why there's syscall.h in each arch.
> > > >
> > > > Still, I ran BusyBox(musl linked) on PowerPC and SH4 machine with
> > > > 2.6.x kernel and they still couldn't use socket() at all. If you look
> > > > at the kernel source code, you'll see that the socket syscall is not
> > > > defined for those arch.
> >
> > Your original email said 2.4. If the syscalls we're using are missing
> > in 2.6 then this is a bug in musl that should be fixed. Can you
> > confirm that this is the case?
> >
> > > > No worries at all. I just switched back to uClibc. Musl looked
> > > > attractive because it produces a bit smaller binaries than uClibc. I
> > > > don't wanna encounter any more bugs like this further down the road.
> > > > No offence.
> > > >
> > > > Thanks.
> > > >
> > > > On Mon, 3 Aug 2020 at 23:03, Rich Felker <dalias@libc.org> wrote:
> > > > >
> > > > > On Mon, Aug 03, 2020 at 05:11:59PM +0930, David Timber wrote:
> > > > > > The following arches do not have __NR_SOCKET in kernel version 2.4.x,
> > > > > > but musl makes call to socket(), resulting in ENOSYS:
> > > > > >
> > > > > > * arm
> > > > > > * powerpc
> > > > > > * powerpc64
> > > > > > * sh (32bit only)
> > > > >
> > > > > The minimum supported kernel version is 2.6.0; anything earlier is on
> > > > > an as-it-happens-to-work basis. In particular threads and a good deal
> > > > > of related functionality will not work at all on 2.4, and on arm
> > > > > nothing at all works on 2.4 because the initial __ARM_NR_set_tls
> > > > > syscall fails with ENOSYS. The others however should work minimally
> > > > > since the thread pointer is kept in a userspace-accessible register.
> > > > > Which arch(s) are you trying where you get far enough to try to open a
> > > > > socket?
> > > > >
> > > > > > Adding `#define SYSCALL_USE_SOCKETCALL` would solve the problem, but
> > > > > > doing so will incur a bit of overhead. uClibc checks if `__NR_socket`
> > > > > > is defined. I suggest you do the same.
> > > > > >
> > > > > > https://git.uclibc.org/uClibc/tree/libc/inet/socketcalls.c#n387
> > > > >
> > > > > This does not work since __NR_socket is always defined. I suppose
> > > > > uClibc does some thing where you're expected to compile with kernel
> > > > > headers from the kernel you intend to run with and things subtly break
> > > > > if they mismatch; musl does not do this and does not use kernel
> > > > > headers at all. This is a runtime issue that can't be decided at
> > > > > compiletime.
> > > > >
> > > > > Since 2.4 is not actually supported, I don't want to do anything that
> > > > > increases complexity here for its sake. However, there have been
> > > > > requests to use recently-added SYS_socket, etc. for archs that
> > > > > previously used SYS_socketcall because the new ones are friendlier to
> > > > > seccomp filtering. The right resolution to the problem you're seeing
> > > > > is probably switching to always using the new syscalls first if
> > > > > they're available, only falling back to SYS_socketcall if they fail
> > > > > with ENOSYS (and only on archs where SYS_socketcall exists to begin
> > > > > with), and getting rid of the SYSCALL_USE_SOCKETCALL macro entirely
> > > > > since we'll be treating all archs the same.
> > > > >
> > > > > Does this sound reasonable?
> > > > >
> > > > > Rich