capset() capget() syscalls

capset() capget() syscalls

[igmar]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 150 bytes --]

Hi,

Attached patch adds types to the existing syscalls, and defines the types 
in the header. All CAP_* defines are also added.


Regards,



 	Igmar

[-- Attachment #2: Type: TEXT/PLAIN, Size: 3006 bytes --]

From 0237fb69ec7c3bf77c6280a005238fc8d0ba7607 Mon Sep 17 00:00:00 2001
From: Igmar Palsenberg <igmar@palsenberg.com>
Date: Mon, 27 Aug 2012 13:47:51 +0200
Subject: [PATCH]] Properly implement capset and capget syscalls
 Define CAP_* defines
 Define kernel ABI structs


Signed-off-by: Igmar Palsenberg <igmar@palsenberg.com>
---
 include/sys/capability.h |   74 ++++++++++++++++++++++++++++++++++++++++++++++
 src/linux/cap.c          |    5 ++-
 2 files changed, 77 insertions(+), 2 deletions(-)
 create mode 100644 include/sys/capability.h

diff --git a/include/sys/capability.h b/include/sys/capability.h
new file mode 100644
index 0000000..68bb4fc
--- /dev/null
+++ b/include/sys/capability.h
@@ -0,0 +1,74 @@
+#ifndef _SYS_CAPABILITY_H
+#define _SYS_CAPABILITY_H
+
+#include <stdint.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LINUX_CAPABILITY_VERSION_1  0x19980330
+#define _LINUX_CAPABILITY_VERSION_2  0x20071026
+#define _LINUX_CAPABILITY_VERSION_3  0x20080522
+
+/* POSIX defined capabilities */
+#define CAP_CHOWN				0
+#define CAP_DAC_OVERRIDE		1
+#define CAP_DAC_READ_SEARCH		2
+#define CAP_FOWNER				3
+#define CAP_FSETID				4
+#define CAP_KILL				5
+#define CAP_SETGID				6
+#define CAP_SETUID				7
+
+/* Linux specific */
+#define CAP_SETPCAP				8
+#define CAP_LINUX_IMMUTABLE		9
+#define CAP_NET_BIND_SERVICE	10
+#define CAP_NET_BROADCAST		11
+#define CAP_NET_ADMIN			12
+#define CAP_NET_RAW				13
+#define CAP_IPC_LOCK			14
+#define	CAP_IPC_OWNER			15
+#define CAP_SYS_MODULE			16
+#define	CAP_SYS_RAWIO			17
+#define	CAP_SYS_CHROOT			18
+#define	CAP_SYS_PTRACE			19
+#define	CAP_SYS_PACCT			20
+#define	CAP_SYS_ADMIN			21
+#define	CAP_SYS_BOOT			22
+#define	CAP_SYS_NICE			23
+#define	CAP_SYS_RESOURCE		24
+#define	CAP_SYS_TIME			25
+#define	CAP_SYS_TTY_CONFIG		26
+#define	CAP_MKNOD				27
+#define	CAP_LEASE				28
+#define	CAP_AUDIT_WRITE			29
+#define	CAP_AUDIT_CONTROL		30
+#define	CAP_SETFCAP				31
+#define	CAP_MAC_OVERRIDE		32
+#define	CAP_MAC_ADMIN			33
+
+typedef struct _user_cap_header_struct * cap_user_header_t;
+typedef struct _user_cap_data_struct * cap_user_data_t;
+
+struct _user_cap_header_struct {
+	uint32_t version;
+	int pid;
+};
+
+struct _user_cap_data_struct {
+	uint32_t effective;
+	uint32_t permitted;
+	uint32_t inheritable;
+};
+
+
+int capget(cap_user_header_t, cap_user_data_t);
+int capset(cap_user_header_t, cap_user_data_t);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/src/linux/cap.c b/src/linux/cap.c
index 8d035e0..b88e530 100644
--- a/src/linux/cap.c
+++ b/src/linux/cap.c
@@ -1,11 +1,12 @@
 #include "syscall.h"
+#include <sys/capability.h>
 
-int capset(void *a, void *b)
+int capset(cap_user_header_t a, cap_user_data_t b)
 {
 	return syscall(SYS_capset, a, b);
 }
 
-int capget(void *a, void *b)
+int capget(cap_user_header_t a, cap_user_data_t b)
 {
 	return syscall(SYS_capget, a, b);
 }
-- 
1.7.1

Re: capset() capget() syscalls

[Rich Felker]

On Mon, Aug 27, 2012 at 01:51:18PM +0200, igmar@s1.palsenberg.com wrote:
> Hi,
> 
> Attached patch adds types to the existing syscalls, and defines the
> types in the header. All CAP_* defines are also added.

I'm curious what applications expect this header and definitions.
glibc does not have such a header. My impression is that it might have
existed at some point but that it was removed a long time ago, and
that any use of capabilities without libcap (which presumably contains
its own copy of the definitions for interfacing with the kernel) was
long ago deprecated.

Can you clear these issues up?

Rich

Re: capset() capget() syscalls

[Igmar Palsenberg]

>> Hi,
>>
>> Attached patch adds types to the existing syscalls, and defines the
>> types in the header. All CAP_* defines are also added.
> I'm curious what applications expect this header and definitions.
> glibc does not have such a header. My impression is that it might have
> existed at some point but that it was removed a long time ago, and
> that any use of capabilities without libcap (which presumably contains
> its own copy of the definitions for interfacing with the kernel) was
> long ago deprecated.
>
> Can you clear these issues up?
My own. Those definitions are part of the kernel syscall. While glibc
does support structure definitions for most syscalls,
capabilities seem to be an exception. (they are in linux/capabilities.h
to be exact). Oh, and I really dislike the interfaces libcap provides.

I pull those definitions in myself now, because pulling in kernel header
files is considered a bad practice (c) Linus Torvalds
Since these won't add a byte of code, it might be a good idea.
I'm open for suggestions however.


    Igmar

Re: capset() capget() syscalls

[Rich Felker]

On Wed, Sep 05, 2012 at 11:28:54AM +0200, Igmar Palsenberg wrote:
> 
> >> Hi,
> >>
> >> Attached patch adds types to the existing syscalls, and defines the
> >> types in the header. All CAP_* defines are also added.
> > I'm curious what applications expect this header and definitions.
> > glibc does not have such a header. My impression is that it might have
> > existed at some point but that it was removed a long time ago, and
> > that any use of capabilities without libcap (which presumably contains
> > its own copy of the definitions for interfacing with the kernel) was
> > long ago deprecated.
> >
> > Can you clear these issues up?
> My own. Those definitions are part of the kernel syscall. While glibc
> does support structure definitions for most syscalls,
> capabilities seem to be an exception. (they are in linux/capabilities.h
> to be exact). Oh, and I really dislike the interfaces libcap provides.

I found them pretty ugly too, but I can't say I like the kernel API
any better... And the API in linux/capabilities.h is just horrible.
The typedefs are pointer types and there's no way to actually declare
the objects they point to without using the _-prefixed struct tag. I
don't see why they needed this hideous struct-pointer-based interface
rather than just passing a few words as syscall arguments...

> I pull those definitions in myself now, because pulling in kernel header
> files is considered a bad practice (c) Linus Torvalds
> Since these won't add a byte of code, it might be a good idea.
> I'm open for suggestions however.

There are a lot of direct-kernel-interface-usage situations that
require pulling in the associated kernel headers, such as fb.h, kb.h,
etc. One issue is that the kernel headers presumably lack prototypes
for the functions, but the most portable way to do what you're doing
is probably going to be:

#include <linux/capabilities.h>
#include <sys/syscall.h>
...
syscall(SYS_capset, x, y);

At least this will work on every Linux libc I know of, and it's
hard for them to break it.

If you really want to see the header in musl, I'm not opposed to
further consideration of the matter, but right now I don't think
adding it promotes portable software; rather it seems to promote
writing software that works on musl and breaks on glibc.

Rich

Re: capset() capget() syscalls

[Justin Cormack]

On Wed, Sep 5, 2012 at 3:24 PM, Rich Felker <dalias@aerifal.cx> wrote:

> There are a lot of direct-kernel-interface-usage situations that
> require pulling in the associated kernel headers, such as fb.h, kb.h,
> etc. One issue is that the kernel headers presumably lack prototypes
> for the functions, but the most portable way to do what you're doing
> is probably going to be:
>
> #include <linux/capabilities.h>
> #include <sys/syscall.h>
> ...
> syscall(SYS_capset, x, y);
>
> At least this will work on every Linux libc I know of, and it's
> hard for them to break it.
>
> If you really want to see the header in musl, I'm not opposed to
> further consideration of the matter, but right now I don't think
> adding it promotes portable software; rather it seems to promote
> writing software that works on musl and breaks on glibc.

It is an unclear situation. Glibc seems inconsistent.

Personally I think Musl should provide the kernel structures and
syscalls for everything that is not deprecated.

I was looking at providing patches for a bunch of missing syscalls
(which are in glibc).

I agree with Linus, provide all the headers in libc. I tried to write
some code to include all syscalls and constants needed for them, and
as far as I can see it is impossible with glibc due to conflicts. If
anyone has a set of includes that works let me know....

Justin

Re: capset() capget() syscalls

[Rich Felker]

On Wed, Sep 05, 2012 at 06:01:11PM +0100, Justin Cormack wrote:
> On Wed, Sep 5, 2012 at 3:24 PM, Rich Felker <dalias@aerifal.cx> wrote:
> 
> > There are a lot of direct-kernel-interface-usage situations that
> > require pulling in the associated kernel headers, such as fb.h, kb.h,
> > etc. One issue is that the kernel headers presumably lack prototypes
> > for the functions, but the most portable way to do what you're doing
> > is probably going to be:
> >
> > #include <linux/capabilities.h>
> > #include <sys/syscall.h>
> > ...
> > syscall(SYS_capset, x, y);
> >
> > At least this will work on every Linux libc I know of, and it's
> > hard for them to break it.
> >
> > If you really want to see the header in musl, I'm not opposed to
> > further consideration of the matter, but right now I don't think
> > adding it promotes portable software; rather it seems to promote
> > writing software that works on musl and breaks on glibc.
> 
> It is an unclear situation. Glibc seems inconsistent.
> 
> Personally I think Musl should provide the kernel structures and
> syscalls for everything that is not deprecated.

In principle, I agree with you.

My impression was that the kernel developers intend for this API to be
deprecated for use by applications, and the only reason they haven't
replaced it with a proper kernelspace API is that they assume you'll
be using libcap which wraps/hides the ugliness (and replaces it with
something else that's just ugly in different ways...).

As such, it's sort of a borderline case. Do we really want to be
promoting this API for use by applications?

> I was looking at providing patches for a bunch of missing syscalls
> (which are in glibc).

OK, maybe I should clean up the source tree in preparation for this.
My intent is that src/linux/*.c should be purely syscall wrappers for
non-POSIX syscalls provided by Linux. Right now src/linux also
contains other misc junk, and some of the syscall wrappers are in
src/misc or perhaps other locations..

> I agree with Linus, provide all the headers in libc. I tried to write
> some code to include all syscalls and constants needed for them, and
> as far as I can see it is impossible with glibc due to conflicts. If
> anyone has a set of includes that works let me know....

Can you explain what you mean here?

Rich

Re: capset() capget() syscalls

[John Spencer]

On 09/06/2012 05:04 AM, Rich Felker wrote:
>
> As such, it's sort of a borderline case. Do we really want to be
> promoting this API for use by applications?
>
no

Re: capset() capget() syscalls

[Rich Felker]

On Thu, Sep 06, 2012 at 05:10:20AM +0200, John Spencer wrote:
> On 09/06/2012 05:04 AM, Rich Felker wrote:
> >
> >As such, it's sort of a borderline case. Do we really want to be
> >promoting this API for use by applications?
> >
> no

Thanks for the input, but I don't think it's quite that clear-cut and
I don't have a good answer. There should be some way to access
capabilities without needing an ugly library which allocates dynamic
memory to report the current capability set (yes, libcap is THAT bad),
but the sys/capabilities.h interfaces are also a complete mess...

Rich

Re: capset() capget() syscalls

[Kurt H Maier]

On Wed, Sep 05, 2012 at 11:20:13PM -0400, Rich Felker wrote:
> 
> Thanks for the input, but I don't think it's quite that clear-cut and
> I don't have a good answer. There should be some way to access
> capabilities without needing an ugly library which allocates dynamic
> memory to report the current capability set (yes, libcap is THAT bad),
> but the sys/capabilities.h interfaces are also a complete mess...
> 

Why not replace libcap instead of doing this in musl?

Re: capset() capget() syscalls

[Rich Felker]

On Wed, Sep 05, 2012 at 11:28:11PM -0400, Kurt H Maier wrote:
> On Wed, Sep 05, 2012 at 11:20:13PM -0400, Rich Felker wrote:
> > 
> > Thanks for the input, but I don't think it's quite that clear-cut and
> > I don't have a good answer. There should be some way to access
> > capabilities without needing an ugly library which allocates dynamic
> > memory to report the current capability set (yes, libcap is THAT bad),
> > but the sys/capabilities.h interfaces are also a complete mess...
> > 
> 
> Why not replace libcap instead of doing this in musl?

That's kind of the direction I was thinking. And the "portable" (to
all known Linux libcs) way to do this would be using <sys/syscall.h>
and SYS_capset/SYS_capget to implement such a library, rather than
having special-case code to do X on glibc and Y on musl...

Alternatively, if the kernel and glibc folks could agree (or at least
the glibc folks), we could develop a new sane API for exposing
capabilities from libc instead of with a third-party library.

Rich

Re: capset() capget() syscalls

[Kurt H Maier]

On Wed, Sep 05, 2012 at 11:41:46PM -0400, Rich Felker wrote:
> 
> Alternatively, if the kernel and glibc folks could agree (or at least
> the glibc folks), we could develop a new sane API for exposing
> capabilities from libc instead of with a third-party library.
> 

I don't think that's a likely event.  I certainly hope musl feature
decisions don't block on awaiting rational behaviour from the glibc
camp, in particular.  I just don't like the idea of shoving that
single-kernel mess into a libc.

Re: capset() capget() syscalls

[Justin Cormack]

On Thu, Sep 6, 2012 at 4:04 AM, Rich Felker <dalias@aerifal.cx> wrote:
>> I agree with Linus, provide all the headers in libc. I tried to write
>> some code to include all syscalls and constants needed for them, and
>> as far as I can see it is impossible with glibc due to conflicts. If
>> anyone has a set of includes that works let me know....
>
> Can you explain what you mean here?

WIth glibc I cannot seem to find a set of headers to include that will
give me access to all the system calls plus the types and constants
needed to call them. There are always conflicts. I am hoping I will be
able to with Musl.

>My impression was that the kernel developers intend for this API to be
>deprecated for use by applications, and the only reason they haven't
>replaced it with a proper kernelspace API is that they assume you'll
>be using libcap which wraps/hides the ugliness (and replaces it with
>something else that's just ugly in different ways...).

I don't see it can be deprecated if it has not been replaced... it is
one of the syscalls not generally intended for everyday use. But it is
a bit unclear.

For reference (this list is not complete), Musl is missing the
following syscalls that glibc has:
fallocate, acct, setns, sync_file_range, readahead, tee,
timerfd_create, timerfd_settime, timerfd_gettime

Musl has the following syscalls that glibc does not provide:
mknod, mknodat, clock_nanosleep

And neither provide the non obsolete
clock_getres, clock_settime, clock_gettime

So I don't think there is much rationale to this set right now...

Justin

Re: capset() capget() syscalls

[Szabolcs Nagy]

* Justin Cormack <justin@specialbusservice.com> [2012-09-06 09:22:58 +0100]:
> WIth glibc I cannot seem to find a set of headers to include that will
> give me access to all the system calls plus the types and constants
> needed to call them. There are always conflicts. I am hoping I will be
> able to with Musl.
> 

so these are functions not present in any standard and any libc

why do you want to include them in musl?
that would mean applications will break using glibc vs musl

why not use a separate library (libcap2)

> I don't see it can be deprecated if it has not been replaced... it is
> one of the syscalls not generally intended for everyday use. But it is
> a bit unclear.
> 

this is the question
and we need an authorative answer

> For reference (this list is not complete), Musl is missing the
> following syscalls that glibc has:
> fallocate, acct, setns, sync_file_range, readahead, tee,
> timerfd_create, timerfd_settime, timerfd_gettime
> 

these are non-standard functions, but the non-broken
ones should be provided eventually

the ugly ones are those which glibc declares in standard
headers (eg readahead, tee)
(timerfd* functions are easy to provide as they live in
their own header)

some of them might be obsolete
(eg posix_fallocate is in musl and can be used instead
of fallocate, and posix_fadvise may be used instead of
readahead)

> And neither provide the non obsolete
> clock_getres, clock_settime, clock_gettime
> 

these are posix standard api and musl provides them
if the necessary feature test macros are defined when
time.h is included

Re: capset() capget() syscalls

[Igmar Palsenberg]

On 9/5/12 4:24 PM, Rich Felker wrote:
> On Wed, Sep 05, 2012 at 11:28:54AM +0200, Igmar Palsenberg wrote:
>>>> Hi,
>>>>
>>>> Attached patch adds types to the existing syscalls, and defines the
>>>> types in the header. All CAP_* defines are also added.
>>> I'm curious what applications expect this header and definitions.
>>> glibc does not have such a header. My impression is that it might have
>>> existed at some point but that it was removed a long time ago, and
>>> that any use of capabilities without libcap (which presumably contains
>>> its own copy of the definitions for interfacing with the kernel) was
>>> long ago deprecated.
>>>
>>> Can you clear these issues up?
>> My own. Those definitions are part of the kernel syscall. While glibc
>> does support structure definitions for most syscalls,
>> capabilities seem to be an exception. (they are in linux/capabilities.h
>> to be exact). Oh, and I really dislike the interfaces libcap provides.
> I found them pretty ugly too, but I can't say I like the kernel API
> any better... And the API in linux/capabilities.h is just horrible.
> The typedefs are pointer types and there's no way to actually declare
> the objects they point to without using the _-prefixed struct tag. I
> don't see why they needed this hideous struct-pointer-based interface
> rather than just passing a few words as syscall arguments...
You only need cap_user_header_t and cap_user_data_t, and the CAP_ defines.
The rest belongs to libcap, and aren't used by the kernel API.

We might consider altering the naming a bit. The __u32 stuff can be
replace by proper typing.

>> I pull those definitions in myself now, because pulling in kernel header
>> files is considered a bad practice (c) Linus Torvalds
>> Since these won't add a byte of code, it might be a good idea.
>> I'm open for suggestions however.
> There are a lot of direct-kernel-interface-usage situations that
> require pulling in the associated kernel headers, such as fb.h, kb.h,
> etc. One issue is that the kernel headers presumably lack prototypes
> for the functions, but the most portable way to do what you're doing
> is probably going to be:
>
> #include <linux/capabilities.h>
> #include <sys/syscall.h>
> ...
> syscall(SYS_capset, x, y);
I prefer leaving out kernel headers in my code. This old project uses an
old netfilter API, and still builds (I pulled the defined in).
It breaks at runtime, but that's a differerent story :)
> At least this will work on every Linux libc I know of, and it's
> hard for them to break it.
>
> If you really want to see the header in musl, I'm not opposed to
> further consideration of the matter, but right now I don't think
> adding it promotes portable software; rather it seems to promote
> writing software that works on musl and breaks on glibc.
The code is in libcap only. Glibc just provides the syscall interfaces
AFAIK.



    Igmar

Re: capset() capget() syscalls

[Igmar Palsenberg]

>> It is an unclear situation. Glibc seems inconsistent.
>>
>> Personally I think Musl should provide the kernel structures and
>> syscalls for everything that is not deprecated.
> In principle, I agree with you.
>
> My impression was that the kernel developers intend for this API to be
> deprecated for use by applications, and the only reason they haven't
> replaced it with a proper kernelspace API is that they assume you'll
> be using libcap which wraps/hides the ugliness (and replaces it with
> something else that's just ugly in different ways...).
>
> As such, it's sort of a borderline case. Do we really want to be
> promoting this API for use by applications?

I haven't seen a discussion on the capset() capget() API in ages in
lkml. My guess : It won't change any time soon.
It's use is very specific, so that narrows the change that it will change.

About the issue if we want to promote this : Yes, when it comes to
capabilities. Most apps don't need full access,
but only a subset of it (who wants to unload kernel modules from an
application ?). I'm a capabilities fan, not a fan of the API.

>> I agree with Linus, provide all the headers in libc. I tried to write
>> some code to include all syscalls and constants needed for them, and
>> as far as I can see it is impossible with glibc due to conflicts. If
>> anyone has a set of includes that works let me know....
> Can you explain what you mean here?
Linus' opinion is that including kernel headers in userspace code is the
road to destruction of this world. Or something
close ;)


Igmar

Re: capset() capget() syscalls

[Rich Felker]

On Thu, Sep 06, 2012 at 01:36:24PM +0200, Igmar Palsenberg wrote:
> On 9/5/12 4:24 PM, Rich Felker wrote:
> > On Wed, Sep 05, 2012 at 11:28:54AM +0200, Igmar Palsenberg wrote:
> >>>> Hi,
> >>>>
> >>>> Attached patch adds types to the existing syscalls, and defines the
> >>>> types in the header. All CAP_* defines are also added.
> >>> I'm curious what applications expect this header and definitions.
> >>> glibc does not have such a header. My impression is that it might have
> >>> existed at some point but that it was removed a long time ago, and
> >>> that any use of capabilities without libcap (which presumably contains
> >>> its own copy of the definitions for interfacing with the kernel) was
> >>> long ago deprecated.
> >>>
> >>> Can you clear these issues up?
> >> My own. Those definitions are part of the kernel syscall. While glibc
> >> does support structure definitions for most syscalls,
> >> capabilities seem to be an exception. (they are in linux/capabilities.h
> >> to be exact). Oh, and I really dislike the interfaces libcap provides.
> > I found them pretty ugly too, but I can't say I like the kernel API
> > any better... And the API in linux/capabilities.h is just horrible.
> > The typedefs are pointer types and there's no way to actually declare
> > the objects they point to without using the _-prefixed struct tag. I
> > don't see why they needed this hideous struct-pointer-based interface
> > rather than just passing a few words as syscall arguments...
> You only need cap_user_header_t and cap_user_data_t, and the CAP_ defines.
> The rest belongs to libcap, and aren't used by the kernel API.

The problem with these is that the typedefs are defined to the
pointer-to-struct type, not the struct type, and the actual struct
type seems to have a name that's not meant to be used by applications
(_-prefixed). This makes it impossible to actually declare objects of
the necessary type. And using typedef to a pointer type instead of the
base object type when you're not dealing with an 100% opaque type is
just Considered Harmful...

> We might consider altering the naming a bit. The __u32 stuff can be
> replace by proper typing.

Yes, that's not what I was talking about; it's easy to fix.

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Thu, Sep 06, 2012 at 11:28:04AM +0200, Szabolcs Nagy wrote:
> > For reference (this list is not complete), Musl is missing the
> > following syscalls that glibc has:
> > fallocate, acct, setns, sync_file_range, readahead, tee,
> > timerfd_create, timerfd_settime, timerfd_gettime
> 
> these are non-standard functions, but the non-broken
> ones should be provided eventually
> 
> the ugly ones are those which glibc declares in standard
> headers (eg readahead, tee)

I'm guessing they're under _GNU_SOURCE, right? If so, that's a good
reason not to make _GNU_SOURCE default in musl.. Anyway, these two
should be easy to add, but readahead is actually deprecated.

> (timerfd* functions are easy to provide as they live in
> their own header)

Yes. Somehow I thought I'd already added that. Must have been thinking
of eventfd... Adding it might fix a few broken programs that assume
timerfd exists when other Linux features that were added earlier were
already detected.

> some of them might be obsolete
> (eg posix_fallocate is in musl and can be used instead
> of fallocate, and posix_fadvise may be used instead of
> readahead)

Yes. Does fallocate need to be a wrapper for posix_fallocate, or can
it just be an alias?

> > And neither provide the non obsolete
> > clock_getres, clock_settime, clock_gettime
> 
> these are posix standard api and musl provides them
> if the necessary feature test macros are defined when
> time.h is included

Yeah, not sure what he was talking about here. The clock_* functions
are used all over musl as basically the only time-related functions we
use; all other time functions are wrappers for them.

Anyway, I'm happy to add the above missing functions.

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
> For reference (this list is not complete), Musl is missing the

Would you be willing to complete this list and submit a patch to add
all the missing syscalls?

> Musl has the following syscalls that glibc does not provide:
> mknod, mknodat, clock_nanosleep

glibc uses a messy __xmknod system; that's probably why you don't see
them. Also, glibc has all the clock_* functions in librt rather than
libc.

Rich

Re: capset() capget() syscalls

[Justin Cormack]

On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
>> For reference (this list is not complete), Musl is missing the
>
> Would you be willing to complete this list and submit a patch to add
> all the missing syscalls?
>

OK stage 1, I have gone through all the syscalls (on x64) to work out
what is missing. As people have been going and adding them there are
now not that many that should definitely be in.

missing and should definitely be in:
accept4
acct
dup3
ppoll
preadv
pwritev
recvmmsg
setdomainname
setns
syncfs

missing but fairly specialist (or not useful), not sure if some should
be exposed, not all in glibc (some are very new):
create_module
exit_group
futex
get_kernel_syms
clock_adjtime
get_thread_area
io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
Linux aio not posix aio)
kexec_load
lookup_dcookie
migrate_pages
mincore
modify_ldt
mqgetsetattr (used internally to provide mq interfaces)
name_to_handle_at
nfsservctl
open_by_handle_at
perf_event_open
getcpu
personality
process_vm_readv
process_vm_writev
query_module
quotactl
remap_file_pages
sched_setaffinity, sched_getaffinity (note glibc uses different
interface to syscalls)
sendmmsg (probably should be in but no man page on my box)
set_robust_list
set_thread_area
set_tid_address
sigaltstack
restart_syscall
tgkill

obsolete or unimplemented in Linux
_sysctl
epoll_ctl_old
epoll_wait_old
eventfd2 (I think this is just the normal call)
fanotify_init
fanotify_mark
futimesat
getpmsg
getrlimit
putpmsg
security
setrlimit
sysfs
tkill
tuxcall
uselib
ustat
vserver

unknown (eg no man page on my box, probably obsolete or should not be
exported, but check)
afs_syscall
arch_prctl
get_robust_list

syscall differs in prototype etc (probably missing some of these):
clone

only exposed as weak alias:
getdents
vfork

These are in libkeyutils even though they are syscalls:
add_key
request_key
keyctl

These are in libnuma, even though they are syscalls
move_pages
mbind
get_mempolicy
set_mempolicy

naming differences (including 64), ignore this list:
fadvise64 is posix_fadvise
fstat
fstatat, newfstatat
lstat
signalfd4
syslog is klogctl

Re: capset() capget() syscalls

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
>> For reference (this list is not complete), Musl is missing the
>
> Would you be willing to complete this list and submit a patch to add
> all the missing syscalls?

Attached is a first set.

setns, acct, accept4, dup3

[-- Attachment #2: calls.diff --]
[-- Type: application/octet-stream, Size: 2670 bytes --]

diff --git a/include/sched.h b/include/sched.h
index 9062772..3df4c7b 100644
--- a/include/sched.h
+++ b/include/sched.h
@@ -59,6 +59,7 @@ int     sched_yield(void);
 #define CLONE_IO	0x80000000
 int clone (int (*)(void *), void *, int, void *, ...);
 int unshare(int);
+int setns(int, int);
 #endif
 
 #ifdef __cplusplus
diff --git a/include/sys/socket.h b/include/sys/socket.h
index 7024d23..4d64336 100644
--- a/include/sys/socket.h
+++ b/include/sys/socket.h
@@ -259,6 +259,10 @@ int sockatmark (int);
 #define SHUT_WR 1
 #define SHUT_RDWR 2
 
+#ifdef _GNU_SOURCE
+int accept4(int, struct sockaddr *__restrict, socklen_t *__restrict, int);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/unistd.h b/include/unistd.h
index b5206a6..4f6953c 100644
--- a/include/unistd.h
+++ b/include/unistd.h
@@ -171,6 +171,7 @@ int daemon(int, int);
 void setusershell(void);
 void endusershell(void);
 char *getusershell(void);
+int acct(const char *);
 #endif
 
 #ifdef _GNU_SOURCE
@@ -181,6 +182,7 @@ int getresuid(uid_t *, uid_t *, uid_t *);
 int getresgid(gid_t *, gid_t *, gid_t *);
 char *get_current_dir_name(void);
 int pipe2(int [2], int);
+int dup3(int, int, int);
 #endif
 
 #if defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE)
diff --git a/src/linux/accept4.c b/src/linux/accept4.c
new file mode 100644
index 0000000..91db284
--- /dev/null
+++ b/src/linux/accept4.c
@@ -0,0 +1,9 @@
+#define _GNU_SOURCE
+#include <sys/socket.h>
+#include "syscall.h"
+#include "libc.h"
+
+int accept4(int fd, struct sockaddr *restrict addr, socklen_t *restrict len, int flg)
+{
+        return syscall(SYS_accept4, fd, addr, len, flg);
+}
diff --git a/src/linux/dup3.c b/src/linux/dup3.c
new file mode 100644
index 0000000..e201ef0
--- /dev/null
+++ b/src/linux/dup3.c
@@ -0,0 +1,10 @@
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <errno.h>
+#include "syscall.h"
+
+int dup3(int old, int new, int flags) {
+        int r;
+        while ((r=__syscall(SYS_dup3, old, new, flags))==-EBUSY);
+        return __syscall_ret(r);
+}
diff --git a/src/linux/setns.c b/src/linux/setns.c
new file mode 100644
index 0000000..af3eb09
--- /dev/null
+++ b/src/linux/setns.c
@@ -0,0 +1,9 @@
+#define _GNU_SOURCE
+#include <sched.h>
+#include "syscall.h"
+#include "libc.h"
+
+int setns(int fd, int nstype)
+{
+        return syscall(SYS_setns, fd, nstype);
+}
diff --git a/src/unistd/acct.c b/src/unistd/acct.c
new file mode 100644
index 0000000..9384712
--- /dev/null
+++ b/src/unistd/acct.c
@@ -0,0 +1,9 @@
+#define _GNU_SOURCE
+#include <unistd.h>
+#include "syscall.h"
+#include "libc.h"
+
+int acct(const char *filename)
+{
+        return syscall(SYS_acct, filename);
+}

Re: capset() capget() syscalls

[Rich Felker]

On Sat, Sep 08, 2012 at 05:02:35PM +0100, Justin Cormack wrote:
> On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
> >> For reference (this list is not complete), Musl is missing the
> >
> > Would you be willing to complete this list and submit a patch to add
> > all the missing syscalls?
> >
> 
> OK stage 1, I have gone through all the syscalls (on x64) to work out
> what is missing. As people have been going and adding them there are
> now not that many that should definitely be in.
> 
> missing and should definitely be in:
> accept4
> acct
> dup3
> ppoll
> preadv
> pwritev
> recvmmsg
> setdomainname
> setns
> syncfs

Agreed. Most of them are trivial to add, but acct may be problematic
unless we also add the ugly header with the struct definitions for
on-disk representation of process accounting info. Otherwise, programs
might assume it exists since acct exists and break when it doesn't.

Also, recvmmsg will require the same ugly kernel bug workarounds as
revcmsg requires, on 64-bit targets. Since there's not just one msghdr
to fix up, but a whole list of them, there may not be temporary
storage available for all of them. I think it might be easier/cleaner
to just unconditionally call recvmsg in a loop in this case,
implementing the MSG_WAITFORONE logic in userspace if possible. Of
course this eliminates all the performance advantages of recvmmsg on
64-bit machines. I think we really need to get the kernel folks to fix
the underlying bugs, and then we can make the workarounds conditional
on kernel version...

> missing but fairly specialist (or not useful), not sure if some should
> be exposed, not all in glibc (some are very new):
> create_module

Deprecated, part of the ancient (2.2? 2.0?) kernel module interface, I
think...

> exit_group

This is actually the syscall for the exit function. The syscall named
"exit" is the syscall for the pthread_exit function. It's confusing.

> futex

glibc does not expose this as far as I know. I would not be opposed to
having sys/futex.h however. It's less ugly than having to use
linux/futex.h and syscall()...

> get_kernel_syms

Probably deprecated..?

> clock_adjtime

Modern, should be exposed.

> get_thread_area

Largely useless, and intended only for use in implementing pthreads.

> io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
> Linux aio not posix aio)

Is there any legitimate user of this horrible API? Last I checked
these functions were only useful for block-aligned IO on block devices
and the impression I got was that they were designed solely for
Oracle's use. (puke) I suppose they could be useful for FUSE drivers
too, however. We can add them if they have legitimate software that
needs them.

> kexec_load

Useful, especially for certain embedded systems.

> lookup_dcookie
> migrate_pages
> mincore

Not sure about these; possibly useful.

> modify_ldt

Probably useful for dosemu or something...

> mqgetsetattr (used internally to provide mq interfaces)

Indeed; we already use it there.

> name_to_handle_at
> nfsservctl
> open_by_handle_at

Useful for userspace nfsd, maybe other similar things.

> perf_event_open
> getcpu
> personality

Not sure about these.

> process_vm_readv
> process_vm_writev

These are supported.

> query_module

Almost surely deprecated.

> quotactl
> remap_file_pages

Probably useful. Don't know details.

> sched_setaffinity, sched_getaffinity (note glibc uses different
> interface to syscalls)

Useful, I think, but we should provide the libc api, not the low-level
syscall api.

> sendmmsg (probably should be in but no man page on my box)

Same issue as recvmmsg. Very useful but likely buggy.

> set_robust_list
> set_thread_area
> set_tid_address

Only meaningful to the pthreads implementation, and already used
internally there. Any direct use of these by an application is likely
to cause dangerous breakage of implementation internals.

> sigaltstack

Already used.

> restart_syscall

This is a kernel implementation detail related to signal handling and
avoiding EINTR.

> tgkill

Used internally in pthread. Not sure if it's useful to provide to apps
too..

> obsolete or unimplemented in Linux
> [...]

Agree.


> unknown (eg no man page on my box, probably obsolete or should not be
> exported, but check)
> afs_syscall
> arch_prctl
> get_robust_list

arch_prctl is used internally for threads on x86_64. The others seem
useless.

> syscall differs in prototype etc (probably missing some of these):
> clone

We provide clone.

> only exposed as weak alias:
> getdents
> vfork

There's no "only" to it; the reason it's weak is so that we can use
the namespace-safe __-prefixed names for these functions without
polluting the external namespace.

> These are in libkeyutils even though they are syscalls:
> add_key
> request_key
> keyctl

Not even sure what they're for. But I don't think they're a priority
if glibc does not even have them internally.

> These are in libnuma, even though they are syscalls
> move_pages
> mbind
> get_mempolicy
> set_mempolicy

Same.

> naming differences (including 64), ignore this list:
> fadvise64 is posix_fadvise
> fstat
> fstatat, newfstatat
> lstat
> signalfd4
> syslog is klogctl

Indeed, these can be ignored.

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Sat, Sep 08, 2012 at 10:25:32PM +0100, Justin Cormack wrote:
> On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
> >> For reference (this list is not complete), Musl is missing the
> >
> > Would you be willing to complete this list and submit a patch to add
> > all the missing syscalls?
> 
> Attached is a first set.
> 
> setns, acct, accept4, dup3

I'd like the associated struct-defining header for acct too; otherwise
I worry apps that see acct will assume it exists and then break...

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Sat, Sep 08, 2012 at 06:15:38PM -0400, Rich Felker wrote:
> On Sat, Sep 08, 2012 at 10:25:32PM +0100, Justin Cormack wrote:
> > On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> > > On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
> > >> For reference (this list is not complete), Musl is missing the
> > >
> > > Would you be willing to complete this list and submit a patch to add
> > > all the missing syscalls?
> > 
> > Attached is a first set.
> > 
> > setns, acct, accept4, dup3
> 
> I'd like the associated struct-defining header for acct too; otherwise
> I worry apps that see acct will assume it exists and then break...

I'm adding it myself and applying your patch.

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Sat, Sep 08, 2012 at 10:25:32PM +0100, Justin Cormack wrote:
> On Fri, Sep 7, 2012 at 5:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Thu, Sep 06, 2012 at 09:22:58AM +0100, Justin Cormack wrote:
> >> For reference (this list is not complete), Musl is missing the
> >
> > Would you be willing to complete this list and submit a patch to add
> > all the missing syscalls?
> 
> Attached is a first set.
> 
> setns, acct, accept4, dup3

Another issue: on i386, accept4 requires an additional socketcall
constant. I'm adding it in, and using socketcall_cp to make the call,
since presumably accept4 should be a cancellation point analogous to
accept. Does that sound reasonable?

Rich

Re: capset() capget() syscalls

[Justin Cormack]

On Sun, Sep 9, 2012 at 1:21 AM, Rich Felker <dalias@aerifal.cx> wrote:
> Another issue: on i386, accept4 requires an additional socketcall
> constant. I'm adding it in, and using socketcall_cp to make the call,
> since presumably accept4 should be a cancellation point analogous to
> accept. Does that sound reasonable?
>
> Rich

Sounds good.

Justin

Re: capset() capget() syscalls

[Justin Cormack]

Here is an updated version of that list.


Syscalls not in Musl

missing and should definitely be in:
ppoll
preadv
pwritev
setdomainname
syncfs
clock_adjtime
remap_file_pages
kexec_load
mincore

some issues, may wait until decide how to resolve
recvmmsg
sendmmsg

useful but perhaps lower priority
futex
mqgetsetattr
lookup_dcookie
modify_ldt
name_to_handle_at
nfsservctl
open_by_handle_at
perf_event_open
getcpu
personality
quotactl
sched_setaffinity, sched_getaffinity (note glibc uses different
interface to syscalls)

may implement if someone has a real use
io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
Linux aio not posix aio)

obsolete or unimplemented in Linux
_sysctl
epoll_ctl_old
epoll_wait_old
eventfd2
fanotify_init
fanotify_mark
futimesat
getpmsg
getrlimit
putpmsg
security
setrlimit
sysfs
tkill
tuxcall
uselib
ustat
vserver
create_module
query_module
afs_syscall
get_robust_list
get_kernel_syms

These are in libkeyutils even though they are syscalls, so will not
implement now
add_key
request_key
keyctl

These are in libnuma, even though they are syscalls, so will not implement now
move_pages
mbind
get_mempolicy
set_mempolicy
migrate_pages

Re: capset() capget() syscalls

[Justin Cormack]

On Sat, Sep 8, 2012 at 11:14 PM, Rich Felker <dalias@aerifal.cx> wrote:
> Is there any legitimate user of this horrible API? Last I checked
> these functions were only useful for block-aligned IO on block devices
> and the impression I got was that they were designed solely for
> Oracle's use. (puke) I suppose they could be useful for FUSE drivers
> too, however. We can add them if they have legitimate software that
> needs them.

It is still the only way to get decent io performance out of an SSD as
far as I can see, being about a factor of 8 faster than anything else
for random reads eg posix aio. Assuming of course our reads are block
aligned.

Justin

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 09:04:02PM +0100, Justin Cormack wrote:
> On Sat, Sep 8, 2012 at 11:14 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > Is there any legitimate user of this horrible API? Last I checked
> > these functions were only useful for block-aligned IO on block devices
> > and the impression I got was that they were designed solely for
> > Oracle's use. (puke) I suppose they could be useful for FUSE drivers
> > too, however. We can add them if they have legitimate software that
> > needs them.
> 
> It is still the only way to get decent io performance out of an SSD as
> far as I can see, being about a factor of 8 faster than anything else
> for random reads eg posix aio. Assuming of course our reads are block
> aligned.

If the normal Linux block cache system is even 25% slower than
low-level IO on the device without any operating system at all, that's
a major bug they need to fix. Adding hideous APIs to for userspace
apps to do their own low-level IO is not a solution. 8x is just
atrocious. Where does that figure come from?

In fairness, if you're comparing to POSIX aio, that's a pretty bad API
too. A thread performing standard blocking pread/pwrite should give
the ideal performance.

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 08:40:25PM +0100, Justin Cormack wrote:
> Here is an updated version of that list.
> 
> 
> Syscalls not in Musl
> 
> missing and should definitely be in:
> ppoll
> preadv
> pwritev
> setdomainname
> mincore

Added all of these; hopefully they work.

> syncfs
> clock_adjtime
> remap_file_pages
> kexec_load

Pending but easy.

BTW, some cleanup in the tree organization is still needed. I noticed
src/linux has things like wait3/wait4 as well as the newly added dup3.
Some of these are historical practice or improved analogues of
standard functions, and probably belong alongside the standard
functionality they go with. I'd eventually like src/linux to be JUST
"linux features" like epoll, timerfd, etc. - all the stuff that's
completely new features/functionality invented as part of Linux.

> some issues, may wait until decide how to resolve
> recvmmsg
> sendmmsg

Indeed. We need to open a dialogue with the kernel folks about this...

> useful but perhaps lower priority
> futex
> mqgetsetattr
> lookup_dcookie
> modify_ldt
> name_to_handle_at
> nfsservctl
> open_by_handle_at
> perf_event_open
> getcpu
> personality
> quotactl
> sched_setaffinity, sched_getaffinity (note glibc uses different
> interface to syscalls)

How did you assign priority? This set looks a lot more _useful_ than
the first set, whereas the first set almost surely appears more widely
in legacy software without an option to omit its use...

> may implement if someone has a real use
> io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
> Linux aio not posix aio)
> 
> obsolete or unimplemented in Linux
> [...]
> fanotify_init
> fanotify_mark

These are obsolete already? I thought fanotify was new...

> [...]
> get_robust_list

This is not obsolete; it's just for implementation-internal use only.

Rich

Re: capset() capget() syscalls

[Justin Cormack]

On Sun, Sep 9, 2012 at 10:02 PM, Rich Felker <dalias@aerifal.cx> wrote:
> On Sun, Sep 09, 2012 at 08:40:25PM +0100, Justin Cormack wrote:
>> Here is an updated version of that list.
>>
>>
>> Syscalls not in Musl
>>
>> missing and should definitely be in:
>> ppoll
>> preadv
>> pwritev
>> setdomainname
>> mincore
>
> Added all of these; hopefully they work.
>
>> syncfs
>> clock_adjtime
>> remap_file_pages
>> kexec_load
>
> Pending but easy.
>
> BTW, some cleanup in the tree organization is still needed. I noticed
> src/linux has things like wait3/wait4 as well as the newly added dup3.
> Some of these are historical practice or improved analogues of
> standard functions, and probably belong alongside the standard
> functionality they go with. I'd eventually like src/linux to be JUST
> "linux features" like epoll, timerfd, etc. - all the stuff that's
> completely new features/functionality invented as part of Linux.

Makes sense.

>> some issues, may wait until decide how to resolve
>> recvmmsg
>> sendmmsg
>
> Indeed. We need to open a dialogue with the kernel folks about this...
>
>> useful but perhaps lower priority
>> futex
>> mqgetsetattr
>> lookup_dcookie
>> modify_ldt
>> name_to_handle_at
>> nfsservctl
>> open_by_handle_at
>> perf_event_open
>> getcpu
>> personality
>> quotactl
>> sched_setaffinity, sched_getaffinity (note glibc uses different
>> interface to syscalls)
>
> How did you assign priority? This set looks a lot more _useful_ than
> the first set, whereas the first set almost surely appears more widely
> in legacy software without an option to omit its use...

Probably no real rationale. The second set are probably more useful to
fewer programs. Merge the lists if you like.

>> may implement if someone has a real use
>> io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
>> Linux aio not posix aio)
>>
>> obsolete or unimplemented in Linux
>> [...]
>> fanotify_init
>> fanotify_mark
>
> These are obsolete already? I thought fanotify was new...

Sorry, my mistake I was misremembering whatever there was before
inotify. These should be in (also need a bunch of structs etc for the
netlink part of the API I think).

>> [...]
>> get_robust_list
>
> This is not obsolete; it's just for implementation-internal use only.

Ah ok.

Justin

Re: capset() capget() syscalls

[James Cloos]

>>>>> "RF" == Rich Felker <dalias@aerifal.cx> writes:

>> futex

RF> glibc does not expose this as far as I know. I would not be opposed
RF> to having sys/futex.h however. It's less ugly than having to use
RF> linux/futex.h and syscall()...

+1.  <sys/futex.h> and a separate library seems to be the best way to
support the futex interface.  Such a lib might try to be portable with
support for other lock and semaphore apis when futex(7)s are unavailable.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: capset() capget() syscalls

[Justin Cormack]

Here is the updated list then without any ordering. Will work on some of them.

Syscalls not in Musl that should be

syncfs
clock_adjtime
remap_file_pages
kexec_load
futex
mqgetsetattr
lookup_dcookie
modify_ldt
name_to_handle_at
nfsservctl
open_by_handle_at
perf_event_open
getcpu
personality
quotactl
sched_setaffinity, sched_getaffinity (note glibc uses different
interface to syscalls)

some issues, may wait until decide how to resolve
recvmmsg
sendmmsg

may implement if someone has a real use
io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
Linux aio not posix aio)

Re: capset() capget() syscalls

[Justin Cormack]

Trying again (missed two)


Syscalls not in Musl that should be

syncfs
clock_adjtime
remap_file_pages
kexec_load
futex
mqgetsetattr
lookup_dcookie
modify_ldt
name_to_handle_at
nfsservctl
open_by_handle_at
perf_event_open
getcpu
personality
quotactl
sched_setaffinity, sched_getaffinity (note glibc uses different
interface to syscalls)
fanotify_init
fanotify_mark

some issues, may wait until decide how to resolve
recvmmsg
sendmmsg

may implement if someone has a real use
io_cancel, io_destroy, io_getevents, io_setup, io_submit (ie native
Linux aio not posix aio)

These are in libkeyutils even though they are syscalls, so will not
implement now
add_key
request_key
keyctl

These are in libnuma, even though they are syscalls, so will not implement now
move_pages
mbind
get_mempolicy
set_mempolicy
migrate_pages

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 05:13:55PM -0400, James Cloos wrote:
> >>>>> "RF" == Rich Felker <dalias@aerifal.cx> writes:
> 
> >> futex
> 
> RF> glibc does not expose this as far as I know. I would not be opposed
> RF> to having sys/futex.h however. It's less ugly than having to use
> RF> linux/futex.h and syscall()...
> 
> +1.  <sys/futex.h> and a separate library seems to be the best way to
> support the futex interface.  Such a lib might try to be portable with
> support for other lock and semaphore apis when futex(7)s are unavailable.

With something like futex that's a trivial syscall, I'm not sure what
qualifies it for a separate lib rather than inclusion in libc. The
library would be <20 bytes of code on most archs... That's not to say
I think we have to expose it in musl, but I think using a separate
library (especially if there's any chance of it being built as a .so
rather than just .a) is a worse choice than just writing
syscall(SYS_futex, ...) in your source...

Rich

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 10:31:28PM +0100, Justin Cormack wrote:
> Trying again (missed two)

And you added mqgetsetattr; this is just the backend of the
corresponding mq_* functions.

> Syscalls not in Musl that should be
> 
> syncfs
> clock_adjtime
> remap_file_pages
> kexec_load
> futex
> mqgetsetattr
> lookup_dcookie
> modify_ldt
> name_to_handle_at
> nfsservctl
> open_by_handle_at
> perf_event_open
> getcpu
> personality
> quotactl
> sched_setaffinity, sched_getaffinity (note glibc uses different
> interface to syscalls)
> fanotify_init
> fanotify_mark

Otherwise, this list looks okay. Would you be interested in preparing
a patch, and/or testing the ones I already added?

Rich

Re: capset() capget() syscalls

[Justin Cormack]

On Sun, Sep 9, 2012 at 10:55 PM, Rich Felker <dalias@aerifal.cx> wrote:
> With something like futex that's a trivial syscall, I'm not sure what
> qualifies it for a separate lib rather than inclusion in libc. The
> library would be <20 bytes of code on most archs... That's not to say
> I think we have to expose it in musl, but I think using a separate
> library (especially if there's any chance of it being built as a .so
> rather than just .a) is a worse choice than just writing
> syscall(SYS_futex, ...) in your source...

The manpage says

       To reiterate, bare futexes are not intended as an easy-to-use
abstraction for end-users.  (There
       is no wrapper function for this system call in glibc.)
Implementors are expected to be assembly
       literate and to have read the sources of the futex userspace
library referenced below.

I thought (from memory) the non-contended case required architecture
specific assembly, the syscall just dealt with contended case. So the
library needs to provide the other stuff to make it usable. Its not a
lot, and must be there to use these for threading I guess, but doesnt
have standard names?

Justin

Re: capset() capget() syscalls

[Justin Cormack]

On Sun, Sep 9, 2012 at 10:58 PM, Rich Felker <dalias@aerifal.cx> wrote:
> On Sun, Sep 09, 2012 at 10:31:28PM +0100, Justin Cormack wrote:
>> Trying again (missed two)
>
> And you added mqgetsetattr; this is just the backend of the
> corresponding mq_* functions.
>
>> Syscalls not in Musl that should be
>>
>> syncfs
>> clock_adjtime
>> remap_file_pages
>> kexec_load
>> futex
>> mqgetsetattr
>> lookup_dcookie
>> modify_ldt
>> name_to_handle_at
>> nfsservctl
>> open_by_handle_at
>> perf_event_open
>> getcpu
>> personality
>> quotactl
>> sched_setaffinity, sched_getaffinity (note glibc uses different
>> interface to syscalls)
>> fanotify_init
>> fanotify_mark
>
> Otherwise, this list looks okay. Would you be interested in preparing
> a patch, and/or testing the ones I already added?
>

Ok almost right then. Happy to do patches and test.

Justin

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 11:12:01PM +0100, Justin Cormack wrote:
> On Sun, Sep 9, 2012 at 10:55 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > With something like futex that's a trivial syscall, I'm not sure what
> > qualifies it for a separate lib rather than inclusion in libc. The
> > library would be <20 bytes of code on most archs... That's not to say
> > I think we have to expose it in musl, but I think using a separate
> > library (especially if there's any chance of it being built as a .so
> > rather than just .a) is a worse choice than just writing
> > syscall(SYS_futex, ...) in your source...
> 
> The manpage says
> 
>        To reiterate, bare futexes are not intended as an easy-to-use
> abstraction for end-users.  (There
>        is no wrapper function for this system call in glibc.)
> Implementors are expected to be assembly
>        literate and to have read the sources of the futex userspace
> library referenced below.
> 
> I thought (from memory) the non-contended case required architecture
> specific assembly, the syscall just dealt with contended case. So the
> library needs to provide the other stuff to make it usable. Its not a
> lot, and must be there to use these for threading I guess, but doesnt
> have standard names?

"GNU C" (GCC-specific) or C11 atomics will work perfectly well in
place of assembly; the whole point of exposing the futex syscall to
the application would be for it to do its own waiting in conjunction
with atomics (whether implemented with GNUC, C11, or asm). The futex
API is mainly for waiting on locks or waking waiters; for the most
part it does not even provide an API for obtaining locks, although the
prio-inheritance stuff can provide full locking in kernelspace, albeit
very inefficiently. We're talking 200x slower than calling
pthread_mutex_lock, so it would be utterly useless..

Rich

Re: capset() capget() syscalls

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

On 9 Sep 2012 23:26, "Rich Felker" <dalias@aerifal.cx> wrote:
>
> On Sun, Sep 09, 2012 at 11:12:01PM +0100, Justin Cormack wrote:
> > On Sun, Sep 9, 2012 at 10:55 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > > With something like futex that's a trivial syscall, I'm not sure what
> > > qualifies it for a separate lib rather than inclusion in libc. The
> > > library would be <20 bytes of code on most archs... That's not to say
> > > I think we have to expose it in musl, but I think using a separate
> > > library (especially if there's any chance of it being built as a .so
> > > rather than just .a) is a worse choice than just writing
> > > syscall(SYS_futex, ...) in your source...
> >
> > The manpage says
> >
> >        To reiterate, bare futexes are not intended as an easy-to-use
> > abstraction for end-users.  (There
> >        is no wrapper function for this system call in glibc.)
> > Implementors are expected to be assembly
> >        literate and to have read the sources of the futex userspace
> > library referenced below.
> >
> > I thought (from memory) the non-contended case required architecture
> > specific assembly, the syscall just dealt with contended case. So the
> > library needs to provide the other stuff to make it usable. Its not a
> > lot, and must be there to use these for threading I guess, but doesnt
> > have standard names?
>
> "GNU C" (GCC-specific) or C11 atomics will work perfectly well in
> place of assembly; the whole point of exposing the futex syscall to
> the application would be for it to do its own waiting in conjunction
> with atomics (whether implemented with GNUC, C11, or asm). The futex
> API is mainly for waiting on locks or waking waiters; for the most
> part it does not even provide an API for obtaining locks, although the
> prio-inheritance stuff can provide full locking in kernelspace, albeit
> very inefficiently. We're talking 200x slower than calling
> pthread_mutex_lock, so it would be utterly useless..
>
> Rich

I should make myself a working example then. It sounds rather simpler than
the wording on the man page...

Thanks

[-- Attachment #2: Type: text/html, Size: 2689 bytes --]

Re: capset() capget() syscalls

[James Cloos]

>>>>> "JC" == Justin Cormack <justin@specialbusservice.com> writes:

JC> I should make myself a working example then. It sounds rather
JC> simpler than the wording on the man page...

Likewise.  The wording in futex(2) also is why I reacted as I did.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 09, 2012 at 11:14:58PM +0100, Justin Cormack wrote:
> On Sun, Sep 9, 2012 at 10:58 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Sun, Sep 09, 2012 at 10:31:28PM +0100, Justin Cormack wrote:
> >> Trying again (missed two)
> >
> > And you added mqgetsetattr; this is just the backend of the
> > corresponding mq_* functions.
> >
> >> Syscalls not in Musl that should be
> >>
> >> syncfs
> >> clock_adjtime
> >> remap_file_pages
> >> kexec_load
> >> futex
> >> mqgetsetattr
> >> lookup_dcookie
> >> modify_ldt
> >> name_to_handle_at
> >> nfsservctl
> >> open_by_handle_at
> >> perf_event_open
> >> getcpu
> >> personality
> >> quotactl
> >> sched_setaffinity, sched_getaffinity (note glibc uses different
> >> interface to syscalls)
> >> fanotify_init
> >> fanotify_mark
> >
> > Otherwise, this list looks okay. Would you be interested in preparing
> > a patch, and/or testing the ones I already added?
> >
> 
> Ok almost right then. Happy to do patches and test.

Any progress?

Rich

Re: capset() capget() syscalls

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 153 bytes --]

> Any progress?
>
> Rich

Sorry was a bit tied up this week. Here are three more, will keep on
working on them, should have a bit more time now.

Justin

[-- Attachment #2: syscalls.patch --]
[-- Type: application/octet-stream, Size: 2524 bytes --]

diff --git a/include/sys/mman.h b/include/sys/mman.h
index 136b45b..9a1e60f 100644
--- a/include/sys/mman.h
+++ b/include/sys/mman.h
@@ -10,6 +10,10 @@ extern "C" {
 #define __NEED_size_t
 #define __NEED_off_t
 
+#if defined(_GNU_SOURCE)
+#define __NEED_ssize_t
+#endif
+
 #include <bits/alltypes.h>
 
 #include <bits/mman.h>
@@ -29,6 +33,7 @@ int munlockall (void);
 
 #ifdef _GNU_SOURCE
 void *mremap (void *, size_t, size_t, int, ...);
+int remap_file_pages (void *, size_t, int, ssize_t, int);
 #endif
 
 #if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
diff --git a/include/sys/timex.h b/include/sys/timex.h
index 1262392..5d9957f 100644
--- a/include/sys/timex.h
+++ b/include/sys/timex.h
@@ -7,6 +7,12 @@ extern "C" {
 
 #include <sys/time.h>
 
+#if defined(_GNU_SOURCE)
+#define __NEED_clockid_t
+#endif
+
+#include <bits/alltypes.h>
+
 struct ntptimeval {
 	struct timeval time;
 	long maxerror, esterror;
@@ -84,6 +90,11 @@ struct timex {
 
 int adjtimex(struct timex *);
 
+
+#if defined(_GNU_SOURCE)
+int clock_adjtime(clockid_t, struct timex *);
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/include/unistd.h b/include/unistd.h
index d3bb781..985d279 100644
--- a/include/unistd.h
+++ b/include/unistd.h
@@ -184,6 +184,7 @@ int getresgid(gid_t *, gid_t *, gid_t *);
 char *get_current_dir_name(void);
 int pipe2(int [2], int);
 int dup3(int, int, int);
+void syncfs(int);
 #endif
 
 #if defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE)
diff --git a/src/linux/clock_adjtime.c b/src/linux/clock_adjtime.c
new file mode 100644
index 0000000..1fc9bef
--- /dev/null
+++ b/src/linux/clock_adjtime.c
@@ -0,0 +1,9 @@
+#define _GNU_SOURCE
+#include <time.h>
+#include <sys/timex.h>
+#include "syscall.h"
+
+int clock_adjtime (clockid_t clock_id, struct timex *utx)
+{
+	return syscall(SYS_clock_adjtime, clock_id, utx);
+}
diff --git a/src/linux/remap_file_pages.c b/src/linux/remap_file_pages.c
new file mode 100644
index 0000000..f95c4cc
--- /dev/null
+++ b/src/linux/remap_file_pages.c
@@ -0,0 +1,8 @@
+#define _GNU_SOURCE
+#include <sys/mman.h>
+#include "syscall.h"
+
+int remap_file_pages(void *addr, size_t size, int prot, ssize_t pgoff, int flags)
+{
+	return syscall(SYS_remap_file_pages, addr, size, prot, pgoff, flags);
+}
diff --git a/src/linux/syncfs.c b/src/linux/syncfs.c
new file mode 100644
index 0000000..fe2b8a7
--- /dev/null
+++ b/src/linux/syncfs.c
@@ -0,0 +1,8 @@
+#define _GNU_SOURCE
+#include <unistd.h>
+#include "syscall.h"
+
+void syncfs(int fd)
+{
+	__syscall(SYS_syncfs, fd);
+}

Re: capset() capget() syscalls

[Rich Felker]

On Sun, Sep 16, 2012 at 03:13:25PM +0100, Justin Cormack wrote:
> > Any progress?
> >
> > Rich
> 
> Sorry was a bit tied up this week. Here are three more, will keep on
> working on them, should have a bit more time now.

Applied, with minor changes; sys/timex.h is not a standard header, and
in most/all cases for such headers musl just exposes all the
functionality whenever they're included.

Rich