Post-1.1.1 plans

Post-1.1.1 plans

[Rich Felker]

A few things I want to work on either adding or making decisions
about, aside from the existing roadmap items:

- Stuff left in the "Open issues at end of April" thread:
  - Renaming _start in dynamic linker (annoying for debugging)
  - Resolving max_align_t issue and adding it
  - Pending __xmknod and __sysv_signal patch (ABI compat)
  - Pending fmtmsg patch (probably ok as-is)
  - Adding recvmmsg/sendmmsg (waiting on Timo, I think)
  - The if_nameindex/getifaddrs issue
- LINE_MAX vs sysconf(_SC_LINE_MAX) issue
- Proposed errc/etc. addition to err.h
- And error.h?
- Whether to add default timezone from /etc?
- reallocarray and explicit_bzero request

There may be other open things I'm missing; let me know if you have a
request that doesn't appear above.

Rich

Re: Post-1.1.1 plans

[Laurent Bercot]

> There may be other open things I'm missing; let me know if you have a
> request that doesn't appear above.

  I would like to have a serious design discussion about how to implement
leap second support in musl. I haven't taken the time to have a serious
go at it since I first mentioned it, but I have thought about it for a
bit and it is indeed more complex than it appears - because the standard
way of supporting them, i.e. in timezone files, is broken. (Timezones are
a process-wide setting since the default can be overriden with TZ, but
interpretation of the system clock as a TAI or UTC clock should be a
system-wide setting.)
  So, before diving in and writing some more code that won't be merged in,
I'd like to assess exactly what would be acceptable for musl and what
would not.

  A probable spin-off of this discussion is a subject in itself: what would
be an acceptable format, a set of conventions to follow, to add
musl-specific extensions outside of POSIX (or whatever superset of POSIX
musl implements).

-- 
  Laurent

Re: Post-1.1.1 plans

[Rich Felker]

On Wed, May 21, 2014 at 02:01:03AM +0100, Laurent Bercot wrote:
> >There may be other open things I'm missing; let me know if you have a
> >request that doesn't appear above.
> 
>  I would like to have a serious design discussion about how to implement
> leap second support in musl. I haven't taken the time to have a serious
> go at it since I first mentioned it, but I have thought about it for a
> bit and it is indeed more complex than it appears - because the standard
> way of supporting them, i.e. in timezone files, is broken. (Timezones are
> a process-wide setting since the default can be overriden with TZ, but
> interpretation of the system clock as a TAI or UTC clock should be a
> system-wide setting.)
>  So, before diving in and writing some more code that won't be merged in,
> I'd like to assess exactly what would be acceptable for musl and what
> would not.
> 
>  A probable spin-off of this discussion is a subject in itself: what would
> be an acceptable format, a set of conventions to follow, to add
> musl-specific extensions outside of POSIX (or whatever superset of POSIX
> musl implements).

I really don't know. My recollection from the previous thread was that
we kept running into situations where the behavior would be
non-conforming. I've since been told by glibc developers that, on
glibc, setting TZ to a zone with leapseconds results in a discrepency
between the results of gmtime() and localtime() -- they don't differ
just by the local offset but also by the leap seconds offset. If
correct, this is obviously atrociously bad behavior, but at least it
seems to be conforming.

So I don't really know what to tell you. As you know I'm rather an
anti-fan of leapseconds/TAI, so while in some ways it's an interesting
and challenging problem, it's not something I'm really excited about
spending my mental energy on.

Probably in the short term, the best thing you can do is keep whatever
patches work for you; it's fine to link them on the wiki, and it would
be great to document what properties of libc they affect (e.g.
thread-safety of interfaces, functions that newly depend on
environment, etc.). This would also help in evaluating whether the
patches would eventually be acceptable upstream.

I also really question whether there's a superior kernel-based
approach to the transformation of time_t. I haven't read it in detail,
but there's a thread here that may be relevant:

http://lkml.iu.edu//hypermail/linux/kernel/1205.2/01644.html

If this interface was committed and works, the way to configure
leapseconds for the libc time_t conversion layer might be simple: ask
the kernel how the system clock is configured. That's certainly
cleaner than having environment variables and inconsistent per-process
settings.

Rich

Re: Post-1.1.1 plans

[Timo Teras]

On Tue, 20 May 2014 18:33:25 -0400
Rich Felker <dalias@libc.org> wrote:

> A few things I want to work on either adding or making decisions
> about, aside from the existing roadmap items:
> 
> - Stuff left in the "Open issues at end of April" thread:
>   - Renaming _start in dynamic linker (annoying for debugging)
>   - Resolving max_align_t issue and adding it
>   - Pending __xmknod and __sysv_signal patch (ABI compat)
>   - Pending fmtmsg patch (probably ok as-is)
>   - Adding recvmmsg/sendmmsg (waiting on Timo, I think)

I don't really have a good idea on how to do it nicely. Perhaps try to
fix kernel on 64-bits?

>   - The if_nameindex/getifaddrs issue
> - LINE_MAX vs sysconf(_SC_LINE_MAX) issue
> - Proposed errc/etc. addition to err.h
> - And error.h?
> - Whether to add default timezone from /etc?

We just default to ":localtime" if TZ is not defined:
http://git.alpinelinux.org/cgit/aports/plain/main/musl/2001-default-to-localtime-timezone-if-TZ-is-undefined.patch

and we make /etc/zoneinfo/localtime a symlink to (or a copy of) the
wanted timezone. Works perfect.

> - reallocarray and explicit_bzero request
> 
> There may be other open things I'm missing; let me know if you have a
> request that doesn't appear above.

Proper backtrace from core dumps of dynamic linked apps. At least to
get analyzing core dumps working. I think adding CFI is 90% of this.
But IIRC, gdb does not reliably detect mappings of shared objects. That
might need something additional.

thread_db support would be also nice, but that falls into gdbing
running processes.

- Timo

Re: Post-1.1.1 plans

[Rich Felker]

On Wed, May 21, 2014 at 09:15:42AM +0300, Timo Teras wrote:
> On Tue, 20 May 2014 18:33:25 -0400
> Rich Felker <dalias@libc.org> wrote:
> 
> > A few things I want to work on either adding or making decisions
> > about, aside from the existing roadmap items:
> > 
> > - Stuff left in the "Open issues at end of April" thread:
> >   - Renaming _start in dynamic linker (annoying for debugging)
> >   - Resolving max_align_t issue and adding it
> >   - Pending __xmknod and __sysv_signal patch (ABI compat)
> >   - Pending fmtmsg patch (probably ok as-is)
> >   - Adding recvmmsg/sendmmsg (waiting on Timo, I think)
> 
> I don't really have a good idea on how to do it nicely. Perhaps try to
> fix kernel on 64-bits?

Repeatedly calling sendmsg does not work because timeout can't be
honored, but repeatedly calling sendmmsg with a count of 1 seems
viable (and likewise for recvmmsg but I don't remember if it's as
difficult or not; need to look back at archives).

As for the const issue, glibc seems to have resolved to remove it. I
think we can do the same but perhaps we should document (when we do)
that the contents of the timespec after return are unspecified.

> >   - The if_nameindex/getifaddrs issue
> > - LINE_MAX vs sysconf(_SC_LINE_MAX) issue
> > - Proposed errc/etc. addition to err.h
> > - And error.h?
> > - Whether to add default timezone from /etc?
> 
> We just default to ":localtime" if TZ is not defined:
> http://git.alpinelinux.org/cgit/aports/plain/main/musl/2001-default-to-localtime-timezone-if-TZ-is-undefined.patch
> 
> and we make /etc/zoneinfo/localtime a symlink to (or a copy of) the
> wanted timezone. Works perfect.

Yes, that probably works fine, but it may be desirable to have more
out-of-the-box compatibility with the way glibc does it for running
binaries on non-musl-based systems. (Not decided on this; just think
some people might prefer it.)

> > - reallocarray and explicit_bzero request
> > 
> > There may be other open things I'm missing; let me know if you have a
> > request that doesn't appear above.
> 
> Proper backtrace from core dumps of dynamic linked apps. At least to
> get analyzing core dumps working. I think adding CFI is 90% of this.
> But IIRC, gdb does not reliably detect mappings of shared objects. That
> might need something additional.

Do you have details on that?

> thread_db support would be also nice, but that falls into gdbing
> running processes.

Have you seen the one-line patch to gdb that seems to make everything
work fine without it?

Rich

Re: Post-1.1.1 plans

[Rich Felker]

On Tue, May 20, 2014 at 06:33:25PM -0400, Rich Felker wrote:
> A few things I want to work on either adding or making decisions
> about, aside from the existing roadmap items:
> 
> - Stuff left in the "Open issues at end of April" thread:
>   - Renaming _start in dynamic linker (annoying for debugging)
>   - Resolving max_align_t issue and adding it
>   - Pending __xmknod and __sysv_signal patch (ABI compat)
>   - Pending fmtmsg patch (probably ok as-is)
>   - Adding recvmmsg/sendmmsg (waiting on Timo, I think)
>   - The if_nameindex/getifaddrs issue
> - LINE_MAX vs sysconf(_SC_LINE_MAX) issue
> - Proposed errc/etc. addition to err.h
> - And error.h?
> - Whether to add default timezone from /etc?
> - reallocarray and explicit_bzero request

One more I omitted: I have a pending patch for siginfo_t that fixes up
some technical issues with the hideous union hackery.

Rich

Re: Post-1.1.1 plans

[Szabolcs Nagy]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

* Rich Felker <dalias@libc.org> [2014-05-22 00:44:45 -0400]:
> On Tue, May 20, 2014 at 06:33:25PM -0400, Rich Felker wrote:
> > A few things I want to work on either adding or making decisions
> > about, aside from the existing roadmap items:
> > 
> > - Stuff left in the "Open issues at end of April" thread:
> >   - Renaming _start in dynamic linker (annoying for debugging)
> >   - Resolving max_align_t issue and adding it
> >   - Pending __xmknod and __sysv_signal patch (ABI compat)
> >   - Pending fmtmsg patch (probably ok as-is)
> >   - Adding recvmmsg/sendmmsg (waiting on Timo, I think)
> >   - The if_nameindex/getifaddrs issue
> > - LINE_MAX vs sysconf(_SC_LINE_MAX) issue
> > - Proposed errc/etc. addition to err.h
> > - And error.h?
> > - Whether to add default timezone from /etc?
> > - reallocarray and explicit_bzero request
> 
> One more I omitted: I have a pending patch for siginfo_t that fixes up
> some technical issues with the hideous union hackery.

i attach a mips RLIM_INFINITY patch so does not get forgotten

[-- Attachment #2: 0001-workaround-for-broken-kernel-side-RLIM_INFINITY-hand.patch --]
[-- Type: text/x-diff, Size: 4101 bytes --]

From b5d429f7cd357a8cf8e49480d58b083892e209f0 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Sat, 24 May 2014 00:14:27 +0200
Subject: [PATCH] workaround for broken kernel side RLIM_INFINITY handling on
 mips

On 32 bit mips the kernel uses -1UL/2 to mark RLIM_INFINITY (and
this is the definition in the userspace api), but since it is in
the middle of the valid range of limits and limits are often
compared with relational operators, various kernel side logic is
broken if larger than -1UL/2 limits are used. So we truncate the
limits to -1UL/2 in get/setrlimit and prlimit.

Even if the kernel side logic consistently treated -1UL/2 as greater
than any other limit value, there wouldn't be any clean workaround
that allowed using large limits:
* using -1UL/2 as RLIM_INFINITY in userspace would mean different
infinity value for get/setrlimt and prlimit (where infinity is always
-1ULL) and userspace logic could break easily (just like the kernel
is broken now) and more special case code would be needed for mips.
* translating -1UL/2 kernel side value to -1ULL in userspace would
mean that -1UL/2 limit cannot be set (eg. -1UL/2+1 had to be passed
to the kernel instead).
---
 arch/mips/bits/resource.h |    2 ++
 src/linux/prlimit.c       |   20 ++++++++++++++++++++
 src/misc/getrlimit.c      |   10 ++++++++++
 src/misc/setrlimit.c      |    6 ++++++
 4 files changed, 38 insertions(+)

diff --git a/arch/mips/bits/resource.h b/arch/mips/bits/resource.h
index 414a405..4d8e43b 100644
--- a/arch/mips/bits/resource.h
+++ b/arch/mips/bits/resource.h
@@ -1,3 +1,5 @@
+#define __broken_RLIM_INFINITY
+
 #define RLIMIT_NOFILE  5
 #define RLIMIT_AS      6
 #define RLIMIT_RSS     7
diff --git a/src/linux/prlimit.c b/src/linux/prlimit.c
index d1639cc..12a9ba9 100644
--- a/src/linux/prlimit.c
+++ b/src/linux/prlimit.c
@@ -3,10 +3,30 @@
 #include "syscall.h"
 #include "libc.h"
 
+#ifdef __broken_RLIM_INFINITY
+#define FIX(x) ((x) >= -1UL/2 ? RLIM_INFINITY : (x))
+int prlimit(pid_t pid, int resource, const struct rlimit *new_limit, struct rlimit *old_limit)
+{
+	struct rlimit tmp;
+	int r;
+	if (new_limit) {
+		tmp.rlim_cur = FIX(new_limit->rlim_cur);
+		tmp.rlim_max = FIX(new_limit->rlim_max);
+		new_limit = &tmp;
+	}
+	r = syscall(SYS_prlimit64, pid, resource, new_limit, old_limit);
+	if (old_limit) {
+		old_limit->rlim_cur = FIX(old_limit->rlim_cur);
+		old_limit->rlim_max = FIX(old_limit->rlim_max);
+	}
+	return r;
+}
+#else
 int prlimit(pid_t pid, int resource, const struct rlimit *new_limit, struct rlimit *old_limit)
 {
 	return syscall(SYS_prlimit64, pid, resource, new_limit, old_limit);
 }
+#endif
 
 #undef prlimit64
 LFS64(prlimit);
diff --git a/src/misc/getrlimit.c b/src/misc/getrlimit.c
index b7bbd06..a6097cf 100644
--- a/src/misc/getrlimit.c
+++ b/src/misc/getrlimit.c
@@ -3,14 +3,24 @@
 #include "syscall.h"
 #include "libc.h"
 
+#ifdef __broken_RLIM_INFINITY
+#define FIX(x) ((x) >= -1UL/2 ? RLIM_INFINITY : (x))
+#else
+#define FIX(x) (x)
+#endif
+
 int getrlimit(int resource, struct rlimit *rlim)
 {
 	unsigned long k_rlim[2];
 	int ret = syscall(SYS_prlimit64, 0, resource, 0, rlim);
+	rlim->rlim_cur = FIX(rlim->rlim_cur);
+	rlim->rlim_max = FIX(rlim->rlim_max);
 	if (!ret || errno != ENOSYS)
 		return ret;
 	if (syscall(SYS_getrlimit, resource, k_rlim) < 0)
 		return -1;
+	k_rlim[0] = FIX(k_rlim[0]);
+	k_rlim[1] = FIX(k_rlim[1]);
 	rlim->rlim_cur = k_rlim[0] == -1UL ? RLIM_INFINITY : k_rlim[0];
 	rlim->rlim_max = k_rlim[1] == -1UL ? RLIM_INFINITY : k_rlim[1];
 	return 0;
diff --git a/src/misc/setrlimit.c b/src/misc/setrlimit.c
index ddc13e9..665b67d 100644
--- a/src/misc/setrlimit.c
+++ b/src/misc/setrlimit.c
@@ -8,6 +8,12 @@
 int __setrlimit(int resource, const struct rlimit *rlim)
 {
 	unsigned long k_rlim[2];
+#ifdef __broken_RLIM_INFINITY
+	struct rlimit tmp;
+	tmp.rlim_cur = MIN(rlim->rlim_cur, -1UL/2);
+	tmp.rlim_max = MIN(rlim->rlim_max, -1UL/2);
+	rlim = &tmp;
+#endif
 	int ret = __syscall(SYS_prlimit64, 0, resource, rlim, 0);
 	if (ret != -ENOSYS) return ret;
 	k_rlim[0] = MIN(rlim->rlim_cur, -1UL);
-- 
1.7.10.4