Preparing for releases 1.1.0 and 1.0.1

Preparing for releases 1.1.0 and 1.0.1

[Rich Felker]

I believe all of the key targets for 1.1.0 have been met, and we also
have fixed several bugs of sufficient severity that a maintenance
release to the 1.0 series is called for.

The commits which I plan to cherry-pick into the 1.0.1 release are:

30c1205acd73c8481ca34f0a41de1d41884d07b5
0a8d98285f46f721dabf38485df916c02d6a4675
91d5aa06572d2660122f9a06ed242fef0383f292
109048e031f39fbb370211fde44ababf6c04c8fb
89740868c9f1c84b8ee528468d12df1fa72cd392
e94d0692864ecf9522fd6a97610a47a2f718d3de
6fbdeff0e51f6afc38fbb1476a4db81322779da4

Note that 5446303328adf4b4e36d9fba21848e6feb55fab4, the malloc brk
exhaustion issue, is not yet in the list. While I'm doubtful that it
has any serious problems, I'd rather get it some more exposure before
putting it into the maintenance-only stable branch.

(BTW, does anyone with git expertise know how I can make git
cherry-pick -x skip a line before the "(cherry picked...)" message?
Perhaps there's a config option or a hook I could use?)

Things I could use help from volunteers on are:

- Reviewing changes, especially the above commits, to make sure they
  don't have any stupid mistakes.

- Running tests and reporting any test regressions.

- Testing x32 and/or sh ports to determine if they're of sufficient
  quality to promote up from "experimental" to the list of officially
  supported archs.

- Informing me of any pending patches or trivial fix requests I might
  have misplaced that could be committed before release.

Rich

Re: Preparing for releases 1.1.0 and 1.0.1

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2014-04-09 22:40:45 -0400]:
> The commits which I plan to cherry-pick into the 1.0.1 release are:
> 
> 30c1205acd73c8481ca34f0a41de1d41884d07b5
> 0a8d98285f46f721dabf38485df916c02d6a4675
> 91d5aa06572d2660122f9a06ed242fef0383f292
> 109048e031f39fbb370211fde44ababf6c04c8fb
> 89740868c9f1c84b8ee528468d12df1fa72cd392
> e94d0692864ecf9522fd6a97610a47a2f718d3de
> 6fbdeff0e51f6afc38fbb1476a4db81322779da4

looks ok

> - Testing x32 and/or sh ports to determine if they're of sufficient
>   quality to promote up from "experimental" to the list of officially
>   supported archs.
> 
> - Informing me of any pending patches or trivial fix requests I might
>   have misplaced that could be committed before release.

i have some uncommitted local modifications/todo items
none of them are critical

* x32 timex is broken (should use long long on x32)

diff --git a/include/sys/timex.h b/include/sys/timex.h
index 2e68888..e404e8b 100644
--- a/include/sys/timex.h
+++ b/include/sys/timex.h
@@ -17,6 +17,9 @@ struct ntptimeval {
 };
 
 struct timex {
+// TODO: x32
+// 7fb30128527a4220f181c2867edd9ac178175a87 2013-12-27
+// x32 adjtimex system call is the same as x86-64 adjtimex system call,
 	unsigned modes;
 	long offset, freq, maxerror, esterror;
 	int status;

* math alias issues on non-x86 archs (about +80bytes)
(either this or __may_alias__)

diff --git a/src/math/modfl.c b/src/math/modfl.c
index f736bba..4b03a4b 100644
--- a/src/math/modfl.c
+++ b/src/math/modfl.c
@@ -3,7 +3,12 @@
 #if LDBL_MANT_DIG == 53 && LDBL_MAX_EXP == 1024
 long double modfl(long double x, long double *iptr)
 {
-	return modf(x, (double *)iptr);
+	double d;
+	long double r;
+
+	r = modf(x, &d);
+	*iptr = d;
+	return r;
 }
 #elif (LDBL_MANT_DIG == 64 || LDBL_MANT_DIG == 113) && LDBL_MAX_EXP == 16384
 #if LDBL_MANT_DIG == 64
diff --git a/src/math/sincosl.c b/src/math/sincosl.c
index 2c60080..d3ac1c4 100644
--- a/src/math/sincosl.c
+++ b/src/math/sincosl.c
@@ -4,7 +4,10 @@
 #if LDBL_MANT_DIG == 53 && LDBL_MAX_EXP == 1024
 void sincosl(long double x, long double *sin, long double *cos)
 {
-	sincos(x, (double *)sin, (double *)cos);
+	double sind, cosd;
+	sincos(x, &sind, &cosd);
+	*sin = sind;
+	*cos = cosd;
 }
 #elif (LDBL_MANT_DIG == 64 || LDBL_MANT_DIG == 113) && LDBL_MAX_EXP == 16384
 void sincosl(long double x, long double *sin, long double *cos)

* use 1/eps for rounding check (with *4 it's nicer, ymmv)

diff --git a/src/stdio/vfprintf.c b/src/stdio/vfprintf.c
index f6e7f38..6e50965 100644
--- a/src/stdio/vfprintf.c
+++ b/src/stdio/vfprintf.c
@@ -343,12 +343,12 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 		x = *d % i;
 		/* Are there any significant digits past j? */
 		if (x || d+1!=z) {
-			long double round = CONCAT(0x1p,LDBL_MANT_DIG);
+			long double round = 4/LDBL_EPSILON;
 			long double small;
-			if (*d/i & 1) round += 2;
-			if (x<i/2) small=0x0.8p0;
-			else if (x==i/2 && d+1==z) small=0x1.0p0;
-			else small=0x1.8p0;
+			if (*d/i & 1) round += 4;
+			if (x<i/2) small=1;
+			else if (x==i/2 && d+1==z) small=2;
+			else small=3;
 			if (pl && *prefix=='-') round*=-1, small*=-1;
 			*d -= x;
 			/* Decide whether to round by probing round+small */

* only use nanosec for entropy

diff --git a/src/temp/__randname.c b/src/temp/__randname.c
index 464b83d..b01bed5 100644
--- a/src/temp/__randname.c
+++ b/src/temp/__randname.c
@@ -12,7 +12,7 @@ char *__randname(char *template)
 	unsigned long r;
 
 	__clock_gettime(CLOCK_REALTIME, &ts);
-	r = ts.tv_nsec*65537 ^ (uintptr_t)&ts / 16 + (uintptr_t)template;
+	r = ts.tv_nsec;
 	for (i=0; i<6; i++, r>>=5)
 		template[i] = 'A'+(r&15)+(r&16)*2;
 

* broken legacy header..

diff --git a/include/sys/procfs.h b/include/sys/procfs.h
index f7936c4..a1fcabf 100644
--- a/include/sys/procfs.h
+++ b/include/sys/procfs.h
@@ -40,7 +40,7 @@ struct elf_prpsinfo
 	char pr_zomb;
 	char pr_nice;
 	unsigned long int pr_flag;
-#if UINTPTR_MAX == 0xffffffff
+#if UINTPTR_MAX == 0xffffffff && !defined __powerpc__
 	unsigned short int pr_uid;
 	unsigned short int pr_gid;
 #else

* linux 3.14 stuff
(sched_setattr/sched_getattr syscall numbers, new sockopt flag, new arphdr type)

* makefile/config changes for out-of-tree build

Re: Preparing for releases 1.1.0 and 1.0.1

[Rich Felker]

On Thu, Apr 10, 2014 at 03:11:23PM +0200, Szabolcs Nagy wrote:
> * x32 timex is broken (should use long long on x32)
> 
> diff --git a/include/sys/timex.h b/include/sys/timex.h
> index 2e68888..e404e8b 100644
> --- a/include/sys/timex.h
> +++ b/include/sys/timex.h
> @@ -17,6 +17,9 @@ struct ntptimeval {
>  };
>  
>  struct timex {
> +// TODO: x32
> +// 7fb30128527a4220f181c2867edd9ac178175a87 2013-12-27
> +// x32 adjtimex system call is the same as x86-64 adjtimex system call,
>  	unsigned modes;
>  	long offset, freq, maxerror, esterror;
>  	int status;

Uhg, so do we need to move this to bits or do some ugly hack? Or
rewrite it in the syscall wrapper code like for timespec?

> * math alias issues on non-x86 archs (about +80bytes)
> (either this or __may_alias__)

I prefer the +80 bytes; the other may_alias uses are optional and have
a portable fallback.

> * use 1/eps for rounding check (with *4 it's nicer, ymmv)

Could you explain why? I would prefer a change that doesn't require so
many lines changed since they're all places errors could hide. Just
getting rid of the CONCAT hack seems preferable to me, but I don't
mind hearing the reason you like the *4.

> * only use nanosec for entropy

I just worry on some archs with bad timer, this could prevent getting
sufficiently many temp names (of course the problem already existed).
Is there more non-valuable entropy we could merge into it? One idea
was the bytes of struct stat from stat() on /proc/self (this is like
using pid, but better). But perhaps there's more we could do with just
time.

> * broken legacy header..
> 
> diff --git a/include/sys/procfs.h b/include/sys/procfs.h
> index f7936c4..a1fcabf 100644
> --- a/include/sys/procfs.h
> +++ b/include/sys/procfs.h
> @@ -40,7 +40,7 @@ struct elf_prpsinfo
>  	char pr_zomb;
>  	char pr_nice;
>  	unsigned long int pr_flag;
> -#if UINTPTR_MAX == 0xffffffff
> +#if UINTPTR_MAX == 0xffffffff && !defined __powerpc__

I'm ok with this hack I think.

>  	unsigned short int pr_uid;
>  	unsigned short int pr_gid;
>  #else
> 
> * linux 3.14 stuff
> (sched_setattr/sched_getattr syscall numbers, new sockopt flag, new arphdr type)

This should probably be held until a later release. We need to
consider ABI issues. I believe we have sufficient room to put a union
(rather than the kernel's silly non-union approach) over top of our
schedparam struct and fit all the values needed without ABI breakage,
but this requires some code to convert to/from the kernel format.

> * makefile/config changes for out-of-tree build

Last I checked you were still finding breakage in it. When I get done
with the release and other higher-priority things I'm trying to get
done, maybe I should look at it and give it a proper review. Sorry I
haven't gotten around to that yet.

Rich

Re: Preparing for releases 1.1.0 and 1.0.1

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2014-04-10 21:39:11 -0400]:
> On Thu, Apr 10, 2014 at 03:11:23PM +0200, Szabolcs Nagy wrote:
> > * x32 timex is broken (should use long long on x32)
> 
> Uhg, so do we need to move this to bits or do some ugly hack? Or
> rewrite it in the syscall wrapper code like for timespec?

move the struct to bits

> > * math alias issues on non-x86 archs (about +80bytes)
> > (either this or __may_alias__)
> 
> I prefer the +80 bytes; the other may_alias uses are optional and have
> a portable fallback.

ok

> > * use 1/eps for rounding check (with *4 it's nicer, ymmv)
> 
> Could you explain why? I would prefer a change that doesn't require so
> many lines changed since they're all places errors could hide. Just
> getting rid of the CONCAT hack seems preferable to me, but I don't
> mind hearing the reason you like the *4.

well integers are special (eg x87 has fld1) so a bit better code
may be generated and they are more familiar

another way to make the code better on some platforms is to use floats:
(and help valgrind)

diff --git a/src/stdio/vfprintf.c b/src/stdio/vfprintf.c
index f6e7f38..546df28 100644
--- a/src/stdio/vfprintf.c
+++ b/src/stdio/vfprintf.c
@@ -13,8 +13,6 @@
 
 #define MAX(a,b) ((a)>(b) ? (a) : (b))
 #define MIN(a,b) ((a)<(b) ? (a) : (b))
-#define CONCAT2(x,y) x ## y
-#define CONCAT(x,y) CONCAT2(x,y)
 
 /* Convenient bit representation for modifier flags, which all fall
  * within 31 codepoints of the space character. */
@@ -343,8 +341,8 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 		x = *d % i;
 		/* Are there any significant digits past j? */
 		if (x || d+1!=z) {
-			long double round = CONCAT(0x1p,LDBL_MANT_DIG);
-			long double small;
+			float_t round = 0x1p24f;
+			float_t small;
 			if (*d/i & 1) round += 2;
 			if (x<i/2) small=0x0.8p0;
 			else if (x==i/2 && d+1==z) small=0x1.0p0;
@@ -352,7 +350,12 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 			if (pl && *prefix=='-') round*=-1, small*=-1;
 			*d -= x;
 			/* Decide whether to round by probing round+small */
-			if (round+small != round) {
+#if FLT_EVAL_METHOD!=0
+			/* avoiding compiler bugs */
+			volatile
+#endif
+			float sum = round+small;
+			if (sum != round) {
 				*d = *d + i;
 				while (*d > 999999999) {
 					*d--=0;


> > * only use nanosec for entropy
> 
> I just worry on some archs with bad timer, this could prevent getting
> sufficiently many temp names (of course the problem already existed).
> Is there more non-valuable entropy we could merge into it? One idea
> was the bytes of struct stat from stat() on /proc/self (this is like
> using pid, but better). But perhaps there's more we could do with just
> time.

stat is ok

if there are different clock sources then you can do more with time
or if we assume the retries take non-predictable time (open syscall
on a crowded /tmp) then that can be measured as well

using the seconds may help a bit

r = ts.tv_sec ^ ts.tv_nsec;

(all the relevant 30bits of r are used so further mixing does not help)


> > * broken legacy header..
> > 
> > diff --git a/include/sys/procfs.h b/include/sys/procfs.h
> > index f7936c4..a1fcabf 100644
> > --- a/include/sys/procfs.h
> > +++ b/include/sys/procfs.h
> > @@ -40,7 +40,7 @@ struct elf_prpsinfo
> >  	char pr_zomb;
> >  	char pr_nice;
> >  	unsigned long int pr_flag;
> > -#if UINTPTR_MAX == 0xffffffff
> > +#if UINTPTR_MAX == 0xffffffff && !defined __powerpc__
> 
> I'm ok with this hack I think.

this is probably still broken on microblaze

my guess is that nobody is actually using this header

> > * linux 3.14 stuff
> > (sched_setattr/sched_getattr syscall numbers, new sockopt flag, new arphdr type)
> 
> This should probably be held until a later release. We need to
> consider ABI issues. I believe we have sufficient room to put a union
> (rather than the kernel's silly non-union approach) over top of our
> schedparam struct and fit all the values needed without ABI breakage,
> but this requires some code to convert to/from the kernel format.

well the kernel did not export any definition for the syscalls
and the sched_attr struct so far, only the syscall numbers

i'm not sure if that was intentional, but currently they cannot
be used reasonably from userspace

> > * makefile/config changes for out-of-tree build
> 
> Last I checked you were still finding breakage in it. When I get done
> with the release and other higher-priority things I'm trying to get
> done, maybe I should look at it and give it a proper review. Sorry I
> haven't gotten around to that yet.

i'd say that there are some tradeoffs..

Re: Preparing for releases 1.1.0 and 1.0.1

[Rich Felker]

On Fri, Apr 11, 2014 at 11:46:16AM +0200, Szabolcs Nagy wrote:
> > > * use 1/eps for rounding check (with *4 it's nicer, ymmv)
> > 
> > Could you explain why? I would prefer a change that doesn't require so
> > many lines changed since they're all places errors could hide. Just
> > getting rid of the CONCAT hack seems preferable to me, but I don't
> > mind hearing the reason you like the *4.
> 
> well integers are special (eg x87 has fld1) so a bit better code
> may be generated and they are more familiar

I see.

> another way to make the code better on some platforms is to use floats:
> (and help valgrind)

Yes but this leads to serious breakage if the compiler does not handle
the request to drop precision in a conforming way, and it still
doesn't help the fact that strtod depends on long double semantics.

> > > * only use nanosec for entropy
> > 
> > I just worry on some archs with bad timer, this could prevent getting
> > sufficiently many temp names (of course the problem already existed).
> > Is there more non-valuable entropy we could merge into it? One idea
> > was the bytes of struct stat from stat() on /proc/self (this is like
> > using pid, but better). But perhaps there's more we could do with just
> > time.
> 
> stat is ok
> 
> if there are different clock sources then you can do more with time
> or if we assume the retries take non-predictable time (open syscall
> on a crowded /tmp) then that can be measured as well
> 
> using the seconds may help a bit
> 
> r = ts.tv_sec ^ ts.tv_nsec;
> 
> (all the relevant 30bits of r are used so further mixing does not help)

This is probably a good mixing since the bits of the seconds that will
change "often" are overlaid with the bits of nsec that might be
missing, but it's probably insufficient if we only have microsecond
time values or worse.

The stat would give a good per-process source of bits, but doesn't
help if a single process needs lots of temp files. And I'm not sure
the double-call to gettime would help if nsec has poor resolution
since the delay between subsequent calls is likely to be on the scale
of ~1us.

Perhaps even with just usec resolution, we get enough names anyway.
I'm open to other ideas.

> > > * broken legacy header..
> > > 
> > > diff --git a/include/sys/procfs.h b/include/sys/procfs.h
> > > index f7936c4..a1fcabf 100644
> > > --- a/include/sys/procfs.h
> > > +++ b/include/sys/procfs.h
> > > @@ -40,7 +40,7 @@ struct elf_prpsinfo
> > >  	char pr_zomb;
> > >  	char pr_nice;
> > >  	unsigned long int pr_flag;
> > > -#if UINTPTR_MAX == 0xffffffff
> > > +#if UINTPTR_MAX == 0xffffffff && !defined __powerpc__
> > 
> > I'm ok with this hack I think.
> 
> this is probably still broken on microblaze
> 
> my guess is that nobody is actually using this header

IIRC gdb is but we should look this up. I have no problem removing it
if it's true that nothing is using it. Do we have history (e.g. on the
list) from when it was added motivating the addition?

> > > * linux 3.14 stuff
> > > (sched_setattr/sched_getattr syscall numbers, new sockopt flag, new arphdr type)
> > 
> > This should probably be held until a later release. We need to
> > consider ABI issues. I believe we have sufficient room to put a union
> > (rather than the kernel's silly non-union approach) over top of our
> > schedparam struct and fit all the values needed without ABI breakage,
> > but this requires some code to convert to/from the kernel format.
> 
> well the kernel did not export any definition for the syscalls
> and the sched_attr struct so far, only the syscall numbers
> 
> i'm not sure if that was intentional, but currently they cannot
> be used reasonably from userspace

OK, then adding just the syscall numbers is no problem.

Rich

Re: Preparing for releases 1.1.0 and 1.0.1

[Morten Welinder]

> Yes but this leads to serious breakage if the compiler does not handle
> the request to drop precision in a conforming way, and it still
> doesn't help the fact that strtod depends on long double semantics.

Are you sure?

If the incoming number has at most 53 bits then there will be at most
24 bits left after the 2^(1+28) reduction.  The scalings by 10^9 = 2^9 * 5^9
require at most log2(5^9) ~ 21 extra bits.

That ought to fit in valgrind's double.

Morten

Re: Preparing for releases 1.1.0 and 1.0.1

[Szabolcs Nagy]

* Morten Welinder <mwelinder@gmail.com> [2014-04-11 11:17:00 -0400]:
> > Yes but this leads to serious breakage if the compiler does not handle
> > the request to drop precision in a conforming way, and it still
> > doesn't help the fact that strtod depends on long double semantics.
> 
> Are you sure?
> 
> If the incoming number has at most 53 bits then there will be at most
> 24 bits left after the 2^(1+28) reduction.  The scalings by 10^9 = 2^9 * 5^9
> require at most log2(5^9) ~ 21 extra bits.
> 
> That ought to fit in valgrind's double.

formatting floats (other than %a) would work with the fixed rounding

but scanning or strtod can't be fixed easily

Re: Preparing for releases 1.1.0 and 1.0.1

[Rich Felker]

On Fri, Apr 11, 2014 at 11:17:00AM -0400, Morten Welinder wrote:
> > Yes but this leads to serious breakage if the compiler does not handle
> > the request to drop precision in a conforming way, and it still
> > doesn't help the fact that strtod depends on long double semantics.
> 
> Are you sure?
> 
> If the incoming number has at most 53 bits then there will be at most
> 24 bits left after the 2^(1+28) reduction.  The scalings by 10^9 = 2^9 * 5^9
> require at most log2(5^9) ~ 21 extra bits.
> 
> That ought to fit in valgrind's double.

Are you basing this on a reading of how long double is used in the
source file, or just generalities? The rounding logic depends on
exactly how many places are kept/dropped when a certain addition is
performed.

Rich

Re: Preparing for releases 1.1.0 and 1.0.1

[Rich Felker]

On Fri, Apr 11, 2014 at 11:48:13AM -0400, Rich Felker wrote:
> On Fri, Apr 11, 2014 at 11:17:00AM -0400, Morten Welinder wrote:
> > > Yes but this leads to serious breakage if the compiler does not handle
> > > the request to drop precision in a conforming way, and it still
> > > doesn't help the fact that strtod depends on long double semantics.
> > 
> > Are you sure?
> > 
> > If the incoming number has at most 53 bits then there will be at most
> > 24 bits left after the 2^(1+28) reduction.  The scalings by 10^9 = 2^9 * 5^9
> > require at most log2(5^9) ~ 21 extra bits.
> > 
> > That ought to fit in valgrind's double.
> 
> Are you basing this on a reading of how long double is used in the
> source file, or just generalities? The rounding logic depends on
> exactly how many places are kept/dropped when a certain addition is
> performed.

Maybe we're unclear on what one another are saying. I'm talking about
strtod and family here, not printf.

Rich

Re: Preparing for releases 1.1.0 and 1.0.1

[Morten Welinder]

> Maybe we're unclear on what one another are saying. I'm talking about
> strtod and family here, not printf.

Ah, that's it.

I was looking at the printf code.  strtod is indeed a different kettle of fish.

Morten