open issues

open issues

[Szabolcs Nagy]

list of issues since last release that don't seem to be resolved.
first ones that looked like bugs to me, then other issues.
list goes backward in time, some issues may be missed.

bug:
- update contributors in copyright file
- fflush(0) does not lock f
  http://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c#n12
- getenv/setenv/putenv ub
  http://www.openwall.com/lists/musl/2017/08/21/1
  http://www.openwall.com/lists/musl/2016/03/13/7
- fix ioctl on mips, add SIOCGSTAMPNS
  http://www.openwall.com/lists/musl/2017/08/13/4
  http://www.openwall.com/lists/musl/2017/08/13/5
- ipc/ftok.c overflowing shift
  http://www.openwall.com/lists/musl/2017/08/12/5
- __progname fallbacks so it's never 0
  http://www.openwall.com/lists/musl/2017/07/28/6
- mbsnrtowcs and mbsnrtowcs confuses byte and wchar counts
  http://www.openwall.com/lists/musl/2017/08/09/1
- memset ub because s[0] = s[0] = c
  http://www.openwall.com/lists/musl/2017/07/10/7
  http://www.openwall.com/lists/musl/2017/07/06/2
- handle whitespace before %% in scanf
  http://www.openwall.com/lists/musl/2017/07/11/1
- mips64 utime issue?
  "tar binary can't fix the modification/access times on any extracted symbolic links,"
  http://www.openwall.com/lists/musl/2017/07/06/1
- oob reads in memmem (and signed << ub)
  http://www.openwall.com/lists/musl/2017/06/29/6
- use-after-free in __unlock of pthread struct
  http://www.openwall.com/lists/musl/2017/06/01/7
- newly created thread may run with signals blocked
  < sergei> there seems to be a race condition in pthread_create.c between lines 134 and 298
  < sergei> if line 298 is executed before 134 (assuming syscall returned 0), startlock will be overwritten with zero, the condition will be evaluated to false and __restore_sigs will not be executed
  < sergei> the newly created thread will run with all signals blocked
  < sergei> i have a patch that fixes the issue for me: https://pastebin.com/T5QSd0C9
- scanf, wrong types in va_arg
  http://www.openwall.com/lists/musl/2017/04/10/3
- missed underflow in fma
  http://www.openwall.com/lists/musl/2017/03/19/6
  new fma, depends on a_clz_64
  http://www.openwall.com/lists/musl/2017/04/23/10
- fix nftw when called with paths ending in slash
  http://www.openwall.com/lists/musl/2017/03/07/1
- fix syscall number differences compared to linux uapi
  http://www.openwall.com/lists/musl/2017/02/18/1
- getservbyport(_r) should not report numeric ports
  http://www.openwall.com/lists/musl/2017/02/06/5
- add s390x and powerpc64 to supported arches
  http://www.openwall.com/lists/musl/2017/02/01/2
- define IPPORT_RESERVED in netinet/in.h and netdb.h
  http://www.openwall.com/lists/musl/2017/01/31/4 
- GLOB_PERIOD is inconsistent with glibc
  http://www.openwall.com/lists/musl/2017/01/12/5
- mmap should not return EPERM when it means ENOMEM
  http://www.openwall.com/lists/musl/2017/01/12/1
- getopt_long does not report failure correctly
  http://www.openwall.com/lists/musl/2017/01/07/4
- make dlsym and reloc time lookup consistent
  http://www.openwall.com/lists/musl/2017/02/16/1
- ldso ctor dependency ordering and recursive dlopen fix
  http://www.openwall.com/lists/musl/2017/01/03/6
- align arm hwcap.h with glibc (nsz)


feature request:
- pending linux uapi updates: v4.10, v4.11, v4.12 (nsz)
- non-thread-safe apis (gethostbyname) can detect some misuse and crash
- avoid mmapping inaccessible parts of a binary as rx
- syslog.h prioritynames compund literal causes problems
  http://www.openwall.com/lists/musl/2017/08/16/4
- align fnmatch \ in bracket semantics with glibc
  http://www.openwall.com/lists/musl/2017/08/15/2
- pthread_attr_init behaviour (to use whatever was set in pthread_setattr_default_np)
  http://www.openwall.com/lists/musl/2017/08/12/6
- fix stdbool.h in c++ code?
  http://www.openwall.com/lists/musl/2017/07/28/1
- add sysconf support for _SC_LEVEL1_DCACHE_LINESIZE
  http://www.openwall.com/lists/musl/2017/07/24/1
- change GMT to UTC
  http://www.openwall.com/lists/musl/2017/07/22/2
- [RFC PATCH 0/5] Add explicit_bzero, vectorize and 'normalize' various string functions
  http://www.openwall.com/lists/musl/2017/07/15/1
- strlen variant without aliasing violation
  http://www.openwall.com/lists/musl/2017/07/10/6
- |32 is not needed in (t|32) != 'c'
  http://www.openwall.com/lists/musl/2017/07/09/5
- __malloc_donate instead of current hack in ldso
  http://www.openwall.com/lists/musl/2017/06/28/4
- optimize __malloc0 to clear memory faster
  http://www.openwall.com/lists/musl/2017/06/26/1
- code size optimize wmemcpy
  http://www.openwall.com/lists/musl/2017/06/25/10
- [PATCH 0/8] the new __lock and follow up patches
  http://www.openwall.com/lists/musl/2017/06/23/2
- more correct va_arg use in fcntl
  http://www.openwall.com/lists/musl/2017/06/21/16
- use better name than index in __tz.c
  http://www.openwall.com/lists/musl/2017/06/11/12
- use O_TMPFILE in tmpfile if possible
  http://www.openwall.com/lists/musl/2017/06/08/2
- fix user.h so gdb builds cleanly on all targets
  http://www.openwall.com/lists/musl/2017/06/11/2
- towlower performance
  http://www.openwall.com/lists/musl/2017/05/27/2
- RES_OPTIONS support for resolv.conf options overriding
  http://www.openwall.com/lists/musl/2017/04/30/1
  related libc-alpha discussion
  http://sourceware.org/ml/libc-alpha/2017-08/msg01097.html
- strptime %z %Z
  http://www.openwall.com/lists/musl/2017/04/19/1
- add microMIPS32 support
  http://www.openwall.com/lists/musl/2017/05/31/2
- IDNA support in name lookups
  http://www.openwall.com/lists/musl/2017/04/02/1
- improve LIBCC detection in configure?
  http://www.openwall.com/lists/musl/2017/03/16/2
- precise complex math
  http://www.openwall.com/lists/musl/2017/02/12/3
- strtoul of width specifier clobbers errno in strftime
  http://www.openwall.com/lists/musl/2017/02/12/1
- provide sgidefs.h on mips
  http://www.openwall.com/lists/musl/2017/02/04/2
- add GLOB_TILDE support
  http://www.openwall.com/lists/musl/2017/01/20/4
- crypt_blowfish: support $2b$ prefix
  http://www.openwall.com/lists/musl/2017/01/12/6
- strftime _-0
  http://www.openwall.com/lists/musl/2017/01/06/1
- static-pie relro
  < fabled> how does static pie musl libc init work?
  < fabled> the reason i ask is, that relro is not enabled then
  < fabled> it should do mprotect() for the relro area
  < nsz> ah i see if you have 'static int *const p = &x;' then that relative reloc should be in relro
- fts64
  http://www.openwall.com/lists/musl/2016/11/23/1
- REG_STARTEND
  http://www.openwall.com/lists/musl/2016/10/05/6
- realpath without /proc
  http://www.openwall.com/lists/musl/2016/11/03/5
- new tsearch
  http://www.openwall.com/lists/musl/2015/12/20/1
- avoid readv in stdio with 0 length buffer because that's broken on /proc
  https://bugzilla.kernel.org/show_bug.cgi?id=195697
- add fortify _chk functions
- memchr optimization

Re: open issues

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26/08/17 23:36, Szabolcs Nagy wrote:
> - add microMIPS32 support 
> http://www.openwall.com/lists/musl/2017/05/31/2

Just for what it's worth, I've just fixed an overlapping memcpy in gas
for microMIPS, so you may need the patch there to ensure no weirdness
(or even a crash under Alpine and similar fortified distros).

https://sourceware.org/bugzilla/show_bug.cgi?id=22014

Hope this helps further the effort.

All the best,
- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZomGEAAoJEMspy1GSK50U/wUQAM402KAgMbQKC86Ef6xD8atT
ENPqnuiiW3kiAmlS2+2h2VnYUo7i++OU06ozI+vUIVSTl4pKVxcu5bwTN28IpnXe
VBxCxEujfMoZkO7nAGVwNktijk52yzIfUpSF83XjIBiVv1NIzzC6ARdkqKUd3HWa
7SQkLSEC3+vg3plDI4PA2FhVt9JTXsKkp5jzOrwJxEro/rSiRSkctTzufCo0UaRv
3D2mEkpxMHBNFigXrbvv5AcexVOobtFjU/qRhrC2/d3PHNyQTyixfCuMdT+8FaQV
9MiG/XJx4rs7HmDf2O28VG3yNLUUfmL+71iiT12Pm+jgMkNoNOF9/gPqmR4qFF1W
uYNzjgYL+gL4RVEPhmDvDUuYOXhgKrPpoBam5v5ZqY7s8ptDFIBT5yBgdZyAu19z
65ecRhuKFnnwnqxg30+anmB2iZToic+ri7ktnT7gzZhAQF9ucNhP8Nx41m3GZvWM
z1y55t11aRd4vJmQopC9DAu5rOM4YRNhYwOrTMWGEQvgf0GBpq+wtV3HfnTCMOI0
B91CBEzdDdqRrlRrNNP8nuqIhEDTBEehjKL3Az+JP/jZsK6pT58uppIdN+WQrAh2
CWhULF8VBPlAZdMvfzybygfjCOPMSqWr6F+fPuZvgJ398+qtRnvZbSwc4SoLjQSA
ct74PLpKPvU2ATcLgXat
=orZU
-----END PGP SIGNATURE-----

Re: open issues

[Rich Felker]

On Sun, Aug 27, 2017 at 06:36:06AM +0200, Szabolcs Nagy wrote:
> list of issues since last release that don't seem to be resolved.
> first ones that looked like bugs to me, then other issues.
> list goes backward in time, some issues may be missed.
> 
> bug:
> - update contributors in copyright file

Will do at release time. Thanks for the reminder though.

> - fflush(0) does not lock f
>   http://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c#n12

Fixing: commit 670d6d01f53b4e85be6b333bf8a137e2be6d3fc3

> - getenv/setenv/putenv ub
>   http://www.openwall.com/lists/musl/2017/08/21/1
>   http://www.openwall.com/lists/musl/2016/03/13/7

Merging this is way overdue. It's larger than the patches I'm merging
while reading through this post right now but I'll come back to it.

> - fix ioctl on mips, add SIOCGSTAMPNS
>   http://www.openwall.com/lists/musl/2017/08/13/4
>   http://www.openwall.com/lists/musl/2017/08/13/5

Committing above patches.

> - ipc/ftok.c overflowing shift
>   http://www.openwall.com/lists/musl/2017/08/12/5

Already committed but I forgot to push.

> - __progname fallbacks so it's never 0
>   http://www.openwall.com/lists/musl/2017/07/28/6

Committing the patch.

> - mbsnrtowcs and mbsnrtowcs confuses byte and wchar counts
>   http://www.openwall.com/lists/musl/2017/08/09/1

I know the current code is wrong but I'm not sure the patches are
fully right either; probably needs testing. But if we don't get to it,
just merging them would be better than leaving the code dangerously
broken.

> - memset ub because s[0] = s[0] = c
>   http://www.openwall.com/lists/musl/2017/07/10/7
>   http://www.openwall.com/lists/musl/2017/07/06/2

Applying patch manually with a short message crediting Pascal Cuoq.
The reason I didn't get to this earlier was that something botched the
email attachment and the patch was corrupt (DOS newlines); I should
have replied and mentioned it at the time.

Fix is in commit 9d4c902c42b3fda368d7ea64bb9575c46228fa7f

> - handle whitespace before %% in scanf
>   http://www.openwall.com/lists/musl/2017/07/11/1

Unless I missed something, I'm waiting for a different form of the
patch that simplifies code rather than making it more complex.

> - mips64 utime issue?
>   "tar binary can't fix the modification/access times on any extracted symbolic links,"
>   http://www.openwall.com/lists/musl/2017/07/06/1

Needs more investigation.

> - oob reads in memmem (and signed << ub)
>   http://www.openwall.com/lists/musl/2017/06/29/6

I think either proposed fix is probably okay, but I'm kinda waiting
for some tests to show that the problem is fixed and that no obvious
new bugs are introduced.

> - use-after-free in __unlock of pthread struct
>   http://www.openwall.com/lists/musl/2017/06/01/7

This could be fixed by merging Jens' new lock implementation or with a
local fix for use of exitlock. I'm not sure which is more work but
either is more than I can do in a couple minutes going through this
list to tick off the easy things.

> - newly created thread may run with signals blocked
>   < sergei> there seems to be a race condition in pthread_create.c between lines 134 and 298
>   < sergei> if line 298 is executed before 134 (assuming syscall returned 0), startlock will be overwritten with zero, the condition will be evaluated to false and __restore_sigs will not be executed
>   < sergei> the newly created thread will run with all signals blocked
>   < sergei> i have a patch that fixes the issue for me: https://pastebin.com/T5QSd0C9

I don't really like this patch, but it might be the best short-term
solution. Ideally all of the wacky random logic with atomics should
just be ripped out and replaced by idiomatic locks of some sort. It
doesn't help that startlock is being used both as a lock and to
reflect other flags (whether the new thread needs to restore its
signal mask and whether it needs to detach+abort). There really should
just be some sort of semaphore here (possibly the existing startlock,
but without overloaded meaning) followed by (after passing it) code to
check and act on the signal and abort conditions _without_ caring
whether the semaphore ever blocked forward progress at first.

> - scanf, wrong types in va_arg
>   http://www.openwall.com/lists/musl/2017/04/10/3

It's not clear what can be done here. Any fix is doomed to be
incomplete because POSIX allows missing specifiers (assuming all
pointers have same variadic representation as void *). When there are
no missing specifiers, we can fix it, but with a lot of "spurious"
code.

> - missed underflow in fma
>   http://www.openwall.com/lists/musl/2017/03/19/6
>   new fma, depends on a_clz_64
>   http://www.openwall.com/lists/musl/2017/04/23/10

IIRC I was plenty convinced that your proposed new fma is a better
approach, but wasn't too fond of adding new a_* primitives (a_clz_64).
Maybe we should just do it for now though.

> - fix nftw when called with paths ending in slash
>   http://www.openwall.com/lists/musl/2017/03/07/1

This is another one that needs more attention that I'm giving items on
the list right now.

> - fix syscall number differences compared to linux uapi
>   http://www.openwall.com/lists/musl/2017/02/18/1

I don't recall whether we checked that the patch doesn't mess up
musl's internal use of the syscall numbers (especially the remapping
logic in src/internal/syscall.h). If that's checked this is probably
ok.

> - getservbyport(_r) should not report numeric ports
>   http://www.openwall.com/lists/musl/2017/02/06/5

This is probably correct, and my comments on simplifying it in the
original thread look wrong. I do wonder if it's inconsistent to make
this change without changing the opposite-direction lookup not to
accept numeric strings though. Thoughts?

> - add s390x and powerpc64 to supported arches
>   http://www.openwall.com/lists/musl/2017/02/01/2

Does s390x need any special notes in INSTALL like most other archs
have? I think probably not.

> - define IPPORT_RESERVED in netinet/in.h and netdb.h
>   http://www.openwall.com/lists/musl/2017/01/31/4 

Fixed in commit 5f7efb87a28a311ad377dd26adf53715dedb096d

> - GLOB_PERIOD is inconsistent with glibc
>   http://www.openwall.com/lists/musl/2017/01/12/5

This needs further consideration of what the right fix is, but I agree
the current behavior is wrong.

> - mmap should not return EPERM when it means ENOMEM
>   http://www.openwall.com/lists/musl/2017/01/12/1

Probably need to work out the specific logic for when the replacement
should be done.

> - getopt_long does not report failure correctly
>   http://www.openwall.com/lists/musl/2017/01/07/4

I'm missing this thread from my copy of the list mail due to downtime
from fire. I think the issue was already fixed correctly (vs with
weird internal-state hacks) in commit
786fda875a901dc1807289c940338487854cd3ba from a few days earlier.

> - make dlsym and reloc time lookup consistent
>   http://www.openwall.com/lists/musl/2017/02/16/1

At least part of this was already merged in commit
c9783e4d32e786c4b76bf77c6030111d9e79dbb7.

I think the other part had sufficient conflicts with other changes
made that it wasn't easy for me to resolve them.

> - ldso ctor dependency ordering and recursive dlopen fix
>   http://www.openwall.com/lists/musl/2017/01/03/6

Yes, I think both of these need to wait til after release since they
involve significant amounts of new development & design that have been
error-prone so far...

> - align arm hwcap.h with glibc (nsz)

I think I'm missing this patch...?

> feature request:

I'm skipping all these for now, but appreciate having them listed.

Rich

Re: open issues

[Rich Felker]

Updated lists after reviewing the list and pushing the changes that
were easy to make/merge:


Still pending due to nontrivial patch to review and/or need for
testing:

- getenv/setenv/putenv ub
  http://www.openwall.com/lists/musl/2017/08/21/1
  http://www.openwall.com/lists/musl/2016/03/13/7
- mbsnrtowcs and mbsnrtowcs confuses byte and wchar counts
  http://www.openwall.com/lists/musl/2017/08/09/1
- oob reads in memmem (and signed << ub)
  http://www.openwall.com/lists/musl/2017/06/29/6
- fix nftw when called with paths ending in slash
  http://www.openwall.com/lists/musl/2017/03/07/1

Pending due to waiting for updated patch incorporating feedback
already given or resolving merge conflicts or similar:

- handle whitespace before %% in scanf
  http://www.openwall.com/lists/musl/2017/07/11/1
- make dlsym and reloc time lookup consistent
  http://www.openwall.com/lists/musl/2017/02/16/1
- newly created thread may run with signals blocked
  < sergei> there seems to be a race condition in pthread_create.c between lines 134 and 298
  < sergei> if line 298 is executed before 134 (assuming syscall returned 0), startlock will be overwritten with zero, the condition will be evaluated to false and __restore_sigs will not be executed
  < sergei> the newly created thread will run with all signals blocked
  < sergei> i have a patch that fixes the issue for me: https://pastebin.com/T5QSd0C9
- missed underflow in fma
  http://www.openwall.com/lists/musl/2017/03/19/6
  new fma, depends on a_clz_64
  http://www.openwall.com/lists/musl/2017/04/23/10

Pending due to need for additional analysis to determine exactly
what/where the bug is:

- mips64 utime issue?
  "tar binary can't fix the modification/access times on any extracted symbolic links,"
  http://www.openwall.com/lists/musl/2017/07/06/1

Pending due to open question about desired behavior:

- getservbyport(_r) should not report numeric ports
  http://www.openwall.com/lists/musl/2017/02/06/5
- mmap should not return EPERM when it means ENOMEM
  http://www.openwall.com/lists/musl/2017/01/12/1
- GLOB_PERIOD is inconsistent with glibc
  http://www.openwall.com/lists/musl/2017/01/12/5
- ldso ctor dependency ordering and recursive dlopen fix
  http://www.openwall.com/lists/musl/2017/01/03/6

Pending due to dependency of fix on larger change:

- use-after-free in __unlock of pthread struct
  http://www.openwall.com/lists/musl/2017/06/01/7

Pending due to need to minor mechanical review:

- fix syscall number differences compared to linux uapi
  http://www.openwall.com/lists/musl/2017/02/18/1

Pending due to missing patch:

- align arm hwcap.h with glibc (nsz)

Punting til later due to lack of serious/any outward symptoms and lack
of any practical way to fix:

- scanf, wrong types in va_arg
  http://www.openwall.com/lists/musl/2017/04/10/3

Re: open issues

[Pascal Cuoq]

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]

Hello,

Punting til later due to lack of serious/any outward symptoms and lack
of any practical way to fix:

- scanf, wrong types in va_arg
 http://www.openwall.com/lists/musl/2017/04/10/3

Joseph Myers has pointed out to me that the POSIX chapter about stdarg.h defines an XSI extension that allows musl's scanf to be written just the way it is:

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/stdarg.h.html  (specifically the sentence “Both types are pointers”)

To me, this means that the issue can be dropped entirely : musl simply needs this XSI extension, as it probably does some others.

Pascal


[-- Attachment #2: Type: text/html, Size: 1480 bytes --]

Re: open issues

[Rich Felker]

On Wed, Aug 30, 2017 at 08:26:46AM +0000, Pascal Cuoq wrote:
> Hello,
> 
> Punting til later due to lack of serious/any outward symptoms and lack
> of any practical way to fix:
> 
> - scanf, wrong types in va_arg
>  http://www.openwall.com/lists/musl/2017/04/10/3
> 
> Joseph Myers has pointed out to me that the POSIX chapter about
> stdarg.h defines an XSI extension that allows musl's scanf to be
> written just the way it is:
> 
> http://pubs.opengroup.org/onlinepubs/009695399/basedefs/stdarg.h.html  (specifically the sentence “Both types are pointers”)
> 
> To me, this means that the issue can be dropped entirely : musl
> simply needs this XSI extension, as it probably does some others.

Great find. I wonder if compiler/diagnostic tool ppl are aware of this
extension to the C language that XSI imposes. It definitely affects
things like UBSan, and of course compilers' optimization based on
assumptions that UB doesn't happen.

Rich

Re: open issues

[Alexander Monakov]

On Wed, 30 Aug 2017, Rich Felker wrote:
> Great find. I wonder if compiler/diagnostic tool ppl are aware of this
> extension to the C language that XSI imposes. It definitely affects
> things like UBSan, and of course compilers' optimization based on
> assumptions that UB doesn't happen.

If musl is going to use this throughout the implementation, I think it
should be documented somewhere: it's of course fair game to use this
guarantee in implementation of interfaces that are themselves XSI-shaded,
but using it to implement more basic interfaces is less obviously so.

Alexander

Re: open issues

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 29/08/17 21:06, Rich Felker wrote:
> Pending due to open question about desired behavior:
> 
> - getservbyport(_r) should not report numeric ports 
> http://www.openwall.com/lists/musl/2017/02/06/5


What's the question?  I'd be happy to answer.

All the best,
- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZpxckAAoJEMspy1GSK50UTlEP/1928p335uUV/kLyCnTxP03X
rQu+0ILB8Ad+ClzeOaOX2QacTxHYaHkfa4IGn9LxqJtXMUkaBWQ5iug1Gvtb/SHX
44TMLqHU5+PVz4SlcVfHIkOr97a7Nj1LJFiM7hOsbOgnKnVxIkcMwzwgE0mM16NJ
Uthf9FOes4stMtCeTf6Uz1Sr9UBaP2mxZUcD62IL8jBsTJ/l1iRDBA6rjnjJfuLq
bpw8VvtSVSWjbYoeXByQRL1Lh+QlzsKPS6F/vODpVwJ29FNLuo2bdjNNmBInKMVq
konIpslYZ4wX8CXlRDf9ot/wtCnUF/f9ADV+ZFsNtAcjyGpL6Vpt6L4tHV40P036
RvQTyQxlSm9ZqKRC6I1cDylyf3NgZLQYHz9FtqjUApKRzfUFpjfhGr89ctMACltx
qn9X3053nBi8IpKoadnUX054ZVq2Sh6R4qHH7iwwDg6EWnyKwAd7sJLWCYvSM7Lg
oWKqcsZDYsAxznKY5ntlAAJK6k29R8cGim3jZMrxHDDe51kYl9+CitqfhCMKyVQE
yrn4RtWyLKAa9D5OWb+iIqjpUD63JpaNbxWKYy+luBtypyoxKSAoHnyWhlD082w1
WrJqmYJ28nPpZ+Oh7fJ/pBqcILS6s9jyK8+DBOplMflHCAkUyln6qX3P8TC6IhJq
6ASV922zjrkMKWYeDkNv
=zi+i
-----END PGP SIGNATURE-----

Re: open issues

[Stefan Sedich]

[-- Attachment #1: Type: text/plain, Size: 7429 bytes --]

>
> list of issues since last release that don't seem to be resolved.
> first ones that looked like bugs to me, then other issues.
> list goes backward in time, some issues may be missed.
>
> bug:
> - update contributors in copyright file
> - fflush(0) does not lock f
>   http://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c#n12
> - getenv/setenv/putenv ub
>   http://www.openwall.com/lists/musl/2017/08/21/1
>   http://www.openwall.com/lists/musl/2016/03/13/7
> - fix ioctl on mips, add SIOCGSTAMPNS
>   http://www.openwall.com/lists/musl/2017/08/13/4
>   http://www.openwall.com/lists/musl/2017/08/13/5
> - ipc/ftok.c overflowing shift
>   http://www.openwall.com/lists/musl/2017/08/12/5
> - __progname fallbacks so it's never 0
>   http://www.openwall.com/lists/musl/2017/07/28/6
> - mbsnrtowcs and mbsnrtowcs confuses byte and wchar counts
>   http://www.openwall.com/lists/musl/2017/08/09/1
> - memset ub because s[0] = s[0] = c
>   http://www.openwall.com/lists/musl/2017/07/10/7
>   http://www.openwall.com/lists/musl/2017/07/06/2
> - handle whitespace before %% in scanf
>   http://www.openwall.com/lists/musl/2017/07/11/1
> - mips64 utime issue?
>   "tar binary can't fix the modification/access times on any extracted
> symbolic links,"
>   http://www.openwall.com/lists/musl/2017/07/06/1
> - oob reads in memmem (and signed << ub)
>   http://www.openwall.com/lists/musl/2017/06/29/6
> - use-after-free in __unlock of pthread struct
>   http://www.openwall.com/lists/musl/2017/06/01/7
> - newly created thread may run with signals blocked
>   < sergei> there seems to be a race condition in pthread_create.c between
> lines 134 and 298
>   < sergei> if line 298 is executed before 134 (assuming syscall returned
> 0), startlock will be overwritten with zero, the condition will be
> evaluated to false and __restore_sigs will not be executed
>   < sergei> the newly created thread will run with all signals blocked
>   < sergei> i have a patch that fixes the issue for me:
> https://pastebin.com/T5QSd0C9
> - scanf, wrong types in va_arg
>   http://www.openwall.com/lists/musl/2017/04/10/3
> - missed underflow in fma
>   http://www.openwall.com/lists/musl/2017/03/19/6
>   new fma, depends on a_clz_64
>   http://www.openwall.com/lists/musl/2017/04/23/10
> - fix nftw when called with paths ending in slash
>   http://www.openwall.com/lists/musl/2017/03/07/1
> - fix syscall number differences compared to linux uapi
>   http://www.openwall.com/lists/musl/2017/02/18/1
> - getservbyport(_r) should not report numeric ports
>   http://www.openwall.com/lists/musl/2017/02/06/5
> - add s390x and powerpc64 to supported arches
>   http://www.openwall.com/lists/musl/2017/02/01/2
> - define IPPORT_RESERVED in netinet/in.h and netdb.h
>   http://www.openwall.com/lists/musl/2017/01/31/4
> - GLOB_PERIOD is inconsistent with glibc
>   http://www.openwall.com/lists/musl/2017/01/12/5
> - mmap should not return EPERM when it means ENOMEM
>   http://www.openwall.com/lists/musl/2017/01/12/1
> - getopt_long does not report failure correctly
>   http://www.openwall.com/lists/musl/2017/01/07/4
> - make dlsym and reloc time lookup consistent
>   http://www.openwall.com/lists/musl/2017/02/16/1
> - ldso ctor dependency ordering and recursive dlopen fix
>   http://www.openwall.com/lists/musl/2017/01/03/6
> - align arm hwcap.h with glibc (nsz)
>
>
> feature request:
> - pending linux uapi updates: v4.10, v4.11, v4.12 (nsz)
> - non-thread-safe apis (gethostbyname) can detect some misuse and crash
> - avoid mmapping inaccessible parts of a binary as rx
> - syslog.h prioritynames compund literal causes problems
>   http://www.openwall.com/lists/musl/2017/08/16/4
> - align fnmatch \ in bracket semantics with glibc
>   http://www.openwall.com/lists/musl/2017/08/15/2
> - pthread_attr_init behaviour (to use whatever was set in
> pthread_setattr_default_np)
>   http://www.openwall.com/lists/musl/2017/08/12/6
> - fix stdbool.h in c++ code?
>   http://www.openwall.com/lists/musl/2017/07/28/1
> - add sysconf support for _SC_LEVEL1_DCACHE_LINESIZE
>   http://www.openwall.com/lists/musl/2017/07/24/1
> - change GMT to UTC
>   http://www.openwall.com/lists/musl/2017/07/22/2
> - [RFC PATCH 0/5] Add explicit_bzero, vectorize and 'normalize' various
> string functions
>   http://www.openwall.com/lists/musl/2017/07/15/1
> - strlen variant without aliasing violation
>   http://www.openwall.com/lists/musl/2017/07/10/6
> - |32 is not needed in (t|32) != 'c'
>   http://www.openwall.com/lists/musl/2017/07/09/5
> - __malloc_donate instead of current hack in ldso
>   http://www.openwall.com/lists/musl/2017/06/28/4
> - optimize __malloc0 to clear memory faster
>   http://www.openwall.com/lists/musl/2017/06/26/1
> - code size optimize wmemcpy
>   http://www.openwall.com/lists/musl/2017/06/25/10
> - [PATCH 0/8] the new __lock and follow up patches
>   http://www.openwall.com/lists/musl/2017/06/23/2
> - more correct va_arg use in fcntl
>   http://www.openwall.com/lists/musl/2017/06/21/16
> - use better name than index in __tz.c
>   http://www.openwall.com/lists/musl/2017/06/11/12
> - use O_TMPFILE in tmpfile if possible
>   http://www.openwall.com/lists/musl/2017/06/08/2
> - fix user.h so gdb builds cleanly on all targets
>   http://www.openwall.com/lists/musl/2017/06/11/2
> - towlower performance
>   http://www.openwall.com/lists/musl/2017/05/27/2
> - RES_OPTIONS support for resolv.conf options overriding
>   http://www.openwall.com/lists/musl/2017/04/30/1
>   related libc-alpha discussion
>   http://sourceware.org/ml/libc-alpha/2017-08/msg01097.html
> - strptime %z %Z
>   http://www.openwall.com/lists/musl/2017/04/19/1
> - add microMIPS32 support
>   http://www.openwall.com/lists/musl/2017/05/31/2
> - IDNA support in name lookups
>   http://www.openwall.com/lists/musl/2017/04/02/1
> - improve LIBCC detection in configure?
>   http://www.openwall.com/lists/musl/2017/03/16/2
> - precise complex math
>   http://www.openwall.com/lists/musl/2017/02/12/3
> - strtoul of width specifier clobbers errno in strftime
>   http://www.openwall.com/lists/musl/2017/02/12/1
> - provide sgidefs.h on mips
>   http://www.openwall.com/lists/musl/2017/02/04/2
> - add GLOB_TILDE support
>   http://www.openwall.com/lists/musl/2017/01/20/4
> - crypt_blowfish: support $2b$ prefix
>   http://www.openwall.com/lists/musl/2017/01/12/6
> - strftime _-0
>   http://www.openwall.com/lists/musl/2017/01/06/1
> - static-pie relro
>   < fabled> how does static pie musl libc init work?
>   < fabled> the reason i ask is, that relro is not enabled then
>   < fabled> it should do mprotect() for the relro area
>   < nsz> ah i see if you have 'static int *const p = &x;' then that
> relative reloc should be in relro
> - fts64
>   http://www.openwall.com/lists/musl/2016/11/23/1
> - REG_STARTEND
>   http://www.openwall.com/lists/musl/2016/10/05/6
> - realpath without /proc
>   http://www.openwall.com/lists/musl/2016/11/03/5
> - new tsearch
>   http://www.openwall.com/lists/musl/2015/12/20/1
> - avoid readv in stdio with 0 length buffer because that's broken on /proc
>   https://bugzilla.kernel.org/show_bug.cgi?id=195697
> - add fortify _chk functions
> - memchr optimization
>

- RES_OPTIONS support for resolv.conf options overriding
  http://www.openwall.com/lists/musl/2017/04/30/1

I would love to sneak this one in if possible, has been sitting around for
a while and something that I want to be able to configure!

[-- Attachment #2: Type: text/html, Size: 14472 bytes --]

Re: open issues

[Rich Felker]

On Wed, Aug 30, 2017 at 10:37:44PM +0300, Alexander Monakov wrote:
> On Wed, 30 Aug 2017, Rich Felker wrote:
> > Great find. I wonder if compiler/diagnostic tool ppl are aware of this
> > extension to the C language that XSI imposes. It definitely affects
> > things like UBSan, and of course compilers' optimization based on
> > assumptions that UB doesn't happen.
> 
> If musl is going to use this throughout the implementation, I think it
> should be documented somewhere: it's of course fair game to use this
> guarantee in implementation of interfaces that are themselves XSI-shaded,
> but using it to implement more basic interfaces is less obviously so.

Have you looked at the specific issue in question? It's scanf's
handling of %n$ positional args. If we want to make scanf
non-dependent on this property of va_arg when called without this form
of arguments, we need to add significant additional code to track and
match the pointer types for each arg. I'm not opposed to that but it's
nontrivial work.

I'd rather *not* make use of this property of va_arg gratuitously in
other parts of musl, since I find it rather contrary to the intent of
the language.

Rich

Re: open issues

[Rich Felker]

On Wed, Aug 30, 2017 at 02:51:05PM -0500, A. Wilcox wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 29/08/17 21:06, Rich Felker wrote:
> > Pending due to open question about desired behavior:
> > 
> > - getservbyport(_r) should not report numeric ports 
> > http://www.openwall.com/lists/musl/2017/02/06/5
> 
> What's the question?  I'd be happy to answer.

I don't recall whether it appeared in the original thread, but I'm
unclear on whether a matching change for consistency is needed to
getservbyname[_r] -- if decimal strings are not considered service
records for the purpose of reverse mapping (as a result of the
proposed change to getservbyport[_r]) then should they also not be
records in the forward mapping? I want to know what other
implementations do and evaluate whether the inconsistency or the
inability to forward-map decimal port strings is more of a problem.

Rich

Re: open issues

[Szabolcs Nagy]

* Stefan Sedich <stefan.sedich@gmail.com> [2017-08-30 19:53:55 +0000]:
> > feature request:
> > - RES_OPTIONS support for resolv.conf options overriding
> >   http://www.openwall.com/lists/musl/2017/04/30/1
> >   related libc-alpha discussion
> >   http://sourceware.org/ml/libc-alpha/2017-08/msg01097.html
...
> 
> - RES_OPTIONS support for resolv.conf options overriding
>   http://www.openwall.com/lists/musl/2017/04/30/1
> 
> I would love to sneak this one in if possible, has been sitting around for
> a while and something that I want to be able to configure!

did you see the libc-alpha discussion?

the current recommended way to override /etc/resolv.conf is using mount
namespace (or chroot), if user namespace is allowed for non-root users
in the kernel then this can be done without root access.

if you can control resolv.conf per process then options can be set too.
(not ideal solution, but may work for you)

Re: open issues

[Stefan Sedich]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

On Thu, Aug 31, 2017 at 2:34 AM Szabolcs Nagy <nsz@port70.net> wrote:

> * Stefan Sedich <stefan.sedich@gmail.com> [2017-08-30 19:53:55 +0000]:
> > > feature request:
> > > - RES_OPTIONS support for resolv.conf options overriding
> > >   http://www.openwall.com/lists/musl/2017/04/30/1
> > >   related libc-alpha discussion
> > >   http://sourceware.org/ml/libc-alpha/2017-08/msg01097.html
> ...
> >
> > - RES_OPTIONS support for resolv.conf options overriding
> >   http://www.openwall.com/lists/musl/2017/04/30/1
> >
> > I would love to sneak this one in if possible, has been sitting around
> for
> > a while and something that I want to be able to configure!
>
> did you see the libc-alpha discussion?
>
> the current recommended way to override /etc/resolv.conf is using mount
> namespace (or chroot), if user namespace is allowed for non-root users
> in the kernel then this can be done without root access.
>
> if you can control resolv.conf per process then options can be set too.
> (not ideal solution, but may work for you)
>

I had not seen that, thanks for the info!

[-- Attachment #2: Type: text/html, Size: 1821 bytes --]

Re: open issues

[Rich Felker]

On Thu, Aug 31, 2017 at 11:34:05AM +0200, Szabolcs Nagy wrote:
> * Stefan Sedich <stefan.sedich@gmail.com> [2017-08-30 19:53:55 +0000]:
> > > feature request:
> > > - RES_OPTIONS support for resolv.conf options overriding
> > >   http://www.openwall.com/lists/musl/2017/04/30/1
> > >   related libc-alpha discussion
> > >   http://sourceware.org/ml/libc-alpha/2017-08/msg01097.html
> ....
> > 
> > - RES_OPTIONS support for resolv.conf options overriding
> >   http://www.openwall.com/lists/musl/2017/04/30/1
> > 
> > I would love to sneak this one in if possible, has been sitting around for
> > a while and something that I want to be able to configure!
> 
> did you see the libc-alpha discussion?
> 
> the current recommended way to override /etc/resolv.conf is using mount
> namespace (or chroot), if user namespace is allowed for non-root users
> in the kernel then this can be done without root access.

Is there a way to do this while still having the process with the
mount namespace see its own uid as the real invoking user and not a
fake uid=0 user in the user namespace?

If not, this seems like a pretty big functionality gap. The mount
namespace approach should work decently for running tests even if it
requires a namespace with fake uid=0, but it's not very friendly to
just allowing normal users to ignore the system resolv.conf nameserver
list and choose their own.

I'm not especially opposed to RES_OPTIONS but I'd rather have a more
complete solution that allows overriding the nameservers too, since
the nameservers are the important part and options are not useful for
much.

Rich

Re: open issues

[Szabolcs Nagy]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

* Rich Felker <dalias@libc.org> [2017-08-29 22:06:52 -0400]:
> Pending due to need to minor mechanical review:
> 
> - fix syscall number differences compared to linux uapi
>   http://www.openwall.com/lists/musl/2017/02/18/1
> 

that patch needed a fix because mips and mipsn32
used SYS_fstatat in syscall_arch.h before that
syscall was defined in internal syscall.h

attached an updated patch, tested on
arm, aarch64, i486, microblaze, mips, mips64,
powerpc, powerpcle, s390x, sh4, x86_64
the stripped libc.so is the same (except for the
version string)

[-- Attachment #2: 0001-make-syscall.h-consistent-with-linux.patch --]
[-- Type: text/x-diff, Size: 10472 bytes --]

From d4180fe34ce6452135f0451176ca59bd6130f200 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Sat, 18 Feb 2017 00:50:09 +0000
Subject: [PATCH] make syscall.h consistent with linux

most of the found naming differences don't matter to musl, because
internally it unifies the syscall names that vary across targets,
but for external code the names should match the kernel uapi.

aarch64:
	__NR_fstatat is called __NR_newfstatat in linux.
	__NR_or1k_atomic got mistakenly copied from or1k.
arm:
	__NR_arm_sync_file_range is an alias for __NR_sync_file_range2
	__NR_fadvise64_64 is called __NR_arm_fadvise64_64 in linux,
	the old non-arm name is kept too, it should not cause issues.
	(powerpc has similar nonstandard fadvise and it uses the
	normal name.)
i386:
	__NR_madvise1 was removed from linux in commit
	303395ac3bf3e2cb488435537d416bc840438fcb 2011-11-11
microblaze:
	__NR_fadvise, __NR_fstatat, __NR_pread, __NR_pwrite
	had different name in linux.
mips:
	__NR_fadvise, __NR_fstatat, __NR_pread, __NR_pwrite, __NR_select
	had different name in linux.
mipsn32:
	__NR_fstatat is called __NR_newfstatat in linux.
or1k:
	__NR__llseek is called __NR_llseek in linux.
	the old name is kept too because that's the name musl uses
	internally.
powerpc:
	__NR_{get,set}res{gid,uid}32 was never present in powerpc linux.
	__NR_timerfd was briefly defined in linux but then got renamed.
---
 arch/aarch64/bits/syscall.h.in    |  3 +--
 arch/arm/bits/syscall.h.in        |  2 ++
 arch/i386/bits/syscall.h.in       |  1 -
 arch/microblaze/bits/syscall.h.in |  8 ++++----
 arch/mips/bits/syscall.h.in       | 10 +++++-----
 arch/mips/syscall_arch.h          |  6 +++---
 arch/mipsn32/bits/syscall.h.in    |  2 +-
 arch/mipsn32/syscall_arch.h       |  6 +++---
 arch/or1k/bits/syscall.h.in       |  1 +
 arch/powerpc/bits/syscall.h.in    |  5 -----
 10 files changed, 20 insertions(+), 24 deletions(-)

diff --git a/arch/aarch64/bits/syscall.h.in b/arch/aarch64/bits/syscall.h.in
index fd388eec..8b56afff 100644
--- a/arch/aarch64/bits/syscall.h.in
+++ b/arch/aarch64/bits/syscall.h.in
@@ -77,7 +77,7 @@
 #define __NR_splice 76
 #define __NR_tee 77
 #define __NR_readlinkat 78
-#define __NR_fstatat 79
+#define __NR_newfstatat 79
 #define __NR_fstat 80
 #define __NR_sync 81
 #define __NR_fsync 82
@@ -242,7 +242,6 @@
 #define __NR_perf_event_open 241
 #define __NR_accept4 242
 #define __NR_recvmmsg 243
-#define __NR_or1k_atomic 244
 #define __NR_wait4 260
 #define __NR_prlimit64 261
 #define __NR_fanotify_init 262
diff --git a/arch/arm/bits/syscall.h.in b/arch/arm/bits/syscall.h.in
index 9b129b23..0096fe7c 100644
--- a/arch/arm/bits/syscall.h.in
+++ b/arch/arm/bits/syscall.h.in
@@ -224,6 +224,7 @@
 #define __NR_tgkill	268
 #define __NR_utimes	269
 #define __NR_fadvise64_64	270
+#define __NR_arm_fadvise64_64	270
 #define __NR_pciconfig_iobase	271
 #define __NR_pciconfig_read	272
 #define __NR_pciconfig_write	273
@@ -295,6 +296,7 @@
 #define __NR_get_robust_list	339
 #define __NR_splice	340
 #define __NR_sync_file_range2	341
+#define __NR_arm_sync_file_range	341
 #define __NR_tee	342
 #define __NR_vmsplice	343
 #define __NR_move_pages	344
diff --git a/arch/i386/bits/syscall.h.in b/arch/i386/bits/syscall.h.in
index 3a1d9270..1217f89a 100644
--- a/arch/i386/bits/syscall.h.in
+++ b/arch/i386/bits/syscall.h.in
@@ -218,7 +218,6 @@
 #define __NR_pivot_root		217
 #define __NR_mincore		218
 #define __NR_madvise		219
-#define __NR_madvise1		219
 #define __NR_getdents64		220
 #define __NR_fcntl64		221
 /* 223 is unused */
diff --git a/arch/microblaze/bits/syscall.h.in b/arch/microblaze/bits/syscall.h.in
index 109fcdeb..e5f928e3 100644
--- a/arch/microblaze/bits/syscall.h.in
+++ b/arch/microblaze/bits/syscall.h.in
@@ -178,8 +178,8 @@
 #define __NR_rt_sigtimedwait 177
 #define __NR_rt_sigqueueinfo 178
 #define __NR_rt_sigsuspend 179
-#define __NR_pread 180
-#define __NR_pwrite 181
+#define __NR_pread64 180
+#define __NR_pwrite64 181
 #define __NR_chown 182
 #define __NR_getcwd 183
 #define __NR_capget 184
@@ -246,7 +246,7 @@
 #define __NR_io_getevents 247
 #define __NR_io_submit 248
 #define __NR_io_cancel 249
-#define __NR_fadvise 250
+#define __NR_fadvise64 250
 #define __NR_exit_group 252
 #define __NR_lookup_dcookie 253
 #define __NR_epoll_create 254
@@ -294,7 +294,7 @@
 #define __NR_mknodat 297
 #define __NR_fchownat 298
 #define __NR_futimesat 299
-#define __NR_fstatat 300
+#define __NR_fstatat64 300
 #define __NR_unlinkat 301
 #define __NR_renameat 302
 #define __NR_linkat 303
diff --git a/arch/mips/bits/syscall.h.in b/arch/mips/bits/syscall.h.in
index 6c9b3d8c..1a2147a7 100644
--- a/arch/mips/bits/syscall.h.in
+++ b/arch/mips/bits/syscall.h.in
@@ -140,7 +140,7 @@
 #define __NR_setfsgid                4139
 #define __NR__llseek                 4140
 #define __NR_getdents                4141
-#define __NR_select                  4142
+#define __NR__newselect              4142
 #define __NR_flock                   4143
 #define __NR_msync                   4144
 #define __NR_readv                   4145
@@ -198,8 +198,8 @@
 #define __NR_rt_sigtimedwait         4197
 #define __NR_rt_sigqueueinfo         4198
 #define __NR_rt_sigsuspend           4199
-#define __NR_pread                   4200
-#define __NR_pwrite                  4201
+#define __NR_pread64                 4200
+#define __NR_pwrite64                4201
 #define __NR_chown                   4202
 #define __NR_getcwd                  4203
 #define __NR_capget                  4204
@@ -252,7 +252,7 @@
 #define __NR_remap_file_pages        4251
 #define __NR_set_tid_address         4252
 #define __NR_restart_syscall         4253
-#define __NR_fadvise                 4254
+#define __NR_fadvise64               4254
 #define __NR_statfs64                4255
 #define __NR_fstatfs64               4256
 #define __NR_timer_create            4257
@@ -290,7 +290,7 @@
 #define __NR_mknodat                 4290
 #define __NR_fchownat                4291
 #define __NR_futimesat               4292
-#define __NR_fstatat                 4293
+#define __NR_fstatat64               4293
 #define __NR_unlinkat                4294
 #define __NR_renameat                4295
 #define __NR_linkat                  4296
diff --git a/arch/mips/syscall_arch.h b/arch/mips/syscall_arch.h
index 3ac4da21..666f413f 100644
--- a/arch/mips/syscall_arch.h
+++ b/arch/mips/syscall_arch.h
@@ -99,7 +99,7 @@ static inline long __syscall4(long n, long a, long b, long c, long d)
 	if (r7) return -r2;
 	long ret = r2;
 	if (n == SYS_stat64 || n == SYS_fstat64 || n == SYS_lstat64) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_fstatat64) __stat_fix(c);
 	return ret;
 }
 
@@ -108,7 +108,7 @@ static inline long __syscall5(long n, long a, long b, long c, long d, long e)
 	long r2 = (__syscall)(n, a, b, c, d, e);
 	if (r2 > -4096UL) return r2;
 	if (n == SYS_stat64 || n == SYS_fstat64 || n == SYS_lstat64) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_fstatat64) __stat_fix(c);
 	return r2;
 }
 
@@ -117,7 +117,7 @@ static inline long __syscall6(long n, long a, long b, long c, long d, long e, lo
 	long r2 = (__syscall)(n, a, b, c, d, e, f);
 	if (r2 > -4096UL) return r2;
 	if (n == SYS_stat64 || n == SYS_fstat64 || n == SYS_lstat64) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_fstatat64) __stat_fix(c);
 	return r2;
 }
 
diff --git a/arch/mipsn32/bits/syscall.h.in b/arch/mipsn32/bits/syscall.h.in
index d6b24e8f..cd843a76 100644
--- a/arch/mipsn32/bits/syscall.h.in
+++ b/arch/mipsn32/bits/syscall.h.in
@@ -253,7 +253,7 @@
 #define __NR_mknodat			6253
 #define __NR_fchownat			6254
 #define __NR_futimesat			6255
-#define __NR_fstatat			6256
+#define __NR_newfstatat			6256
 #define __NR_unlinkat			6257
 #define __NR_renameat			6258
 #define __NR_linkat			6259
diff --git a/arch/mipsn32/syscall_arch.h b/arch/mipsn32/syscall_arch.h
index 37e71a7e..93a026f6 100644
--- a/arch/mipsn32/syscall_arch.h
+++ b/arch/mipsn32/syscall_arch.h
@@ -97,7 +97,7 @@ static inline long __syscall4(long n, long a, long b, long c, long d)
 	if (r7) return -r2;
 	long ret = r2;
 	if (n == SYS_stat || n == SYS_fstat || n == SYS_lstat) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_newfstatat) __stat_fix(c);
 	return ret;
 }
 
@@ -106,7 +106,7 @@ static inline long __syscall5(long n, long a, long b, long c, long d, long e)
 	long r2 = (__syscall)(n, a, b, c, d, e);
 	if (r2 > -4096UL) return r2;
 	if (n == SYS_stat || n == SYS_fstat || n == SYS_lstat) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_newfstatat) __stat_fix(c);
 	return r2;
 }
 
@@ -115,7 +115,7 @@ static inline long __syscall6(long n, long a, long b, long c, long d, long e, lo
 	long r2 = (__syscall)(n, a, b, c, d, e, f);
 	if (r2 > -4096UL) return r2;
 	if (n == SYS_stat || n == SYS_fstat || n == SYS_lstat) __stat_fix(b);
-	if (n == SYS_fstatat) __stat_fix(c);
+	if (n == SYS_newfstatat) __stat_fix(c);
 	return r2;
 }
 
diff --git a/arch/or1k/bits/syscall.h.in b/arch/or1k/bits/syscall.h.in
index 89788a9d..54581fb4 100644
--- a/arch/or1k/bits/syscall.h.in
+++ b/arch/or1k/bits/syscall.h.in
@@ -61,6 +61,7 @@
 #define __NR_quotactl 60
 #define __NR_getdents64 61
 #define __NR__llseek 62
+#define __NR_llseek 62
 #define __NR_read 63
 #define __NR_write 64
 #define __NR_readv 65
diff --git a/arch/powerpc/bits/syscall.h.in b/arch/powerpc/bits/syscall.h.in
index 9d022321..35f7033e 100644
--- a/arch/powerpc/bits/syscall.h.in
+++ b/arch/powerpc/bits/syscall.h.in
@@ -162,16 +162,12 @@
 #define __NR_sched_rr_get_interval  161
 #define __NR_nanosleep              162
 #define __NR_mremap                 163
-#define __NR_setresuid32            164
 #define __NR_setresuid              164
-#define __NR_getresuid32            165
 #define __NR_getresuid              165
 #define __NR_query_module           166
 #define __NR_poll                   167
 #define __NR_nfsservctl             168
-#define __NR_setresgid32            169
 #define __NR_setresgid              169
-#define __NR_getresgid32            170
 #define __NR_getresgid              170
 #define __NR_prctl                  171
 #define __NR_rt_sigreturn           172
@@ -306,7 +302,6 @@
 #define __NR_epoll_pwait	303
 #define __NR_utimensat		304
 #define __NR_signalfd		305
-#define __NR_timerfd		306
 #define __NR_timerfd_create     306
 #define __NR_eventfd		307
 #define __NR_sync_file_range2	308
-- 
2.14.1

Re: open issues

[Rich Felker]

On Tue, Aug 29, 2017 at 10:06:52PM -0400, Rich Felker wrote:
> Updated lists after reviewing the list and pushing the changes that
> were easy to make/merge:

Further updates:

> Still pending due to nontrivial patch to review and/or need for
> testing:
> 
> - getenv/setenv/putenv ub
>   http://www.openwall.com/lists/musl/2017/08/21/1
>   http://www.openwall.com/lists/musl/2016/03/13/7

Committed new version of the patches by Alexander Monakov.

> - mbsnrtowcs and mbsnrtowcs confuses byte and wchar counts
>   http://www.openwall.com/lists/musl/2017/08/09/1

Committed patch by Mikhail Kremnyov.

> - oob reads in memmem (and signed << ub)
>   http://www.openwall.com/lists/musl/2017/06/29/6

Committed Alexander Monakov's fix. I'm not sure it's my favorite but
upon rereading it was the version that I could easily say doesn't
change behavior (except for fixing the bug) and looks like it doesn't
change any performance characteristics.

> - fix nftw when called with paths ending in slash
>   http://www.openwall.com/lists/musl/2017/03/07/1

Reviewed and I understand the issue and patch well now. I replied to
the thread with a possible simpler patch that just fixes the bug
without other behavioral change and raised a couple small issues about
behavior.

> Pending due to waiting for updated patch incorporating feedback
> already given or resolving merge conflicts or similar:
> 
> - handle whitespace before %% in scanf
>   http://www.openwall.com/lists/musl/2017/07/11/1

Committed patch by Bartosz Brachaczek. Previously I thought a
different approach would be cleaner but it turned out not to be.

> - make dlsym and reloc time lookup consistent
>   http://www.openwall.com/lists/musl/2017/02/16/1

Remains open.

> - newly created thread may run with signals blocked
>   < sergei> there seems to be a race condition in pthread_create.c between lines 134 and 298
>   < sergei> if line 298 is executed before 134 (assuming syscall returned 0), startlock will be overwritten with zero, the condition will be evaluated to false and __restore_sigs will not be executed
>   < sergei> the newly created thread will run with all signals blocked
>   < sergei> i have a patch that fixes the issue for me: https://pastebin.com/T5QSd0C9

While I still don't like the original direct use of atomics, the
proposed patch (especially once a no-op change is removed from it) is
simple and fairly clearly fixes the bug. Committing something similar
with added comments.

> - missed underflow in fma
>   http://www.openwall.com/lists/musl/2017/03/19/6
>   new fma, depends on a_clz_64
>   http://www.openwall.com/lists/musl/2017/04/23/10

New proposal is pending review.

> Pending due to need for additional analysis to determine exactly
> what/where the bug is:
> 
> - mips64 utime issue?
>   "tar binary can't fix the modification/access times on any extracted symbolic links,"
>   http://www.openwall.com/lists/musl/2017/07/06/1

Still needs analysis.

> Pending due to open question about desired behavior:
> 
> - getservbyport(_r) should not report numeric ports
>   http://www.openwall.com/lists/musl/2017/02/06/5

Committed with corresponding change in opposite direction and added
comments.

> - mmap should not return EPERM when it means ENOMEM
>   http://www.openwall.com/lists/musl/2017/01/12/1

Fixed in commit da438ee1fc516c41ba1790cef7be551a9e244397

> - GLOB_PERIOD is inconsistent with glibc
>   http://www.openwall.com/lists/musl/2017/01/12/5

Fixed in commit 8c4be3e2209d2a1d3874b8bc2b474668fcbbbac6

> - ldso ctor dependency ordering and recursive dlopen fix
>   http://www.openwall.com/lists/musl/2017/01/03/6

This is going to have to be a punt until next release cycle.

> Pending due to dependency of fix on larger change:
> 
> - use-after-free in __unlock of pthread struct
>   http://www.openwall.com/lists/musl/2017/06/01/7

Still need to decide what to do on this.

> Pending due to need to minor mechanical review:
> 
> - fix syscall number differences compared to linux uapi
>   http://www.openwall.com/lists/musl/2017/02/18/1

Committed new patch.

> Pending due to missing patch:
> 
> - align arm hwcap.h with glibc (nsz)

Got patch and committed.

Rich

Re: open issues

[Rich Felker]

Updated status:

Pending decision(s) of what do to:

- fix nftw when called with paths ending in slash
  http://www.openwall.com/lists/musl/2017/03/07/1
- use-after-free in __unlock of pthread struct
  http://www.openwall.com/lists/musl/2017/06/01/7
- missed underflow in fma
  http://www.openwall.com/lists/musl/2017/03/19/6
  new fma, depends on a_clz_64
  http://www.openwall.com/lists/musl/2017/04/23/10
  [new patch on top of old one submitted, still not ideal
  but no volatile]

Pending updated patch:

- make dlsym and reloc time lookup consistent
  http://www.openwall.com/lists/musl/2017/02/16/1

Pending analysis of cause:

- mips64 utime issue?
  "tar binary can't fix the modification/access times on any extracted symbolic links,"
  http://www.openwall.com/lists/musl/2017/07/06/1

Punting til next release cycle:

- ldso ctor dependency ordering and recursive dlopen fix
  http://www.openwall.com/lists/musl/2017/01/03/6

Additional issue that was left out that I promised for a long time to
review; pending tests to ensure it doesn't break anything in the
standard functionality it touches:

- strftime extensions
  http://www.openwall.com/lists/musl/2016/11/22/1

Rich

Re: open issues

[Szabolcs Nagy]

[-- Attachment #1: Type: text/plain, Size: 371 bytes --]

* Rich Felker <dalias@libc.org> [2017-09-06 22:32:01 -0400]:
> - missed underflow in fma
>   http://www.openwall.com/lists/musl/2017/03/19/6
>   new fma, depends on a_clz_64
>   http://www.openwall.com/lists/musl/2017/04/23/10
>   [new patch on top of old one submitted, still not ideal
>   but no volatile]

attached latest version, i could not find a simpler solution.

[-- Attachment #2: 0001-math-rewrite-fma-with-mostly-int-arithmetics.patch --]
[-- Type: text/x-diff, Size: 18428 bytes --]

From 29a046cd846a05aeca9a497505124c768cc3c465 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Sun, 17 Sep 2017 17:31:20 +0000
Subject: [PATCH] math: rewrite fma with mostly int arithmetics

the freebsd fma code failed to raise underflow exception in some
cases in nearest rounding mode (affects fmal too) e.g.

  fma(-0x1p-1000, 0x1.000001p-74, 0x1p-1022)

and the inexact exception may be raised spuriously since the fenv
is not saved/restored around the exact multiplication algorithm
(affects x86 fma too).

another issue is that the underflow behaviour when the rounded result
is the minimal normal number is target dependent, ieee754 allows two
ways to raise underflow for inexact results: raise if the result before
rounding is in the subnormal range (e.g. aarch64, arm, powerpc) or if
the result after rounding with infinite exponent range is in the
subnormal range (e.g. x86, mips, sh).

to avoid all these issues the algorithm was rewritten with mostly int
arithmetics and float arithmetics is only used to get correct rounding
and raise exceptions according to the behaviour of the target without
any fenv.h dependency. it also unifies x86 and non-x86 fma.

fmaf is not affected, fmal need to be fixed too.

this algorithm depends on a_clz_64 and it required a few spurious
instructions to make sure underflow exception is raised in a particular
corner case. (normally FORCE_EVAL(tiny*tiny) would be used for this,
but on i386 gcc is broken if the expression is constant
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57245
and there is no easy portable fix for the macro.)
---
 src/math/fma.c | 585 +++++++++++++++------------------------------------------
 1 file changed, 154 insertions(+), 431 deletions(-)

diff --git a/src/math/fma.c b/src/math/fma.c
index 741ccd75..0c6f90c9 100644
--- a/src/math/fma.c
+++ b/src/math/fma.c
@@ -1,460 +1,183 @@
-#include <fenv.h>
-#include "libm.h"
+#include <stdint.h>
+#include <float.h>
+#include <math.h>
+#include "atomic.h"
 
-#if LDBL_MANT_DIG==64 && LDBL_MAX_EXP==16384
-/* exact add, assumes exponent_x >= exponent_y */
-static void add(long double *hi, long double *lo, long double x, long double y)
-{
-	long double r;
-
-	r = x + y;
-	*hi = r;
-	r -= x;
-	*lo = y - r;
-}
-
-/* exact mul, assumes no over/underflow */
-static void mul(long double *hi, long double *lo, long double x, long double y)
-{
-	static const long double c = 1.0 + 0x1p32L;
-	long double cx, xh, xl, cy, yh, yl;
+#define ASUINT64(x) ((union {double f; uint64_t i;}){x}).i
+#define ZEROINFNAN (0x7ff-0x3ff-52-1)
 
-	cx = c*x;
-	xh = (x - cx) + cx;
-	xl = x - xh;
-	cy = c*y;
-	yh = (y - cy) + cy;
-	yl = y - yh;
-	*hi = x*y;
-	*lo = (xh*yh - *hi) + xh*yl + xl*yh + xl*yl;
-}
+struct num { uint64_t m; int e; int sign; };
 
-/*
-assume (long double)(hi+lo) == hi
-return an adjusted hi so that rounding it to double (or less) precision is correct
-*/
-static long double adjust(long double hi, long double lo)
+static struct num normalize(double x)
 {
-	union ldshape uhi, ulo;
-
-	if (lo == 0)
-		return hi;
-	uhi.f = hi;
-	if (uhi.i.m & 0x3ff)
-		return hi;
-	ulo.f = lo;
-	if ((uhi.i.se & 0x8000) == (ulo.i.se & 0x8000))
-		uhi.i.m++;
-	else {
-		/* handle underflow and take care of ld80 implicit msb */
-		if (uhi.i.m << 1 == 0) {
-			uhi.i.m = 0;
-			uhi.i.se--;
-		}
-		uhi.i.m--;
+	uint64_t ix = ASUINT64(x);
+	int e = ix>>52;
+	int sign = e & 0x800;
+	e &= 0x7ff;
+	if (!e) {
+		ix = ASUINT64(x*0x1p63);
+		e = ix>>52 & 0x7ff;
+		e = e ? e-63 : 0x800;
 	}
-	return uhi.f;
+	ix &= (1ull<<52)-1;
+	ix |= 1ull<<52;
+	ix <<= 1;
+	e -= 0x3ff + 52 + 1;
+	return (struct num){ix,e,sign};
 }
 
-/* adjusted add so the result is correct when rounded to double (or less) precision */
-static long double dadd(long double x, long double y)
+static void mul(uint64_t *hi, uint64_t *lo, uint64_t x, uint64_t y)
 {
-	add(&x, &y, x, y);
-	return adjust(x, y);
-}
-
-/* adjusted mul so the result is correct when rounded to double (or less) precision */
-static long double dmul(long double x, long double y)
-{
-	mul(&x, &y, x, y);
-	return adjust(x, y);
-}
-
-static int getexp(long double x)
-{
-	union ldshape u;
-	u.f = x;
-	return u.i.se & 0x7fff;
+	uint64_t t1,t2,t3;
+	uint64_t xlo = (uint32_t)x, xhi = x>>32;
+	uint64_t ylo = (uint32_t)y, yhi = y>>32;
+
+	t1 = xlo*ylo;
+	t2 = xlo*yhi + xhi*ylo;
+	t3 = xhi*yhi;
+	*lo = t1 + (t2<<32);
+	*hi = t3 + (t2>>32) + (t1 > *lo);
 }
 
 double fma(double x, double y, double z)
 {
 	#pragma STDC FENV_ACCESS ON
-	long double hi, lo1, lo2, xy;
-	int round, ez, exy;
 
-	/* handle +-inf,nan */
-	if (!isfinite(x) || !isfinite(y))
+	/* normalize so top 10bits and last bit are 0 */
+	struct num nx, ny, nz;
+	nx = normalize(x);
+	ny = normalize(y);
+	nz = normalize(z);
+
+	if (nx.e >= ZEROINFNAN || ny.e >= ZEROINFNAN)
 		return x*y + z;
-	if (!isfinite(z))
+	if (nz.e >= ZEROINFNAN) {
+		if (nz.e > ZEROINFNAN) /* z==0 */
+			return x*y + z;
 		return z;
-	/* handle +-0 */
-	if (x == 0.0 || y == 0.0)
-		return x*y + z;
-	round = fegetround();
-	if (z == 0.0) {
-		if (round == FE_TONEAREST)
-			return dmul(x, y);
-		return x*y;
 	}
 
-	/* exact mul and add require nearest rounding */
-	/* spurious inexact exceptions may be raised */
-	fesetround(FE_TONEAREST);
-	mul(&xy, &lo1, x, y);
-	exy = getexp(xy);
-	ez = getexp(z);
-	if (ez > exy) {
-		add(&hi, &lo2, z, xy);
-	} else if (ez > exy - 12) {
-		add(&hi, &lo2, xy, z);
-		if (hi == 0) {
-			/*
-			xy + z is 0, but it should be calculated with the
-			original rounding mode so the sign is correct, if the
-			compiler does not support FENV_ACCESS ON it does not
-			know about the changed rounding mode and eliminates
-			the xy + z below without the volatile memory access
-			*/
-			volatile double z_;
-			fesetround(round);
-			z_ = z;
-			return (xy + z_) + lo1;
+	/* mul: r = x*y */
+	uint64_t rhi, rlo, zhi, zlo;
+	mul(&rhi, &rlo, nx.m, ny.m);
+	/* either top 20 or 21 bits of rhi and last 2 bits of rlo are 0 */
+
+	/* align exponents */
+	int e = nx.e + ny.e;
+	int d = nz.e - e;
+	/* shift bits z<<=kz, r>>=kr, so kz+kr == d, set e = e+kr (== ez-kz) */
+	if (d > 0) {
+		if (d < 64) {
+			zlo = nz.m<<d;
+			zhi = nz.m>>64-d;
+		} else {
+			zlo = 0;
+			zhi = nz.m;
+			e = nz.e - 64;
+			d -= 64;
+			if (d == 0) {
+			} else if (d < 64) {
+				rlo = rhi<<64-d | rlo>>d | !!(rlo<<64-d);
+				rhi = rhi>>d;
+			} else {
+				rlo = 1;
+				rhi = 0;
+			}
 		}
 	} else {
-		/*
-		ez <= exy - 12
-		the 12 extra bits (1guard, 11round+sticky) are needed so with
-			lo = dadd(lo1, lo2)
-		elo <= ehi - 11, and we use the last 10 bits in adjust so
-			dadd(hi, lo)
-		gives correct result when rounded to double
-		*/
-		hi = xy;
-		lo2 = z;
-	}
-	/*
-	the result is stored before return for correct precision and exceptions
-
-	one corner case is when the underflow flag should be raised because
-	the precise result is an inexact subnormal double, but the calculated
-	long double result is an exact subnormal double
-	(so rounding to double does not raise exceptions)
-
-	in nearest rounding mode dadd takes care of this: the last bit of the
-	result is adjusted so rounding sees an inexact value when it should
-
-	in non-nearest rounding mode fenv is used for the workaround
-	*/
-	fesetround(round);
-	if (round == FE_TONEAREST)
-		z = dadd(hi, dadd(lo1, lo2));
-	else {
-#if defined(FE_INEXACT) && defined(FE_UNDERFLOW)
-		int e = fetestexcept(FE_INEXACT);
-		feclearexcept(FE_INEXACT);
-#endif
-		z = hi + (lo1 + lo2);
-#if defined(FE_INEXACT) && defined(FE_UNDERFLOW)
-		if (getexp(z) < 0x3fff-1022 && fetestexcept(FE_INEXACT))
-			feraiseexcept(FE_UNDERFLOW);
-		else if (e)
-			feraiseexcept(FE_INEXACT);
-#endif
-	}
-	return z;
-}
-#else
-/* origin: FreeBSD /usr/src/lib/msun/src/s_fma.c */
-/*-
- * Copyright (c) 2005-2011 David Schultz <das@FreeBSD.ORG>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-/*
- * A struct dd represents a floating-point number with twice the precision
- * of a double.  We maintain the invariant that "hi" stores the 53 high-order
- * bits of the result.
- */
-struct dd {
-	double hi;
-	double lo;
-};
-
-/*
- * Compute a+b exactly, returning the exact result in a struct dd.  We assume
- * that both a and b are finite, but make no assumptions about their relative
- * magnitudes.
- */
-static inline struct dd dd_add(double a, double b)
-{
-	struct dd ret;
-	double s;
-
-	ret.hi = a + b;
-	s = ret.hi - a;
-	ret.lo = (a - (ret.hi - s)) + (b - s);
-	return (ret);
-}
-
-/*
- * Compute a+b, with a small tweak:  The least significant bit of the
- * result is adjusted into a sticky bit summarizing all the bits that
- * were lost to rounding.  This adjustment negates the effects of double
- * rounding when the result is added to another number with a higher
- * exponent.  For an explanation of round and sticky bits, see any reference
- * on FPU design, e.g.,
- *
- *     J. Coonen.  An Implementation Guide to a Proposed Standard for
- *     Floating-Point Arithmetic.  Computer, vol. 13, no. 1, Jan 1980.
- */
-static inline double add_adjusted(double a, double b)
-{
-	struct dd sum;
-	union {double f; uint64_t i;} uhi, ulo;
-
-	sum = dd_add(a, b);
-	if (sum.lo != 0) {
-		uhi.f = sum.hi;
-		if ((uhi.i & 1) == 0) {
-			/* hibits += (int)copysign(1.0, sum.hi * sum.lo) */
-			ulo.f = sum.lo;
-			uhi.i += 1 - ((uhi.i ^ ulo.i) >> 62);
-			sum.hi = uhi.f;
+		zhi = 0;
+		d = -d;
+		if (d == 0) {
+			zlo = nz.m;
+		} else if (d < 64) {
+			zlo = nz.m>>d | !!(nz.m<<64-d);
+		} else {
+			zlo = 1;
 		}
 	}
-	return (sum.hi);
-}
-
-/*
- * Compute ldexp(a+b, scale) with a single rounding error. It is assumed
- * that the result will be subnormal, and care is taken to ensure that
- * double rounding does not occur.
- */
-static inline double add_and_denormalize(double a, double b, int scale)
-{
-	struct dd sum;
-	union {double f; uint64_t i;} uhi, ulo;
-	int bits_lost;
-
-	sum = dd_add(a, b);
-
-	/*
-	 * If we are losing at least two bits of accuracy to denormalization,
-	 * then the first lost bit becomes a round bit, and we adjust the
-	 * lowest bit of sum.hi to make it a sticky bit summarizing all the
-	 * bits in sum.lo. With the sticky bit adjusted, the hardware will
-	 * break any ties in the correct direction.
-	 *
-	 * If we are losing only one bit to denormalization, however, we must
-	 * break the ties manually.
-	 */
-	if (sum.lo != 0) {
-		uhi.f = sum.hi;
-		bits_lost = -((int)(uhi.i >> 52) & 0x7ff) - scale + 1;
-		if ((bits_lost != 1) ^ (int)(uhi.i & 1)) {
-			/* hibits += (int)copysign(1.0, sum.hi * sum.lo) */
-			ulo.f = sum.lo;
-			uhi.i += 1 - (((uhi.i ^ ulo.i) >> 62) & 2);
-			sum.hi = uhi.f;
-		}
-	}
-	return scalbn(sum.hi, scale);
-}
-
-/*
- * Compute a*b exactly, returning the exact result in a struct dd.  We assume
- * that both a and b are normalized, so no underflow or overflow will occur.
- * The current rounding mode must be round-to-nearest.
- */
-static inline struct dd dd_mul(double a, double b)
-{
-	static const double split = 0x1p27 + 1.0;
-	struct dd ret;
-	double ha, hb, la, lb, p, q;
-
-	p = a * split;
-	ha = a - p;
-	ha += p;
-	la = a - ha;
-
-	p = b * split;
-	hb = b - p;
-	hb += p;
-	lb = b - hb;
-
-	p = ha * hb;
-	q = ha * lb + la * hb;
-
-	ret.hi = p + q;
-	ret.lo = p - ret.hi + q + la * lb;
-	return (ret);
-}
 
-/*
- * Fused multiply-add: Compute x * y + z with a single rounding error.
- *
- * We use scaling to avoid overflow/underflow, along with the
- * canonical precision-doubling technique adapted from:
- *
- *      Dekker, T.  A Floating-Point Technique for Extending the
- *      Available Precision.  Numer. Math. 18, 224-242 (1971).
- *
- * This algorithm is sensitive to the rounding precision.  FPUs such
- * as the i387 must be set in double-precision mode if variables are
- * to be stored in FP registers in order to avoid incorrect results.
- * This is the default on FreeBSD, but not on many other systems.
- *
- * Hardware instructions should be used on architectures that support it,
- * since this implementation will likely be several times slower.
- */
-double fma(double x, double y, double z)
-{
-	#pragma STDC FENV_ACCESS ON
-	double xs, ys, zs, adj;
-	struct dd xy, r;
-	int oround;
-	int ex, ey, ez;
-	int spread;
-
-	/*
-	 * Handle special cases. The order of operations and the particular
-	 * return values here are crucial in handling special cases involving
-	 * infinities, NaNs, overflows, and signed zeroes correctly.
-	 */
-	if (!isfinite(x) || !isfinite(y))
-		return (x * y + z);
-	if (!isfinite(z))
-		return (z);
-	if (x == 0.0 || y == 0.0)
-		return (x * y + z);
-	if (z == 0.0)
-		return (x * y);
-
-	xs = frexp(x, &ex);
-	ys = frexp(y, &ey);
-	zs = frexp(z, &ez);
-	oround = fegetround();
-	spread = ex + ey - ez;
-
-	/*
-	 * If x * y and z are many orders of magnitude apart, the scaling
-	 * will overflow, so we handle these cases specially.  Rounding
-	 * modes other than FE_TONEAREST are painful.
-	 */
-	if (spread < -DBL_MANT_DIG) {
-#ifdef FE_INEXACT
-		feraiseexcept(FE_INEXACT);
-#endif
-#ifdef FE_UNDERFLOW
-		if (!isnormal(z))
-			feraiseexcept(FE_UNDERFLOW);
-#endif
-		switch (oround) {
-		default: /* FE_TONEAREST */
-			return (z);
-#ifdef FE_TOWARDZERO
-		case FE_TOWARDZERO:
-			if (x > 0.0 ^ y < 0.0 ^ z < 0.0)
-				return (z);
-			else
-				return (nextafter(z, 0));
-#endif
-#ifdef FE_DOWNWARD
-		case FE_DOWNWARD:
-			if (x > 0.0 ^ y < 0.0)
-				return (z);
-			else
-				return (nextafter(z, -INFINITY));
-#endif
-#ifdef FE_UPWARD
-		case FE_UPWARD:
-			if (x > 0.0 ^ y < 0.0)
-				return (nextafter(z, INFINITY));
-			else
-				return (z);
-#endif
+	/* add */
+	int sign = nx.sign^ny.sign;
+	int samesign = !(sign^nz.sign);
+	int nonzero = 1;
+	if (samesign) {
+		/* r += z */
+		rlo += zlo;
+		rhi += zhi + (rlo < zlo);
+	} else {
+		/* r -= z */
+		uint64_t t = rlo;
+		rlo -= zlo;
+		rhi = rhi - zhi - (t < rlo);
+		if (rhi>>63) {
+			rlo = -rlo;
+			rhi = -rhi-!!rlo;
+			sign = !sign;
 		}
+		nonzero = !!rhi;
 	}
-	if (spread <= DBL_MANT_DIG * 2)
-		zs = scalbn(zs, -spread);
-	else
-		zs = copysign(DBL_MIN, zs);
-
-	fesetround(FE_TONEAREST);
 
-	/*
-	 * Basic approach for round-to-nearest:
-	 *
-	 *     (xy.hi, xy.lo) = x * y           (exact)
-	 *     (r.hi, r.lo)   = xy.hi + z       (exact)
-	 *     adj = xy.lo + r.lo               (inexact; low bit is sticky)
-	 *     result = r.hi + adj              (correctly rounded)
-	 */
-	xy = dd_mul(xs, ys);
-	r = dd_add(xy.hi, zs);
-
-	spread = ex + ey;
-
-	if (r.hi == 0.0) {
-		/*
-		 * When the addends cancel to 0, ensure that the result has
-		 * the correct sign.
-		 */
-		fesetround(oround);
-		volatile double vzs = zs; /* XXX gcc CSE bug workaround */
-		return xy.hi + vzs + scalbn(xy.lo, spread);
+	/* set rhi to top 63bit of the result (last bit is sticky) */
+	if (nonzero) {
+		e += 64;
+		d = a_clz_64(rhi)-1;
+		/* note: d > 0 */
+		rhi = rhi<<d | rlo>>64-d | !!(rlo<<d);
+	} else if (rlo) {
+		d = a_clz_64(rlo)-1;
+		if (d < 0)
+			rhi = rlo>>1 | (rlo&1);
+		else
+			rhi = rlo<<d;
+	} else {
+		/* exact +-0 */
+		return x*y + z;
 	}
-
-	if (oround != FE_TONEAREST) {
-		/*
-		 * There is no need to worry about double rounding in directed
-		 * rounding modes.
-		 * But underflow may not be raised properly, example in downward rounding:
-		 * fma(0x1.000000001p-1000, 0x1.000000001p-30, -0x1p-1066)
-		 */
-		double ret;
-#if defined(FE_INEXACT) && defined(FE_UNDERFLOW)
-		int e = fetestexcept(FE_INEXACT);
-		feclearexcept(FE_INEXACT);
-#endif
-		fesetround(oround);
-		adj = r.lo + xy.lo;
-		ret = scalbn(r.hi + adj, spread);
-#if defined(FE_INEXACT) && defined(FE_UNDERFLOW)
-		if (ilogb(ret) < -1022 && fetestexcept(FE_INEXACT))
-			feraiseexcept(FE_UNDERFLOW);
-		else if (e)
-			feraiseexcept(FE_INEXACT);
-#endif
-		return ret;
+	e -= d;
+
+	/* convert to double */
+	int64_t i = rhi; /* i is in [1<<62,(1<<63)-1] */
+	if (sign)
+		i = -i;
+	double r = i; /* |r| is in [0x1p62,0x1p63] */
+
+	if (e < -1022-62) {
+		/* result is subnormal before rounding */
+		if (e == -1022-63) {
+			double c = 0x1p63;
+			if (sign)
+				c = -c;
+			if (r == c) {
+				/* min normal after rounding, underflow depends
+				   on arch behaviour which can be imitated by
+				   a double to float conversion */
+				float fltmin = 0x0.ffffff8p-63*FLT_MIN * r;
+				return DBL_MIN/FLT_MIN * fltmin;
+			}
+			/* one bit is lost when scaled, add another top bit to
+			   only round once at conversion if it is inexact */
+			if (rhi << 53) {
+				i = rhi>>1 | (rhi&1) | 1ull<<62;
+				if (sign)
+					i = -i;
+				r = i;
+				r = 2*r - c; /* remove top bit */
+
+				/* raise underflow portably, such that it
+				   cannot be optimized away */
+				{
+					double_t tiny = DBL_MIN/FLT_MIN * r;
+					r += (double)(tiny*tiny) * (r-r);
+				}
+			}
+		} else {
+			/* only round once when scaled */
+			d = 10;
+			i = ( rhi>>d | !!(rhi<<64-d) ) << d;
+			if (sign)
+				i = -i;
+			r = i;
+		}
 	}
-
-	adj = add_adjusted(r.lo, xy.lo);
-	if (spread + ilogb(r.hi) > -1023)
-		return scalbn(r.hi + adj, spread);
-	else
-		return add_and_denormalize(r.hi, adj, spread);
+	return scalbn(r, e);
 }
-#endif
-- 
2.14.1

Re: open issues

[Rich Felker]

On Wed, Sep 06, 2017 at 10:32:01PM -0400, Rich Felker wrote:
> Updated status:
> 
> Pending decision(s) of what do to:
> 
> - fix nftw when called with paths ending in slash
>   http://www.openwall.com/lists/musl/2017/03/07/1

Planning to just commit my minimal fix for now since further changes
are still incomplete/pending discussion.

> - use-after-free in __unlock of pthread struct
>   http://www.openwall.com/lists/musl/2017/06/01/7

I really want to adopt the new lock but the release is way past due
without having thoroughly reviewed and tested. I have a trivial patch
to just make pthread_detach always call __wake rather than using the
waiters count which fixes the bug for now. We can switch back to lock
primitives once the new lock is in use.

> - missed underflow in fma
>   http://www.openwall.com/lists/musl/2017/03/19/6
>   new fma, depends on a_clz_64
>   http://www.openwall.com/lists/musl/2017/04/23/10
>   [new patch on top of old one submitted, still not ideal
>   but no volatile]

Applying nsz's latest patch that fixed the volatile hacks.

> Pending updated patch:
> 
> - make dlsym and reloc time lookup consistent
>   http://www.openwall.com/lists/musl/2017/02/16/1

Punting to next release cycle.

> Pending analysis of cause:
> 
> - mips64 utime issue?
>   "tar binary can't fix the modification/access times on any extracted symbolic links,"
>   http://www.openwall.com/lists/musl/2017/07/06/1

Punting to next release cycle.

> Additional issue that was left out that I promised for a long time to
> review; pending tests to ensure it doesn't break anything in the
> standard functionality it touches:
> 
> - strftime extensions
>   http://www.openwall.com/lists/musl/2016/11/22/1

I'm still waiting for tests to show that this doesn't introduce
regressions, so won't be including the patch.

But for the purpose of Alpine or any other distros that want to patch
it themselves without introducing incompatibility, please consider it
"accepted" in that the interface (the extensions) will make it
upstream; it's just the implementation that still needs checking (and
may need changes if there are problems).

I think this mostly concludes what's pending for release.

Rich

Re: open issues

[Rich Felker]

On Wed, Sep 06, 2017 at 10:18:12PM -0400, Rich Felker wrote:
> > - GLOB_PERIOD is inconsistent with glibc
> >   http://www.openwall.com/lists/musl/2017/01/12/5
> 
> Fixed in commit 8c4be3e2209d2a1d3874b8bc2b474668fcbbbac6

This was wrong and introduced a regression: literal . and .. in paths
passed into glob result in no results. I really should have held off
on it because determining the preferred behavior is complicated. But
in any case, a change which alters behavior without GLOB_PERIOD being
set in the flags was wrong. I'll be committing a fix soon.

Due to this and a couple small other bugs/regressions, I think it will
make sense to make another release right away, to have a solid "works
out of the box without patching" one as the latest. Please let me know
if you spot other regressions so fixes can be included.

Rich

Re: open issues

[Alexander Monakov]

On Tue, 29 Aug 2017, Rich Felker wrote:
> On Sun, Aug 27, 2017 at 06:36:06AM +0200, Szabolcs Nagy wrote:
> > - update contributors in copyright file
> 
> Will do at release time. Thanks for the reminder though.

I believe this wasn't addressed in neither of the two releases since then.

Alexander

Re: open issues

[Rich Felker]

On Thu, Jan 04, 2018 at 03:01:56PM +0300, Alexander Monakov wrote:
> On Tue, 29 Aug 2017, Rich Felker wrote:
> > On Sun, Aug 27, 2017 at 06:36:06AM +0200, Szabolcs Nagy wrote:
> > > - update contributors in copyright file
> > 
> > Will do at release time. Thanks for the reminder though.
> 
> I believe this wasn't addressed in neither of the two releases since then.

Thanks for the reminder. I'll try to get it done soon.

Rich