mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Invalid pointer subtractions in __shlim and __shgetc
@ 2020-04-17 15:56 Pascal Cuoq
  2020-04-17 16:13 ` Rich Felker
  0 siblings, 1 reply; 8+ messages in thread
From: Pascal Cuoq @ 2020-04-17 15:56 UTC (permalink / raw)
  To: musl

[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]

?Hello,

both functions `__shlim` and `__shgetc` subtract the members
named `buf` and `rpos` of the struct they manipulate.

In `__shlim`, this happens in the statement `f->shcnt = f->buf - f->rpos;`.
And in `__shgetc`, in happens inside the `shcnt` macro:

#define shcnt(f) ((f)->shcnt + ((f)->rpos - (f)->buf))

In our tests, while running `testsuite` in `libc-testsuite`,
both the `__shlim` and `__shgetc` functions are reached
with `f->buf` non-null and `f->rpos` a null pointer.

This can be made visible on execution platforms other than ours
by adding a statement at the beginning of the functions:

+      if (f->buf && !f->rpos) dprintf (2, "XXX Problem in __shlim\n");
+      if (f->buf && !f->rpos) dprintf (2, "XXX Problem in __shgetc\n");

Then if, running `libc-testsuite`, you see the following, it means that
`f->buf` was non-null and `f->rpos` was null when these points were
reached:

$ ./testsuite
fdopen test passed
fcntl test passed
fnmatch test passed
XXX Problem in __shlim
XXX Problem in __shgetc
XXX Problem in __shlim
XXX Problem in __shgetc
XXX Problem in __shlim
XXX Problem in __shgetc
XXX Problem in __shlim
XXX Problem in __shgetc
XXX Problem in __shlim
XXX Problem in __shgetc
XXX Problem in __shlim
XXX Problem in __shgetc
fscanf test passed
(...)

This has been tested on the (tag: v1.2.0) branch of git://git.musl-libc.org/musl

These pointer subtractions are undefined behavior. This is slightly worse
than computing `(char*)0-(char*)0`, which is undefined in C and defined in C++,
because compilers for both C and C++ are unlikely to exploit this one
for optimization. Subtracting between a non-null pointer and a null pointer
on the other hand is undefined behavior in both languages, and it is
plausible that doing it may someday have unexpected consequences.

I mention this because similar undefined behaviors that were extremely
unlikely to cause harm have been fixed in musl in recent months,
so that this looks like something you may want to fix too.

Pascal




[-- Attachment #2: Type: text/html, Size: 3661 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2020-04-24 14:12 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-17 15:56 [musl] Invalid pointer subtractions in __shlim and __shgetc Pascal Cuoq
2020-04-17 16:13 ` Rich Felker
2020-04-17 16:48   ` Rich Felker
2020-04-17 17:56     ` Rich Felker
2020-04-23 11:34       ` Pascal Cuoq
2020-04-23 16:14         ` Rich Felker
2020-04-24  9:40           ` Pascal Cuoq
2020-04-24 14:12             ` Rich Felker

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).