dirname() / basename() - musl vs FreeBSD and OpenBSD

dirname() / basename() - musl vs FreeBSD and OpenBSD

[Daniel Cegiełka]

Hi,

I came across a very strange problem when I ports code from OpenBSD to
musl-libc, and it seems, that a lot of problems can be caused by
dirname().

http://pubs.opengroup.org/onlinepubs/009695399/functions/dirname.html

"The dirname() function >>> may <<< modify the string pointed to by
path, and may return a pointer to static storage that may then be
overwritten by subsequent calls to dirname()."


OpenBSD and FreeBSD:

http://man.openbsd.org/OpenBSD-5.8/dirname.3
https://www.freebsd.org/cgi/man.cgi?query=dirname&sektion=3

"dirname() returns a pointer to internal static storage space that
will be overwritten by subsequent calls (each function has its own
separate storage).

Other vendor implementations of dirname() may modify the contents of
the string passed to dirname(); this should be taken into account when
writing code which calls this function if portability is desired."

NetBSD:

http://netbsd.gw.com/cgi-bin/man-cgi?dirname+3+NetBSD-7.0

"BUGS
(...)
     The dirname() function returns a pointer to static storage that
may be overwritten by subse-
     quent calls to dirname(). This is not strictly a bug; it is
explicitly allowed by IEEE Std
     1003.1-2001 (``POSIX.1'')."


so:

#include <libgen.h> /* musl libc dirname() */
#include <stdio.h>

int main()
{
    char s1[] = "/usr/lib/";
    char s2[] = "/usr/lib/";
    char *p1, *p2;

    p1 = dirname(s1);
    p2 = openbsd_dirname(s2);

    printf("musl: s1: %s, p1: %s\n", s1, p1);
    printf("openbsd_dirname: s2: %s, p2: %s\n", s2, p2);
    return 0;
}

# ./a.out
musl: s1: /usr, p1: /usr
openbsd_dirname: s2: /usr/lib/, p2: /usr

So if you use the code from OpenBSD or FreeBSD, then you should be
very careful... grep, sed, patch, diff... etc. everything is
potentially error prone.

musl has very good support for code from *BSD, so is the ability that
dirname() in musl does not overwrite argument of the function? It will
not change anything in relation to the IEEE Std 1003.1-2001, but it
will be much safer for the code from FreeBSD and OpenBSD.


btw. the same problem applies to basename():

http://netbsd.gw.com/cgi-bin/man-cgi?basename+3+NetBSD-7.0

http://man.openbsd.org/OpenBSD-5.8/man3/basename.3

https://www.freebsd.org/cgi/man.cgi?query=basename&apropos=0&sektion=3&manpath=FreeBSD+10.3-RELEASE+and+Ports&arch=default&format=html


Daniel

Re: dirname() / basename() - musl vs FreeBSD and OpenBSD

[Rich Felker]

On Sun, Jul 24, 2016 at 09:12:49PM +0200, Daniel Cegiełka wrote:
> Hi,
> 
> I came across a very strange problem when I ports code from OpenBSD to
> musl-libc, and it seems, that a lot of problems can be caused by
> dirname().
> 
> http://pubs.opengroup.org/onlinepubs/009695399/functions/dirname.html
> 
> "The dirname() function >>> may <<< modify the string pointed to by
> path, and may return a pointer to static storage that may then be
> overwritten by subsequent calls to dirname()."

There is actually no other option for implementing this function. The
contract does not require that the argument string be a valid pathname
or have a bounded length like PATH_MAX; it operates on general
strings. And "No errors are defined", so failure is not an option. The
only way to implement this function is for it to modify its argument.

> OpenBSD and FreeBSD:
> 
> http://man.openbsd.org/OpenBSD-5.8/dirname.3
> https://www.freebsd.org/cgi/man.cgi?query=dirname&sektion=3
> 
> "dirname() returns a pointer to internal static storage space that
> will be overwritten by subsequent calls (each function has its own
> separate storage).
> 
> Other vendor implementations of dirname() may modify the contents of
> the string passed to dirname(); this should be taken into account when
> writing code which calls this function if portability is desired."
> 
> NetBSD:
> 
> http://netbsd.gw.com/cgi-bin/man-cgi?dirname+3+NetBSD-7.0
> 
> "BUGS
> (...)
>      The dirname() function returns a pointer to static storage that
> may be overwritten by subse-
>      quent calls to dirname(). This is not strictly a bug; it is
> explicitly allowed by IEEE Std
>      1003.1-2001 (``POSIX.1'')."

It is a bug because it necessarily returns wrong results for extremely
long strings.

> so:
> 
> #include <libgen.h> /* musl libc dirname() */
> #include <stdio.h>
> 
> int main()
> {
>     char s1[] = "/usr/lib/";
>     char s2[] = "/usr/lib/";
>     char *p1, *p2;
> 
>     p1 = dirname(s1);
>     p2 = openbsd_dirname(s2);
> 
>     printf("musl: s1: %s, p1: %s\n", s1, p1);
>     printf("openbsd_dirname: s2: %s, p2: %s\n", s2, p2);
>     return 0;
> }
> 
> # ./a.out
> musl: s1: /usr, p1: /usr
> openbsd_dirname: s2: /usr/lib/, p2: /usr
> 
> So if you use the code from OpenBSD or FreeBSD, then you should be
> very careful... grep, sed, patch, diff... etc. everything is
> potentially error prone.
> 
> musl has very good support for code from *BSD, so is the ability that
> dirname() in musl does not overwrite argument of the function? It will
> not change anything in relation to the IEEE Std 1003.1-2001, but it
> will be much safer for the code from FreeBSD and OpenBSD.

Not an option, for the above reason.

> btw. the same problem applies to basename():
> 
> http://netbsd.gw.com/cgi-bin/man-cgi?basename+3+NetBSD-7.0
> 
> http://man.openbsd.org/OpenBSD-5.8/man3/basename.3
> 
> https://www.freebsd.org/cgi/man.cgi?query=basename&apropos=0&sektion=3&manpath=FreeBSD+10.3-RELEASE+and+Ports&arch=default&format=html

Same applies there.

Rich

Re: dirname() / basename() - musl vs FreeBSD and OpenBSD

[Daniel Cegiełka]

2016-07-24 21:21 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Sun, Jul 24, 2016 at 09:12:49PM +0200, Daniel Cegiełka wrote:
>> Hi,
>>
>> I came across a very strange problem when I ports code from OpenBSD to
>> musl-libc, and it seems, that a lot of problems can be caused by
>> dirname().
>>
>> http://pubs.opengroup.org/onlinepubs/009695399/functions/dirname.html
>>
>> "The dirname() function >>> may <<< modify the string pointed to by
>> path, and may return a pointer to static storage that may then be
>> overwritten by subsequent calls to dirname()."
>
> There is actually no other option for implementing this function. The
> contract does not require that the argument string be a valid pathname
> or have a bounded length like PATH_MAX; it operates on general
> strings. And "No errors are defined", so failure is not an option. The
> only way to implement this function is for it to modify its argument.

Thank you for your answer. dirname() and basename() in OpenBSD compare
length against PATH_MAX:

http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/lib/libc/gen/dirname.c?rev=1.15

http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/lib/libc/gen/basename.c?rev=1.15

So if someone uses the code from OpenBSD/FreeBSD needs to be careful
not to use the argument after dirname() or basename().

Best regards,
Daniel


>
> Rich