[musl] vfprintf(..., "%lc", (wint_t)0) fails to output a \0

[musl] vfprintf(..., "%lc", (wint_t)0) fails to output a \0

[Peter Kasting]

[-- Attachment #1: Type: text/plain, Size: 2381 bytes --]

(I'm not subscribed here, please CC me on any responses.)

I believe* the following program will fail on musl:

#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <wchar.h>

int main() {
  char buf[16];
  memset(buf, 1, 16);
  int len = sprintf(buf, "%lc", (wint_t)0);
  assert(len > 0);
  assert(buf[0] == 0);
  return 0;
}

This should write a \0 to the buffer, but will write nothing.

*I don't have direct access to a musl environment to compile and test the
code above, so this is speculation. I reproduced this bug indirectly via a
Linux Alpine test environment here at Google while trying to make changes
to Abseil's string-handling implementation.

From code inspection, the problem occurs at
https://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n611. Here
the loop guard tests `*ws` to ensure it stops on null terminators. However,
when we fall through from handling this input case (of "%lc", 0), *ws ==
wc[0] == 0, so wctomb() is never called.

The C99 spec here arguably allows this behavior, but I believe its intent
is to specify the behavior glibc's implementation exhibits (where a 0 is
written). Section 7.19.6.1.8 says regarding %lc, "...the wint_t argument is
converted as if by an ls conversion specification with no precision and an
argument that points to the initial element of a two-element array of
wchar_t, the first element containing the wint_t argument to the lc
conversion specification and the second a null wide character." And
regarding %ls, "...the argument shall be a pointer to the initial element
of an array of wchar_t type. Wide characters from the array are converted
to multibyte characters...up to and including a terminating null wide
character. The resulting multibyte characters are written up to (but not
including) the terminating null character (byte)." One could argue that the
first (zero) element in the array, being a 0, is "a terminating null wide
character" that should be converted to a multibyte character, but
subsequently not written (since the conversion will result in solely a
terminating null character). But it seems like the intent of the spec was
to say that a %lc argument is always converted and written, with the second
array element always treated as "the null terminator". I don't know if
there is further clarifying language/discussion in some mailing list or
archives somewhere.

PK

[-- Attachment #2: Type: text/html, Size: 2891 bytes --]

Re: [musl] vfprintf(..., "%lc", (wint_t)0) fails to output a \0

[Rich Felker]

On Thu, Oct 26, 2023 at 05:56:09PM -0700, Peter Kasting wrote:
> (I'm not subscribed here, please CC me on any responses.)
> 
> I believe* the following program will fail on musl:
> 
> #include <assert.h>
> #include <stdio.h>
> #include <string.h>
> #include <wchar.h>
> 
> int main() {
>   char buf[16];
>   memset(buf, 1, 16);
>   int len = sprintf(buf, "%lc", (wint_t)0);
>   assert(len > 0);
>   assert(buf[0] == 0);
>   return 0;
> }
> 
> This should write a \0 to the buffer, but will write nothing.
> 
> *I don't have direct access to a musl environment to compile and test the
> code above, so this is speculation. I reproduced this bug indirectly via a
> Linux Alpine test environment here at Google while trying to make changes
> to Abseil's string-handling implementation.
> 
> >From code inspection, the problem occurs at
> https://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n611. Here
> the loop guard tests `*ws` to ensure it stops on null terminators. However,
> when we fall through from handling this input case (of "%lc", 0), *ws ==
> wc[0] == 0, so wctomb() is never called.
> 
> The C99 spec here arguably allows this behavior, but I believe its intent
> is to specify the behavior glibc's implementation exhibits (where a 0 is
> written). Section 7.19.6.1.8 says regarding %lc, "...the wint_t argument is
> converted as if by an ls conversion specification with no precision and an
> argument that points to the initial element of a two-element array of
> wchar_t, the first element containing the wint_t argument to the lc
> conversion specification and the second a null wide character." And
> regarding %ls, "...the argument shall be a pointer to the initial element
> of an array of wchar_t type. Wide characters from the array are converted
> to multibyte characters...up to and including a terminating null wide
> character. The resulting multibyte characters are written up to (but not
> including) the terminating null character (byte)." One could argue that the
> first (zero) element in the array, being a 0, is "a terminating null wide
> character" that should be converted to a multibyte character, but
> subsequently not written (since the conversion will result in solely a
> terminating null character). But it seems like the intent of the spec was
> to say that a %lc argument is always converted and written, with the second
> array element always treated as "the null terminator". I don't know if
> there is further clarifying language/discussion in some mailing list or
> archives somewhere.

As I understand it, this was raised with the committee and since musl
was found to be the only conforming implementation, they've opted to
change the spec to what you expect was the intent. So we'll be
changing this at some point to follow.

Rich

Re: [musl] vfprintf(..., "%lc", (wint_t)0) fails to output a \0

[Peter Kasting]

[-- Attachment #1: Type: text/plain, Size: 3082 bytes --]

Sounds good. Thanks!

PK

On Thu, Oct 26, 2023, 6:38 PM Rich Felker <dalias@libc.org> wrote:

> On Thu, Oct 26, 2023 at 05:56:09PM -0700, Peter Kasting wrote:
> > (I'm not subscribed here, please CC me on any responses.)
> >
> > I believe* the following program will fail on musl:
> >
> > #include <assert.h>
> > #include <stdio.h>
> > #include <string.h>
> > #include <wchar.h>
> >
> > int main() {
> >   char buf[16];
> >   memset(buf, 1, 16);
> >   int len = sprintf(buf, "%lc", (wint_t)0);
> >   assert(len > 0);
> >   assert(buf[0] == 0);
> >   return 0;
> > }
> >
> > This should write a \0 to the buffer, but will write nothing.
> >
> > *I don't have direct access to a musl environment to compile and test the
> > code above, so this is speculation. I reproduced this bug indirectly via
> a
> > Linux Alpine test environment here at Google while trying to make changes
> > to Abseil's string-handling implementation.
> >
> > >From code inspection, the problem occurs at
> > https://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n611. Here
> > the loop guard tests `*ws` to ensure it stops on null terminators.
> However,
> > when we fall through from handling this input case (of "%lc", 0), *ws ==
> > wc[0] == 0, so wctomb() is never called.
> >
> > The C99 spec here arguably allows this behavior, but I believe its intent
> > is to specify the behavior glibc's implementation exhibits (where a 0 is
> > written). Section 7.19.6.1.8 says regarding %lc, "...the wint_t argument
> is
> > converted as if by an ls conversion specification with no precision and
> an
> > argument that points to the initial element of a two-element array of
> > wchar_t, the first element containing the wint_t argument to the lc
> > conversion specification and the second a null wide character." And
> > regarding %ls, "...the argument shall be a pointer to the initial element
> > of an array of wchar_t type. Wide characters from the array are converted
> > to multibyte characters...up to and including a terminating null wide
> > character. The resulting multibyte characters are written up to (but not
> > including) the terminating null character (byte)." One could argue that
> the
> > first (zero) element in the array, being a 0, is "a terminating null wide
> > character" that should be converted to a multibyte character, but
> > subsequently not written (since the conversion will result in solely a
> > terminating null character). But it seems like the intent of the spec was
> > to say that a %lc argument is always converted and written, with the
> second
> > array element always treated as "the null terminator". I don't know if
> > there is further clarifying language/discussion in some mailing list or
> > archives somewhere.
>
> As I understand it, this was raised with the committee and since musl
> was found to be the only conforming implementation, they've opted to
> change the spec to what you expect was the intent. So we'll be
> changing this at some point to follow.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 3893 bytes --]