wchar_t and -fshort-wchar

wchar_t and -fshort-wchar

[Sergey Dmitrouk]

Hi,

musl seems to build fine with -fshort-wchar, but when client applications
are built against musl all uses of wide character literals fail due to wide
type defined internally by a compiler differs from the type of `wchar_t` in
musl headers.

I faced this on ARM where `wchar_t` is defined as `unsigned int` by
musl but it's `unsigned short` from compilers point of view.  I'd expect
similar issues with other targets.

Would it make sense to use `__WCHAR_TYPE__` for `wchar_t` when it's
available (it's already used for i386, but for different reason)?
Presumably, as compiler is responsible for creating wide literals, libc
would better agree with it on the type.

Of course, this makes sense only if you intend to support builds with
`-fshort-wchar` flag, which are not very common I believe.

Best regards,
Sergey

Re: wchar_t and -fshort-wchar

[Szabolcs Nagy]

* Sergey Dmitrouk <sdmitrouk@accesssoftek.com> [2015-02-18 12:53:37 +0200]:
> musl seems to build fine with -fshort-wchar, but when client applications

that cannot possibly work

wchar_t is assumed to be a unicode code point so short wchar_t is broken

> are built against musl all uses of wide character literals fail due to wide
> type defined internally by a compiler differs from the type of `wchar_t` in
> musl headers.
> 
> I faced this on ARM where `wchar_t` is defined as `unsigned int` by
> musl but it's `unsigned short` from compilers point of view.  I'd expect
> similar issues with other targets.
> 

http://arminfo.emea.arm.com/help/topic/com.arm.doc.ihi0042e/IHI0042E_aapcs.pdf

says

"The preferred type of wchar_t is unsigned int. However,
a virtual platform may elect to use unsigned short instead.
A platform standard must document its choice"

on musl/glibc/.. the platform standard is unsigned int

> Would it make sense to use `__WCHAR_TYPE__` for `wchar_t` when it's
> available (it's already used for i386, but for different reason)?
> Presumably, as compiler is responsible for creating wide literals, libc
> would better agree with it on the type.
> 
> Of course, this makes sense only if you intend to support builds with
> `-fshort-wchar` flag, which are not very common I believe.
> 

this is an abi change so a different loader path name etc should be
used then (you should create a new subarch in musl's terminology)

but i'm not sure how you plan to fix up mb to wc functions for
such a subarch..

> Best regards,
> Sergey

Re: wchar_t and -fshort-wchar

[Sergey Dmitrouk]

OK, I see, thanks for the explanation.  I agree there are problems with
short wchar_t, but in my specific case they are mostly irrelevant (no need
for dynamic loader or conversions that involve code points with values
above 0xffff).  I mainly wanted to inform you about this build issue, in
case you would like to do something about it.

On Wed, Feb 18, 2015 at 03:54:13AM -0800, Szabolcs Nagy wrote:
> * Sergey Dmitrouk <sdmitrouk@accesssoftek.com> [2015-02-18 12:53:37 +0200]:
> > musl seems to build fine with -fshort-wchar, but when client applications
>
> that cannot possibly work
>
> wchar_t is assumed to be a unicode code point so short wchar_t is broken
>
> > are built against musl all uses of wide character literals fail due to wide
> > type defined internally by a compiler differs from the type of `wchar_t` in
> > musl headers.
> >
> > I faced this on ARM where `wchar_t` is defined as `unsigned int` by
> > musl but it's `unsigned short` from compilers point of view.  I'd expect
> > similar issues with other targets.
> >
>
> http://arminfo.emea.arm.com/help/topic/com.arm.doc.ihi0042e/IHI0042E_aapcs.pdf
>
> says
>
> "The preferred type of wchar_t is unsigned int. However,
> a virtual platform may elect to use unsigned short instead.
> A platform standard must document its choice"
>
> on musl/glibc/.. the platform standard is unsigned int
>
> > Would it make sense to use `__WCHAR_TYPE__` for `wchar_t` when it's
> > available (it's already used for i386, but for different reason)?
> > Presumably, as compiler is responsible for creating wide literals, libc
> > would better agree with it on the type.
> >
> > Of course, this makes sense only if you intend to support builds with
> > `-fshort-wchar` flag, which are not very common I believe.
> >
>
> this is an abi change so a different loader path name etc should be
> used then (you should create a new subarch in musl's terminology)
>
> but i'm not sure how you plan to fix up mb to wc functions for
> such a subarch..
>
> > Best regards,
> > Sergey

Re: wchar_t and -fshort-wchar

[Rich Felker]

On Wed, Feb 18, 2015 at 12:53:37PM +0200, Sergey Dmitrouk wrote:
> Hi,
> 
> musl seems to build fine with -fshort-wchar, but when client applications
> are built against musl all uses of wide character literals fail due to wide
> type defined internally by a compiler differs from the type of `wchar_t` in
> musl headers.
> 
> I faced this on ARM where `wchar_t` is defined as `unsigned int` by
> musl but it's `unsigned short` from compilers point of view.  I'd expect
> similar issues with other targets.
> 
> Would it make sense to use `__WCHAR_TYPE__` for `wchar_t` when it's
> available (it's already used for i386, but for different reason)?
> Presumably, as compiler is responsible for creating wide literals, libc
> would better agree with it on the type.
> 
> Of course, this makes sense only if you intend to support builds with
> `-fshort-wchar` flag, which are not very common I believe.

musl does not support configurations with under-sized definitions of
types like 16-bit wchar_t or 32-bit off_t. Only the sizes that can
represent the full range of values are supported. musl does however
have the C11 uchar.h functions which can operate on char16_t, so you
could use char16_t and the corresponding c16 conversion functions
instead of the wc versions if that helps. What are you trying to
achieve?

Rich

Re: wchar_t and -fshort-wchar

[Sergey Dmitrouk]

On Thu, Feb 19, 2015 at 08:26:02PM -0800, Rich Felker wrote:
> musl does not support configurations with under-sized definitions of
> types like 16-bit wchar_t or 32-bit off_t. Only the sizes that can
> represent the full range of values are supported. musl does however
> have the C11 uchar.h functions which can operate on char16_t, so you
> could use char16_t and the corresponding c16 conversion functions
> instead of the wc versions if that helps. What are you trying to
> achieve?

Nothing particular, -fshort-wchar is dictated by some compatibility
requirements, which are probably caused by building things on Windows,
where sizeof(wchar_t) usually equals to 2.  Maybe the flag will go away
later, if not, will try using uchar.h in case there will be a need, thanks
for the pointer.

-- 
Sergey

Re: wchar_t and -fshort-wchar

[Rich Felker]

On Fri, Feb 20, 2015 at 08:53:02AM +0200, Sergey Dmitrouk wrote:
> On Thu, Feb 19, 2015 at 08:26:02PM -0800, Rich Felker wrote:
> > musl does not support configurations with under-sized definitions of
> > types like 16-bit wchar_t or 32-bit off_t. Only the sizes that can
> > represent the full range of values are supported. musl does however
> > have the C11 uchar.h functions which can operate on char16_t, so you
> > could use char16_t and the corresponding c16 conversion functions
> > instead of the wc versions if that helps. What are you trying to
> > achieve?
> 
> Nothing particular, -fshort-wchar is dictated by some compatibility
> requirements, which are probably caused by building things on Windows,
> where sizeof(wchar_t) usually equals to 2.  Maybe the flag will go away
> later, if not, will try using uchar.h in case there will be a need, thanks
> for the pointer.

musl won't work with -fshort-wchar, so the flag needs to go away. If
you compile musl with that you'll just get a broken (and potentially
even dangerous; it's hard to say) libc build. If you compile the
application (but not libc) with that option, you'll get mismatching
ABI and things will break if you call any functions that operate on
wchar_t arrays, but if you're not actually using these functions you
might not even see any breakage. If there's no specific reason this
option is being used I think it just needs to be removed from the
build and everything should work fine.

Rich