RE: There is no tests for musl,

RE: There is no tests for musl,

[writeonce]

On 05/08/2015 11:41 AM, 罗勇刚(Yonggang Luo)  wrote:>>>> solution, I
think there is no one would use
>>>> wchar_t for cross text processing, cause, on some system, wchar_t is
>>>> just 8bit  width!
>>>
>>> anybody would use wchar_t who cares about standard conformant
>>> implementations.
>>>
>>> non-standard broken platforms may get an unmaintained #ifdef
>>> as usual..
>>
>> I think we (and midipix) have a different perspective from Yonggang
>> Luo on portable development. Our view is that you write to a POSIX (or
>> nearly-POSIX) target with fully working Unicode support and fix the
>> small number of targets (i.e. just Windows) that don't already provide
> Small is relative, if counting the distribution count, well, Unix wins.
>> these things. Yonggang Luo's perspective seems to be more of a
>> traditional Windows approach with #ifdef and lots of OS-specific code,
>> but just making the Windows branch of the #ifdefs less hideous than it
>> was before. :)
> If getting wchar_t to be 32 bit on win32, then truly will be a lot of
> #ifdef, I am not so sure
> if you have experience on Win32 API development, I hope we discussing
> the problems in a
>   more objective way.
> 

One primary objective of code portability and posix-compatibility layer
for win32 is to _remove_ the need for OS-specific code-paths. A wchar_t
that is anything short (no pun intended) of a 32-bit integer will render
it impossible to build out of the box many pieces of commonly-used
software, including, but not limited to musl libc, the curses library,
and anything that expects wchar_t to cover the entire unicode range.

As for your suggested framework: there are currently at least three
compilers that can produce optimized code for the target platform (gcc,
clang, and cparser), and which work very well with most open-source
software out there. As an aside, if you are interested in an 8-byte long
on 64-bit windows then an open-source compiler is probably your only
option. To compile musl with msvc, on the other hand, you'd have to make
so many changes to the source code that you might as well write your own
libc from scratch. To see why, please attempt to compile some ten or
fifteen core libc headers (stdio.h, unistd.h, etc.) with msvc. If that
goes well (spoiler: it won't), then the next step would be to compile a
subset of the source files (src/pthread or src/stdio, for instance) and
remove any remaining obstacles.

m.


>>
>> Rich
> 
> 
> 

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

I think maybe the best way is add char16_t and char32_t variant for
All string related APIs
But that not standard yet.

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

>> > solution, I think there is no one would use
>> > wchar_t for cross text processing, cause, on some system, wchar_t is
>> > just 8bit  width!
>>
>> anybody would use wchar_t who cares about standard conformant
>> implementations.
>>
>> non-standard broken platforms may get an unmaintained #ifdef
>> as usual..
>
> I think we (and midipix) have a different perspective from Yonggang
> Luo on portable development. Our view is that you write to a POSIX (or
> nearly-POSIX) target with fully working Unicode support and fix the
> small number of targets (i.e. just Windows) that don't already provide
Small is relative, if counting the distribution count, well, Unix wins.
> these things. Yonggang Luo's perspective seems to be more of a
> traditional Windows approach with #ifdef and lots of OS-specific code,
> but just making the Windows branch of the #ifdefs less hideous than it
> was before. :)
If getting wchar_t to be 32 bit on win32, then truly will be a lot of
#ifdef, I am not so sure
if you have experience on Win32 API development, I hope we discussing
the problems in a
 more objective way.

>
> Rich



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Rich Felker]

On Fri, May 08, 2015 at 05:20:45PM +0200, Szabolcs Nagy wrote:
> * ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 22:17:58 +0800]:
> > solution, I think there is no one would use
> > wchar_t for cross text processing, cause, on some system, wchar_t is
> > just 8bit  width!
> 
> anybody would use wchar_t who cares about standard conformant
> implementations.
> 
> non-standard broken platforms may get an unmaintained #ifdef
> as usual..

I think we (and midipix) have a different perspective from Yonggang
Luo on portable development. Our view is that you write to a POSIX (or
nearly-POSIX) target with fully working Unicode support and fix the
small number of targets (i.e. just Windows) that don't already provide
these things. Yonggang Luo's perspective seems to be more of a
traditional Windows approach with #ifdef and lots of OS-specific code,
but just making the Windows branch of the #ifdefs less hideous than it
was before. :)

Rich

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

Yeap, using wchar_t always intent for platform-compatible, not used
for cross-platform developing.

that's the relationship between int32_t and int.
int32_t is cross-platform and int is not.

2015-05-08 23:20 GMT+08:00 Szabolcs Nagy <nsz@port70.net>:
> * ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 22:17:58 +0800]:
>> solution, I think there is no one would use
>> wchar_t for cross text processing, cause, on some system, wchar_t is
>> just 8bit  width!
>
> anybody would use wchar_t who cares about standard conformant
> implementations.
>
> non-standard broken platforms may get an unmaintained #ifdef
> as usual..



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Szabolcs Nagy]

* ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 22:17:58 +0800]:
> solution, I think there is no one would use
> wchar_t for cross text processing, cause, on some system, wchar_t is
> just 8bit  width!

anybody would use wchar_t who cares about standard conformant
implementations.

non-standard broken platforms may get an unmaintained #ifdef
as usual..

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

2015-05-08 22:10 GMT+08:00 Rich Felker <dalias@libc.org>:
> On Fri, May 08, 2015 at 10:02:53PM +0800, 罗勇刚(Yonggang Luo)  wrote:
>> Thanks for the detail explain, I know all the shortcut of 16bit
>> wchar_t, considerate that 4bit wchar_t is rarely used in unix world,
>> but 16bit wchar_t is frequently in
>> Windows/Qt/Java/Javascript, so I think it's better not change the
>> 16bit wchar_t to 32 bit, and that's would confusing those people
>> already use 16bit wchar_t on Windows platform, and this would affect
>> Unix- world, by default, on win32, wchar_t is 16 bit, this is a de
>> facto.
>
> That's a choice you can make in the system you're developing, but you
> should be aware that it makes it impossible to support full Unicode
> with the standard APIs and thus requires apps to either limit
> themselves to supporting only the BMP or using nonstandard APIs.
>
>> >From this point of view,  getting wchar_t to be 32bit on win32 is
>> useless and cause more problems.
>> The main point to port musl on win32 is add posix support and utf8
>> support on win32.
>> The wchar_t is useless for those people need cross-platform text
>> processing, and should using char32_t instead. That's a design
>> principle.
>
> Again, char32_t can't support full Unicode if wchar_t is only 16-bit.
> It's stuck supporting just the BMP and the upper 16 bits are always
> zeros. Any attempt to make char32_t support the full range would
> produce inconsistent and nonconforming results. :(
I can predict the shortcoming of this, and there is a lot of ways to
resolve this, anyway,
for win32, preserve the original wchar_t semantics is the best
solution, I think there is no one would use
wchar_t for cross text processing, cause, on some system, wchar_t is
just 8bit  width!
for some os platform:(

refere to http://en.wikipedia.org/wiki/Wide_character#Programming_specifics
We should not use wchar_t for cross-platform text processing.

Anyway, deal with ANSI things on win32 is the maddest things on the
world, cause wchar_t at lease is Unicode after all:)
either utf8,utf16 or utf32.

>
> Rich



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Rich Felker]

On Fri, May 08, 2015 at 10:02:53PM +0800, 罗勇刚(Yonggang Luo)  wrote:
> Thanks for the detail explain, I know all the shortcut of 16bit
> wchar_t, considerate that 4bit wchar_t is rarely used in unix world,
> but 16bit wchar_t is frequently in
> Windows/Qt/Java/Javascript, so I think it's better not change the
> 16bit wchar_t to 32 bit, and that's would confusing those people
> already use 16bit wchar_t on Windows platform, and this would affect
> Unix- world, by default, on win32, wchar_t is 16 bit, this is a de
> facto.

That's a choice you can make in the system you're developing, but you
should be aware that it makes it impossible to support full Unicode
with the standard APIs and thus requires apps to either limit
themselves to supporting only the BMP or using nonstandard APIs.

> >From this point of view,  getting wchar_t to be 32bit on win32 is
> useless and cause more problems.
> The main point to port musl on win32 is add posix support and utf8
> support on win32.
> The wchar_t is useless for those people need cross-platform text
> processing, and should using char32_t instead. That's a design
> principle.

Again, char32_t can't support full Unicode if wchar_t is only 16-bit.
It's stuck supporting just the BMP and the upper 16 bits are always
zeros. Any attempt to make char32_t support the full range would
produce inconsistent and nonconforming results. :(

Rich

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

Thanks for the detail explain, I know all the shortcut of 16bit
wchar_t, considerate that 4bit wchar_t is rarely used in unix world,
but 16bit wchar_t is frequently in
Windows/Qt/Java/Javascript, so I think it's better not change the
16bit wchar_t to 32 bit, and that's would confusing those people
already use 16bit wchar_t on Windows platform, and this would affect
Unix- world, by default, on win32, wchar_t is 16 bit, this is a de
facto.

From this point of view,  getting wchar_t to be 32bit on win32 is
useless and cause more problems.
The main point to port musl on win32 is add posix support and utf8
support on win32.
The wchar_t is useless for those people need cross-platform text
processing, and should using char32_t instead. That's a design
principle.



2015-05-08 21:35 GMT+08:00 Rich Felker <dalias@libc.org>:
> On Fri, May 08, 2015 at 09:10:14PM +0800, 罗勇刚(Yonggang Luo)  wrote:
>> I've seen all the codes of midipix, it's seems not active developing now.
>
> Perhaps you mean not finished/available rather than not active. It's
> very active.
>
>> I was trying to the other way, that getting musl can be compiled with msvc.
>
> That's unlikely to be possible since MSVC is not a C compiler; it's a
> C++ compiler that supports a variant of C that's basically just a
> subset of C++. Since the license is free you're welcome to do your own
> work taking code from musl and making it work on MSVC, but patches for
> MSVC compatibility won't be acceptable upstream.
>
>> For easily debugging on win32 and preserve the wchar_t  on Win32.
>>
>> I know the advantage of wchar_t to be 32bit, but I think this would be
>> double-edged.
>> Cause 16 bit wchar_t doesn't not good for Linux, but also 32bit
>> wchar_t doesn't good for
>> Win32.
>> We should leave the all of Unicode to the work of char32_t. That's my design.
>
> If wchar_t is 16-bit then you _can't_ support all of Unicode, at all.
> The C language is such that all defined characters must have
> representations in wchar_t, and "multi-wchar_t-characters" are
> fundamentally not possible due to the API (and this is stated
> explicitly, too). MS ignores this by deprecating the standard C
> language functions and providing their own WinAPI functions you have
> to use instead.
>
> Rich



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Rich Felker]

On Fri, May 08, 2015 at 09:10:14PM +0800, 罗勇刚(Yonggang Luo)  wrote:
> I've seen all the codes of midipix, it's seems not active developing now.

Perhaps you mean not finished/available rather than not active. It's
very active.

> I was trying to the other way, that getting musl can be compiled with msvc.

That's unlikely to be possible since MSVC is not a C compiler; it's a
C++ compiler that supports a variant of C that's basically just a
subset of C++. Since the license is free you're welcome to do your own
work taking code from musl and making it work on MSVC, but patches for
MSVC compatibility won't be acceptable upstream.

> For easily debugging on win32 and preserve the wchar_t  on Win32.
> 
> I know the advantage of wchar_t to be 32bit, but I think this would be
> double-edged.
> Cause 16 bit wchar_t doesn't not good for Linux, but also 32bit
> wchar_t doesn't good for
> Win32.
> We should leave the all of Unicode to the work of char32_t. That's my design.

If wchar_t is 16-bit then you _can't_ support all of Unicode, at all.
The C language is such that all defined characters must have
representations in wchar_t, and "multi-wchar_t-characters" are
fundamentally not possible due to the API (and this is stated
explicitly, too). MS ignores this by deprecating the standard C
language functions and providing their own WinAPI functions you have
to use instead.

Rich

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

I've seen all the codes of midipix, it's seems not active developing now.
I was trying to the other way, that getting musl can be compiled with msvc.
For easily debugging on win32 and preserve the wchar_t  on Win32.

I know the advantage of wchar_t to be 32bit, but I think this would be
double-edged.
Cause 16 bit wchar_t doesn't not good for Linux, but also 32bit
wchar_t doesn't good for
Win32.
We should leave the all of Unicode to the work of char32_t. That's my design.


2015-05-08 20:49 GMT+08:00 Rich Felker <dalias@libc.org>:
> On Fri, May 08, 2015 at 04:52:28PM +0800, 罗勇刚(Yonggang Luo)  wrote:
>> That's looks good, except the wchar_t part.
>> I am not so sure that's was a good idea, cause all Windows API wchar_t
>> are UTF16...
>
> It's necessary to support all of Unicode and not just the BMP.
>
> On midipix you still have all the raw Windows API functions that need
> UTF-16 "WCHAR" arrays as input, so you can convert manually to WCHAR
> (which happens to also match the standard C type char16_t) and call
> them. But there's also a wrapper layer that lets you pass UTF-8
> strings to WinAPI wrappers, so you never have to deal with wchar_t or
> WCHAR at all. I don't know all the details but you could ask on
> #midipix.
>
> Rich



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Rich Felker]

On Fri, May 08, 2015 at 04:52:28PM +0800, 罗勇刚(Yonggang Luo)  wrote:
> That's looks good, except the wchar_t part.
> I am not so sure that's was a good idea, cause all Windows API wchar_t
> are UTF16...

It's necessary to support all of Unicode and not just the BMP.

On midipix you still have all the raw Windows API functions that need
UTF-16 "WCHAR" arrays as input, so you can convert manually to WCHAR
(which happens to also match the standard C type char16_t) and call
them. But there's also a wrapper layer that lets you pass UTF-8
strings to WinAPI wrappers, so you never have to deal with wchar_t or
WCHAR at all. I don't know all the details but you could ask on
#midipix.

Rich

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

That's looks good, except the wchar_t part.
I am not so sure that's was a good idea, cause all Windows API wchar_t
are UTF16...

2015-05-08 16:36 GMT+08:00 Szabolcs Nagy <nsz@port70.net>:
> * ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 16:03:20 +0800]:
>> Understood, Indeed, I wanna to port musl to win32, doesn't know if
>> that's was a good idea.
>>
>
> i think you should try to contact the midipix project
>
> http://www.midipix.org/
>
> it tries to provide posix environment on windows using musl
>
> there is a #midipix channel on the freenode irc network.



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Szabolcs Nagy]

* ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 16:03:20 +0800]:
> Understood, Indeed, I wanna to port musl to win32, doesn't know if
> that's was a good idea.
> 

i think you should try to contact the midipix project

http://www.midipix.org/

it tries to provide posix environment on windows using musl

there is a #midipix channel on the freenode irc network.

Re: There is no tests for musl,

[罗勇刚(Yonggang Luo)]

Understood, Indeed, I wanna to port musl to win32, doesn't know if
that's was a good idea.

2015-05-08 16:01 GMT+08:00 Szabolcs Nagy <nsz@port70.net>:
> * ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 15:38:33 +0800]:
>> That's really not so good for the stable.
>>
>
> it is a good thing
>
> a libc should provide portable semantics so implementation
> independent tests should be used not "tests for musl"
>
> http://wiki.musl-libc.org/wiki/Libc-Test



-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

Re: There is no tests for musl,

[Szabolcs Nagy]

* ?????????(Yonggang Luo)  <luoyonggang@gmail.com> [2015-05-08 15:38:33 +0800]:
> That's really not so good for the stable.
> 

it is a good thing

a libc should provide portable semantics so implementation
independent tests should be used not "tests for musl"

http://wiki.musl-libc.org/wiki/Libc-Test

There is no tests for musl,

[罗勇刚(Yonggang Luo)]

That's really not so good for the stable.

-- 
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo