Re: [PATCH] Update ctype data to Unicode 12.1.0

[Eleftherios Kritikos <el01049@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2805 bytes --]

Thanks for all this good work!

There is no time pressure for me btw as I am not depending on this patch. I
am using musl for static binaries though, and that's great!


On Fri, 25 Oct 2019, 5:15 pm Rich Felker, <dalias@libc.org> wrote:

> On Wed, Oct 23, 2019 at 07:21:35PM +0300, Eleftherios Kritikos wrote:
> > Hi all,
> >
> > I wanted to mention that I have used the code for `wcwidth`[1] and for
> > generating Unicode data tables[2] from musl in the Haskell library
> > vty[3] (a ncurses style library).
> >
> > Relevant files in the MR:
> >  *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-ab3908e00d1c13397ed03e5c2213ad8bR5
> >  *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-a06fd5aeeca6d7dac0278c2537eb1950R1
> >  *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-86acb7ffecd1a09c5f55892bd0ce13b1R1
> >  *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-dc77683ad25ad6f509fb58a397c93f4aR1
> >  *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-9879d6db96fd29134fc802214163b95aR32
> >
> > Thanks Rich Felker and everyone else for all the good work that has
> > gone into musl!
> >
> > Please let me know if you think attribution was not properly given.
> >
> > 1.
> http://git.musl-libc.org/cgit/musl/tree/src/ctype/wcwidth.c?id=9b2921bea1d5017832e1b45d1fd64220047a9802
> > 2.https://github.com/richfelker/musl-chartable-tools/tree/master/ctype
> > 3. https://github.com/jtdaugherty/vty
>
> Great! I love seeing code/concepts from musl getting adopted elsewhere
> especially in places where the classic solutions were all much larger.
>
> Just a quick update on why I haven't merged this yet: I went to do the
> case mappings too, and found that at least one range, I believe the
> one that would be CASEMAP(0x1c90,0x1cba,0x10d0), is not representable
> in the current code that requires updating by hand (it could be done
> on a char-by-char basis but continuing to expand that part makes the
> file grow larger and slower very quickly).
>
> So, I'm pulling back up the proposed replacement code from April 2018
> that never got finished and merged. The old thread is here:
> https://www.openwall.com/lists/musl/2018/04/05/1
>
> It's moderately larger -- ~4.8k instead of ~1.5k for Unicode 10 -- but
> O(1) rather than O(n) (n = # of case mappings), about 10x faster, and
> programmatically generated from UnicodeData.txt. I'll add the (awful,
> ugly, just like everything else in musl-chartable-tools) code for
> generating the table to musl-chartable-tools when I merge it so it's
> not a black box.
>
> I have it working now, so as long as I don't hit any unexpected
> problems testing I'll get this (and your patch, and updating case
> mappings to Unicode 12) merged soon.
>
> Thanks again for sending the patch and pinging this.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 4602 bytes --]