* [PATCH] Update ctype data to Unicode 12.1.0
@ 2019-10-12 21:27 u_quark
2019-10-12 22:39 ` Rich Felker
0 siblings, 1 reply; 12+ messages in thread
From: u_quark @ 2019-10-12 21:27 UTC (permalink / raw)
To: musl; +Cc: u_quark
---
src/ctype/alpha.h | 159 +++++++++++++++++++++-------------------
src/ctype/nonspacing.h | 88 ++++++++++++-----------
src/ctype/punct.h | 160 ++++++++++++++++++++++-------------------
src/ctype/wide.h | 26 +++----
4 files changed, 232 insertions(+), 201 deletions(-)
diff --git a/src/ctype/alpha.h b/src/ctype/alpha.h
index 299277c7..4167f387 100644
--- a/src/ctype/alpha.h
+++ b/src/ctype/alpha.h
@@ -8,17 +8,17 @@
17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
17,17,17,17,17,17,17,63,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,64,65,17,66,67,
-68,69,70,71,72,73,74,17,75,76,77,78,79,80,16,16,16,81,82,83,84,85,86,87,88,89,
-16,90,16,91,92,16,16,17,17,17,93,94,95,16,16,16,16,16,16,16,16,16,16,17,17,17,
-17,96,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,17,97,16,16,16,16,16,16,
+68,69,70,71,72,73,74,17,75,76,77,78,79,80,81,16,82,83,84,85,86,87,88,89,90,91,
+92,93,16,94,95,96,16,17,17,17,97,98,99,16,16,16,16,16,16,16,16,16,16,17,17,17,
+17,100,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,17,101,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,17,17,98,99,16,16,16,100,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
-17,17,17,17,17,17,17,101,17,17,102,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,103,
-104,16,16,16,16,16,16,16,16,16,105,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,16,16,106,107,108,109,16,16,16,16,16,16,16,16,110,16,16,
-16,16,16,16,16,111,112,16,16,16,16,113,16,16,114,16,16,16,16,16,16,16,16,16,
-16,16,16,16,
+16,16,17,17,102,103,16,16,104,105,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
+17,17,17,17,17,17,17,17,17,106,17,17,107,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,
+108,109,16,16,16,16,16,16,16,16,16,110,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,16,16,16,16,16,16,16,111,112,113,114,16,16,16,16,16,16,16,16,115,116,
+117,16,16,16,16,16,118,119,16,16,16,16,120,16,16,121,16,16,16,16,16,16,16,16,
+16,16,16,16,16,
16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,0,0,0,0,0,0,0,0,254,255,255,7,254,
@@ -27,8 +27,8 @@
255,195,255,3,0,31,80,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,0,223,188,64,215,255,255,
251,255,255,255,255,255,255,255,255,255,191,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,3,252,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,254,255,255,255,127,2,254,255,255,
-255,255,0,0,0,0,0,255,191,182,0,255,255,255,7,7,0,0,0,255,7,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,254,255,255,255,127,2,255,255,255,
+255,255,1,0,0,0,0,255,191,182,0,255,255,255,135,7,0,0,0,255,7,255,255,255,255,
255,255,255,254,255,195,255,255,255,255,255,255,255,255,255,255,255,255,239,
31,254,225,255,
159,0,0,255,255,255,255,255,255,0,224,255,255,255,255,255,255,255,255,255,255,
@@ -42,54 +42,55 @@
255,0,0,239,223,253,255,255,253,239,227,223,29,96,64,207,255,6,0,239,223,253,
255,255,255,255,231,223,93,240,128,207,255,0,252,236,255,127,252,255,255,251,
47,127,128,95,255,192,255,12,0,254,255,255,255,255,127,255,7,63,32,255,3,0,0,
-0,0,150,37,240,254,174,236,255,59,95,32,255,243,0,0,0,
+0,0,214,247,255,255,175,255,255,59,95,32,255,243,0,0,0,
0,1,0,0,0,255,3,0,0,255,254,255,255,255,31,254,255,3,255,255,254,255,255,255,
-31,0,0,0,0,0,0,0,0,255,255,255,255,255,255,127,249,255,3,255,255,231,193,255,
-255,127,64,255,51,255,255,255,255,191,32,255,255,255,255,255,247,255,255,255,
+31,0,0,0,0,0,0,0,0,255,255,255,255,255,255,127,249,255,3,255,255,255,255,255,
+255,255,255,255,63,255,255,255,255,191,32,255,255,255,255,255,247,255,255,255,
255,255,255,255,255,255,61,127,61,255,255,255,255,255,61,255,255,255,255,61,
127,61,255,127,255,255,255,255,255,255,255,61,255,255,255,255,255,255,255,255,
-135,0,0,0,0,255,255,0,0,255,255,255,255,255,255,255,255,255,255,63,63,254,255,
+7,0,0,0,0,255,255,0,0,255,255,255,255,255,255,255,255,255,255,63,63,254,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,159,255,255,254,255,255,7,255,255,255,255,255,255,255,255,
255,199,255,1,255,223,15,0,255,255,15,0,255,255,15,0,255,223,13,0,255,255,255,
255,255,255,207,255,255,1,128,16,255,3,0,0,0,0,255,3,255,255,255,255,255,255,
-255,255,255,255,255,0,255,255,255,255,255,7,255,255,255,255,255,255,255,255,
+255,255,255,255,255,1,255,255,255,255,255,7,255,255,255,255,255,255,255,255,
63,
0,255,255,255,127,255,15,255,1,192,255,255,255,255,63,31,0,255,255,255,255,
255,15,255,255,255,3,255,3,0,0,0,0,255,255,255,15,255,255,255,255,255,255,255,
127,254,255,31,0,255,3,255,3,128,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,
255,239,255,239,15,255,3,0,0,0,0,255,255,255,255,255,243,255,255,255,255,255,
-255,191,255,3,0,255,255,255,255,255,255,63,0,255,227,255,255,255,255,255,63,
-255,1,0,0,0,0,0,0,0,0,0,0,0,222,111,0,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,0,0,0,128,255,31,0,
-255,255,63,63,255,255,255,255,63,63,255,170,255,255,255,63,255,255,255,255,
-255,255,223,95,220,31,207,15,255,31,220,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,128,
-0,0,255,31,0,0,0,0,0,0,0,0,0,0,0,0,132,252,47,62,80,189,255,243,224,67,0,0,
-255,255,255,255,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,
+255,191,255,3,0,255,255,255,255,255,255,127,0,255,227,255,255,255,255,255,63,
+255,1,255,255,255,255,255,231,0,0,0,0,0,222,111,4,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,0,0,0,
+128,255,31,0,255,255,63,63,255,255,255,255,63,63,255,170,255,255,255,63,255,
+255,255,255,255,255,223,95,220,31,207,15,255,31,220,31,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,2,128,0,0,255,31,0,0,0,0,0,0,0,0,0,0,0,0,132,252,47,62,80,189,255,243,
+224,67,0,0,255,255,255,255,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,192,255,255,255,255,255,255,3,0,
0,255,255,255,255,255,127,255,255,255,255,255,127,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,31,120,12,0,255,255,255,255,191,32,255,
255,255,255,255,255,255,128,0,0,255,255,127,0,127,127,127,127,127,127,127,127,
255,255,255,255,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,224,0,0,0,254,3,62,31,254,255,255,255,255,255,255,255,255,255,127,224,254,
-255,255,255,255,255,255,255,255,255,255,247,224,255,255,255,255,127,254,255,
+255,255,255,255,255,255,255,255,255,255,247,224,255,255,255,255,255,254,255,
255,255,255,255,255,255,255,255,255,127,0,0,255,255,255,7,0,0,0,0,0,0,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,63,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,7,0,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,
0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,0,0,
0,0,0,0,0,0,255,255,255,255,255,63,255,31,255,255,255,15,0,0,255,255,255,255,
255,127,240,143,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,0,0,
0,128,255,252,255,255,255,255,255,255,255,255,255,255,255,255,249,255,255,255,
-127,255,0,0,0,0,0,0,0,128,255,187,247,255,255,255,0,0,0,255,255,255,255,255,
-255,15,0,255,255,255,255,255,255,255,255,47,0,255,3,0,0,252,40,255,255,255,
-255,255,7,255,255,255,255,7,0,255,255,255,31,255,255,255,255,255,255,247,255,
-0,128,255,3,223,255,255,127,255,255,255,255,255,255,127,0,255,63,255,3,255,
-255,127,196,255,255,255,255,255,255,255,127,5,0,0,56,255,255,60,0,126,126,126,
-0,127,127,255,255,255,255,255,247,63,0,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,7,255,3,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,15,0,255,255,127,248,255,255,255,255,255,
+255,255,255,124,0,0,0,0,0,128,255,191,255,255,255,255,0,0,0,255,255,255,255,
+255,255,15,0,255,255,255,255,255,255,255,255,47,0,255,3,0,0,252,232,255,255,
+255,255,255,7,255,255,255,255,7,0,255,255,255,31,255,255,255,255,255,255,247,
+255,0,128,255,3,255,255,255,127,255,255,255,255,255,255,127,0,255,63,255,3,
+255,255,127,252,255,255,255,255,255,255,255,127,5,0,0,56,255,255,60,0,126,126,
+126,0,127,127,255,255,255,255,255,247,255,0,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,7,255,3,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,15,0,255,255,127,248,255,255,255,255,
+255,
15,255,255,255,255,255,255,255,255,255,255,255,255,255,63,255,255,255,255,255,
255,255,255,255,255,255,255,255,3,0,0,0,0,127,0,248,224,255,253,127,95,219,
255,255,255,255,255,255,255,255,255,255,255,255,255,3,0,0,0,248,255,255,255,
@@ -109,55 +110,63 @@
0,0,0,0,0,0,0,0,0,0,0,0,63,253,255,255,255,255,191,145,255,255,63,0,255,255,
127,0,255,255,255,127,0,0,0,0,0,0,0,0,255,255,55,0,255,255,63,0,255,255,255,3,
0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,192,0,0,0,0,0,0,0,0,111,240,239,
-254,255,255,15,0,0,0,0,0,255,255,255,31,255,255,255,31,0,0,0,0,255,254,255,
+254,255,255,63,0,0,0,0,0,255,255,255,31,255,255,255,31,0,0,0,0,255,254,255,
255,31,0,0,0,255,255,255,255,255,255,63,0,255,255,63,0,255,255,7,0,255,255,3,
0,0,0,0,0,0,0,0,0,0,0,0,
0,255,255,255,255,255,255,255,255,255,1,0,0,0,0,0,0,255,255,255,255,255,255,7,
-0,255,255,255,255,255,255,7,0,255,255,255,255,255,255,255,255,63,0,0,0,192,
-255,0,0,252,255,255,255,255,255,255,1,0,0,255,255,255,1,255,3,255,255,255,255,
-255,255,199,255,0,0,255,255,255,255,71,0,255,255,255,255,255,255,255,255,30,0,
-255,23,0,0,0,0,255,255,251,255,255,255,159,64,0,0,0,0,0,0,0,0,127,189,255,191,
-255,1,255,255,255,255,255,255,255,1,255,3,239,159,249,255,255,253,237,227,159,
-25,129,224,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,
-255,255,187,7,255,3,0,0,0,0,255,255,255,255,255,255,255,255,179,0,255,3,0,0,0,
+0,255,255,255,255,255,255,7,0,255,255,255,255,255,0,255,3,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,31,128,0,255,255,63,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,255,255,127,0,255,255,255,255,255,255,255,255,63,0,0,0,
+192,255,0,0,252,255,255,255,255,255,255,1,0,0,255,255,255,1,255,3,255,255,255,
+255,255,255,199,255,112,0,255,255,255,255,71,0,255,255,255,255,255,255,255,
+255,30,0,255,23,0,0,0,0,255,255,251,255,255,255,159,64,0,0,0,0,0,0,0,0,127,
+189,255,191,255,1,255,255,255,255,255,255,255,1,255,3,239,159,249,255,255,253,
+237,227,159,25,129,224,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,
+255,255,255,255,255,187,7,255,131,0,0,0,0,255,255,255,255,255,255,255,255,179,
+0,255,3,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,63,127,0,0,0,63,0,0,
0,0,255,255,255,255,255,255,255,127,17,0,255,3,0,0,0,0,255,255,255,255,255,
-255,63,0,255,3,0,0,0,0,0,
-0,255,255,255,227,255,7,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,3,
-0,128,255,255,255,255,255,255,231,127,0,0,255,255,255,255,255,255,207,255,255,
-0,0,0,0,0,255,255,255,255,255,255,255,1,255,253,255,255,255,255,127,127,1,0,
-255,3,0,0,252,255,255,255,252,255,255,254,127,0,0,0,0,0,0,0,0,0,127,251,255,
-255,255,255,127,180,203,0,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
+255,63,1,255,3,0,0,0,0,0,0,255,255,255,231,255,7,255,3,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,1,0,0,0,0,0,0,0,0,0,0,0,
+0,255,255,255,255,255,255,255,255,255,3,0,128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,255,252,255,255,255,255,255,252,26,0,0,0,255,255,255,255,255,255,231,
+127,0,0,255,255,255,255,255,255,255,255,255,32,0,0,0,0,255,255,255,255,255,
+255,255,1,255,253,255,255,255,255,127,127,1,0,255,3,0,0,252,255,255,255,252,
+255,255,254,127,0,0,0,0,0,0,0,0,0,127,251,255,255,255,255,127,180,203,0,255,3,
+191,253,255,255,255,127,123,1,255,3,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,127,0,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,3,0,0,
0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,127,0,
0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,255,255,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-255,255,255,255,255,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,255,255,255,255,255,255,255,1,255,255,255,127,255,3,0,0,0,0,0,0,0,0,0,0,0,
-0,255,255,255,63,0,0,255,255,255,255,255,255,127,0,15,0,255,3,248,255,255,224,
-255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,31,0,255,
-255,255,255,255,127,0,0,248,255,0,0,0,0,0,0,0,0,3,0,0,0,255,255,255,255,255,
+255,255,255,255,255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
+255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
+255,255,255,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
+255,255,255,255,255,255,1,255,255,255,127,255,3,0,0,0,0,0,0,0,0,0,0,0,0,255,
+255,255,63,0,0,255,255,255,255,255,255,0,0,15,0,255,3,248,255,255,224,255,255,
+0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,255,255,255,255,255,255,255,255,255,135,255,255,255,255,255,255,255,128,
+255,255,0,0,0,0,0,0,0,0,11,0,0,0,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,31,0,0,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,7,0,
-255,255,255,127,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,
+255,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,7,0,255,255,255,127,0,0,0,0,0,
+0,7,0,240,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,15,255,255,255,255,255,
-255,255,255,255,255,255,255,255,7,255,31,255,1,255,67,0,0,0,0,0,0,0,0,0,0,0,0,
-255,255,255,255,255,255,255,255,255,255,223,255,255,255,255,255,255,255,255,
-223,100,222,255,235,239,255,255,255,255,255,255,255,191,231,223,223,255,255,
-255,123,95,252,253,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,15,255,255,255,255,
+255,255,255,255,255,255,255,255,255,7,255,31,255,1,255,67,0,0,0,0,0,0,0,0,0,0,
+0,0,255,255,255,255,255,255,255,255,255,255,223,255,255,255,255,255,255,255,
+255,223,100,222,255,235,239,255,255,255,255,255,255,
+255,191,231,223,223,255,255,255,123,95,252,253,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,63,255,255,255,253,255,255,247,255,255,255,
-247,255,255,223,255,255,255,223,255,255,127,255,255,255,127,255,255,255,253,
-255,255,255,253,255,255,247,207,255,255,255,255,255,255,127,255,255,249,219,7,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,0,
-0,0,0,0,0,
-0,255,255,255,255,255,255,255,255,143,0,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,239,255,255,255,150,254,247,10,132,234,150,170,150,247,247,94,255,251,
-255,15,238,251,255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,3,255,255,255,3,
-255,255,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,63,255,255,255,
+253,255,255,247,255,255,255,247,255,255,223,255,255,255,223,255,255,127,255,
+255,255,127,255,255,255,253,255,255,255,253,255,255,247,207,255,255,255,255,
+255,255,127,255,255,249,219,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,255,255,255,255,255,31,128,63,255,67,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,
+15,255,3,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,31,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,
+143,8,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,239,255,255,255,150,254,247,10,132,234,150,170,150,247,247,94,255,251,255,
+15,238,251,255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,3,255,255,255,3,255,
+255,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
diff --git a/src/ctype/nonspacing.h b/src/ctype/nonspacing.h
index 48231e73..5d05a3d1 100644
--- a/src/ctype/nonspacing.h
+++ b/src/ctype/nonspacing.h
@@ -8,16 +8,16 @@
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,49,16,16,50,
-51,16,52,53,54,16,16,16,16,16,16,55,16,16,16,16,16,56,57,58,59,60,61,62,63,16,
-16,64,16,65,66,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+51,16,52,53,54,16,16,16,16,16,16,55,16,16,56,16,57,58,59,60,61,62,63,64,65,66,
+67,68,16,69,70,71,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,72,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,73,74,16,16,16,75,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,67,68,16,16,16,69,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,70,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,71,72,16,16,16,16,16,16,16,73,16,16,16,16,16,74,16,16,16,16,16,16,16,75,
-76,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,16,16,16,16,76,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,77,78,16,16,16,16,16,16,16,79,16,16,16,16,16,80,81,82,16,16,16,16,16,83,
+84,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
@@ -25,16 +25,16 @@
0,0,0,0,0,0,0,248,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,254,255,255,255,255,191,182,0,0,0,0,0,0,0,63,0,255,23,0,0,0,0,0,248,255,
255,0,0,1,0,0,0,0,0,0,0,0,0,0,0,192,191,159,61,0,0,0,128,2,0,0,0,255,255,255,
-7,0,0,0,0,0,0,0,0,0,0,192,255,1,0,0,0,0,0,0,248,15,0,0,0,192,251,239,62,0,0,0,
-0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,255,255,255,255,
+7,0,0,0,0,0,0,0,0,0,0,192,255,1,0,0,0,0,0,0,248,15,32,0,0,192,251,239,62,0,0,
+0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,255,255,255,255,
255,7,0,0,0,0,0,0,20,254,33,254,0,12,0,0,0,2,0,0,0,0,0,0,16,30,32,0,0,12,0,0,
-0,6,0,0,0,0,0,0,16,134,57,2,0,0,0,35,0,6,0,0,0,0,0,0,16,190,33,0,0,12,0,0,252,
-2,0,0,0,0,0,0,144,30,32,64,0,12,0,0,0,4,0,0,0,0,0,0,0,1,32,0,0,0,0,0,0,1,0,0,
-0,0,0,0,192,193,61,96,0,12,0,0,0,2,0,0,0,0,0,0,144,64,48,0,0,12,0,0,0,3,0,0,0,
-0,0,0,24,30,32,0,0,12,0,0,0,0,0,0,0,0,0,0,0,0,4,92,0,0,0,0,0,0,0,0,0,0,0,242,
-7,128,127,0,0,0,0,0,0,0,0,0,0,0,0,242,27,0,63,0,0,0,0,0,0,0,0,0,3,0,0,160,2,0,
-0,0,0,0,0,254,127,223,224,255,254,255,255,255,31,64,0,0,0,0,0,0,0,0,0,0,0,0,
-224,253,102,0,0,0,195,1,0,30,0,100,32,0,32,0,0,0,0,0,0,0,0,0,0,0,
+64,6,0,0,0,0,0,0,16,134,57,2,0,0,0,35,0,6,0,0,0,0,0,0,16,190,33,0,0,12,0,0,
+252,2,0,0,0,0,0,0,144,30,32,64,0,12,0,0,0,4,0,0,0,0,0,0,0,1,32,0,0,0,0,0,0,17,
+0,0,0,0,0,0,192,193,61,96,0,12,0,0,0,2,0,0,0,0,0,0,144,64,48,0,0,12,0,0,0,3,0,
+0,0,0,0,0,24,30,32,0,0,12,0,0,0,0,0,0,0,0,0,0,0,0,4,92,0,0,0,0,0,0,0,0,0,0,0,
+242,7,128,127,0,0,0,0,0,0,0,0,0,0,0,0,242,31,0,63,0,0,0,0,0,0,0,0,0,3,0,0,160,
+2,0,0,0,0,0,0,254,127,223,224,255,254,255,255,255,31,64,0,0,0,0,0,0,0,0,0,0,0,
+0,224,253,102,0,0,0,195,1,0,30,0,100,32,0,32,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,224,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,28,0,
0,0,28,0,0,0,12,0,0,0,12,0,0,0,0,0,0,0,176,63,64,254,15,32,0,0,0,0,0,120,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,96,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,1,4,14,0,
@@ -48,9 +48,9 @@
0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,0,0,0,0,
0,60,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,128,247,63,0,0,0,192,0,0,0,0,0,0,0,0,0,0,3,0,68,8,0,0,96,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,48,0,0,0,255,255,3,0,0,0,0,0,192,63,0,0,128,255,3,0,0,
-0,0,0,7,0,0,0,0,0,200,19,0,0,0,0,32,0,0,0,0,0,0,0,0,126,102,0,8,16,0,0,0,0,0,
-16,0,0,0,0,0,0,157,193,2,0,0,0,0,48,64,
+0,0,0,0,0,0,0,0,0,0,0,0,48,0,0,0,255,255,3,128,0,0,0,0,192,63,0,0,128,255,3,0,
+0,0,0,0,7,0,0,0,0,0,200,51,0,0,0,0,32,0,0,0,0,0,0,0,0,126,102,0,8,16,0,0,0,0,
+0,16,0,0,0,0,0,0,157,193,2,0,0,0,0,48,64,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,33,0,0,0,0,0,64,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,0,0,255,255,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,0,
@@ -58,24 +58,32 @@
0,0,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,192,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,110,240,0,0,0,0,0,135,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,0,0,
-0,2,0,0,0,0,0,0,255,127,0,0,0,0,0,0,128,3,0,0,0,0,0,120,38,0,0,0,0,0,0,0,0,7,
-0,0,0,128,239,31,0,0,0,0,0,0,0,8,0,3,0,0,0,0,0,192,127,0,28,0,0,0,0,0,0,0,0,0,
-0,0,128,211,64,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,248,7,0,0,3,0,0,0,0,
-0,0,16,1,0,0,0,192,31,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
-92,0,0,0,0,0,0,0,0,0,0,0,0,0,248,133,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,60,176,1,0,0,48,0,0,0,0,0,0,0,0,0,0,248,167,1,0,0,0,0,0,0,
-0,0,0,0,0,0,40,191,0,0,0,0,0,0,0,0,0,0,0,0,224,188,15,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,126,6,0,0,0,0,248,121,128,0,126,14,0,0,0,0,0,252,127,3,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,127,191,0,0,0,0,0,0,0,0,0,0,252,255,255,252,109,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,126,180,191,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,0,0,0,0,0,0,0,127,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,128,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-96,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,3,248,255,231,15,0,0,
-0,60,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
-255,255,255,255,127,248,255,255,255,255,255,31,32,0,16,0,0,248,254,255,0,0,0,
-0,0,0,0,0,0,0,127,255,255,249,219,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,
-0,0,0,0,0,0,0,240,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,240,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,192,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,255,
+127,0,0,0,0,0,0,128,3,0,0,0,0,0,120,38,0,32,0,0,0,0,0,0,7,0,0,0,128,239,31,0,
+0,0,0,0,0,0,8,0,3,0,0,0,0,0,192,127,0,30,0,0,0,0,0,0,0,0,0,0,0,128,211,64,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,248,7,0,0,3,0,0,0,0,0,0,24,1,0,0,0,192,
+31,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,92,0,0,64,0,0,0,0,0,
+0,0,0,0,0,248,133,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,60,176,1,0,0,48,0,0,0,
+0,0,0,0,0,0,0,248,167,1,0,0,0,0,0,0,0,0,0,0,0,0,40,191,0,0,0,0,0,0,0,0,0,0,0,
+0,224,188,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+128,255,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,12,1,0,0,0,254,7,0,0,0,0,248,121,128,0,
+126,14,0,0,0,0,0,252,127,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,191,0,0,0,
+0,0,0,0,0,0,0,252,255,255,252,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,126,180,191,0,
+0,0,0,0,0,0,0,0,163,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,24,
+0,0,0,0,0,0,0,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,0,0,0,0,0,0,0,127,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,
+0,128,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,15,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,3,248,255,231,15,0,0,0,60,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,255,255,255,255,255,255,127,248,255,255,255,255,255,31,32,0,16,0,0,248,
+254,255,0,0,0,0,0,0,0,0,0,
+0,127,255,255,249,219,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,240,7,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,
diff --git a/src/ctype/punct.h b/src/ctype/punct.h
index 7a623940..67929470 100644
--- a/src/ctype/punct.h
+++ b/src/ctype/punct.h
@@ -8,17 +8,17 @@
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,64,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,65,16,16,66,16,67,68,
-69,16,70,71,72,16,73,16,16,74,75,76,77,78,16,79,16,80,81,82,83,84,85,86,87,88,
-16,89,16,90,91,16,16,16,16,16,16,92,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+69,16,70,71,72,16,73,16,16,74,75,76,77,78,16,79,80,81,82,83,84,85,86,87,88,89,
+90,91,16,92,93,94,95,16,16,16,16,96,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,97,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,98,99,16,16,100,101,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,93,94,16,16,16,95,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,96,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,97,98,99,100,16,16,101,102,17,17,103,16,16,16,16,16,16,16,16,16,16,16,16,
-16,104,105,16,16,16,16,106,16,107,108,109,17,17,17,110,111,112,113,16,16,16,
-16,16,
+16,16,16,16,16,16,16,16,102,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
+16,16,16,103,104,105,106,16,16,107,108,17,17,109,16,16,16,16,16,16,110,111,16,
+16,16,16,16,112,113,16,16,114,115,116,16,117,118,119,17,17,17,120,121,122,123,
+124,16,16,16,16,
16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,0,0,0,0,254,255,0,252,1,0,0,248,1,
@@ -28,25 +28,25 @@
0,0,0,0,0,0,0,0,0,0,0,0,0,0,252,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,252,0,0,0,0,0,230,254,255,255,255,0,64,73,0,0,0,0,0,24,0,255,255,0,216,
0,0,0,0,0,0,0,1,0,60,0,0,0,0,0,0,0,0,0,0,0,0,16,224,1,30,0,
-96,255,191,0,0,0,0,0,0,255,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,207,3,
-0,0,0,3,0,32,255,127,0,0,0,78,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,7,252,0,0,0,0,0,
-0,0,0,0,16,0,32,30,0,48,0,1,0,0,0,0,0,0,0,0,16,0,32,0,0,0,0,252,47,0,0,0,0,0,
-0,0,16,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,16,0,32,0,0,0,0,3,224,0,0,0,0,0,0,0,16,
-0,32,0,0,0,0,253,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,255,7,0,0,0,0,0,0,0,0,0,32,0,
-0,0,0,0,255,0,0,0,0,0,0,0,16,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,24,0,160,0,127,0,
-0,255,3,0,0,0,0,0,0,0,0,0,4,0,0,0,0,16,0,0,0,0,0,0,128,0,128,192,223,0,12,0,0,
-0,0,0,0,0,0,0,0,0,0,0,31,0,0,0,0,0,
+96,255,191,0,0,0,0,0,0,255,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,207,
+227,0,0,0,3,0,32,255,127,0,0,0,78,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,7,252,0,0,0,
+0,0,0,0,0,0,16,0,32,30,0,48,0,1,0,0,0,0,0,0,0,0,16,0,32,0,0,0,0,252,111,0,0,0,
+0,0,0,0,16,0,32,0,0,0,0,64,0,0,0,0,0,0,0,0,16,0,32,0,0,0,0,3,224,0,0,0,0,0,0,
+0,16,0,32,0,0,0,0,253,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,255,7,16,0,0,0,0,0,0,0,0,
+32,0,0,0,0,128,255,16,0,0,0,0,0,0,16,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,24,0,160,
+0,127,0,0,255,3,0,0,0,0,0,0,0,0,0,4,0,0,0,0,16,0,0,0,0,0,0,128,0,128,192,223,
+0,12,0,0,0,0,0,0,0,0,0,0,0,4,0,31,0,0,0,0,0,
0,254,255,255,255,0,252,255,255,0,0,0,0,0,0,0,0,252,0,0,0,0,0,0,192,255,223,
-255,7,0,0,0,0,0,0,0,0,0,0,128,6,0,252,0,0,24,62,0,0,128,191,0,204,0,0,0,0,0,0,
-0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,96,255,255,255,31,0,0,255,3,0,0,0,0,0,0,0,0,
-0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,96,0,0,1,0,0,24,0,0,0,0,0,0,0,0,0,56,0,0,0,0,16,0,0,0,112,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,48,0,0,254,127,47,0,0,255,3,255,127,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,14,49,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,196,255,255,255,
+255,7,0,0,0,0,0,0,0,0,0,0,128,6,0,252,0,0,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,0,0,
+0,0,8,0,0,0,0,0,0,0,0,0,0,0,224,255,255,255,31,0,0,255,3,0,0,0,0,0,0,0,0,0,0,
+0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,96,0,0,1,0,0,24,0,0,0,0,0,0,0,0,0,56,0,0,0,0,16,0,0,0,112,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,48,0,0,254,127,47,0,0,255,3,255,127,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,14,49,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,196,255,255,255,
255,0,0,0,192,0,0,0,0,0,0,0,0,1,0,224,159,0,0,0,0,127,63,255,127,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,16,0,16,0,0,252,255,255,255,31,0,0,0,0,0,12,0,0,0,0,0,0,64,0,
-12,240,0,0,0,0,0,0,192,248,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,0,255,0,255,255,
+12,240,0,0,0,0,0,0,128,248,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,0,255,0,255,255,
255,33,144,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,
127,0,224,251,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,160,3,224,0,224,0,
224,0,96,128,248,255,255,255,252,255,255,255,255,255,127,223,255,241,127,255,
@@ -55,22 +55,23 @@
255,255,255,255,255,127,0,0,0,255,7,0,0,255,255,255,255,255,255,255,255,255,
255,63,0,0,0,0,0,0,252,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,207,255,255,255,
-63,255,255,255,255,227,255,253,7,0,0,240,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,224,135,3,254,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,128,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,127,255,255,255,3,0,0,0,0,0,0,
-255,255,255,251,255,255,255,255,255,255,255,255,255,255,15,0,255,255,255,255,
+63,255,255,255,255,255,255,255,255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,224,135,3,254,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
+128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,127,255,255,255,255,0,
+0,0,0,0,0,255,255,255,251,255,255,255,255,255,255,255,255,255,255,15,0,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,63,0,0,0,255,15,30,255,255,255,1,252,193,224,0,0,0,0,0,0,0,0,0,0,
-0,30,1,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,0,0,
-0,0,255,255,255,255,15,0,0,0,255,255,255,127,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,63,0,0,0,255,15,30,255,255,255,1,252,193,224,0,0,0,0,
+0,0,0,0,0,0,0,30,1,0,0,0,0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+255,255,0,0,0,0,255,255,255,255,15,0,0,0,255,255,255,127,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,
+255,255,255,
+255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,
255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,127,0,0,0,
0,0,0,192,0,224,0,0,0,0,0,0,0,0,0,0,0,128,15,112,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
255,0,255,255,127,0,3,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-68,8,0,0,0,15,255,3,0,0,0,0,0,0,240,0,0,0,0,0,0,0,0,0,16,192,0,0,255,255,3,23,
-0,0,0,0,0,248,0,0,0,0,8,128,0,0,0,0,0,0,0,0,0,0,8,0,255,63,0,192,32,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,240,0,0,128,59,0,0,0,0,0,0,0,128,2,0,0,192,0,0,67,0,0,0,0,0,
+64,0,0,0,0,15,255,3,0,0,0,0,0,0,240,0,0,0,0,0,0,0,0,0,16,192,0,0,255,255,3,23,
+0,0,0,0,0,248,0,0,0,0,8,128,0,0,0,0,0,0,0,0,0,0,8,0,255,63,0,192,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,240,0,0,128,3,0,0,0,0,0,0,0,128,2,0,0,192,0,0,67,0,0,0,0,0,
0,0,0,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,56,0,
0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,2,0,0,0,0,0,0,
@@ -84,46 +85,57 @@
0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
128,255,0,0,128,255,0,0,0,0,128,255,0,0,0,0,0,0,0,0,0,248,0,0,192,143,0,0,0,
128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,48,255,255,252,255,255,255,255,255,0,0,0,0,
-0,0,0,135,255,0,255,1,0,0,0,224,0,0,0,224,0,0,0,0,0,1,0,0,96,248,127,0,0,0,0,
+0,0,0,135,255,1,255,1,0,0,0,224,0,0,0,224,0,0,0,0,0,1,0,0,96,248,127,0,0,0,0,
0,0,0,0,254,0,0,0,255,0,0,0,255,0,0,0,30,0,254,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,252,0,0,0,0,0,0,0,0,0,0,0,
0,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,192,63,252,255,63,0,0,128,3,0,0,0,0,0,0,254,3,0,0,0,0,0,0,0,
-0,0,0,0,0,0,24,0,15,0,0,0,0,0,56,0,0,0,0,0,0,0,0,0,225,63,0,232,254,255,31,0,
-0,0,0,0,0,0,96,63,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,
-0,16,0,32,0,0,192,31,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,
-248,0,40,0,0,0,0,0,0,0,0,0,0,0,0,76,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,128,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,128,14,0,0,0,255,31,
-0,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,0,252,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,224,127,0,0,0,192,255,255,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,192,63,252,255,63,0,0,128,3,0,0,0,0,0,0,254,3,32,0,0,0,0,0,0,0,
+0,0,0,0,0,24,0,15,0,0,0,0,0,56,0,0,0,0,0,0,0,0,0,225,63,0,232,254,255,31,0,0,
+0,0,0,0,0,96,63,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,
+24,0,32,0,0,192,31,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,
+248,0,104,0,0,0,0,0,0,0,0,0,0,0,0,76,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,128,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,128,14,0,0,0,255,
+31,0,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,8,0,252,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,252,7,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0,0,0,0,24,128,255,0,0,0,0,0,
+0,0,0,0,0,223,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,62,0,0,252,255,31,3,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,52,0,0,0,0,0,0,0,0,0,128,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,128,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,
+255,3,
+128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,192,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,63,0,0,0,0,0,0,0,255,255,48,0,0,248,
+3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
+255,255,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,176,15,0,0,0,0,0,0,
+0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,63,
+0,255,255,255,255,127,254,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,1,0,0,255,255,255,255,255,255,255,255,
+63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,15,0,255,255,255,255,255,255,
+255,255,255,255,127,0,255,255,255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,8,0,0,0,8,0,0,32,0,0,0,32,0,0,128,
+0,0,0,128,0,0,0,2,0,0,0,2,0,0,8,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,15,0,248,254,255,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,127,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,0,
+128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,255,127,0,0,0,0,0,0,0,
+0,0,0,0,0,0,112,7,0,192,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,254,255,255,255,255,255,255,255,31,0,0,0,0,0,0,0,0,0,254,255,
+255,255,255,255,255,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,255,255,255,255,255,
+15,255,255,255,255,255,255,255,255,255,255,255,255,15,0,255,127,254,255,254,
+255,254,255,255,255,63,0,255,31,255,255,255,255,0,0,0,252,0,0,0,28,0,0,0,252,
+255,255,255,31,0,0,0,0,0,0,192,255,255,255,7,0,255,255,255,255,255,15,255,1,3,
+0,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,63,0,255,31,255,7,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,15,0,255,255,255,255,255,255,255,255,255,255,255,1,
+255,15,0,0,255,15,255,255,255,255,255,255,255,0,255,3,255,255,255,255,255,0,
+255,255,255,63,0,0,0,0,0,0,0,0,0,0,255,239,255,255,255,255,255,255,255,255,
+255,255,255,255,123,252,255,255,255,255,231,199,255,255,255,231,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,15,0,255,63,15,7,7,0,63,0,
0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,252,7,0,0,0,0,0,0,
-0,24,128,255,0,0,0,0,0,0,0,0,0,0,223,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-128,62,0,0,252,255,31,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,52,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,192,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,63,0,0,0,0,0,0,0,128,255,48,0,0,248,3,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,7,0,0,0,0,0,0,0,0,0,0,0,
-0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,176,15,0,0,0,0,0,0,0,0,0,0,0,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,63,0,255,255,255,255,127,254,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,1,0,0,255,255,255,255,255,255,255,255,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,127,0,255,255,3,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,
-0,8,0,0,0,8,0,0,32,0,0,0,32,0,0,128,0,0,0,128,0,0,0,2,0,0,0,2,0,0,8,0,0,0,0,0,
-0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,15,0,
-248,254,255,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,255,127,0,0,0,0,0,0,0,0,
-0,0,0,0,0,112,7,0,192,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,255,255,255,255,255,15,255,
-255,255,255,255,255,255,255,255,255,255,255,15,0,255,127,254,255,254,255,254,
-255,255,255,63,0,255,31,255,255,255,127,0,0,0,252,0,0,0,12,0,0,0,252,255,255,
-255,31,0,0,0,0,0,0,192,255,255,255,7,0,255,255,255,255,255,15,255,1,3,0,63,0,
-0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,0,255,31,
-255,1,255,255,255,255,255,255,255,255,255,255,255,255,255,255,15,0,255,255,
-255,255,255,255,255,255,255,255,31,0,0,0,0,
-0,255,15,255,255,255,255,255,255,255,0,255,3,255,255,255,255,255,0,255,255,
-255,63,0,0,0,0,0,0,0,0,0,0,255,15,255,255,255,255,255,127,255,31,255,255,255,
-15,0,0,255,255,255,0,0,0,0,0,1,0,255,255,127,0,0,0,
diff --git a/src/ctype/wide.h b/src/ctype/wide.h
index e4672b23..e403c9a5 100644
--- a/src/ctype/wide.h
+++ b/src/ctype/wide.h
@@ -17,7 +17,7 @@
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,38,39,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
-16,16,16,16,16,16,16,40,41,42,43,44,45,46,16,16,47,16,16,16,16,16,
+16,16,16,16,16,16,16,40,41,42,43,44,45,46,47,16,48,49,16,16,16,16,
16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
@@ -31,10 +31,10 @@
255,255,255,15,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,63,0,0,0,255,15,255,255,255,255,
255,255,255,127,254,255,255,255,255,255,255,255,255,255,127,254,255,255,255,
-255,255,255,255,255,255,255,255,255,224,255,255,255,255,127,254,255,255,255,
+255,255,255,255,255,255,255,255,255,224,255,255,255,255,255,254,255,255,255,
255,255,255,255,255,255,255,127,255,255,255,255,255,7,255,255,255,255,15,0,
255,255,255,255,255,127,255,255,255,255,255,0,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,127,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,
0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,31,255,255,255,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
@@ -43,13 +43,13 @@
255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,255,3,0,0,255,255,255,255,247,255,127,15,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,254,255,255,255,255,255,255,255,255,255,255,
255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
-0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,31,0,
-0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,7,0,255,255,255,127,0,0,0,0,0,0,0,
-0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+0,0,0,0,0,0,0,0,0,0,0,0,15,0,0,0,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,7,0,255,255,255,127,0,0,0,0,0,
+0,7,0,240,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
-255,255,255,255,255,255,255,255,255,255,255,255,
+255,255,255,255,255,255,255,255,255,255,255,255,255,255,
15,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,64,254,7,0,0,0,0,0,0,0,0,0,0,0,0,7,0,255,255,255,
255,255,15,255,1,3,0,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,
@@ -58,6 +58,8 @@
255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
159,255,255,255,255,255,255,255,63,0,120,255,255,255,0,0,4,0,0,96,0,16,0,0,0,
0,0,0,0,0,0,0,248,255,255,255,255,255,255,255,255,255,255,0,0,0,0,0,0,255,255,
-255,255,255,255,255,255,63,16,7,0,0,24,240,1,0,0,255,255,255,255,255,127,255,
-31,255,255,255,15,0,0,255,255,255,0,0,0,0,0,1,0,255,255,127,0,0,
-0,
+255,255,255,255,255,255,63,16,39,0,0,24,240,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,0,0,0,0,0,0,0,0,0,0,0,255,15,0,
+0,0,224,255,255,255,255,255,255,255,255,255,255,255,255,123,252,255,255,255,
+255,231,199,255,255,255,231,255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,
+0,15,7,7,0,63,0,0,0,0,0,0,0,0,0,0,0,0,0,
--
2.23.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-12 21:27 [PATCH] Update ctype data to Unicode 12.1.0 u_quark
@ 2019-10-12 22:39 ` Rich Felker
2019-10-12 22:56 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Rich Felker @ 2019-10-12 22:39 UTC (permalink / raw)
To: musl
On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
> ---
> src/ctype/alpha.h | 159 +++++++++++++++++++++-------------------
> src/ctype/nonspacing.h | 88 ++++++++++++-----------
> src/ctype/punct.h | 160 ++++++++++++++++++++++-------------------
> src/ctype/wide.h | 26 +++----
> 4 files changed, 232 insertions(+), 201 deletions(-)
Is this done just by dropping the new Unicode files into
musl-chartable-tools and running make?
Rich
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-12 22:39 ` Rich Felker
@ 2019-10-12 22:56 ` Eleftherios Kritikos
2019-10-14 13:07 ` Rich Felker
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-12 22:56 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 678 bytes --]
Yes. I also created a merge request for musl-chartable-tools
https://github.com/richfelker/musl-chartable-tools/pull/2
On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
> On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
> > ---
> > src/ctype/alpha.h | 159 +++++++++++++++++++++-------------------
> > src/ctype/nonspacing.h | 88 ++++++++++++-----------
> > src/ctype/punct.h | 160 ++++++++++++++++++++++-------------------
> > src/ctype/wide.h | 26 +++----
> > 4 files changed, 232 insertions(+), 201 deletions(-)
>
> Is this done just by dropping the new Unicode files into
> musl-chartable-tools and running make?
>
> Rich
>
[-- Attachment #2: Type: text/html, Size: 1168 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-12 22:56 ` Eleftherios Kritikos
@ 2019-10-14 13:07 ` Rich Felker
2019-10-14 13:51 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Rich Felker @ 2019-10-14 13:07 UTC (permalink / raw)
To: musl
On Sat, Oct 12, 2019 at 11:56:44PM +0100, Eleftherios Kritikos wrote:
> Yes. I also created a merge request for musl-chartable-tools
> https://github.com/richfelker/musl-chartable-tools/pull/2
Thanks. stdc-predef.h also needs to be updated with a new value for
__STD_ISO_10646__. Do you know the right yyyymm value it should have
for this version of Unicode?
Rich
> On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
>
> > On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
> > > ---
> > > src/ctype/alpha.h | 159 +++++++++++++++++++++-------------------
> > > src/ctype/nonspacing.h | 88 ++++++++++++-----------
> > > src/ctype/punct.h | 160 ++++++++++++++++++++++-------------------
> > > src/ctype/wide.h | 26 +++----
> > > 4 files changed, 232 insertions(+), 201 deletions(-)
> >
> > Is this done just by dropping the new Unicode files into
> > musl-chartable-tools and running make?
> >
> > Rich
> >
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-14 13:07 ` Rich Felker
@ 2019-10-14 13:51 ` Eleftherios Kritikos
2019-10-14 13:57 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-14 13:51 UTC (permalink / raw)
To: Rich Felker; +Cc: musl
[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]
From what I read from here:
https://stackoverflow.com/questions/12590255/what-does-stdc-iso-10646-exactly-mean
and here:
https://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
it seems like the latest ISO/IEC 10646 standard that most closely match
Unicode 12.1.0 is ISO/IEC 10646:2017/Amd 2:2019 (fifth edition 2017,
amendment 2).
From what I read in the document here:
https://standards.iso.org/ittf/PubliclyAvailableStandards/c073773_ISO_IEC_10646_2017_Amd_2_2019%20(E).zip
on first page, this amendment was made on 2019-06. So I would guess that
the correct value should be:
```
#define __STDC_ISO_10646__ 201906L
```
All this with a grain of salt as this is the first time I am looking at
ISO/IEC 10646.
Thanks for looking into this!
Regards,
Lefteris
On Mon, Oct 14, 2019 at 2:07 PM Rich Felker <dalias@libc.org> wrote:
> On Sat, Oct 12, 2019 at 11:56:44PM +0100, Eleftherios Kritikos wrote:
> > Yes. I also created a merge request for musl-chartable-tools
> > https://github.com/richfelker/musl-chartable-tools/pull/2
>
> Thanks. stdc-predef.h also needs to be updated with a new value for
> __STD_ISO_10646__. Do you know the right yyyymm value it should have
> for this version of Unicode?
>
> Rich
>
>
> > On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
> >
> > > On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
> > > > ---
> > > > src/ctype/alpha.h | 159
> +++++++++++++++++++++-------------------
> > > > src/ctype/nonspacing.h | 88 ++++++++++++-----------
> > > > src/ctype/punct.h | 160
> ++++++++++++++++++++++-------------------
> > > > src/ctype/wide.h | 26 +++----
> > > > 4 files changed, 232 insertions(+), 201 deletions(-)
> > >
> > > Is this done just by dropping the new Unicode files into
> > > musl-chartable-tools and running make?
> > >
> > > Rich
> > >
>
[-- Attachment #2: Type: text/html, Size: 3241 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-14 13:51 ` Eleftherios Kritikos
@ 2019-10-14 13:57 ` Eleftherios Kritikos
2019-10-20 8:53 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-14 13:57 UTC (permalink / raw)
To: Rich Felker; +Cc: musl
[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]
Actually now that I read the ISO spec again, it seems to correspond to
Unicode 11 :( so I think ISO/IEC 10646 has not catch up with Unicode 12
yet. Not sure what we should do in this case.
On Mon, Oct 14, 2019 at 2:51 PM Eleftherios Kritikos <el01049@gmail.com>
wrote:
> From what I read from here:
>
>
> https://stackoverflow.com/questions/12590255/what-does-stdc-iso-10646-exactly-mean
>
> and here:
>
> https://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
>
> it seems like the latest ISO/IEC 10646 standard that most closely match
> Unicode 12.1.0 is ISO/IEC 10646:2017/Amd 2:2019 (fifth edition 2017,
> amendment 2).
>
> From what I read in the document here:
> https://standards.iso.org/ittf/PubliclyAvailableStandards/c073773_ISO_IEC_10646_2017_Amd_2_2019%20(E).zip
>
> on first page, this amendment was made on 2019-06. So I would guess that
> the correct value should be:
>
> ```
> #define __STDC_ISO_10646__ 201906L
> ```
>
> All this with a grain of salt as this is the first time I am looking at
> ISO/IEC 10646.
>
> Thanks for looking into this!
>
> Regards,
> Lefteris
>
>
>
> On Mon, Oct 14, 2019 at 2:07 PM Rich Felker <dalias@libc.org> wrote:
>
>> On Sat, Oct 12, 2019 at 11:56:44PM +0100, Eleftherios Kritikos wrote:
>> > Yes. I also created a merge request for musl-chartable-tools
>> > https://github.com/richfelker/musl-chartable-tools/pull/2
>>
>> Thanks. stdc-predef.h also needs to be updated with a new value for
>> __STD_ISO_10646__. Do you know the right yyyymm value it should have
>> for this version of Unicode?
>>
>> Rich
>>
>>
>> > On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
>> >
>> > > On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
>> > > > ---
>> > > > src/ctype/alpha.h | 159
>> +++++++++++++++++++++-------------------
>> > > > src/ctype/nonspacing.h | 88 ++++++++++++-----------
>> > > > src/ctype/punct.h | 160
>> ++++++++++++++++++++++-------------------
>> > > > src/ctype/wide.h | 26 +++----
>> > > > 4 files changed, 232 insertions(+), 201 deletions(-)
>> > >
>> > > Is this done just by dropping the new Unicode files into
>> > > musl-chartable-tools and running make?
>> > >
>> > > Rich
>> > >
>>
>
[-- Attachment #2: Type: text/html, Size: 3778 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-14 13:57 ` Eleftherios Kritikos
@ 2019-10-20 8:53 ` Eleftherios Kritikos
2019-10-20 14:59 ` Rich Felker
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-20 8:53 UTC (permalink / raw)
To: Rich Felker; +Cc: musl
[-- Attachment #1: Type: text/plain, Size: 2453 bytes --]
Is anything else we could do to get this merged?
Kind regards,
Lefteris
On Mon, Oct 14, 2019 at 4:57 PM Eleftherios Kritikos <el01049@gmail.com>
wrote:
> Actually now that I read the ISO spec again, it seems to correspond to
> Unicode 11 :( so I think ISO/IEC 10646 has not catch up with Unicode 12
> yet. Not sure what we should do in this case.
>
> On Mon, Oct 14, 2019 at 2:51 PM Eleftherios Kritikos <el01049@gmail.com>
> wrote:
>
>> From what I read from here:
>>
>>
>> https://stackoverflow.com/questions/12590255/what-does-stdc-iso-10646-exactly-mean
>>
>> and here:
>>
>> https://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
>>
>> it seems like the latest ISO/IEC 10646 standard that most closely match
>> Unicode 12.1.0 is ISO/IEC 10646:2017/Amd 2:2019 (fifth edition 2017,
>> amendment 2).
>>
>> From what I read in the document here:
>> https://standards.iso.org/ittf/PubliclyAvailableStandards/c073773_ISO_IEC_10646_2017_Amd_2_2019%20(E).zip
>>
>> on first page, this amendment was made on 2019-06. So I would guess that
>> the correct value should be:
>>
>> ```
>> #define __STDC_ISO_10646__ 201906L
>> ```
>>
>> All this with a grain of salt as this is the first time I am looking at
>> ISO/IEC 10646.
>>
>> Thanks for looking into this!
>>
>> Regards,
>> Lefteris
>>
>>
>>
>> On Mon, Oct 14, 2019 at 2:07 PM Rich Felker <dalias@libc.org> wrote:
>>
>>> On Sat, Oct 12, 2019 at 11:56:44PM +0100, Eleftherios Kritikos wrote:
>>> > Yes. I also created a merge request for musl-chartable-tools
>>> > https://github.com/richfelker/musl-chartable-tools/pull/2
>>>
>>> Thanks. stdc-predef.h also needs to be updated with a new value for
>>> __STD_ISO_10646__. Do you know the right yyyymm value it should have
>>> for this version of Unicode?
>>>
>>> Rich
>>>
>>>
>>> > On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
>>> >
>>> > > On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
>>> > > > ---
>>> > > > src/ctype/alpha.h | 159
>>> +++++++++++++++++++++-------------------
>>> > > > src/ctype/nonspacing.h | 88 ++++++++++++-----------
>>> > > > src/ctype/punct.h | 160
>>> ++++++++++++++++++++++-------------------
>>> > > > src/ctype/wide.h | 26 +++----
>>> > > > 4 files changed, 232 insertions(+), 201 deletions(-)
>>> > >
>>> > > Is this done just by dropping the new Unicode files into
>>> > > musl-chartable-tools and running make?
>>> > >
>>> > > Rich
>>> > >
>>>
>>
[-- Attachment #2: Type: text/html, Size: 4286 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-20 8:53 ` Eleftherios Kritikos
@ 2019-10-20 14:59 ` Rich Felker
2019-10-20 15:26 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Rich Felker @ 2019-10-20 14:59 UTC (permalink / raw)
To: musl
On Sun, Oct 20, 2019 at 11:53:15AM +0300, Eleftherios Kritikos wrote:
> Is anything else we could do to get this merged?
Thanks for pinging. I think we just need to go with the previous
__STDC_ISO_10646__ macro value for now until there's a new one. I
don't think it's a conformance problem to support additional new
characters we don't advertise support for with the macro, so it's not
a big deal. I can add it as a separate commit and update later with
another once there's a new value to be used.
Sorry for not getting to this sooner. I do want to run over the
chartable tools change and make sure there's nothing unexpected that
could cause problems, and didn't want to get bogged down in that
before release.
BTW are there any new case mappings needed too? I can handle those if
so. (The current case mapping code needs manual work to update; I have
a draft replacement that won't need manual work I should really get
around to integrating.)
Rich
> On Mon, Oct 14, 2019 at 4:57 PM Eleftherios Kritikos <el01049@gmail.com>
> wrote:
>
> > Actually now that I read the ISO spec again, it seems to correspond to
> > Unicode 11 :( so I think ISO/IEC 10646 has not catch up with Unicode 12
> > yet. Not sure what we should do in this case.
> >
> > On Mon, Oct 14, 2019 at 2:51 PM Eleftherios Kritikos <el01049@gmail.com>
> > wrote:
> >
> >> From what I read from here:
> >>
> >>
> >> https://stackoverflow.com/questions/12590255/what-does-stdc-iso-10646-exactly-mean
> >>
> >> and here:
> >>
> >> https://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
> >>
> >> it seems like the latest ISO/IEC 10646 standard that most closely match
> >> Unicode 12.1.0 is ISO/IEC 10646:2017/Amd 2:2019 (fifth edition 2017,
> >> amendment 2).
> >>
> >> From what I read in the document here:
> >> https://standards.iso.org/ittf/PubliclyAvailableStandards/c073773_ISO_IEC_10646_2017_Amd_2_2019%20(E).zip
> >>
> >> on first page, this amendment was made on 2019-06. So I would guess that
> >> the correct value should be:
> >>
> >> ```
> >> #define __STDC_ISO_10646__ 201906L
> >> ```
> >>
> >> All this with a grain of salt as this is the first time I am looking at
> >> ISO/IEC 10646.
> >>
> >> Thanks for looking into this!
> >>
> >> Regards,
> >> Lefteris
> >>
> >>
> >>
> >> On Mon, Oct 14, 2019 at 2:07 PM Rich Felker <dalias@libc.org> wrote:
> >>
> >>> On Sat, Oct 12, 2019 at 11:56:44PM +0100, Eleftherios Kritikos wrote:
> >>> > Yes. I also created a merge request for musl-chartable-tools
> >>> > https://github.com/richfelker/musl-chartable-tools/pull/2
> >>>
> >>> Thanks. stdc-predef.h also needs to be updated with a new value for
> >>> __STD_ISO_10646__. Do you know the right yyyymm value it should have
> >>> for this version of Unicode?
> >>>
> >>> Rich
> >>>
> >>>
> >>> > On Sat, 12 Oct 2019, 11:40 pm Rich Felker, <dalias@libc.org> wrote:
> >>> >
> >>> > > On Sat, Oct 12, 2019 at 10:27:42PM +0100, u_quark wrote:
> >>> > > > ---
> >>> > > > src/ctype/alpha.h | 159
> >>> +++++++++++++++++++++-------------------
> >>> > > > src/ctype/nonspacing.h | 88 ++++++++++++-----------
> >>> > > > src/ctype/punct.h | 160
> >>> ++++++++++++++++++++++-------------------
> >>> > > > src/ctype/wide.h | 26 +++----
> >>> > > > 4 files changed, 232 insertions(+), 201 deletions(-)
> >>> > >
> >>> > > Is this done just by dropping the new Unicode files into
> >>> > > musl-chartable-tools and running make?
> >>> > >
> >>> > > Rich
> >>> > >
> >>>
> >>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-20 14:59 ` Rich Felker
@ 2019-10-20 15:26 ` Eleftherios Kritikos
2019-10-23 16:21 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-20 15:26 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 872 bytes --]
Thanks a lot for looking into this.
On Sun, Oct 20, 2019 at 5:59 PM Rich Felker <dalias@libc.org> wrote:
> Thanks for pinging. I think we just need to go with the previous
> __STDC_ISO_10646__ macro value for now until there's a new one. I
> don't think it's a conformance problem to support additional new
> characters we don't advertise support for with the macro, so it's not
> a big deal. I can add it as a separate commit and update later with
> another once there's a new value to be used.
>
I also think the value of the macro does not matter much.
> BTW are there any new case mappings needed too? I can handle those if
> so. (The current case mapping code needs manual work to update; I have
> a draft replacement that won't need manual work I should really get
> around to integrating.)
>
There might be! I haven't looked into that.
Kind regards
Lefteris
[-- Attachment #2: Type: text/html, Size: 1534 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-20 15:26 ` Eleftherios Kritikos
@ 2019-10-23 16:21 ` Eleftherios Kritikos
2019-10-25 14:15 ` Rich Felker
0 siblings, 1 reply; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-23 16:21 UTC (permalink / raw)
To: musl, Rich Felker
Hi all,
I wanted to mention that I have used the code for `wcwidth`[1] and for
generating Unicode data tables[2] from musl in the Haskell library
vty[3] (a ncurses style library).
Relevant files in the MR:
* https://github.com/jtdaugherty/vty/pull/179/files#diff-ab3908e00d1c13397ed03e5c2213ad8bR5
* https://github.com/jtdaugherty/vty/pull/179/files#diff-a06fd5aeeca6d7dac0278c2537eb1950R1
* https://github.com/jtdaugherty/vty/pull/179/files#diff-86acb7ffecd1a09c5f55892bd0ce13b1R1
* https://github.com/jtdaugherty/vty/pull/179/files#diff-dc77683ad25ad6f509fb58a397c93f4aR1
* https://github.com/jtdaugherty/vty/pull/179/files#diff-9879d6db96fd29134fc802214163b95aR32
Thanks Rich Felker and everyone else for all the good work that has
gone into musl!
Please let me know if you think attribution was not properly given.
1.http://git.musl-libc.org/cgit/musl/tree/src/ctype/wcwidth.c?id=9b2921bea1d5017832e1b45d1fd64220047a9802
2.https://github.com/richfelker/musl-chartable-tools/tree/master/ctype
3. https://github.com/jtdaugherty/vty
Kind regards
Lefteris
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-23 16:21 ` Eleftherios Kritikos
@ 2019-10-25 14:15 ` Rich Felker
2019-10-25 14:29 ` Eleftherios Kritikos
0 siblings, 1 reply; 12+ messages in thread
From: Rich Felker @ 2019-10-25 14:15 UTC (permalink / raw)
To: musl
On Wed, Oct 23, 2019 at 07:21:35PM +0300, Eleftherios Kritikos wrote:
> Hi all,
>
> I wanted to mention that I have used the code for `wcwidth`[1] and for
> generating Unicode data tables[2] from musl in the Haskell library
> vty[3] (a ncurses style library).
>
> Relevant files in the MR:
> * https://github.com/jtdaugherty/vty/pull/179/files#diff-ab3908e00d1c13397ed03e5c2213ad8bR5
> * https://github.com/jtdaugherty/vty/pull/179/files#diff-a06fd5aeeca6d7dac0278c2537eb1950R1
> * https://github.com/jtdaugherty/vty/pull/179/files#diff-86acb7ffecd1a09c5f55892bd0ce13b1R1
> * https://github.com/jtdaugherty/vty/pull/179/files#diff-dc77683ad25ad6f509fb58a397c93f4aR1
> * https://github.com/jtdaugherty/vty/pull/179/files#diff-9879d6db96fd29134fc802214163b95aR32
>
> Thanks Rich Felker and everyone else for all the good work that has
> gone into musl!
>
> Please let me know if you think attribution was not properly given.
>
> 1.http://git.musl-libc.org/cgit/musl/tree/src/ctype/wcwidth.c?id=9b2921bea1d5017832e1b45d1fd64220047a9802
> 2.https://github.com/richfelker/musl-chartable-tools/tree/master/ctype
> 3. https://github.com/jtdaugherty/vty
Great! I love seeing code/concepts from musl getting adopted elsewhere
especially in places where the classic solutions were all much larger.
Just a quick update on why I haven't merged this yet: I went to do the
case mappings too, and found that at least one range, I believe the
one that would be CASEMAP(0x1c90,0x1cba,0x10d0), is not representable
in the current code that requires updating by hand (it could be done
on a char-by-char basis but continuing to expand that part makes the
file grow larger and slower very quickly).
So, I'm pulling back up the proposed replacement code from April 2018
that never got finished and merged. The old thread is here:
https://www.openwall.com/lists/musl/2018/04/05/1
It's moderately larger -- ~4.8k instead of ~1.5k for Unicode 10 -- but
O(1) rather than O(n) (n = # of case mappings), about 10x faster, and
programmatically generated from UnicodeData.txt. I'll add the (awful,
ugly, just like everything else in musl-chartable-tools) code for
generating the table to musl-chartable-tools when I merge it so it's
not a black box.
I have it working now, so as long as I don't hit any unexpected
problems testing I'll get this (and your patch, and updating case
mappings to Unicode 12) merged soon.
Thanks again for sending the patch and pinging this.
Rich
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Update ctype data to Unicode 12.1.0
2019-10-25 14:15 ` Rich Felker
@ 2019-10-25 14:29 ` Eleftherios Kritikos
0 siblings, 0 replies; 12+ messages in thread
From: Eleftherios Kritikos @ 2019-10-25 14:29 UTC (permalink / raw)
To: musl, Rich Felker
[-- Attachment #1: Type: text/plain, Size: 2805 bytes --]
Thanks for all this good work!
There is no time pressure for me btw as I am not depending on this patch. I
am using musl for static binaries though, and that's great!
On Fri, 25 Oct 2019, 5:15 pm Rich Felker, <dalias@libc.org> wrote:
> On Wed, Oct 23, 2019 at 07:21:35PM +0300, Eleftherios Kritikos wrote:
> > Hi all,
> >
> > I wanted to mention that I have used the code for `wcwidth`[1] and for
> > generating Unicode data tables[2] from musl in the Haskell library
> > vty[3] (a ncurses style library).
> >
> > Relevant files in the MR:
> > *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-ab3908e00d1c13397ed03e5c2213ad8bR5
> > *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-a06fd5aeeca6d7dac0278c2537eb1950R1
> > *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-86acb7ffecd1a09c5f55892bd0ce13b1R1
> > *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-dc77683ad25ad6f509fb58a397c93f4aR1
> > *
> https://github.com/jtdaugherty/vty/pull/179/files#diff-9879d6db96fd29134fc802214163b95aR32
> >
> > Thanks Rich Felker and everyone else for all the good work that has
> > gone into musl!
> >
> > Please let me know if you think attribution was not properly given.
> >
> > 1.
> http://git.musl-libc.org/cgit/musl/tree/src/ctype/wcwidth.c?id=9b2921bea1d5017832e1b45d1fd64220047a9802
> > 2.https://github.com/richfelker/musl-chartable-tools/tree/master/ctype
> > 3. https://github.com/jtdaugherty/vty
>
> Great! I love seeing code/concepts from musl getting adopted elsewhere
> especially in places where the classic solutions were all much larger.
>
> Just a quick update on why I haven't merged this yet: I went to do the
> case mappings too, and found that at least one range, I believe the
> one that would be CASEMAP(0x1c90,0x1cba,0x10d0), is not representable
> in the current code that requires updating by hand (it could be done
> on a char-by-char basis but continuing to expand that part makes the
> file grow larger and slower very quickly).
>
> So, I'm pulling back up the proposed replacement code from April 2018
> that never got finished and merged. The old thread is here:
> https://www.openwall.com/lists/musl/2018/04/05/1
>
> It's moderately larger -- ~4.8k instead of ~1.5k for Unicode 10 -- but
> O(1) rather than O(n) (n = # of case mappings), about 10x faster, and
> programmatically generated from UnicodeData.txt. I'll add the (awful,
> ugly, just like everything else in musl-chartable-tools) code for
> generating the table to musl-chartable-tools when I merge it so it's
> not a black box.
>
> I have it working now, so as long as I don't hit any unexpected
> problems testing I'll get this (and your patch, and updating case
> mappings to Unicode 12) merged soon.
>
> Thanks again for sending the patch and pinging this.
>
> Rich
>
[-- Attachment #2: Type: text/html, Size: 4602 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-10-25 14:29 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-12 21:27 [PATCH] Update ctype data to Unicode 12.1.0 u_quark
2019-10-12 22:39 ` Rich Felker
2019-10-12 22:56 ` Eleftherios Kritikos
2019-10-14 13:07 ` Rich Felker
2019-10-14 13:51 ` Eleftherios Kritikos
2019-10-14 13:57 ` Eleftherios Kritikos
2019-10-20 8:53 ` Eleftherios Kritikos
2019-10-20 14:59 ` Rich Felker
2019-10-20 15:26 ` Eleftherios Kritikos
2019-10-23 16:21 ` Eleftherios Kritikos
2019-10-25 14:15 ` Rich Felker
2019-10-25 14:29 ` Eleftherios Kritikos
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).