[musl] On musl-locales, gettext, and easying translatable strings detections

[musl] On musl-locales, gettext, and easying translatable strings detections

[Pablo Correa Gomez]

Hello everybody,

I've been working on musl-locales with the general goal of improving
localization in musl. I'd hope that some of the locale-related patches
in the pipeline could be merged if we improve the locales, and prove
that we can maintain them. 

While going through the current status, I've realized that the ".pot"
file contains a multitude of strings that are not supposed to be
localized in musl (e.g: strings in src/misc/fmtmsg.c), and it has
missing some others (like key-related errors from
src/errno/__strerror.h). Manually crafting such a file is complete
madness, and gettext can help here, but we need to be able to provide
it with some way to identify what is a translatable string, as it does
not understand static strings with '\0' as separator. I've come with an
example of something that would work for gettext, but would maintain
the behavior of static strings. Would this be something that might be
considered, given a complete patch is provided? 
          
==============================================================        
diff --git i/src/network/hstrerror.c w/src/network/hstrerror.c
index a4d001c5..c86e3e8f 100644
--- i/src/network/hstrerror.c
+++ w/src/network/hstrerror.c
@@ -3,11 +3,14 @@
 #include "locale_impl.h"
 
 static const char msgs[] =
-       "Host not found\0"
-       "Try again\0"
-       "Non-recoverable error\0"
-       "Address not available\0"
-       "\0Unknown error";
+#define MSG(m) m"\0"
+       MSG("Host not found")
+       MSG("Try again")
+       MSG("Non-recoverable error")
+       MSG("Address not available")
+       "\0"
+       MSG("Unknown error");
+#undef MSG
 
 const char *hstrerror(int ecode)
 {
==============================================================

Would probably mean applying the same logic to the relevant strings in 

 src/network/gai_strerror.c                                           
 src/network/hstrerror.c                                              
 src/regex/regerror.c                                                 
 src/string/strsignal.c                                               
 src/locale/langinfo.c                                                
 
Anything else I might have missed?

Best regards,
Pablo Correa Gomez.

Re: [musl] On musl-locales, gettext, and easying translatable strings detections

[Konstantin P.]

[-- Attachment #1: Type: text/plain, Size: 2600 bytes --]

I am an original author of musl-locales.

1. We need a solution for May month, because in some languages it have more
than 3 letters
2. I crafted those musl-po files mostly manually, with a little help of
gettext
3. We need to be able to translate currency and digital symbol somehow.

Maybe a solution is an embedding a .mo file into musl?

On Tue, Jan 30, 2024 at 2:02 AM Pablo Correa Gomez <
pabloyoyoista@postmarketos.org> wrote:

> Hello everybody,
>
> I've been working on musl-locales with the general goal of improving
> localization in musl. I'd hope that some of the locale-related patches
> in the pipeline could be merged if we improve the locales, and prove
> that we can maintain them.
>
> While going through the current status, I've realized that the ".pot"
> file contains a multitude of strings that are not supposed to be
> localized in musl (e.g: strings in src/misc/fmtmsg.c), and it has
> missing some others (like key-related errors from
> src/errno/__strerror.h). Manually crafting such a file is complete
> madness, and gettext can help here, but we need to be able to provide
> it with some way to identify what is a translatable string, as it does
> not understand static strings with '\0' as separator. I've come with an
> example of something that would work for gettext, but would maintain
> the behavior of static strings. Would this be something that might be
> considered, given a complete patch is provided?
>
> ==============================================================
> diff --git i/src/network/hstrerror.c w/src/network/hstrerror.c
> index a4d001c5..c86e3e8f 100644
> --- i/src/network/hstrerror.c
> +++ w/src/network/hstrerror.c
> @@ -3,11 +3,14 @@
>  #include "locale_impl.h"
>
>  static const char msgs[] =
> -       "Host not found\0"
> -       "Try again\0"
> -       "Non-recoverable error\0"
> -       "Address not available\0"
> -       "\0Unknown error";
> +#define MSG(m) m"\0"
> +       MSG("Host not found")
> +       MSG("Try again")
> +       MSG("Non-recoverable error")
> +       MSG("Address not available")
> +       "\0"
> +       MSG("Unknown error");
> +#undef MSG
>
>  const char *hstrerror(int ecode)
>  {
> ==============================================================
>
> Would probably mean applying the same logic to the relevant strings in
>
>  src/network/gai_strerror.c
>  src/network/hstrerror.c
>  src/regex/regerror.c
>  src/string/strsignal.c
>  src/locale/langinfo.c
>
> Anything else I might have missed?
>
> Best regards,
> Pablo Correa Gomez.
>

[-- Attachment #2: Type: text/html, Size: 3682 bytes --]

Re: [musl] On musl-locales, gettext, and easying translatable strings detections

[Pablo Correa Gomez]

Hi Konstantin,

Thank you very much for your input here.

I guess that might be an option. The other alternative would be to
maybe have some place-holder values? And change a bit some of the logic
so that if no message is found, the placeholders are not returned? 

In any case, I'm happy to look into implementing some of this
solutions, but need feedback mostly from Rich into what could possibly
be accepted into musl. So I can know in which directions would be worth
to investigate.

Best,
Pablo.

El mar, 30-01-2024 a las 15:12 +0300, Konstantin P. escribió:
> I am an original author of musl-locales.
> 
> 1. We need a solution for May month, because in some languages it
> have more than 3 letters
> 2. I crafted those musl-po files mostly manually, with a little help
> of gettext
> 3. We need to be able to translate currency and digital symbol
> somehow.
> 
> Maybe a solution is an embedding a .mo file into musl?
> 
> On Tue, Jan 30, 2024 at 2:02 AM Pablo Correa Gomez
> <pabloyoyoista@postmarketos.org> wrote:
> > Hello everybody,
> > 
> > I've been working on musl-locales with the general goal of
> > improving
> > localization in musl. I'd hope that some of the locale-related
> > patches
> > in the pipeline could be merged if we improve the locales, and
> > prove
> > that we can maintain them. 
> > 
> > While going through the current status, I've realized that the
> > ".pot"
> > file contains a multitude of strings that are not supposed to be
> > localized in musl (e.g: strings in src/misc/fmtmsg.c), and it has
> > missing some others (like key-related errors from
> > src/errno/__strerror.h). Manually crafting such a file is complete
> > madness, and gettext can help here, but we need to be able to
> > provide
> > it with some way to identify what is a translatable string, as it
> > does
> > not understand static strings with '\0' as separator. I've come
> > with an
> > example of something that would work for gettext, but would
> > maintain
> > the behavior of static strings. Would this be something that might
> > be
> > considered, given a complete patch is provided? 
> >           
> > ==============================================================     
> >   
> > diff --git i/src/network/hstrerror.c w/src/network/hstrerror.c
> > index a4d001c5..c86e3e8f 100644
> > --- i/src/network/hstrerror.c
> > +++ w/src/network/hstrerror.c
> > @@ -3,11 +3,14 @@
> >  #include "locale_impl.h"
> > 
> >  static const char msgs[] =
> > -       "Host not found\0"
> > -       "Try again\0"
> > -       "Non-recoverable error\0"
> > -       "Address not available\0"
> > -       "\0Unknown error";
> > +#define MSG(m) m"\0"
> > +       MSG("Host not found")
> > +       MSG("Try again")
> > +       MSG("Non-recoverable error")
> > +       MSG("Address not available")
> > +       "\0"
> > +       MSG("Unknown error");
> > +#undef MSG
> > 
> >  const char *hstrerror(int ecode)
> >  {
> > ==============================================================
> > 
> > Would probably mean applying the same logic to the relevant strings
> > in 
> > 
> >  src/network/gai_strerror.c                                       
> >    
> >  src/network/hstrerror.c                                           
> >   
> >  src/regex/regerror.c                                             
> >    
> >  src/string/strsignal.c                                           
> >    
> >  src/locale/langinfo.c                                             
> >   
> > 
> > Anything else I might have missed?
> > 
> > Best regards,
> > Pablo Correa Gomez.