iconv output formats other than UTF-8

iconv output formats other than UTF-8

[William Pitcock]

Hello,

A common complaint is that applications on Alpine cannot output in
encodings other than UTF-8.

This causes problems with pre-existing IRC networks and other legacy
services where a different text encoding is either preferred or
required.

As such, this makes the iconv in musl deficient for these use cases.

A solution could be to link things like IRC clients against GNU
libiconv instead, which we have done in the past on uClibc, but that
could lead to other problems when a user observes inconsistent
behavior with regards to iconv (one app works, the next app does not).

The better solution would be to somehow enable musl to output in
encodings other than UTF-8.

We are open to contributing patches to enable this, but would like
suggestions on how to proceed.

If it is unwanted work, please let us know, and we will just ship GNU
libiconv instead.

William

Re: iconv output formats other than UTF-8

[Rich Felker]

On Mon, Nov 13, 2017 at 10:32:35PM -0600, William Pitcock wrote:
> Hello,
> 
> A common complaint is that applications on Alpine cannot output in
> encodings other than UTF-8.
> 
> This causes problems with pre-existing IRC networks and other legacy
> services where a different text encoding is either preferred or
> required.
> 
> As such, this makes the iconv in musl deficient for these use cases.
> 
> A solution could be to link things like IRC clients against GNU
> libiconv instead, which we have done in the past on uClibc, but that
> could lead to other problems when a user observes inconsistent
> behavior with regards to iconv (one app works, the next app does not).
> 
> The better solution would be to somehow enable musl to output in
> encodings other than UTF-8.
> 
> We are open to contributing patches to enable this, but would like
> suggestions on how to proceed.
> 
> If it is unwanted work, please let us know, and we will just ship GNU
> libiconv instead.

It's only the legacy cjk encodings that don't havd output support in
iconv. This is because they're sufficiently large that reverse mapping
via linear search would be sliw, likely offensively slow. But it could
be added trivially.

Instead, we're researching how to make the mapping efficient without
huge reverse tables, and I think a good solution will be found.

BTW conversion *from* iso-2022-jp was just added in git master.

Rich