[ruby-core:120279] [Ruby master Feature#20959] Add a way to get codepage of console.

[ruby-core:120279] [Ruby master Feature#20959] Add a way to get codepage of console.

[YO4 (Yoshinao Muramatsu) via ruby-core]

Issue #20959 has been reported by YO4 (Yoshinao Muramatsu).

----------------------------------------
Feature #20959: Add a way to get codepage of console.
https://bugs.ruby-lang.org/issues/20959

* Author: YO4 (Yoshinao Muramatsu)
* Status: Open
----------------------------------------
# Abstract

Add a way to retrieve code pages of console.
On Windows, Encoding.find("locale") returns the console codepage.
To prepare for future changes, specify ```console``` instead of ```locale``` to get the encoding when a console code page is needed.

# Background

On Windows, Encoding.find("locale") returns the console codepage.
This is different from locale in other environments. Also the name and content do not seem to match.
In the future, if we change the locale_encoding of the Windows port to the locale codepage of the C runtime library, we need to get the encoding for the console.
Strings received via pipe from cmd.exe or powershell are encoded in the console codepage.
This would be necessary when communicating with other programs via pipes.

# Proposal

Make Encoding.find("console") return the encoding that represents the console codepage.

# Background(continued)

Since Windows 10, UTF-8 support seems to be enhanced in the commandline environment.
* build 17134
  ucrt supports UTF-8 locale codepage ```setlocale(LC_CTYPE, ".utf8")```
  Windows support UTF-8 ANSI Codepage(experimental) ```Beta: Use Unicode UTF-8 for worldwide language support```
* build 18362
  support ```Set a process code page to UTF-8``` via manifest.
* build 19041
  Time zone name holds in wchar_t internaly, so proper tzname can be obtained regardless of the Windows language setting.

This should not be all.
Through these, I feel that the Microsoft team recommends the use of utf-8. It also seems that Microsoft, which has previously depricated the ANSI version of the API, is treating it as a valid alternative to use in UTF-8.
If it becomes widespread on Windows for libraries to respect the C runtime library locale, ruby would do better to follow it.

Of course, it is good to be able to get the encoding of the console by specifying a console.

# Future plan

By reducing differences from other platforms, bugs and extra code are hoped to be reduced.
* Do ```setlocale(LC_CTYPE, ".utf8");``` in main.c (or refer LC_* environment variables).
* ```Encoding.find("locale")``` returns C runtime library locale.

Since strings obtained from the Windows system have a Unicode code range, the API to obtain fixed UTF-8 encoding remains unchanged.

# Discussion

The code page we can get from Windows also has ACP and OEMCP, but are these necessary?
Is it reasonable to get locale_encoding if Encoding.find("console") is called on other platforms?




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:120402] [Ruby master Feature#20959] Add a way to get codepage of console.

[nobu (Nobuyoshi Nakada) via ruby-core]

Issue #20959 has been updated by nobu (Nobuyoshi Nakada).

Assignee set to windows
Target version set to 3.5

----------------------------------------
Feature #20959: Add a way to get codepage of console.
https://bugs.ruby-lang.org/issues/20959#change-111182

* Author: YO4 (Yoshinao Muramatsu)
* Status: Open
* Assignee: windows
* Target version: 3.5
----------------------------------------
# Abstract

Add a way to retrieve code pages of console.
On Windows, Encoding.find("locale") returns the console codepage.
To prepare for future changes, specify ```console``` instead of ```locale``` to get the encoding when a console code page is needed.

# Background

On Windows, Encoding.find("locale") returns the console codepage.
This is different from locale in other environments. Also the name and content do not seem to match.
In the future, if we change the locale_encoding of the Windows port to the locale codepage of the C runtime library, we need to get the encoding for the console.
Strings received via pipe from cmd.exe or powershell are encoded in the console codepage.
This would be necessary when communicating with other programs via pipes.

# Proposal

Make Encoding.find("console") return the encoding that represents the console codepage.

# Background(continued)

Since Windows 10, UTF-8 support seems to be enhanced in the commandline environment.
* build 17134
  ucrt supports UTF-8 locale codepage ```setlocale(LC_CTYPE, ".utf8")```
  Windows support UTF-8 ANSI Codepage(experimental) ```Beta: Use Unicode UTF-8 for worldwide language support```
* build 18362
  support ```Set a process code page to UTF-8``` via manifest.
* build 19041
  Time zone name holds in wchar_t internaly, so proper tzname can be obtained regardless of the Windows language setting.

This should not be all.
Through these, I feel that the Microsoft team recommends the use of utf-8. It also seems that Microsoft, which has previously depricated the ANSI version of the API, is treating it as a valid alternative to use in UTF-8.
If it becomes widespread on Windows for libraries to respect the C runtime library locale, ruby would do better to follow it.

Of course, it is good to be able to get the encoding of the console by specifying a console.

# Future plan

By reducing differences from other platforms, bugs and extra code are hoped to be reduced.
* Do ```setlocale(LC_CTYPE, ".utf8");``` in main.c (or refer LC_* environment variables).
* ```Encoding.find("locale")``` returns C runtime library locale.

Since strings obtained from the Windows system have a Unicode code range, the API to obtain fixed UTF-8 encoding remains unchanged.

# Discussion

The code page we can get from Windows also has ACP and OEMCP, but are these necessary?
Is it reasonable to get locale_encoding if Encoding.find("console") is called on other platforms?




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/