RE: [Caml-list] OCaml 4.06.0+rc1

[David Allsopp <dra-news@metastack.com>]

Mattias Engdegård wrote:
> 30 okt. 2017 kl. 19.25 skrev David Allsopp <dra-news@metastack.com>:
> >
> > One of the big changes for the native Windows ports of OCaml is that
> we now have support for UTF-8 filenames in the runtime.
> >
> > TL;DR functions like Sys.readdir no longer return garbage (or at best
> locale-specific strings).
> 
> This sounds like very good news, but just for curiosity, how are
> unpaired surrogates in file names handled? (I have no Windows machine
> handy.) Does readdir raise an exception, or return useless rubbish, or
> somehow encode the surrogates in a form that can be used by other file
> handling functions?

This is a good question - and indeed is a section I should add to the post. At the moment, you will get a Sys_error exception informing you that "No mapping for the Unicode character exists in the target multi-byte code page" - in other words, the runtime refuses to give invalid UTF-8 in this case. 

If you configure OCaml with WINDOWS_UNICODE=0 then, as in previous versions, you will get a nonsense string, but no exception.

We plan in 4.07.0 to address this but it hasn't been dealt with in this release mainly because unpaired surrogates in filenames are a highly parasitic case and also because this issue does not (I believe) only affect Windows.


David