[Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Soegtrop, Michael]

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

Dear Ocaml users,

I am working on compiling core with opam on cygwin. My plan is to integrate this with the work of whitequark (MinGW cross opam for Linux) to have a MinGW cross opam on cygwin.

I already have a convenient script to setup cygwin with all modules and to compile opam and it works fine. Just when compiling core, there are two Unix features not implemented in cygwin. One is O_ASYNC. It is easy to patch it away and also easy to make this system dependent in the C file (unix_stubs.c), but I wonder how I could handle this in a portable way in the OCaml wrapper. It looks like this:

module Open_flags = struct
  external append    : unit -> Int63.t = "unix_O_APPEND"
  external async     : unit -> Int63.t = "unix_O_ASYNC"
  external cloexec   : unit -> Int63.t = "unix_O_CLOEXEC"

Is there some way to have async on normal Linux and not have it on cygwin?

Btw.: Opam, batteries and 34 out of the 35 modules of core compile fine on cygwin and the remaining module just has two errors, so I think it is worthwhile to fix this.
Best regards,
Michael

Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

[-- Attachment #2: Type: text/html, Size: 3987 bytes --]

Re: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Yaron Minsky]

Looping in the Core mailing list, since this seems like a core-specific issue.

y

On Fri, May 6, 2016 at 4:48 AM, Soegtrop, Michael
<michael.soegtrop@intel.com> wrote:
> Dear Ocaml users,
>
>
>
> I am working on compiling core with opam on cygwin. My plan is to integrate
> this with the work of whitequark (MinGW cross opam for Linux) to have a
> MinGW cross opam on cygwin.
>
>
>
> I already have a convenient script to setup cygwin with all modules and to
> compile opam and it works fine. Just when compiling core, there are two Unix
> features not implemented in cygwin. One is O_ASYNC. It is easy to patch it
> away and also easy to make this system dependent in the C file
> (unix_stubs.c), but I wonder how I could handle this in a portable way in
> the OCaml wrapper. It looks like this:
>
>
>
> module Open_flags = struct
>
>   external append    : unit -> Int63.t = "unix_O_APPEND"
>
>   external async     : unit -> Int63.t = "unix_O_ASYNC"
>
>   external cloexec   : unit -> Int63.t = "unix_O_CLOEXEC"
>
>
>
> Is there some way to have async on normal Linux and not have it on cygwin?
>
>
>
> Btw.: Opam, batteries and 34 out of the 35 modules of core compile fine on
> cygwin and the remaining module just has two errors, so I think it is
> worthwhile to fix this.
>
> Best regards,
>
> Michael
>
>
>
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de
> Managing Directors: Christin Eisenschmid, Christian Lamprechter
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928

Re: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Jérôme Benoit]

[-- Attachment #1: Type: text/plain, Size: 1375 bytes --]

On Fri, 6 May 2016 10:33:32 -0400 in
<CACLX4jRYhcZ4XZCDKGODYd1PPBcqCcfGUU2aVU=VJskipxiLUg@mail.gmail.com>,
Yaron Minsky <yminsky@janestreet.com> wrote:

> Looping in the Core mailing list, since this seems like a
> core-specific issue.

> > I already have a convenient script to setup cygwin with all modules
> > and to compile opam and it works fine. Just when compiling core,
> > there are two Unix features not implemented in cygwin. One is
> > O_ASYNC. It is easy to patch it away and also easy to make this
> > system dependent in the C file (unix_stubs.c), but I wonder how I
> > could handle this in a portable way in the OCaml wrapper. It looks
> > like this:
> >

Most people that need to make portable code handle flag dependencies on
open() - or other flags on standard functions - in the buildsystem
with a little piece of C code that open() a file created by the
buildsystem with different flags and catch the return code : 

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
...
int retasync = open(/path/to/test/file, O_ASYNC);
...
 
then define a boolean inside the buildsystem to make conditional
compilation based on the boolean. 

Cheers. 

-- 
Jérôme Benoit aka fraggle
Piment Noir - http://piment-noir.org
OpenPGP Key ID : 9FE9161D
Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D

[-- Attachment #2: Signature digitale OpenPGP --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

RE: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Soegtrop, Michael]

Dear Jérôme,

it is easy to detect with preprocessor defines if O_ASYNC is supported or not on the C level. I wondered what would be the most appropriate way to inform OCaml of the situation. I guess I should just call caml_failwith("O_ASYNC not supported by Cygwin") in the async function on cygwin? Maybe a "not supported" standard exception would be a nice thing to handle system dependencies.

Best regards,

Michael
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Jérôme Benoit]

[-- Attachment #1: Type: text/plain, Size: 1350 bytes --]

On Sat, 7 May 2016 08:37:22 +0000 in
<0F7D3B1B3C4B894D824F5B822E3E5A172CEEFECE@IRSMSX102.ger.corp.intel.com>,
"Soegtrop, Michael" <michael.soegtrop@intel.com> wrote:

> Dear Jérôme,
> 
> it is easy to detect with preprocessor defines if O_ASYNC is
> supported or not on the C level. I wondered what would be the most
> appropriate way to inform OCaml of the situation. I guess I should
> just call caml_failwith("O_ASYNC not supported by Cygwin") in the
> async function on cygwin? 

So you want to catch the system call specificities at runtime inside
OCaml ? 

Not sure it's a good idea to maintain a list of supported features, std
functions flags, etc. linked to uname -o output inside OCaml. 

Most FOSS software simply compile out code not supported by system X
and rely on error handling in the underlying layer and propagate the
error number, message in the upper layer.  


> Maybe a "not supported" standard exception
> would be a nice thing to handle system dependencies.


Maybe I fail to understand what you want to achieve, Cygwin do not
report an error that OCaml propagate if you use an unsupported flag
with open() ?

Regards,  

-- 
Jérôme Benoit aka fraggle
Piment Noir - http://piment-noir.org
OpenPGP Key ID : 9FE9161D
Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D

[-- Attachment #2: Signature digitale OpenPGP --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

RE: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Soegtrop, Michael]

Dear Jérôme,

> Maybe I fail to understand what you want to achieve, Cygwin do not report
> an error that OCaml propagate if you use an unsupported flag with open() ?

the issue is that the O_ASYNC flag isn't even defined in the Cygwin headers, so I can't simply pass an unsupported flag and wait for open to flag it. I could invent a value for the flag, but I am not sure if this is a good idea. So I thought the cleanest way is to raise an exception in the function which returns the value of the flag.

Best regards,

Michael
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: [Caml-list] How to handle system decencies in Ocaml (like O_ASYNC not implemented in cygwin)

[Adrien Nader]

On Mon, May 09, 2016, Soegtrop, Michael wrote:
> Dear Jérôme,
> 
> > Maybe I fail to understand what you want to achieve, Cygwin do not report
> > an error that OCaml propagate if you use an unsupported flag with open() ?
> 
> the issue is that the O_ASYNC flag isn't even defined in the Cygwin headers, so I can't simply pass an unsupported flag and wait for open to flag it. I could invent a value for the flag, but I am not sure if this is a good idea. So I thought the cleanest way is to raise an exception in the function which returns the value of the flag.
> 

Considering this is cygwin and therefore also Windows, I wouldn't be
surprised this isn't supported at all and not merely not exposed.
Asynchronous I/O on Windows is profitable only for job safety and
psychiatrists.

At first, I actually had O_NONBLOCK in mind. After some checking I now
see the difference and I really really think you should simply avoid
this.

Since this flag is basically never used, it's probably worth detailing
things a bit. From "man open" on Linux:

  Enable signal-driven I/O: generate a signal (SIGIO  by  default,
  but  this  can  be  changed  via  fcntl(2)) when input or output
  becomes possible on  this  file  descriptor.   This  feature  is
  available  only  for  terminals,  pseudoterminals,  sockets, and
  (since Linux 2.6) pipes and FIFOs.   See  fcntl(2)  for  further
  details.  See also BUGS, below.

  [...]

  Currently, it is not possible to enable signal-driven I/O by
  specifying O_ASYNC when calling open(); use fcntl(2) to enable this
  flag.


And from
http://stackoverflow.com/questions/6260114/whats-the-difference-between-async-and-nonblocking-in-unix-socket :

  O_ASYNC is not used often:
  - it is extremely difficult to properly handle IO in signal handlers;
    they are best left as tiny as possible
  - because signals interrupt the control flow of the program, they
    'cost more' to run than standard system calls, such as select(2) or
    poll(2)
  - signals provide less information than other calls: they only report
    one fd ready vs many fds that might be ready.

This is going to be useful for serial communications. Clearly. You could
even use it for other things like sockets and files but the more work
you'll have to do and the less sane it will be. Two sockets? Sure. But
nothing more.

On Windows you should forget signals besides SIGINT and maybe a couple
others. If you end up requiring Cygwin at runtime, you'll never see any
user because besides license constraints, you're also adding overhead
for syscalls and the likes, which is something one were probably trying
to optimize when settling on O_ASYNC (well, and it's probably going to
be slow).

I'd advise to kill that, preferably with fire. I'm really wondering how
many are using that in Core. I actually expect a few because of the
serial handling stuff but not more. I would move this to a separate
module which is completely optional in Core. Additionally you can also
raise an exception when the function is used; maybe it breaks things at
runtime but the number of users on Windows who haven't already seen all
the warnings signs should be really really low.

-- 
Adrien Nader