Re: [Caml-list] Multicore runtime

[Anil Madhavapeddy <anil@recoil.org>]

On 8 Sep 2014, at 15:49, Jesper Louis Andersen <jesper.louis.andersen@gmail.com> wrote:

> 
> On Mon, Sep 8, 2014 at 12:05 PM, Stephen Dolan <stephen.dolan@cl.cam.ac.uk> wrote:
> Generally, we're going to try to avoid blocking C calls and instead
> use select/poll/kqueue/epoll to handle blocking I/O. From the fiber's
> point of view, this looks like normal blocking I/O, except if a system
> call returns EWOULDBLOCK we'll switch to another fiber until the I/O
> is ready.
> 
> 
> The problem is that for file descriptors representing (kernel) file objects, they never return EWOULDBLOCK since you can always write to them (rather subtle semantics). But once you write to them, that thread is off to kernelspace and you only get it back milliseconds later when the write is complete. In the meantime that thread is placed in uninterruptible sleep. The async File I/O implementations are not portable either, which is why everyone ends up with having process pools for this or ends up spawning threads. Go's rationale is that file I/O is probably going to take so long anyway that it doesn't really cost too much to create another thread (given that you also run a thread cache of where to hand off the domain). This URI has a high-level overview of the Go style:
> 
> http://morsmachine.dk/go-scheduler
> 
> and there are nice isomorphisms to the concept of fiber (G) and domain (P) and thread (M).


Indeed, our fiber model is compatible with filesystem I/O, but does not
*mandate* that implementations support threads if a better-than-POSIX
model is available on the target operating system.  Async I/O works fine
on Windows for instance, and does not require a thread-per-operation.

Much of the platform portability code (which requires excruciating testing
and is not something we want to replicate) for async filesystem I/O is
already present in libuv upstream:

  https://github.com/joyent/libuv

I'd be most interested in hearing from anyone that would like to work on
a Ctypes binding to libuv, since that could provide the basis for an
excellent platform-independent I/O library as an alternative to the Unix
module.

best,
Anil