From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 4EFD87FACC for ; Mon, 8 Sep 2014 16:49:40 +0200 (CEST) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of jesper.louis.andersen@gmail.com) identity=pra; client-ip=209.85.223.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="jesper.louis.andersen@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of jesper.louis.andersen@gmail.com designates 209.85.223.174 as permitted sender) identity=mailfrom; client-ip=209.85.223.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="jesper.louis.andersen@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ie0-f174.google.com) identity=helo; client-ip=209.85.223.174; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="postmaster@mail-ie0-f174.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqMBALfADVTRVd+um2dsb2JhbABZDoNSVwSCeLEEg3oIkW6HUAGBCQgWEAEBAQEBBgsLCRQqhAQBAQMBEhEdARsdAQMBCwYFBAc3AgIiAREBBQEcBhMiiAsBAwkIDZlta4swgXKDEIh6ChknDWaFRQERAQUOjz8HgnmBUwWSQYMvhwKBX5FbGCmEUEQ7LwGCTgEBAQ X-IPAS-Result: AqMBALfADVTRVd+um2dsb2JhbABZDoNSVwSCeLEEg3oIkW6HUAGBCQgWEAEBAQEBBgsLCRQqhAQBAQMBEhEdARsdAQMBCwYFBAc3AgIiAREBBQEcBhMiiAsBAwkIDZlta4swgXKDEIh6ChknDWaFRQERAQUOjz8HgnmBUwWSQYMvhwKBX5FbGCmEUEQ7LwGCTgEBAQ X-IronPort-AV: E=Sophos;i="5.04,486,1406584800"; d="scan'208";a="78183169" Received: from mail-ie0-f174.google.com ([209.85.223.174]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 08 Sep 2014 16:49:39 +0200 Received: by mail-ie0-f174.google.com with SMTP id at20so18046054iec.33 for ; Mon, 08 Sep 2014 07:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Ym7GnR8OCiBT9T1r1nQLgvsXUZf0HR5zjzRwidj5ImQ=; b=mRoYjIUhAkzsLPkgSjKzj3Dmnng7eYM5BzKpMcYWMUrdoc5jTs++K5DK7hJod8sW4E bspijzGOyNAZOWWWug6UXQywhLRnwUrOOqkRn+K9u1jN7BpAouotD/PA/KyeHGQN+AF9 0qepyuc/+p1uc0O13E6h7ZleJy7fRD4oKWRzJhcvVL2dab2TRMedSHuHWln1Jh7bvNtm Ny5viXYn0I67ox6Rwlo4B6b5th34tyKRYtEXUlvXUcXnS1ytCrh9HjLYZko+JZ+kwJWo trLLzM8Zouzx/2F0qLwf3pwB9WgVXOuLMWj83r70/b1L8WcnSi/2y7zaMB/z9FASX+Hv 7xzQ== X-Received: by 10.42.204.76 with SMTP id fl12mr1983935icb.80.1410187777610; Mon, 08 Sep 2014 07:49:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.57.230 with HTTP; Mon, 8 Sep 2014 07:49:22 -0700 (PDT) In-Reply-To: References: From: Jesper Louis Andersen Date: Mon, 8 Sep 2014 16:49:22 +0200 Message-ID: To: Stephen Dolan Cc: Jiten Pathy , Caml List Content-Type: multipart/alternative; boundary=20cf3040e2d68d1bce05028eeda9 Subject: Re: [Caml-list] Multicore runtime --20cf3040e2d68d1bce05028eeda9 Content-Type: text/plain; charset=UTF-8 On Mon, Sep 8, 2014 at 12:05 PM, Stephen Dolan wrote: > Generally, we're going to try to avoid blocking C calls and instead > use select/poll/kqueue/epoll to handle blocking I/O. From the fiber's > point of view, this looks like normal blocking I/O, except if a system > call returns EWOULDBLOCK we'll switch to another fiber until the I/O > is ready. > This doesn't work for a (portable) implementation of File I/O which is why everyone else (Go, Erlang, Haskell, ...) either has a pool of threads for handling these (Erlang), or they just create another thread for handling it, up to a point (Go). Network I/O is simple however, since you can batch those on a standard epoll/kqueue/poll/select loop. The problem is that for file descriptors representing (kernel) file objects, they never return EWOULDBLOCK since you can always write to them (rather subtle semantics). But once you write to them, that thread is off to kernelspace and you only get it back milliseconds later when the write is complete. In the meantime that thread is placed in uninterruptible sleep. The async File I/O implementations are not portable either, which is why everyone ends up with having process pools for this or ends up spawning threads. Go's rationale is that file I/O is probably going to take so long anyway that it doesn't really cost too much to create another thread (given that you also run a thread cache of where to hand off the domain). This URI has a high-level overview of the Go style: http://morsmachine.dk/go-scheduler and there are nice isomorphisms to the concept of fiber (G) and domain (P) and thread (M). -- J. --20cf3040e2d68d1bce05028eeda9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On Mon, Sep 8, 2014 at 12:05 PM, Stephen Dolan <stephen.dolan@cl.= cam.ac.uk> wrote:
Generally, we're going to try to avoid = blocking C calls and instead
use select/poll/kqueue/epoll to handle blocking I/O. From the fiber's point of view, this looks like normal blocking I/O, except if a system
call returns EWOULDBLOCK we'll switch to another fiber until the I/O
is ready.

This doesn't work for a (port= able) implementation of File I/O which is why everyone else (Go, Erlang, Ha= skell, ...) either has a pool of threads for handling these (Erlang), or th= ey just create another thread for handling it, up to a point (Go). Network = I/O is simple however, since you can batch those on a standard epoll/kqueue= /poll/select loop.

The problem is that for file descriptors representing (kernel)= file objects, they never return EWOULDBLOCK since you can always write to = them (rather subtle semantics). But once you write to them, that thread is = off to kernelspace and you only get it back milliseconds later when the wri= te is complete. In the meantime that thread is placed in uninterruptible sl= eep. The async File I/O implementations are not portable either, which is w= hy everyone ends up with having process pools for this or ends up spawning = threads. Go's rationale is that file I/O is probably going to take so l= ong anyway that it doesn't really cost too much to create another threa= d (given that you also run a thread cache of where to hand off the domain).= This URI has a high-level overview of the Go style:

http://morsmachine.dk/go-scheduler

and there are nice is= omorphisms to the concept of fiber (G) and domain (P) and thread (M).

--
J.
--20cf3040e2d68d1bce05028eeda9--