From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 67C8B7F75C for ; Mon, 8 Sep 2014 17:57:21 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of anil@recoil.org) identity=pra; client-ip=89.16.177.154; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="anil@recoil.org"; x-sender="anil@recoil.org"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of anil@recoil.org) identity=mailfrom; client-ip=89.16.177.154; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="anil@recoil.org"; x-sender="anil@recoil.org"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@dark.recoil.org) identity=helo; client-ip=89.16.177.154; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="anil@recoil.org"; x-sender="postmaster@dark.recoil.org"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjwFAJ/RDVRZELGa/2dsb2JhbABZg2BXsWKYD4dQAYEoeIQDAQEEATo/EAsYLiEoAQ0GE4guAwkMCbRmDYZcAReKAIMggXozB4MvgR0FlXCEcoIQgV+NFIY5g2I8LwGCTgEBAQ X-IPAS-Result: AjwFAJ/RDVRZELGa/2dsb2JhbABZg2BXsWKYD4dQAYEoeIQDAQEEATo/EAsYLiEoAQ0GE4guAwkMCbRmDYZcAReKAIMggXozB4MvgR0FlXCEcoIQgV+NFIY5g2I8LwGCTgEBAQ X-IronPort-AV: E=Sophos;i="5.04,486,1406584800"; d="scan'208";a="93590826" Received: from recoil.dh.bytemark.co.uk (HELO dark.recoil.org) ([89.16.177.154]) by mail2-smtp-roc.national.inria.fr with SMTP; 08 Sep 2014 17:57:09 +0200 Received: (qmail 1388 invoked by uid 634); 8 Sep 2014 15:56:57 -0000 X-Spam-Level: * X-Spam-Check-By: dark.recoil.org Received: from cpc7-cmbg14-2-0-cust238.5-4.cable.virginm.net (HELO [192.168.1.113]) (86.30.244.239) (smtp-auth username remote@recoil.org, mechanism cram-md5) by dark.recoil.org (qpsmtpd/0.84) with ESMTPA; Mon, 08 Sep 2014 16:56:57 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Anil Madhavapeddy In-Reply-To: Date: Mon, 8 Sep 2014 16:56:52 +0100 Cc: Stephen Dolan , Jiten Pathy , Caml List Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jesper Louis Andersen X-Mailer: Apple Mail (2.1878.6) X-Virus-Checked: Checked by ClamAV on dark.recoil.org Subject: Re: [Caml-list] Multicore runtime On 8 Sep 2014, at 15:49, Jesper Louis Andersen wrote: >=20 > On Mon, Sep 8, 2014 at 12:05 PM, Stephen Dolan wrote: > Generally, we're going to try to avoid blocking C calls and instead > use select/poll/kqueue/epoll to handle blocking I/O. From the fiber's > point of view, this looks like normal blocking I/O, except if a system > call returns EWOULDBLOCK we'll switch to another fiber until the I/O > is ready. >=20 >=20 > The problem is that for file descriptors representing (kernel) file objec= ts, they never return EWOULDBLOCK since you can always write to them (rathe= r subtle semantics). But once you write to them, that thread is off to kern= elspace and you only get it back milliseconds later when the write is compl= ete. In the meantime that thread is placed in uninterruptible sleep. The as= ync File I/O implementations are not portable either, which is why everyone= ends up with having process pools for this or ends up spawning threads. Go= 's rationale is that file I/O is probably going to take so long anyway that= it doesn't really cost too much to create another thread (given that you a= lso run a thread cache of where to hand off the domain). This URI has a hig= h-level overview of the Go style: >=20 > http://morsmachine.dk/go-scheduler >=20 > and there are nice isomorphisms to the concept of fiber (G) and domain (P= ) and thread (M). Indeed, our fiber model is compatible with filesystem I/O, but does not *mandate* that implementations support threads if a better-than-POSIX model is available on the target operating system. Async I/O works fine on Windows for instance, and does not require a thread-per-operation. Much of the platform portability code (which requires excruciating testing and is not something we want to replicate) for async filesystem I/O is already present in libuv upstream: https://github.com/joyent/libuv I'd be most interested in hearing from anyone that would like to work on a Ctypes binding to libuv, since that could provide the basis for an excellent platform-independent I/O library as an alternative to the Unix module. best, Anil=