From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 4694E7FE53 for ; Wed, 9 Mar 2016 15:38:25 +0100 (CET) IronPort-PHdr: 9a23:chktOxAZwYlq4w3D2IMwUyQJP3N1i/DPJgcQr6AfoPdwSP7zpsbcNUDSrc9gkEXOFd2CrakU1KyG7eu9BCQp2tWojjMrSNR0TRgLiMEbzUQLIfWuLgnFFsPsdDEwB89YVVVorDmROElRH9viNRWJ+iXhpQAbFhi3DwdpPOO9QteU1JTokbDssMCOKyxzxxODIppKZC2sqgvQssREyaBDEY0WjiXzn31TZu5NznlpL1/A1zz158O34YIxu38I46Fp34d6XK77Z6U1S6BDRHRjajhtpZ6jiR6WYgaV6jMnTmISih9BBQ6NuBD8UJDZvSbguq9mxC6eJcj/S7ZyVTn0vIlxTxq9rS4DPDk99Snyg9B5iKFS6EakohVjyorXaamaMfN/euXWetZMFjkJZdpYSyEUWtD0VIAIFedUeL8A94Q= Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=mmatalka@gmail.com; spf=Pass smtp.mailfrom=mmatalka@gmail.com; spf=None smtp.helo=postmaster@mail-wm0-f44.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.44; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.44 as permitted sender) identity=mailfrom; client-ip=74.125.82.44; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wm0-f44.google.com) identity=helo; client-ip=74.125.82.44; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-wm0-f44.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0B7AADsM+BWiyxSfUpehA1tr22KYwENgWkXCoVuAoFEOBQBAQEBAQEBARABAQEICwsJHzGCLYIUAQEBAwESER0BGx0BAwELBgULAwoCAgUhAgIPAQQPEQEFAQ4BExMih2wBAwoIBAqid4ExPjGLNoFqgleFVAoZJw1Rg3gBAQEBAQEBAQEBAQEBAQEBAQEBEQEBBAoFbYMXggWEQoJagV6CSjgTgScFlzeFZYYbgXKBZIc5MYUujSAvgQ8eAQGCOB6BUGqJUwEBAQ X-IPAS-Result: A0B7AADsM+BWiyxSfUpehA1tr22KYwENgWkXCoVuAoFEOBQBAQEBAQEBARABAQEICwsJHzGCLYIUAQEBAwESER0BGx0BAwELBgULAwoCAgUhAgIPAQQPEQEFAQ4BExMih2wBAwoIBAqid4ExPjGLNoFqgleFVAoZJw1Rg3gBAQEBAQEBAQEBAQEBAQEBAQEBEQEBBAoFbYMXggWEQoJagV6CSjgTgScFlzeFZYYbgXKBZIc5MYUujSAvgQ8eAQGCOB6BUGqJUwEBAQ X-IronPort-AV: E=Sophos;i="5.24,311,1454972400"; d="scan'208";a="206829178" Received: from mail-wm0-f44.google.com ([74.125.82.44]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 09 Mar 2016 15:38:24 +0100 Received: by mail-wm0-f44.google.com with SMTP id p65so73761614wmp.0 for ; Wed, 09 Mar 2016 06:38:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=9DgTt5G/EvUcTpZvWLk61tUVM9VP+rEL4QY5mzxVQX8=; b=NQPHag69QjEkq3S98kBY+Ab8ysMMXdCSqMdQgfvyWOPk3RIYGHObIbVcYppzloVPAV ujCtd8Uz+1yaZFF5BUhAaucjPtRjnRy9x4yRk37bNVO/Cl8ipz8YGp89MwztuZNz3TPA hgZ5R4V3T+yFERoOJzY4SQbrcc087gBt3Bh5rTPgSzDSIRPJ5s8VOwoHQDE3640D32P9 a+JswntCmO9EOjxzCYy+Etjz5AKRf8XlXmoisp8NVAOrTjSraCSXEljrjxzTRkh/nb42 06tSxRC/jSYPU60ZruO6BtTlwM3K0s0F3z0GHj/Jvn8RlStH2IetcoYhA+l+9PFiMENZ XGTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=9DgTt5G/EvUcTpZvWLk61tUVM9VP+rEL4QY5mzxVQX8=; b=eq7eajqYhNml6hEBwb3vZROwDriGkI8JYkGb4y2G8Fusy7kR4XNEZlLmJpi2qsmoJM XgOqOpAFFLFksxu0Qt0g/fa3lnIy8/6+9kljqp7th/qy7y812BzfC5/sX72BS6NwHPBZ F2rSgbJyZb/qSpBqJBreZqY0KfunzBRzWbMj2zrlRUBL2DZzEIvYEZBTaBYiJzcyzlyV W9liv51k0GK+DP7q+sfWAnrhmCsPJTvO+5q7bLtnfK5Ubm5y0TohwJAtkbSz561IsQ0K S9hQGYp5wFc3muN+iA+h6GVRMh68CHwW3vypgJD7cAsLvmf0qnMPtuA6ukXTH7bA+56E A5Ug== X-Gm-Message-State: AD7BkJJ3enprZ0HmRcr48gkN515XG17MH9QjWiJjd2yVeypHQl6iuCHHm8nZbZAykFaPVQ== X-Received: by 10.28.48.137 with SMTP id w131mr26891009wmw.73.1457534304364; Wed, 09 Mar 2016 06:38:24 -0800 (PST) Received: from localhost ([37.153.108.22]) by smtp.gmail.com with ESMTPSA id 63sm8727039wms.1.2016.03.09.06.38.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Mar 2016 06:38:23 -0800 (PST) From: Malcolm Matalka To: Gerd Stolpmann Cc: Jeremie Dimino , Yaron Minsky , Yotam Barnoy , Jesper Louis Andersen , Ocaml Mailing List References: <86h9gi9msc.fsf@gmail.com> <86d1r69ho4.fsf@gmail.com> <868u1ta25r.fsf@gmail.com> <86vb4w85o2.fsf@gmail.com> <1457519028.13223.20.camel@e130.lan.sumadev.de> Date: Wed, 09 Mar 2016 14:37:53 +0000 In-Reply-To: <1457519028.13223.20.camel@e130.lan.sumadev.de> (Gerd Stolpmann's message of "Wed, 09 Mar 2016 11:23:48 +0100") Message-ID: <86mvq790ou.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Caml-list] Question about Lwt/Async Gerd Stolpmann writes: > Am Mittwoch, den 09.03.2016, 07:35 +0000 schrieb Malcolm Matalka: >> Jeremie Dimino writes: >>=20 >> > On Tue, Mar 8, 2016 at 12:47 PM, Yaron Minsky >> > wrote: >> > >> >> Jeremie, other than having some different back-ends available (e.g., = glib >> >> main loop), how different are the approaches to backend management be= tween >> >> Async and Lwt? >> >> >> > >> > =E2=80=8BThe backend interfaces are slightly different=E2=80=8B, but w= e just need a bit of >> > glue in the middle. Essentially the difference is that with Lwt you pr= ovide >> > one callback per fd and watch (read or write), while with Async you ha= ve a >> > global callback. >> > >> > =E2=80=8BRight now what we need to change in Async to make this work i= s: >> > >> > - allow to provide a backend =E2=80=8Bprogrammatically; right now you = can only >> > choose between the predefined epoll and select ones >> > - make the scheduler ignore fds returned by the backend that are not >> > handled by async >>=20 >> For what it's worth, which isn't much right now, I've been slowly >> developing an interface point for event loops and user facing code. The >> rough idea is to present "asynchronous system calls" like an OS would, >> so user facing code has an interface to program against and the >> underlying event loop can change as someone wants, libev, libuv, direct >> epoll or kqueue, etc. So Async and Lwt libraries could be implemented >> in terms of this interface and share the same event loop, to cooperate >> nicely. So far I haven't implemented anything using the interface >> except for a barely functional test to demonstrate that it even works, >> so it's quite raw. And it's clearly deficient on a few things, but I >> think the idea is sound and would alleviate some of the pain of deciding >> to use Lwt or Async and if it works on JS or Windows or My Favorite OS >> (just flip out the underlying scheduler implementation). >>=20 >> The work in progress around the interface can be found below, any >> constructive feedback would be appreciated. >>=20 >> https://bitbucket.org/acslab/abb_scheduler_inf/src >>=20 > > Very academic. The reality is different. Most of these operations are > only provided as synchronous calls anyway in all OS I know (and you can > only provide a non-blocking version by using helper threads). The only > operations you can do something about are those reading/writing a file > descriptor, but even here there is a strong OS dependency, e.g. on > Windows async operations are very restricted, limiting implementation > options drastically. The truth is that you cannot abstract the OS > away. I'm not sure how thoroughly you read the link, but each call takes a callback to indicate when the operation is complete. I put "system call" in quotes because it's attempting to define a what an asynchronous set of system calls would look like. It also defines a "Scheduler" which maps to an event loop. > > And what if I need linkat and not link? And what about calling my > favorite C library that uses blocking I/O? E.g. I'm often preferring a > variant of Unix.read/write using bigarrays as buffer. Deferring something to a thread is lacking currently, one of the "obvious deficiencies" I mentioned. I'm not sure how this is any different from the given state of event loops in Ocaml. > > I'd prefer a reduced approach for interoperability: Focus on event loops > and ways to read/write, and accept that everything else must be dealt > with using helper threads. This is entirely possible given the interface I linked to, I don't see this as being a reason not to like the interface. In fact, the demo implementation of the interface in the link I gave does this, even dirtier. It runs everything in a thread and uses some queues and cond variables to kick off callbacks. > > Sorry for not being constructive. I don't like the approach (and I also > don't like Lwt and Async, and by the way these are not the only kids on > the block). Definitely, I'd like more options, the hope is that an interface like this would provide a meeting point for the various libraries. > > Gerd