From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id B967F7FF9F for ; Mon, 7 Mar 2016 19:16:28 +0100 (CET) IronPort-PHdr: 9a23:x0GY9xd/v3qKFK1F71dl27NSlGMj4u6mDksu8pMizoh2WeGdxc6yZR7h7PlgxGXEQZ/co6odzbGG7OawACdZuczJmUtBWaIPfidNsd8RkQ0kDZzNImzAB9muURYHGt9fXkRu5XCxPBsdMs//Y1rPvi/6tmZKSV3BPAZ4bt74BpTVx5zukbvipNuDOk4R3GD1SIgxBSv1hD2ZjtMRj4pmJ/R54TryiVwMRd5rw3h1L0mYhRf265T41pdi9yNNp6BprJYYAu3SNp41Rr1ADTkgL3t9pIiy7UGCHkOz4S4tW38RlFJtAg7e7wCyCob0sy3htftV2iCcMNbqV705RXKp6KI9GzHyjyJSEjc9+2bTj4RVhb5SpBGo70h6xofIaYWWPdJxe6rceZURQm8XDZUZbDBIHo7pN9hHNOEGJ+sN6tCl/1Y= Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=mmatalka@gmail.com; spf=Pass smtp.mailfrom=mmatalka@gmail.com; spf=None smtp.helo=postmaster@mail-wm0-f50.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.50; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.50 as permitted sender) identity=mailfrom; client-ip=74.125.82.50; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wm0-f50.google.com) identity=helo; client-ip=74.125.82.50; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-wm0-f50.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BUAAAvxN1WjjJSfUpchAxtukABDYFpFwaFcgKBKzgUAQEBAQEBAQEQAQEBAQcLCwkfMYItghQBAQEDARIuARsdAQMBCwYFCwMKCSUPAQQNAhEBBQEOARMTGweHbAEDCggECqFxgTE+MY0ggleFKwoZJw1Rg3EBAQEBAQEBAQIBAQEBAQEBARIBAQQKBIQEggWEPYI9gVeCbkuBJwWXKoVjhhWBdYkbhV+HDIYLL4EPHgEBgjgNEQiBSGoBiAGBOwEBAQ X-IPAS-Result: A0BUAAAvxN1WjjJSfUpchAxtukABDYFpFwaFcgKBKzgUAQEBAQEBAQEQAQEBAQcLCwkfMYItghQBAQEDARIuARsdAQMBCwYFCwMKCSUPAQQNAhEBBQEOARMTGweHbAEDCggECqFxgTE+MY0ggleFKwoZJw1Rg3EBAQEBAQEBAQIBAQEBAQEBARIBAQQKBIQEggWEPYI9gVeCbkuBJwWXKoVjhhWBdYkbhV+HDIYLL4EPHgEBgjgNEQiBSGoBiAGBOwEBAQ X-IronPort-AV: E=Sophos;i="5.22,552,1449529200"; d="scan'208";a="206433019" Received: from mail-wm0-f50.google.com ([74.125.82.50]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 07 Mar 2016 19:16:27 +0100 Received: by mail-wm0-f50.google.com with SMTP id p65so119432803wmp.1 for ; Mon, 07 Mar 2016 10:16:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=twUmslcjKETVerlJwNiEJIb6Mij6JWhGPPbwUP99m9c=; b=IrocEVPiqy3KRuVW1DPpMezj0fOOA1qWUmnl1kGgPtPdl+e/ftNWr3FkRJKpfXr5EJ /hIuD8ZACw/QQMTG2iKv7dbS0nPAJ159ZW8syj7Mw/Ta9aR6oUA8cUZ3sjpM6Zz3roND RBdFwbaUI1G/4M3OAFp8s5VwHuFWKKzqmNYOlcMzOAs5x26z9GOLGXI8BJNMumLAlSTb Pr2pXFEPnjEts69yaZX3TCwMH1NfoXsnO1JDTL1z+OKPETKDN9OOZOc9ZCtZLQ/vELTV kILIPWK4ka8PR9aLRTjDa303Z/2SBaqHy03bj+GoqnLQwz7y+oxT3gudFx3UgClblrjZ 8WJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=twUmslcjKETVerlJwNiEJIb6Mij6JWhGPPbwUP99m9c=; b=FmXtvMvqrJFMBInWIi1KFiakDWArMrC72q1iEskW2Au/Cc5549tGwIGuCSBhalW4zK C1+IZC/q7HaZ3OLHGG9rsVdQIXpRiSAPPdA3tzyTkYxffsv2Bhk09qO33u1CINH//pfC m24ZkrJX/F/0j9JKl6paArpJTbKRIkqBHEXoz3i+IQdJw/zfIrOYkySaEs2svW8VodVQ +oCCVBtEqIaLauZF8Poj2ZTguVeBjWAkQHcTHIHpC1C9Py9OwsiDA8rAMnw45svAGAyN UORyzx9v1b6NkhsghQR00HDnH7L+VU0kIxBlnLiIUsF2tw7CoP6AsnIrR/m2JLfobPW2 CgnQ== X-Gm-Message-State: AD7BkJJqQRDOIlfDYBiR+OvXV6O+7k3+2JZhBKwJXiUFZKT888v52K2/KSO32AV8fccDrA== X-Received: by 10.28.20.145 with SMTP id 139mr15021818wmu.76.1457374587497; Mon, 07 Mar 2016 10:16:27 -0800 (PST) Received: from localhost ([37.153.108.22]) by smtp.gmail.com with ESMTPSA id lh1sm19056035wjb.20.2016.03.07.10.16.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Mar 2016 10:16:26 -0800 (PST) From: Malcolm Matalka To: Yaron Minsky Cc: Jesper Louis Andersen , Yotam Barnoy , Ocaml Mailing List References: Date: Mon, 07 Mar 2016 18:16:03 +0000 In-Reply-To: (Yaron Minsky's message of "Mon, 7 Mar 2016 12:03:44 -0500") Message-ID: <86h9gi9msc.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Caml-list] Question about Lwt/Async Yaron Minsky writes: > This is definitely a fraught topic, and it's unfortunate that there's > no clear solution. > > To add a bit more information: > > - Async is more portable than it once was. There's now Core_kernel, > Async_kernel and Async_rpc_kernel, which allows us to do things like > run Async applications in the browser. I would think Windows > support would be pretty doable by someone who understands that world > well. > > That said, the chain of dependencies brought in by Async is still > quite big. This is something that could perhaps be improved, either > with better dead code analysis in OCaml, or some tweaks to > Async_kernel and Core_kernel themselves. When I last looked at the scheduler it was limited to [select] or [epoll], is this still the case? How difficult would it be to expand on those? > > - There are things we could contemplate to make it easier to bridge > the divide. Jeremie Dimino did a proof of concept rewrite of lwt to > use async as its implementation, where an Lwt.t and a Deferred.t are > equal at the type level. > > https://github.com/janestreet/lwt-async > > Another possibility, and one that might be easier to write, would be > to allow Lwt code to run using the Async scheduler as another > possible back-end. This would allow one to have programs that used > both Async and Lwt together in one program, without running on > different threads. > > It's worth mentioning if that there is interest in making Async more > suitable for a wider variety of goals, we're happy to work with > outside contributors on it. For example, if someone wanted to work on > Windows support for Async, we'd be happy to help out on integrating > that work. > > Probably the biggest issue is executable size. That will get better > when we release an unpacked version of our external libraries. But > even then, the module-level granularity captures more things than > would be ideal. > > y > > On Mon, Mar 7, 2016 at 10:16 AM, Jesper Louis Andersen > wrote: >> >> On Mon, Mar 7, 2016 at 2:38 AM, Yotam Barnoy wrote: >>> >>> Also, what happens to general utility functions that aren't rewritten for >>> Async/Lwt -- as far as I can tell, being in non-monadic code, they will >>> always starve other threads, since they cannot yield to another Async/Lwt >>> thread. Is this perception correct? >> >> >> Yes. >> >> On one hand, your observation is negative in the sense that now your code >> has "color" in the sense that it is written for one library only. And you >> have to transform code to having the right color before it can be used. This >> is not the case if the concurrency model is at a lower level[0]. >> >> On the other hand, your observation is positive: cooperative scheduling >> makes the points in which the code can switch explicit. This gives the >> programmer far more control over when you are done with a task and start to >> process the next task. You can also avoid the preemption check in the code >> all the time. If your code manipulates lots of shared data, it also >> simplifies things since you don't usually have to protect data with a mutex >> in a single-threaded context as much[1]. Cooperative models, if carefully >> managed, can exploit structure in the problem domain, whereas a preemptive >> model needs to fit all. >> >> My personal opinion is that the preemptive model eventually wins over the >> cooperative model, much like it has in most (all popular) operating systems. >> It is simply more productive to take an up-front performance hit as a >> sacrifice for a system which is more robust against stray code misbehaving. >> If a cooperative system fails, it is fails catastrophically. If a preemptive >> system fails, it degrades in performance. >> >> But given I have more than 10 years of Erlang programming behind me by now, >> I'm obviously biased toward certain computational models :) >> >> [0] Erlang would be one such example, where the system is preemptively >> scheduling for you and you can use any code in any place without having to >> worry about blocking for latency. Go is quasi-preemptive because it checks >> on function calls, but in contrast to Erlang a loop is not forced to factor >> through a recursion, so it can in principle run indefinitely. Haskell (GHC) >> is quasi-preemptive as well, checking on memory allocation boundaries. So >> the thing to look out for in GHC is latency from processing large arrays >> with no allocation, say. >> >> [1] Erlang has two VM runtimes for this reason. One is single-threaded and >> can avoid lots of locks which is far faster for certain workloads, or on >> embedded devices with a single core only. >> >> -- >> J.