From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id B777B7FF9F for ; Mon, 7 Mar 2016 21:07:00 +0100 (CET) IronPort-PHdr: 9a23:sSsCEh1mk5r/rHGqsmDT+DRfVm0co7zxezQtwd8ZsegeK/ad9pjvdHbS+e9qxAeQG96LtLQU0qGP6PiocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC0ILnjqvroMWbSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoJvvRNWqTifqk+UacQTHF/azh0t4XXskzyRBGI4DM5U2MNkQsAVxnA7RfhXYbZsCL8u/FhwiSXIYv9SrViChq46KI+bRbsgyADMnYc+X3ejs95xPZepRu9rhh8yqbbZYiUMLx1eaaLLoBSfnZIQssED38JOYi7dYZaSrNZZes= Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=mmatalka@gmail.com; spf=Pass smtp.mailfrom=mmatalka@gmail.com; spf=None smtp.helo=postmaster@mail-wm0-f46.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.46 as permitted sender) identity=mailfrom; client-ip=74.125.82.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wm0-f46.google.com) identity=helo; client-ip=74.125.82.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-wm0-f46.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CEAAAE3t1Wji5SfUpchAxtukABDYFpFwaFcgKBLjgUAQEBAQEBAQEQAQEBAQcLCwkfMYItghQBAQEDARIuARsPDgEDAQsGBQsDCgklDwEEDQIRAQUBDgETExsHh2wBAwoIBAqhXoExPjGNIIJXhT0KGScNUYNxAQEBAQEBAQECAQEBAQEBAQESAQEECgSEBIIFhD2CPYFXgm5LgScFlyqFY4YVgXWJG4VfhwyGCy+BDx4BAYI4DREIgUhqAYgBgTsBAQE X-IPAS-Result: A0CEAAAE3t1Wji5SfUpchAxtukABDYFpFwaFcgKBLjgUAQEBAQEBAQEQAQEBAQcLCwkfMYItghQBAQEDARIuARsPDgEDAQsGBQsDCgklDwEEDQIRAQUBDgETExsHh2wBAwoIBAqhXoExPjGNIIJXhT0KGScNUYNxAQEBAQEBAQECAQEBAQEBAQESAQEECgSEBIIFhD2CPYFXgm5LgScFlyqFY4YVgXWJG4VfhwyGCy+BDx4BAYI4DREIgUhqAYgBgTsBAQE X-IronPort-AV: E=Sophos;i="5.22,553,1449529200"; d="scan'208";a="206446818" Received: from mail-wm0-f46.google.com ([74.125.82.46]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 07 Mar 2016 21:06:59 +0100 Received: by mail-wm0-f46.google.com with SMTP id n186so102510422wmn.1 for ; Mon, 07 Mar 2016 12:06:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=0Gj6ASH2K7sktsUollVU/21JcYd12qsy4FYKPfadxEo=; b=ygl8qoE68PGKv8aX6YzhUEmUjqLaPp7202GqOt3Kegy5hPZFdqquh0XF3XcqdaR/Px cQsBigxogYrwUH3JAEBiJTA7smfHFSw4KYljXGjw7sgECPKfnKEoDcl7brk78JSCPhHT RQ7nr25ZDKOlBQxXNZhWxkqummf6bjCOWZJjeCNL5vQ+2g6lQV4sZj8zNNi5QNFIw0PA jRU8hzyEjqkW8ait/28Aja789BRsyyyGrRyxqm7fcpjW6H3f5VQvjrydlhpq9ZUI9m4E R8PRXNsuekuFUWCn8nhcOrih2g7RWYpaqjuLrdOBtZy5Yq1c8U0rcapmXKlCETvj3CQ/ tZtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=0Gj6ASH2K7sktsUollVU/21JcYd12qsy4FYKPfadxEo=; b=N9USVvvuS3aart1EDTaDxHJQP9wDuNHwOoZfxJl5IlD/9WSg4x7RcPu6YBqyISb3O6 T5PeNWxvqICbYgXxBBJj1QaPsRrOC4Dp9ajrqdQT+f2aJ2IZayFqfSkuhkPQN2xZmXXU YYOfbMJXJcAGVkS7FzchlXP3uKw7tYpHc4KsJT3n7WpvQIfdoxMmo6bzlJaIayCQcEL1 FGGW6H/k4JMZ6UfAllQpG4kl2APOVCUzQVIk+6UmLa48X7+9JE3A3IX0Ieh2mAHNnv6l TuVh51TjjB9tTBaxSIJVGodikxHe43zOjIdTu41pRsSy75dD0tX2UC4sLaXQoaIidhuH i2tA== X-Gm-Message-State: AD7BkJI7NPBjpy+to95x+NgFxDatoTulVzrAvSjNr5GCK2ba06O6m1oF6f0671X52oNYpw== X-Received: by 10.194.84.171 with SMTP id a11mr23848323wjz.91.1457381219681; Mon, 07 Mar 2016 12:06:59 -0800 (PST) Received: from localhost ([37.153.108.22]) by smtp.gmail.com with ESMTPSA id hq2sm19623929wjb.3.2016.03.07.12.06.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Mar 2016 12:06:58 -0800 (PST) From: Malcolm Matalka To: Yaron Minsky Cc: Jesper Louis Andersen , Yotam Barnoy , Ocaml Mailing List References: <86h9gi9msc.fsf@gmail.com> Date: Mon, 07 Mar 2016 20:06:35 +0000 In-Reply-To: (Yaron Minsky's message of "Mon, 7 Mar 2016 13:41:22 -0500") Message-ID: <86d1r69ho4.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Caml-list] Question about Lwt/Async Yaron Minsky writes: > Right now, only select and epoll are supported, but adding support for > something else isn't hard. The Async_unix library has an interface > called File_descr_watcher_intf.S, which both select and epoll go > through. Adding support for another shouldn't be difficult if someone > with the right OS expertise wants to do it. > > Is there a particular kernel API you want support for? kqueue, I run most things on FreeBSD and select is sadly mostly useless for anything serious. I've played with the idea of adding kqueue support myself but haven't had the time. > > y > > On Mon, Mar 7, 2016 at 1:16 PM, Malcolm Matalka wrote: >> Yaron Minsky writes: >> >>> This is definitely a fraught topic, and it's unfortunate that there's >>> no clear solution. >>> >>> To add a bit more information: >>> >>> - Async is more portable than it once was. There's now Core_kernel, >>> Async_kernel and Async_rpc_kernel, which allows us to do things like >>> run Async applications in the browser. I would think Windows >>> support would be pretty doable by someone who understands that world >>> well. >>> >>> That said, the chain of dependencies brought in by Async is still >>> quite big. This is something that could perhaps be improved, either >>> with better dead code analysis in OCaml, or some tweaks to >>> Async_kernel and Core_kernel themselves. >> >> When I last looked at the scheduler it was limited to [select] or >> [epoll], is this still the case? How difficult would it be to expand on >> those? >> >>> >>> - There are things we could contemplate to make it easier to bridge >>> the divide. Jeremie Dimino did a proof of concept rewrite of lwt to >>> use async as its implementation, where an Lwt.t and a Deferred.t are >>> equal at the type level. >>> >>> https://github.com/janestreet/lwt-async >>> >>> Another possibility, and one that might be easier to write, would be >>> to allow Lwt code to run using the Async scheduler as another >>> possible back-end. This would allow one to have programs that used >>> both Async and Lwt together in one program, without running on >>> different threads. >>> >>> It's worth mentioning if that there is interest in making Async more >>> suitable for a wider variety of goals, we're happy to work with >>> outside contributors on it. For example, if someone wanted to work on >>> Windows support for Async, we'd be happy to help out on integrating >>> that work. >>> >>> Probably the biggest issue is executable size. That will get better >>> when we release an unpacked version of our external libraries. But >>> even then, the module-level granularity captures more things than >>> would be ideal. >>> >>> y >>> >>> On Mon, Mar 7, 2016 at 10:16 AM, Jesper Louis Andersen >>> wrote: >>>> >>>> On Mon, Mar 7, 2016 at 2:38 AM, Yotam Barnoy wrote: >>>>> >>>>> Also, what happens to general utility functions that aren't rewritten for >>>>> Async/Lwt -- as far as I can tell, being in non-monadic code, they will >>>>> always starve other threads, since they cannot yield to another Async/Lwt >>>>> thread. Is this perception correct? >>>> >>>> >>>> Yes. >>>> >>>> On one hand, your observation is negative in the sense that now your code >>>> has "color" in the sense that it is written for one library only. And you >>>> have to transform code to having the right color before it can be used. This >>>> is not the case if the concurrency model is at a lower level[0]. >>>> >>>> On the other hand, your observation is positive: cooperative scheduling >>>> makes the points in which the code can switch explicit. This gives the >>>> programmer far more control over when you are done with a task and start to >>>> process the next task. You can also avoid the preemption check in the code >>>> all the time. If your code manipulates lots of shared data, it also >>>> simplifies things since you don't usually have to protect data with a mutex >>>> in a single-threaded context as much[1]. Cooperative models, if carefully >>>> managed, can exploit structure in the problem domain, whereas a preemptive >>>> model needs to fit all. >>>> >>>> My personal opinion is that the preemptive model eventually wins over the >>>> cooperative model, much like it has in most (all popular) operating systems. >>>> It is simply more productive to take an up-front performance hit as a >>>> sacrifice for a system which is more robust against stray code misbehaving. >>>> If a cooperative system fails, it is fails catastrophically. If a preemptive >>>> system fails, it degrades in performance. >>>> >>>> But given I have more than 10 years of Erlang programming behind me by now, >>>> I'm obviously biased toward certain computational models :) >>>> >>>> [0] Erlang would be one such example, where the system is preemptively >>>> scheduling for you and you can use any code in any place without having to >>>> worry about blocking for latency. Go is quasi-preemptive because it checks >>>> on function calls, but in contrast to Erlang a loop is not forced to factor >>>> through a recursion, so it can in principle run indefinitely. Haskell (GHC) >>>> is quasi-preemptive as well, checking on memory allocation boundaries. So >>>> the thing to look out for in GHC is latency from processing large arrays >>>> with no allocation, say. >>>> >>>> [1] Erlang has two VM runtimes for this reason. One is single-threaded and >>>> can avoid lots of locks which is far faster for certain workloads, or on >>>> embedded devices with a single core only. >>>> >>>> -- >>>> J.