From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 9241E7F787 for ; Tue, 8 Nov 2016 13:16:14 +0100 (CET) Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=gabriel.scherer@gmail.com; spf=Pass smtp.mailfrom=gabriel.scherer@gmail.com; spf=None smtp.helo=postmaster@mail-it0-f46.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.214.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.214.46 as permitted sender) identity=mailfrom; client-ip=209.85.214.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-it0-f46.google.com) identity=helo; client-ip=209.85.214.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-it0-f46.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3A+OowohQWnFHzeCq+fMVjELt/b9psv+yvbD5Q0YIu?= =?us-ascii?q?jvd0So/mwa64Yx2N2/xhgRfzUJnB7Loc0qyN4vqmCDJLuMfJmUtBWaQEbwUCh8?= =?us-ascii?q?QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYsExnyfTB4?= =?us-ascii?q?Ov7yUtaLyZ/mjabjpdaKMk1hv3mUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO?= =?us-ascii?q?9MxGlldhq5lhf44dqsrtY4q3wD89pozcNLUL37cqIkVvQYSW1+ayFmrPHs4DbZ?= =?us-ascii?q?QASI/GpUd2wSnxFICkCR4xTzWpr6siLSue902S3cNsrzG+MaQzOnuohiQgXphS?= =?us-ascii?q?NPDDU5/XvakIQkg6tRuhOso1pkyI7ZeoyPHPV7d6LZO9gdQDwSDY5qSyVdD9bk?= =?us-ascii?q?PMM0BO0bMLMd9tGlqg=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AkAAC1wCFYhi7WVdFdGwEBAQMBAQEJA?= =?us-ascii?q?QEBFwEBBAEBCgEBgwQBAQEBAXdvEAeNMpcDgmCMWYUagggphXsCgg4HPxQBAQE?= =?us-ascii?q?BAQEBAQEBARIBAQEICwsJHTCCMwQBFQEEgg8BAQECAQESER0BGxILAQMBCwYDA?= =?us-ascii?q?gsDCioCAiEBAREBBQEcBhMUDogfAQMPCA6UPI9NgTI/MotPggwGAS2DGgWDbgo?= =?us-ascii?q?ZJw1UgxABAQEBAQEBAwEBAQEBAQEBFwIGEIYuhFeCSByBMAYRAYMgglwFj1KKK?= =?us-ascii?q?DWBa4RLhleDO4I8jViJDYQjgkMTHoESDw+BDQ0yglssIIIGIDSFDV+BTgEBAQ?= X-IPAS-Result: =?us-ascii?q?A0AkAAC1wCFYhi7WVdFdGwEBAQMBAQEJAQEBFwEBBAEBCgE?= =?us-ascii?q?BgwQBAQEBAXdvEAeNMpcDgmCMWYUagggphXsCgg4HPxQBAQEBAQEBAQEBARIBA?= =?us-ascii?q?QEICwsJHTCCMwQBFQEEgg8BAQECAQESER0BGxILAQMBCwYDAgsDCioCAiEBARE?= =?us-ascii?q?BBQEcBhMUDogfAQMPCA6UPI9NgTI/MotPggwGAS2DGgWDbgoZJw1UgxABAQEBA?= =?us-ascii?q?QEBAwEBAQEBAQEBFwIGEIYuhFeCSByBMAYRAYMgglwFj1KKKDWBa4RLhleDO4I?= =?us-ascii?q?8jViJDYQjgkMTHoESDw+BDQ0yglssIIIGIDSFDV+BTgEBAQ?= X-IronPort-AV: E=Sophos;i="5.31,462,1473112800"; d="scan'208,217";a="199611562" Received: from mail-it0-f46.google.com ([209.85.214.46]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 08 Nov 2016 13:16:13 +0100 Received: by mail-it0-f46.google.com with SMTP id e187so152143951itc.0 for ; Tue, 08 Nov 2016 04:16:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=H11Jm9MqJzMETEf1AMCsr1kcUh/T1EhWXD8xfp6ajwQ=; b=Ct69/ZhOmNPF3SR5AvMLP9Q2dc965zBpa/oANGSoFZ/8pwblqJJYPI1yZRC3bacPXJ KFXR8eZDtLeHevIlncaf1FFTlxd/mux+aE8dxhK+hHGmk/2+WAe5V2Ia6XmMTJMJuTKS h9G6K2sTUkzTyfS2VRbFYKOkPwlckzUbPznvN+T7yt/vnZWnwoyGMprefEwFwFcajAPw fPik7MnZMwDGnCmEwQQvyDNBXO8Vi82FA5brOSxkWR5Po2kLxXmOOTfT2dRHujOr+ipM 50HIwZr5vKzo1g2z/MwqZcs8oXP+pYLnMJmUluBl0Vstyq4ksulGyRMHkn4XAlrd6xL+ 5ygg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=H11Jm9MqJzMETEf1AMCsr1kcUh/T1EhWXD8xfp6ajwQ=; b=h929mZ0kM4ZITIB1QsLC2AZPtAmRLcChq8DkOTqkM3cfaBgpDUUR9ySDqY/KSbzsov fyuRonamX7D9a4oW5j2PT0Eb9h/oYOin8PHlctCpPd6wnqy8ymj/U5Y2XRRs95z64/7k ykA3ed7r3Qm4tcbKRI/y9sGFcoMQTCj+n7dAh2rJaoV6B4wyGpQMCFwElOLm7iMy6Van WD53fqigON+CXnNTfc94/OmC/s2OQ1MMj3Uqmi6RbXvB7g9KqkOvyHmXsIwShlOl2otk uxa6mS5UiioClEyfYjrQ46B4UjiAbY3OIi+rT0/HCWPkMqq2dHwN5wH5117bHOsnGeuf INRg== X-Gm-Message-State: ABUngveAmNvFpJLMr1dzvWSaFiVo6QWZ6eCxNgpi7dyaqw8H7wER9337a8llzYw7DuSntsLh0pNk/3zuTVHzLA== X-Received: by 10.36.220.130 with SMTP id q124mr10976930itg.16.1478607369103; Tue, 08 Nov 2016 04:16:09 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.30.129 with HTTP; Tue, 8 Nov 2016 04:15:28 -0800 (PST) In-Reply-To: References: <20161108120716.GA6073@Magus.localnet> From: Gabriel Scherer Date: Tue, 8 Nov 2016 07:15:28 -0500 Message-ID: To: Gregory Malecha Cc: Oleg , Yaron Minsky , Jonas Jensen , caml users Content-Type: multipart/alternative; boundary=94eb2c0b1e64ff6e730540c91aac Subject: Re: [Caml-list] The fastest stream library [Was: Question about Optimization] --94eb2c0b1e64ff6e730540c91aac Content-Type: text/plain; charset=UTF-8 Hi Oleg, Thanks for the work, it does look very nice and I will have a look. My remarks below concern the benchmarks. I regret not being pointed to this work earlier, because I think that measuring the performance of Enum as a representative OCaml stream library performance is not the best choice : Enum is designed to be flexible in a bit too many ways to be efficient on pure-streaming scenarios (it supports a generic "clone", and effectful generators, that makes the codebase too complex for its own good; I think that Batteries community is aware that Enum is not as good as it should be right now). There are other, more efficient streaming libraries out there, including BatSeq in Batteries that should already be sensibly faster; Core and Containers also have more efficient streaming libraries. I think that it would be good to try to use at least BatSeq for your benchmarks. I won't have time this week or the next one, but then maybe I could look at it. I'm not surprised that neither POPL reviewing nor POPL artifact evaluation caught this issue, but next time you benchmark using Batteries, feel free to send an email to the batteries-users mailing-list to point it out. On Tue, Nov 8, 2016 at 7:05 AM, Gregory Malecha wrote: > Thanks. This is interesting, I'll have to take a closer look at it. > > On Tue, Nov 8, 2016, 7:01 AM Oleg wrote: > >> >> > On 21 April 2016 at 09:13, Gregory Malecha wrote: >> > I'm wondering if there is any work (and interest) on supporting >> > user-defined optimizations similar to GHC's rewrite rules in the Ocaml >> > compiler. For example, a standard example would be specifying map >> fusion: >> >> to which Gabriel Scherer commented on Thu, 21 Apr 2016 12:02:14 -0400 >> >> > Another approach that might be worth trying (sorry for not thinking >> > about it earlier) is MetaOCaml. I tend of think of it as a tool to >> > explicitly specify and control partial evaluation strategies. >> >> Indeed. We'd like to point out an application of MetaOCaml, not just >> to map fusion -- but also concat_map fusion and zip fusion, etc. We >> present a streams library that supports the wide set of combinators -- >> from map and filter to concat_map (flat_map) and zip -- and produces >> the hand-written quality code. It is faster than Batteries by up to more >> than two orders of magnitude. >> >> http://okmij.org/ftp/meta-programming/strymonas.pdf >> http://strymonas.github.io/ >> >> Unlike GHC Rules, we guarantee the performance. >> > -- > > - gregory malecha > gmalecha.github.io > --94eb2c0b1e64ff6e730540c91aac Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Oleg,

Thanks for the work, = it does look very nice and I will have a look. My remarks below concern the= benchmarks.

I regret not being pointed to this wor= k earlier, because I think that measuring the performance of Enum as a repr= esentative OCaml stream library performance is not the best choice : Enum i= s designed to be flexible in a bit too many ways to be efficient on pure-st= reaming scenarios (it supports a generic "clone", and effectful g= enerators, that makes the codebase too complex for its own good; I think th= at Batteries community is aware that Enum is not as good as it should be ri= ght now). There are other, more efficient streaming libraries out there, in= cluding BatSeq in Batteries that should already be sensibly faster; Core an= d Containers also have more efficient streaming libraries.

I think that it would be good to try to use at least BatSeq for your be= nchmarks. I won't have time this week or the next one, but then maybe I= could look at it. I'm not surprised that neither POPL reviewing nor PO= PL artifact evaluation caught this issue, but next time you benchmark using= Batteries, feel free to send an email to the batteries-users mailing-list = to point it out.

On Tue, Nov 8, 2016 at 7:05 AM, Gregory Malecha <gmalecha@gmail.com> wrote:

Thanks= . This is interesting, I'll have to take a closer look at it.


On Tue, Nov 8, 2016, 7:01 A= M Oleg <oleg@okmij.o= rg> wrote:

> On 21 April 2016 at 09:13, Gregory Malecha <gmalecha@gmail.com> wrote:
> I'm wondering if there is any work (and interest) on supporting > user-defined optimizations similar to GHC's rewrite rules in the O= caml
> compiler. For example, a standard example would be specifying map fusi= on:

to which Gabriel Scherer commented on Thu, 21 Apr 2016 12:02:14 -0400

> Another approach that might be worth trying (sorry for not thinking > about it earlier) is MetaOCaml. I tend of think of it as a tool to
> explicitly specify and control partial evaluation strategies.

Indeed. We'd like to point out an application of MetaOCaml, not just to map fusion -- but also concat_map fusion and zip fusion, etc. We
present a streams library that supports the wide set of combinators --
from map and filter to concat_map (flat_map) and zip -- and produces
the hand-written quality code. It is faster than Batteries by up to more than two orders of magnitude.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 http://okmij.org/ftp/meta-programming/strymonas.pdf
=C2=A0 =C2=A0 =C2=A0 =C2=A0 http://strymonas.github.io/

Unlike GHC Rules, we guarantee the performance.
=
--

- gregory malecha
=C2=A0 gmalecha.git= hub.io


--94eb2c0b1e64ff6e730540c91aac--