From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=5.0 tests=AWL,DNS_FROM_RFC_POST, HTML_MESSAGE,SPF_NEUTRAL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 065D4BB84 for ; Tue, 17 Feb 2009 13:26:34 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: As4BAB88mknRVdrYmWdsb2JhbACCQi+KHYZ5PwEBAQEBCAsKBxGtFo8lAQMBA4QQBoQ0 X-IronPort-AV: E=Sophos;i="4.38,223,1233529200"; d="ml'?scan'208";a="23079119" Received: from mail-bw0-f216.google.com ([209.85.218.216]) by mail3-smtp-sop.national.inria.fr with ESMTP; 17 Feb 2009 13:26:33 +0100 Received: by bwz12 with SMTP id 12so8520511bwz.9 for ; Tue, 17 Feb 2009 04:26:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=Y3kEhHNOVZYRabUY4NO+1xL+CDcXLd+tFECVxKPhvHU=; b=q6oADWWR3HqPHURobonfM2LEFzKtDQkugh3NwI3Kw1lj4W9TMA5sd9U3spLo7fX4fU Xn9H4LfiwiCNuKdbj7iEcpnDoPqcZeeaJmr+X1TUj/mDoHyJ5vbiKo09WXzoNlpE5iBy unQsfKm9UoxE2P2sKQrRDNt8Xwh3axH8HVWos= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=VTTowgL5EIYH/WtzFk0hqmayz8nShB3lnXKZBB62eKOqXXo69wbgHkYQG4gB8GgFqD ic8AkExC8XtesrH360J71/SGWr/J1L7D+lzf8ZuGn06hdKLZY3yzH1HhIue4xLHdk11u AW7umSI65s7Flvm93QV1wj117ZhJ17dzMmbs0= MIME-Version: 1.0 Sender: remidewitte@gmail.com Received: by 10.223.115.16 with SMTP id g16mr1028518faq.93.1234873592708; Tue, 17 Feb 2009 04:26:32 -0800 (PST) In-Reply-To: <891bd3390902170420i6e7647ccl44c55456952d3f5a@mail.gmail.com> References: <2184b2340902160715y1f935b5ehc0e6195b3f75b66b@mail.gmail.com> <891bd3390902160847p25ad3bf1pe59da620dfc667f2@mail.gmail.com> <2184b2340902160937i53b8f3fbga01eaf14ed829f8f@mail.gmail.com> <2184b2340902162340s540c5ac7g9f42b59d03f643cb@mail.gmail.com> <891bd3390902170420i6e7647ccl44c55456952d3f5a@mail.gmail.com> Date: Tue, 17 Feb 2009 13:26:32 +0100 X-Google-Sender-Auth: 1f2a3a3877dc37f5 Message-ID: <2184b2340902170426n381d4d7dv2d4d3ea2bad6701f@mail.gmail.com> Subject: Re: [Caml-list] Re: Threads performance issue. From: =?UTF-8?Q?R=C3=A9mi_Dewitte?= To: yminsky@gmail.com Cc: Sylvain Le Gall , caml-list@inria.fr Content-Type: multipart/mixed; boundary=001636c5afe1d5ffee04631c690c X-Spam: no; 0.00; descriptors:01 bigarray:01 ocaml:01 yaron:01 minsky:01 yminsky:01 buffer:01 buffer:01 descriptors:01 le-gall:01 ocaml:01 beginner's:01 bug:01 beginner's:01 bug:01 X-Attachments: cset="UTF-8" cset="UTF-8" type="text/x-ocaml" name="transi2.ml" name="transi2.ml" type="text/x-ocaml" name="transimm.ml" name="transimm.ml" --001636c5afe1d5ffee04631c690c Content-Type: multipart/alternative; boundary=001636c5afe1d5ffe904631c690a --001636c5afe1d5ffe904631c690a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Not using channels with either file descriptors or bigarray works well in m= y case. Good to know when working with ocaml to take care of channels ;) ! R=C3=A9mi 2009/2/17 Yaron Minsky > Interestingly, this probably has nothing to do with the size of the > buffer. input_char actually acquires and releases a lock for every singl= e > call, whether or not an underlying system call is required to fill the > buffer. This has always struck me as an odd aspect of the in/out channel > implementation, and means that IO is a lot more expensive in a threaded > context than it should be. > > At Jane Street, performance-sensitive code tends to use other libraries > that we've built directly on top of file descriptors that batches the IO = and > doesn't require constant lock acquisition. > > y > > > On Tue, Feb 17, 2009 at 5:07 AM, Sylvain Le Gall wro= te: > >> On 17-02-2009, R=C3=A9mi Dewitte wrote: >> > >> > test.csv is a 21mo file with ~13k rows and a thousands of columns on a >> 15rp=3D >> > m >> > disk. >> > >> > ocaml version : 3.11.0 >> > >> >> You are using input_char and standard IO channel. This is a good choice >> for non-threaded program. But in your case, I will use Unix.read with a >> big buffer (32KB to 4MB) and change your program to use it. As >> benchmarked by John Harrop, you are spending most of your time in >> caml_enter|leave_blocking section. I think it comes from reading using >> std IO channel which use 4k buffer. Using a bigger buffer will allow >> less call to this two functions (but you won't win time at the end, I >> think you will just reduce the difference between non-threaded and >> threaded code). >> >> Regards >> Sylvain Le Gall >> >> _______________________________________________ >> Caml-list mailing list. Subscription management: >> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list >> Archives: http://caml.inria.fr >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> Bug reports: http://caml.inria.fr/bin/caml-bugs >> > > > _______________________________________________ > Caml-list mailing list. Subscription management: > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list > Archives: http://caml.inria.fr > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > > --001636c5afe1d5ffe904631c690a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Not using channels with either file descriptors or bigarray works well in m= y case.

Good to know when working with ocaml to take care of channel= s ;) !

R=C3=A9mi

2009/2/17 Yaron M= insky <yminsky@gm= ail.com>
Interestingly, th= is probably has nothing to do with the size of the buffer.  input_char= actually acquires and releases a lock for every single call, whether or no= t an underlying system call is required to fill the buffer.  This has = always struck me as an odd aspect of the in/out channel implementation, and= means that IO is a lot more expensive in a threaded context than it should= be. 

At Jane Street, performance-sensitive code tends to use other libraries= that we've built directly on top of file descriptors that batches the = IO and doesn't require constant lock acquisition.

y


On Tue, Feb 17, 2009 at 5:07 AM, Sylvain Le Gall <sylvain@le-gall.net> wrote:
On 17-02-2009, R=C3=A9mi Dewitte <remi@gide.net> wrote:
>
> test.csv is a 21mo file with ~13k rows and a thousands of columns on a= 15rp=3D
> m
> disk.
>
> ocaml version : 3.11.0
>

You are using input_char and standard IO channel. This is a good choi= ce
for non-threaded program. But in your case, I will use Unix.read with a
big buffer (32KB to 4MB) and change your program to use it. As
benchmarked by John Harrop, you are spending most of your time in
caml_enter|leave_blocking section. I think it comes from reading using
std IO channel which use 4k buffer. Using a bigger buffer will allow
less call to this two functions (but you won't win time at the end, I think you will just reduce the difference between non-threaded and
threaded code).

Regards
Sylvain Le Gall

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.in= ria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.in= ria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


--001636c5afe1d5ffe904631c690a-- --001636c5afe1d5ffee04631c690c Content-Type: text/x-ocaml; charset=US-ASCII; name="transi2.ml" Content-Disposition: attachment; filename="transi2.ml" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fraju5p40 KCogb3BlbiBFeHRMaWIgKikKCigqKiBTbGl0aGx5IG1vZGlmaWVkIGNvcHkgZnJvbSBtb2R1bGUg Q1NWICopCmV4Y2VwdGlvbiBCYWRfQ1NWX2ZpbGUgb2Ygc3RyaW5nCgp0eXBlIHN0YXRlX3QgPSBT dGFydEZpZWxkCgkgICAgICAgfCBJblVucXVvdGVkRmllbGQKCSAgICAgICB8IEluUXVvdGVkRmll bGQKCSAgICAgICB8IEluUXVvdGVkRmllbGRBZnRlclF1b3RlCgpsZXQgbG9hZF9yb3dzID8oc2Vw YXJhdG9yID0gJywnKSA/KG5yZWFkID0gLTEpIGYgZmlsZSA9CiAgbGV0IG5yID0gcmVmIDAgaW4K ICBsZXQgcm93ID0gcmVmIFtdIGluCQkJKCogQ3VycmVudCByb3cuICopCiAgbGV0IGZpZWxkID0g cmVmIFtdIGluCQkJKCogQ3VycmVudCBmaWVsZC4gKikKICBsZXQgc3RhdGUgPSByZWYgU3RhcnRG aWVsZCBpbgkJKCogQ3VycmVudCBzdGF0ZS4gKikKICBsZXQgZW5kX29mX2ZpZWxkICgpID0KICAg IGxldCBmaWVsZF9saXN0ID0gTGlzdC5yZXYgIWZpZWxkIGluCiAgICBsZXQgZmllbGRfbGVuID0g TGlzdC5sZW5ndGggZmllbGRfbGlzdCBpbgogICAgbGV0IGZpZWxkX3N0ciA9IFN0cmluZy5jcmVh dGUgZmllbGRfbGVuIGluCiAgICBsZXQgcmVjIGxvb3AgaSA9IGZ1bmN0aW9uCglbXSAtPiAoKQog ICAgICB8IHggOjogeHMgLT4KCSAgZmllbGRfc3RyLltpXSA8LSB4OwoJICBsb29wIChpKzEpIHhz CiAgICBpbgogICAgbG9vcCAwIGZpZWxkX2xpc3Q7CiAgICByb3cgOj0gKFNvbWUgZmllbGRfc3Ry KSA6OiAhcm93OwogICAgZmllbGQgOj0gW107CiAgICBzdGF0ZSA6PSBTdGFydEZpZWxkCiAgaW4K ICBsZXQgZW1wdHlfZmllbGQgKCkgPQogICAgcm93IDo9IE5vbmUgOjogIXJvdzsKICAgIGZpZWxk IDo9IFtdOwogICAgc3RhdGUgOj0gU3RhcnRGaWVsZAogIGluCiAgbGV0IGVuZF9vZl9yb3cgKCkg PQogICAgbGV0IHJvd19saXN0ID0gTGlzdC5yZXYgIXJvdyBpbgogICAgZiByb3dfbGlzdDsKICAg IHJvdyA6PSBbXTsKICAgIHN0YXRlIDo9IFN0YXJ0RmllbGQ7CiAgICBuciA6PSAhbnIgKyAxOwog IGluCiAgbGV0IHByb2Nlc3MgYyA9CiAgICBpZiBjICE9ICdccicgdGhlbiAoCQkJKCogQWx3YXlz IGlnbm9yZSBcciBjaGFyYWN0ZXJzLiAqKQogICAgICBtYXRjaCAhc3RhdGUgd2l0aAoJICBTdGFy dEZpZWxkIC0+CQkJKCogRXhwZWN0aW5nIHF1b3RlIG9yIG90aGVyIGNoYXIuICopCgkgICAgaWYg YyA9ICciJyB0aGVuICgKCSAgICAgIHN0YXRlIDo9IEluUXVvdGVkRmllbGQ7CgkgICAgICBmaWVs ZCA6PSBbXQoJICAgICkgZWxzZSBpZiBjID0gc2VwYXJhdG9yIHRoZW4gKCogRW1wdHkgZmllbGQu ICopCgkgICAgICBlbXB0eV9maWVsZCAoKQoJICAgIGVsc2UgaWYgYyA9ICdcbicgdGhlbiAoCSgq IEVtcHR5IGZpZWxkLCBlbmQgb2Ygcm93LiAqKQoJICAgICAgZW1wdHlfZmllbGQgKCk7CgkgICAg ICBlbmRfb2Zfcm93ICgpCgkgICAgKSBlbHNlICgKCSAgICAgIHN0YXRlIDo9IEluVW5xdW90ZWRG aWVsZDsKCSAgICAgIGZpZWxkIDo9IFtjXQoJICAgICkKCXwgSW5VbnF1b3RlZEZpZWxkIC0+CQko KiBSZWFkaW5nIGNoYXJzIHRvIGVuZCBvZiBmaWVsZC4gKikKCSAgICBpZiBjID0gc2VwYXJhdG9y IHRoZW4JKCogRW5kIG9mIGZpZWxkLiAqKQoJICAgICAgZW5kX29mX2ZpZWxkICgpCgkgICAgZWxz ZSBpZiBjID0gJ1xuJyB0aGVuICgJKCogRW5kIG9mIGZpZWxkIGFuZCBlbmQgb2Ygcm93LiAqKQoJ ICAgICAgZW5kX29mX2ZpZWxkICgpOwoJICAgICAgZW5kX29mX3JvdyAoKQoJICAgICkgZWxzZQoJ ICAgICAgZmllbGQgOj0gYyA6OiAhZmllbGQKCXwgSW5RdW90ZWRGaWVsZCAtPgkJKCogUmVhZGlu ZyBjaGFycyB0byBlbmQgb2YgZmllbGQuICopCgkgICAgaWYgYyA9ICciJyB0aGVuCgkgICAgICBz dGF0ZSA6PSBJblF1b3RlZEZpZWxkQWZ0ZXJRdW90ZQoJICAgIGVsc2UKCSAgICAgIGZpZWxkIDo9 IGMgOjogIWZpZWxkCgl8IEluUXVvdGVkRmllbGRBZnRlclF1b3RlIC0+CgkgICAgaWYgYyA9ICci JyB0aGVuICgJCSgqIERvdWJsZWQgcXVvdGUuICopCgkgICAgICBmaWVsZCA6PSBjIDo6ICFmaWVs ZDsKCSAgICAgIHN0YXRlIDo9IEluUXVvdGVkRmllbGQKCSAgICApIGVsc2UgaWYgYyA9ICcwJyB0 aGVuICgJKCogUXVvdGUtMCBpcyBBU0NJSSBOVUwuICopCgkgICAgICBmaWVsZCA6PSAnXDAwMCcg OjogIWZpZWxkOwoJICAgICAgc3RhdGUgOj0gSW5RdW90ZWRGaWVsZAoJICAgICkgZWxzZSBpZiBj ID0gc2VwYXJhdG9yIHRoZW4gKCogRW5kIG9mIGZpZWxkLiAqKQoJICAgICAgZW5kX29mX2ZpZWxk ICgpCgkgICAgZWxzZSBpZiBjID0gJ1xuJyB0aGVuICgJKCogRW5kIG9mIGZpZWxkIGFuZCBlbmQg b2Ygcm93LiAqKQoJICAgICAgZW5kX29mX2ZpZWxkICgpOwoJICAgICAgZW5kX29mX3JvdyAoKQoJ ICAgICkgZWxzZSAoCQkJKCogQmFkIHNpbmdsZSBxdW90ZSBpbiBmaWVsZC4gKikKCSAgICAgIGZp ZWxkIDo9IGMgOjogJyInIDo6ICFmaWVsZDsKCSAgICAgIHN0YXRlIDo9IEluUXVvdGVkRmllbGQK CSAgICApCiAgICApICgqIGVuZCBvZiBtYXRjaCAqKQogIGluCiAgbGV0IGNvbnRpbnVlID0gcmVm IHRydWUgaW4KICBsZXQgZmlsZV9pbiA9IFVuaXgub3BlbmZpbGUgZmlsZSBbVW5peC5PX1JET05M WV0gMG82NDAgaW4KICBsZXQgZW5kX3Byb2Nlc3NpbmcgKCkgPQogICAgY29udGludWUgOj0gZmFs c2U7CiAgICB0cnkgVW5peC5jbG9zZSBmaWxlX2luIHdpdGggXyAtPiAoKTsKICAgIChtYXRjaCAh c3RhdGUgd2l0aAogICAgICB8IFN0YXJ0RmllbGQgLT4KCSBpZiAhcm93IDw+IFtdIHRoZW4KCSAg ICggZW1wdHlfZmllbGQgKCk7IGVuZF9vZl9yb3cgKCkgKQogICAgICB8IEluVW5xdW90ZWRGaWVs ZCB8IEluUXVvdGVkRmllbGRBZnRlclF1b3RlIC0+CgkgICAgZW5kX29mX2ZpZWxkICgpOyBlbmRf b2Zfcm93ICgpCiAgICAgIHwgSW5RdW90ZWRGaWVsZCAtPgoJICAgcmFpc2UgKEJhZF9DU1ZfZmls ZSAiTWlzc2luZyBlbmQgcXVvdGUgYWZ0ZXIgcXVvdGVkIGZpZWxkLiIpCiAgICApCiAgaW4KICBs ZXQgYnVmZmVyX2xlbmd0aCA9IDIgKiAxMDI0ICogMTAyNCBpbgogIGxldCBidWZmZXIgPSBTdHJp bmcubWFrZSBidWZmZXJfbGVuZ3RoICdcMDAwJyBpbgogIGxldCBwcm9jZXNzX2J1ZmZlciBsID0g CigqICAgICBmb3IgaSA9IDAgdG8gbCBkbyAqKQogICAgbGV0IGlpID0gcmVmIDAgaW4KICAgIHdo aWxlICghY29udGludWUpICYmICghaWkpIDw9IGwgZG8KICAgICAgbGV0IGkgPSAhaWkgaW4KICAg ICAgcHJvY2VzcyBidWZmZXIuW2ldOwogICAgICBpaSA6PSBpICsgMTsKICAgICAgaWYoIG5yZWFk ID4gMCAmJiAhbnIgPSBucmVhZCApIHRoZW4gZW5kX3Byb2Nlc3NpbmcgKCkgZWxzZSAoKQogICAg ZG9uZQogIGluCiAgd2hpbGUgIShjb250aW51ZSkKICBkbyAKICAgIGxldCBuID0gVW5peC5yZWFk IGZpbGVfaW4gYnVmZmVyIDAgYnVmZmVyX2xlbmd0aCBpbgogICAgaWYgKG4gPiAwICkKICAgIHRo ZW4gcHJvY2Vzc19idWZmZXIgbgogICAgZWxzZSBlbmRfcHJvY2Vzc2luZyAoKQogIGRvbmUKCmxl dCBydW5fdGhyZWFkZWQgZiA9IFRocmVhZC5jcmVhdGUgKGZ1biAoKSAtPiBmICgpOyBUaHJlYWQu ZXhpdCAoKSkgKCkKCmxldCB0MSA9IGxvYWRfcm93cyAoZnVuIF8gLT4gKCkpICJ0ZXN0LmNzdiIK bGV0IHQyID0gbG9hZF9yb3dzIChmdW4gXyAtPiAoKSkgInRlc3QyLmNzdiIKbGV0IHQzID0gbG9h ZF9yb3dzIChmdW4gXyAtPiAoKSkgInRlc3QzLmNzdiI= --001636c5afe1d5ffee04631c690c Content-Type: text/x-ocaml; charset=US-ASCII; name="transimm.ml" Content-Disposition: attachment; filename="transimm.ml" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fraju7n41 KCogb3BlbiBFeHRMaWIgKikKCm9wZW4gQmlnYXJyYXkKCigqKiBTbGl0aGx5IG1vZGlmaWVkIGNv cHkgZnJvbSBtb2R1bGUgQ1NWICopCmV4Y2VwdGlvbiBCYWRfQ1NWX2ZpbGUgb2Ygc3RyaW5nCgp0 eXBlIHN0YXRlX3QgPSBTdGFydEZpZWxkCgkgICAgICAgfCBJblVucXVvdGVkRmllbGQKCSAgICAg ICB8IEluUXVvdGVkRmllbGQKCSAgICAgICB8IEluUXVvdGVkRmllbGRBZnRlclF1b3RlCgpsZXQg bG9hZF9yb3dzID8oc2VwYXJhdG9yID0gJywnKSA/KG5yZWFkID0gLTEpIGYgZmlsZSA9CiAgbGV0 IG5yID0gcmVmIDAgaW4KICBsZXQgcm93ID0gcmVmIFtdIGluCQkJKCogQ3VycmVudCByb3cuICop CiAgbGV0IGZpZWxkID0gcmVmIFtdIGluCQkJKCogQ3VycmVudCBmaWVsZC4gKikKICBsZXQgc3Rh dGUgPSByZWYgU3RhcnRGaWVsZCBpbgkJKCogQ3VycmVudCBzdGF0ZS4gKikKICBsZXQgZW5kX29m X2ZpZWxkICgpID0KICAgIGxldCBmaWVsZF9saXN0ID0gTGlzdC5yZXYgIWZpZWxkIGluCiAgICBs ZXQgZmllbGRfbGVuID0gTGlzdC5sZW5ndGggZmllbGRfbGlzdCBpbgogICAgbGV0IGZpZWxkX3N0 ciA9IFN0cmluZy5jcmVhdGUgZmllbGRfbGVuIGluCiAgICBsZXQgcmVjIGxvb3AgaSA9IGZ1bmN0 aW9uCglbXSAtPiAoKQogICAgICB8IHggOjogeHMgLT4KCSAgZmllbGRfc3RyLltpXSA8LSB4OwoJ ICBsb29wIChpKzEpIHhzCiAgICBpbgogICAgbG9vcCAwIGZpZWxkX2xpc3Q7CiAgICByb3cgOj0g KFNvbWUgZmllbGRfc3RyKSA6OiAhcm93OwogICAgZmllbGQgOj0gW107CiAgICBzdGF0ZSA6PSBT dGFydEZpZWxkCiAgaW4KICBsZXQgZW1wdHlfZmllbGQgKCkgPQogICAgcm93IDo9IE5vbmUgOjog IXJvdzsKICAgIGZpZWxkIDo9IFtdOwogICAgc3RhdGUgOj0gU3RhcnRGaWVsZAogIGluCiAgbGV0 IGVuZF9vZl9yb3cgKCkgPQogICAgbGV0IHJvd19saXN0ID0gTGlzdC5yZXYgIXJvdyBpbgogICAg ZiByb3dfbGlzdDsKICAgIHJvdyA6PSBbXTsKICAgIHN0YXRlIDo9IFN0YXJ0RmllbGQ7CiAgICBu ciA6PSAhbnIgKyAxOwogIGluCiAgbGV0IHByb2Nlc3MgYyA9CiAgICBpZiBjICE9ICdccicgdGhl biAoCQkJKCogQWx3YXlzIGlnbm9yZSBcciBjaGFyYWN0ZXJzLiAqKQogICAgICBtYXRjaCAhc3Rh dGUgd2l0aAoJICBTdGFydEZpZWxkIC0+CQkJKCogRXhwZWN0aW5nIHF1b3RlIG9yIG90aGVyIGNo YXIuICopCgkgICAgaWYgYyA9ICciJyB0aGVuICgKCSAgICAgIHN0YXRlIDo9IEluUXVvdGVkRmll bGQ7CgkgICAgICBmaWVsZCA6PSBbXQoJICAgICkgZWxzZSBpZiBjID0gc2VwYXJhdG9yIHRoZW4g KCogRW1wdHkgZmllbGQuICopCgkgICAgICBlbXB0eV9maWVsZCAoKQoJICAgIGVsc2UgaWYgYyA9 ICdcbicgdGhlbiAoCSgqIEVtcHR5IGZpZWxkLCBlbmQgb2Ygcm93LiAqKQoJICAgICAgZW1wdHlf ZmllbGQgKCk7CgkgICAgICBlbmRfb2Zfcm93ICgpCgkgICAgKSBlbHNlICgKCSAgICAgIHN0YXRl IDo9IEluVW5xdW90ZWRGaWVsZDsKCSAgICAgIGZpZWxkIDo9IFtjXQoJICAgICkKCXwgSW5VbnF1 b3RlZEZpZWxkIC0+CQkoKiBSZWFkaW5nIGNoYXJzIHRvIGVuZCBvZiBmaWVsZC4gKikKCSAgICBp ZiBjID0gc2VwYXJhdG9yIHRoZW4JKCogRW5kIG9mIGZpZWxkLiAqKQoJICAgICAgZW5kX29mX2Zp ZWxkICgpCgkgICAgZWxzZSBpZiBjID0gJ1xuJyB0aGVuICgJKCogRW5kIG9mIGZpZWxkIGFuZCBl bmQgb2Ygcm93LiAqKQoJICAgICAgZW5kX29mX2ZpZWxkICgpOwoJICAgICAgZW5kX29mX3JvdyAo KQoJICAgICkgZWxzZQoJICAgICAgZmllbGQgOj0gYyA6OiAhZmllbGQKCXwgSW5RdW90ZWRGaWVs ZCAtPgkJKCogUmVhZGluZyBjaGFycyB0byBlbmQgb2YgZmllbGQuICopCgkgICAgaWYgYyA9ICci JyB0aGVuCgkgICAgICBzdGF0ZSA6PSBJblF1b3RlZEZpZWxkQWZ0ZXJRdW90ZQoJICAgIGVsc2UK CSAgICAgIGZpZWxkIDo9IGMgOjogIWZpZWxkCgl8IEluUXVvdGVkRmllbGRBZnRlclF1b3RlIC0+ CgkgICAgaWYgYyA9ICciJyB0aGVuICgJCSgqIERvdWJsZWQgcXVvdGUuICopCgkgICAgICBmaWVs ZCA6PSBjIDo6ICFmaWVsZDsKCSAgICAgIHN0YXRlIDo9IEluUXVvdGVkRmllbGQKCSAgICApIGVs c2UgaWYgYyA9ICcwJyB0aGVuICgJKCogUXVvdGUtMCBpcyBBU0NJSSBOVUwuICopCgkgICAgICBm aWVsZCA6PSAnXDAwMCcgOjogIWZpZWxkOwoJICAgICAgc3RhdGUgOj0gSW5RdW90ZWRGaWVsZAoJ ICAgICkgZWxzZSBpZiBjID0gc2VwYXJhdG9yIHRoZW4gKCogRW5kIG9mIGZpZWxkLiAqKQoJICAg ICAgZW5kX29mX2ZpZWxkICgpCgkgICAgZWxzZSBpZiBjID0gJ1xuJyB0aGVuICgJKCogRW5kIG9m IGZpZWxkIGFuZCBlbmQgb2Ygcm93LiAqKQoJICAgICAgZW5kX29mX2ZpZWxkICgpOwoJICAgICAg ZW5kX29mX3JvdyAoKQoJICAgICkgZWxzZSAoCQkJKCogQmFkIHNpbmdsZSBxdW90ZSBpbiBmaWVs ZC4gKikKCSAgICAgIGZpZWxkIDo9IGMgOjogJyInIDo6ICFmaWVsZDsKCSAgICAgIHN0YXRlIDo9 IEluUXVvdGVkRmllbGQKCSAgICApCiAgICApICgqIGVuZCBvZiBtYXRjaCAqKQogIGluCiAgbGV0 IGZpbGVfaW4gPSBVbml4Lm9wZW5maWxlIGZpbGUgW1VuaXguT19SRE9OTFldIDBvNjQwIGluCiAg bGV0IGVuZF9wcm9jZXNzaW5nICgpID0KICAgIHRyeSBVbml4LmNsb3NlIGZpbGVfaW4gd2l0aCBf IC0+ICgpOwogICAgKG1hdGNoICFzdGF0ZSB3aXRoCiAgICAgIHwgU3RhcnRGaWVsZCAtPgoJIGlm ICFyb3cgPD4gW10gdGhlbgoJICAgKCBlbXB0eV9maWVsZCAoKTsgZW5kX29mX3JvdyAoKSApCiAg ICAgIHwgSW5VbnF1b3RlZEZpZWxkIHwgSW5RdW90ZWRGaWVsZEFmdGVyUXVvdGUgLT4KCSAgICBl bmRfb2ZfZmllbGQgKCk7IGVuZF9vZl9yb3cgKCkKICAgICAgfCBJblF1b3RlZEZpZWxkIC0+Cgkg ICByYWlzZSAoQmFkX0NTVl9maWxlICJNaXNzaW5nIGVuZCBxdW90ZSBhZnRlciBxdW90ZWQgZmll bGQuIikKICAgICkKICBpbgogIGxldCBtbWFwID0gQmlnYXJyYXkuQXJyYXkxLm1hcF9maWxlIGZp bGVfaW4gQmlnYXJyYXkuY2hhciBCaWdhcnJheS5jX2xheW91dCBmYWxzZSAoLTEpIGluCiAgbGV0 IGwgPSAoQmlnYXJyYXkuQXJyYXkxLmRpbSBtbWFwKSBpbgogIGxldCBjb250aW51ZSA9IHJlZiB0 cnVlIGluCiAgbGV0IGkgPSByZWYgMCBpbgogIHdoaWxlICFjb250aW51ZSBkbwogICAgcHJvY2Vz cyAoQXJyYXkxLigqdW5zYWZlXyopZ2V0IG1tYXAgIWkpOwogICAgaSA6PSAhaSArIDE7CiAgICBj b250aW51ZSA6PSAobnJlYWQgPCAwIHx8ICFuciA8IG5yZWFkICkgJiYgIWkgPCBsCiAgZG9uZTsK ICBlbmRfcHJvY2Vzc2luZyAoKQo7OwoKbGV0IHJ1bl90aHJlYWRlZCBmID0gVGhyZWFkLmNyZWF0 ZSAoZnVuICgpIC0+IGYgKCk7IFRocmVhZC5leGl0ICgpKTs7Cgpsb2FkX3Jvd3MgKGZ1biBfIC0+ ICgpKSAidGVzdC5jc3YiIDs7CmxvYWRfcm93cyAoZnVuIF8gLT4gKCkpICJ0ZXN0Mi5jc3YiOzsK bG9hZF9yb3dzIChmdW4gXyAtPiAoKSkgInRlc3QzLmNzdiI7Ow== --001636c5afe1d5ffee04631c690c--