From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id C47728018B for ; Thu, 6 Jul 2017 08:52:14 +0200 (CEST) Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=shayne.fletcher.50@gmail.com; spf=Pass smtp.mailfrom=shayne.fletcher.50@gmail.com; spf=None smtp.helo=postmaster@mail-vk0-f46.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of shayne.fletcher.50@gmail.com) identity=pra; client-ip=209.85.213.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="shayne.fletcher.50@gmail.com"; x-sender="shayne.fletcher.50@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of shayne.fletcher.50@gmail.com designates 209.85.213.46 as permitted sender) identity=mailfrom; client-ip=209.85.213.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="shayne.fletcher.50@gmail.com"; x-sender="shayne.fletcher.50@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-vk0-f46.google.com) identity=helo; client-ip=209.85.213.46; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="shayne.fletcher.50@gmail.com"; x-sender="postmaster@mail-vk0-f46.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AE/yg/BGXT2mIGnTBkWFwo51GYnF86YWxBRYc798d?= =?us-ascii?q?s5kLTJ7zoMWwAkXT6L1XgUPTWs2DsrQf2rWQ6/iocFdDyK7JiGoFfp1IWk1Nou?= =?us-ascii?q?QttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXdrXKo8DEdBAj0OxZr?= =?us-ascii?q?KeTpAI7SiNm82/yv95HJbQhFgDiwbaluIBmqsA7cqtQYjYx+J6gr1xDHuGFIe+?= =?us-ascii?q?NYxWNpIVKcgRPx7dqu8ZBg7ipdpesv+9ZPXqvmcas4S6dYDCk9PGAu+MLrrxjD?= =?us-ascii?q?QhCR6XYaT24bjwBHAwnB7BH9Q5fxri73vfdz1SWGIcH7S60/VC+85Kl3VhDnlC?= =?us-ascii?q?YHNyY48G7JjMxwkLlbqw+lqxBm3oLYfJ2ZOP94c6jAf90VWHBBU95TWCxPAo2y?= =?us-ascii?q?YYgBAfcfM+lEtITyvUcCoAGkCAWwGO/iyDlFjWL2060g1OQhFBnL0hEgH90UsH?= =?us-ascii?q?XUqsv6P7oSX+Cx0qbI0SnMb/NI1jfn9YPFdQshofGWUrNwd8re11MjFw3BjlqO?= =?us-ascii?q?tYzqIzOV2foXs2SB7+dsSfmii2Eiqw5rozivwt0ghZXOhoIQ013J8zhyzogyJd?= =?us-ascii?q?29UkF7YNikHYNItyGbL4t5XNsuQ3lpuCYi1LIKo4K7fC8SxJQ/xh7fc+KIf5KS?= =?us-ascii?q?4h39TumePC90hGl5eLKlhhay90egx+vhXce3yFZHtjRJnsXIu3wX1BHe6tKLRu?= =?us-ascii?q?Vg8kqvwzqC2AHe5+dZKk4uj6XbMYQuwrsom5oTr0vDGij2lV3zjKCMd0Uk/vGk?= =?us-ascii?q?6/rkYrn6v5OcOYB5hwDkPqQhncy/Bus4MgwQUGSB5eu807jj8VX4QLVMkPI2jr?= =?us-ascii?q?HUvI7GKckfvKK0AA9Y3pw95xqiATqqytsVkHgfIFJAYh2HjozpO1/UIPD/CPey?= =?us-ascii?q?m0+skDdxyPHGOL3uHI/BLnbHkLj/YbZy905cxxQpzdxF559YFqoMIP32WkDrtd?= =?us-ascii?q?zYCgU1PBCzw+biENl9zJ8RWXqTAq+FN6PfqUOH6fgqI+mIfYMVvDf9K+M55/P1?= =?us-ascii?q?ln84mVodfbGz0pcNaXC4GO5mI0SDbnb2jNcBCzRCgg1rfeXwj1vKdD9VfHWjF/?= =?us-ascii?q?Yt4zA9E4G3JYXKQI22nKaM0Tv9FZpTMDNoEFeJREzofpSCVr8ibC+OKc9siHRQ?= =?us-ascii?q?TrioWokskx+quRPxy7d9aLeM0iIdvJPnktNy4ruAxlkJ6TVoApHFgCm2RGZukz?= =?us-ascii?q?ZNHmdu0Q=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CIAwCf3V1Zhi7VVdFcHQEFAQsBGAEFA?= =?us-ascii?q?QsBhBGBEAeFH5otH41eiCWCESeFdQKDGQdBFgEBAQEBAQEBAQEBEgEBAQgLCwg?= =?us-ascii?q?oL4IzJAGCQAEBAQECASMEGQEbHQEDAQsGAwILAwoqAgIhAQERAQUBHAYTCIoOA?= =?us-ascii?q?QMNCBCSSJEaP4wKgWwYBQEcgwgFg2UKGScNVoMnAQEBAQEFAQEBAQEBGgIGEoE?= =?us-ascii?q?MggmIUYJXT4RXFoJLAQSBLAEBlX6HJDEIAQGBTYV6h1SEaoIMhUqDcYZXSIstN?= =?us-ascii?q?4dCFB+BFSYJgTJSI14aW4FXgjgPEAwZgWokNgGJAgEBAQ?= X-IPAS-Result: =?us-ascii?q?A0CIAwCf3V1Zhi7VVdFcHQEFAQsBGAEFAQsBhBGBEAeFH5o?= =?us-ascii?q?tH41eiCWCESeFdQKDGQdBFgEBAQEBAQEBAQEBEgEBAQgLCwgoL4IzJAGCQAEBA?= =?us-ascii?q?QECASMEGQEbHQEDAQsGAwILAwoqAgIhAQERAQUBHAYTCIoOAQMNCBCSSJEaP4w?= =?us-ascii?q?KgWwYBQEcgwgFg2UKGScNVoMnAQEBAQEFAQEBAQEBGgIGEoEMggmIUYJXT4RXF?= =?us-ascii?q?oJLAQSBLAEBlX6HJDEIAQGBTYV6h1SEaoIMhUqDcYZXSIstN4dCFB+BFSYJgTJ?= =?us-ascii?q?SI14aW4FXgjgPEAwZgWokNgGJAgEBAQ?= X-IronPort-AV: E=Sophos;i="5.40,315,1496095200"; d="scan'208,217";a="282263486" Received: from mail-vk0-f46.google.com ([209.85.213.46]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 06 Jul 2017 08:51:58 +0200 Received: by mail-vk0-f46.google.com with SMTP id r125so4818822vkf.1 for ; Wed, 05 Jul 2017 23:51:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Ch/S7/fKM/24Cs7p6L1FFGGiPULH03uO7PbKziTLvz0=; b=melVhPNxMhTM7al1AvNAyYoJTKUrmKDsuSoqnlbsdMRnAHMehNKLUvlfOhcucQoin+ PejD4P4bJXaC7CO4x6tLwMEEE+6jOAF3OLhQkCdVKdjjgSNwsHlbfxtMg49H60XxdwI2 g1DT3+bM/0GqmwkXu80+HWCp+ZhfusQKcFdd8vAsZM1LNBThD95NXPKd8n3A2C7yG5iu tv1S09ZOguB9qe1tJwkYynfxgGauXMbyV52BWgD6LT7vUwdgT6hb2k6SXllqyZLhYKZ5 zO0RUkWj/2Qmpk1uWqjYPBpaG8qNjiibUaKux3+m9OnTYvVoEnH7I5NA0wH270fxnX/i 25yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Ch/S7/fKM/24Cs7p6L1FFGGiPULH03uO7PbKziTLvz0=; b=fnnG6b+g76SayRLUEYQVCDLcEufJw6HfWeXU5y0dKhnnccOdbMwGFkY6RKHorNM3tf V1W4CBm1BdOMDBbQ3yuVY/VQb38+6MaKeSVeyFtKZU0v37/lbk7oMc7dmVGT5wZu4i7j A/8Zzw0LwOJ5MHu3HtUm/++DHEQX5AS7f23lZvfr0SOtRID0iEvirPJ7OVVro61h2uEo LZes0K0LpwkhvmswyaHlL16kqMhIwUxYZkaPLdoJRiGMA6pXirLuAgxdPNPMSHTPu4AV CCevALhyOtPev/cxLpVvBJ4u5sB4z8cjUiwqWoUcQWJ4HKeBSx8pA05TPNyWwnZ+wTRy BRUA== X-Gm-Message-State: AKS2vOwx3hnjBh8hopaTcxv1TKKtSRCUOMGwYfYcGT6OnQble9rdEbT8 +QzJn3vFPf2+lfknHNDBHgMEuqB8EPsR X-Received: by 10.31.146.211 with SMTP id u202mr24289247vkd.66.1499323917039; Wed, 05 Jul 2017 23:51:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.81.41 with HTTP; Wed, 5 Jul 2017 23:51:36 -0700 (PDT) In-Reply-To: References: From: Shayne Fletcher Date: Thu, 6 Jul 2017 02:51:36 -0400 Message-ID: To: Yaron Minsky Cc: =?UTF-8?Q?Nicol=C3=A1s_Ojeda_B=C3=A4r?= , "caml-list@inria.fr users" Content-Type: multipart/alternative; boundary="001a11435db87a7daf0553a08dcf" Subject: Re: [Caml-list] [Async / Cohttp_async] Problem getting all of the data in a response body --001a11435db87a7daf0553a08dcf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jul 6, 2017 at 1:37 AM, Yaron Minsky wrote: > The fact that you're ignoring the result of Pipe.iter is concerning. That > function should return a deferred that becomes determined once the > iteration is complete. It's hard to say without having a bit more of the > program, but I suspect the bug is related to that. > > y > > =E2=80=8BAh!=E2=80=8B Thank-you Yaron. Is something close to this schematic= looking more plausible? let run : Command.t =3D let cmd_proc : (unit -> unit Deferred.Or_error.t) Param.t =3D let open Command.Let_syntax in [%map_open let uri =3D ... and ... in fun () -> Async_client.post ... uri >>| fun (_, body) -> let r =3D Cohttp_async.Body.to_pipe body in Async.Pipe.iter r ~continue_on_error:true ~f:(fun s -> Async.return (printf "%s" s)) >>| fun () -> Ok () ] in Command.async_or_error' ~summary:"..." cmd_proc let main : Command.t =3D Command.group ~summary:"..." [ "run", run ] let () =3D Command.run ~v main On Thu, Jul 6, 2017 at 8:23 AM, Nicol=C3=A1s Ojeda B=C3=A4r < > nicolas.ojeda.bar@lexifi.com> wrote: > >> Hi Shayne, >> >> I am not very familiar with Async, but did you remember to run the >> scheduler (Scheduler.go () or similar) ? >> >> Cheers, >> Nicolas >> >> >> On Thu, Jul 6, 2017 at 12:55 AM, Shayne Fletcher < >> shayne.fletcher.50@gmail.com> wrote: >> >>> Can anyone help me out with this? >>> >>> In short, I'm doing a HTTP POST with a handler along the lines of: >>> ``` >>> fun ((resp : Cohttp_async.Response.t) >>> , (body : Cohttp_async.Body.t)) : unit Or_error.t -> >>> let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe b= ody >>> in >>> let _ =3D >>> Async.Pipe.iter >>> r >>> ~continue_on_error:true >>> ~f:(fun s -> Async.return (printf "%s" s)) in >>> (Ok () : unit Or_error.t) >>> >>> ``` >>> >>> Each time I invoke the program I get, more or less output (very >>> occasionally none) but never all of it. The response header says it's >>> "fixed 2700816" and indeed, if I replace the code above with >>> ``` >>> fun ((resp : Cohttp_async.Response.t) >>> , (body : Cohttp_async.Body.t)) : unit Or_error.t -> >>> let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe body= in >>> let n : int Deferred.t =3D >>> Async.Pipe.fold >>> r >>> ~init:0 >>> ~f:(fun acc s -> >>> printf "acc : %d\n" acc; Async.return (acc + String.length = s) >>> ) in >>> let _ : unit Deferred.t =3D n >>| fun n -> printf "Chars read : %d" = n in >>> (Ok () : unit Or_error.t) >>> >>> ``` >>> then (for example), on the first run I might see >>> ``` >>> acc : 0 >>> acc : 1118 >>> acc : 7503 >>> acc : 8780 >>> >>> ``` >>> >>> and then on the second run I might see >>> ``` >>> acc : 0 >>> acc : 1118 >>> acc : 4949 >>> >>> ``` >>> but never have I seen "Chars read : 2700816". >>> >>> I've tried a bunch of different things: >>> >>> - `Pipe.read_all`: >>> ``` >>> let _ =3D >>> Async.Pipe.read_all r >>| >>> fun q -> Queue.iter q ~f:(fun s -> printf "%s" s) in >>> ... >>> >>> ``` >>> No output. >>> >>> - `Pipe.drain_and_count`: >>> ``` >>> let _ =3D >>> Async.Pipe.drain_and_count r >>| >>> fun n -> printf "Count %d\n" n in >>> ... >>> >>> ``` >>> No output. >>> >>> - `Pipe.to_list`: >>> ``` >>> let _ =3D >>> Async.Pipe.to_list r >>| >>> fun l -> printf "%s" (String.concat ~sep:"" l) in >>> ... >>> >>> ``` >>> No output. >>> >>> Also tried the obvious: >>> >>> - `Cohttp_async.Body.to_string`: >>> ``` >>> let s =3D Cohttp_async.Body.to_string body in >>> let _ =3D s >>| fun s -> printf "%s" s in >>> ... >>> >>> ``` >>> No output. >>> >>> I'm new to this and obviously missing something fundamental :) Is there >>> a kind soul out there who can give me a hint on how to proceed please? >>> >>> Thanks! >>> >>> -- >>> Shayne Fletcher >>> >> >> > --=20 Shayne Fletcher --001a11435db87a7daf0553a08dcf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Jul 6, 2017 at 1:37 AM, Yaron M= insky <yminsky@janestreet.com> wrote:
The fact that you'= re ignoring the result of Pipe.iter is concerning. That function should ret= urn a deferred that becomes determined once the iteration is complete. It&#= 39;s hard to say without having a bit more of the program, but I suspect th= e bug is related to that.

y


=E2=80=8BAh!=E2=80=8B Thank-you Yaron. Is something close = to this schematic looking more plausible?

l= et run : Command.t =3D
=C2=A0 let cmd_proc : (unit -> unit Deferred.Or_= error.t) Param.t =3D
=C2=A0 =C2=A0 let open Command.Let_syntax in
=C2=A0 = =C2=A0 [%map_open
=C2=A0 =C2=A0 =C2=A0 =C2=A0let uri =3D ...=C2=A0
=C2=A0 = =C2=A0 =C2=A0 =C2=A0and ... in
=C2=A0 =C2=A0 =C2=A0 =C2=A0fun () ->
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Async_client.post ... uri >>|<= /div>
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 fun (_, body) ->
=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 let r =3D Cohttp_async.Body.to_pipe body in
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 Async.Pipe.iter
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 r
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ~conti= nue_on_error:true
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ~f:(fun = s -> Async.return (printf "%s" s)) >>|
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 fun () -> Ok ()
=C2=A0 =C2=A0 ] in
=
=C2=A0 Comma= nd.async_or_error' ~summary:"..." cmd_proc

<= div class=3D"gmail_default">let main : Co= mmand.t =3D
=C2=A0 Command.group ~summary:"..."
=C2=A0 [
=C2=A0 = =C2=A0 "run", run
=C2=A0 ]

let () =3D Command.run ~v main


On Thu, Jul 6, 2017 at 8:23 AM, Nicol=C3=A1s Ojeda B=C3=A4= r <nicolas.ojeda.bar@lexifi.com> wrote:
Hi Shayne,
I am not very familiar with Async, but did you remember to = run the scheduler (Scheduler.go () or similar) ?

C= heers,
Nicolas


On Thu, Jul 6, 2017 at= 12:55 AM, Shayne Fletcher <shayne.fletcher.50@gmail.com>= ; wrote:
Can anyone help me o= ut with this?

In short, I'm doing a HTT= P POST with a handler along the lines of:
```
fu= n ((resp : Cohttp_async.Response.t)
=C2=A0 =C2=A0 , (body : Cohttp_async.Body.t)) : unit Or_error.t= ->
=C2=A0 =C2=A0 =C2= =A0let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe body in=
=C2=A0 =C2=A0 =C2=A0let= _ =3D
=C2=A0 =C2=A0 =C2= =A0 Async.Pipe.iter
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 r
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ~continue_on_error:true
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ~f:(fun s -> = Async.return (printf "%s" s)) in
=C2=A0 =C2=A0 =C2=A0 (Ok () : unit Or_error.t)

```

Each time I invoke the program I get, more or less output (very occa= sionally none) but never all of it.=C2=A0The response header says it's "fixed =C2=A02700= 816" and indeed, if I replace the code above with
```
fun ((resp : Cohttp_async.Response.t)
=C2=A0 =C2=A0 , (body : Cohttp_async.Body.t)) : un= it Or_error.t ->
=C2= =A0 let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe body i= n
=C2=A0 let n : int Def= erred.t =3D
=C2=A0 =C2= =A0 Async.Pipe.fold
=C2= =A0 =C2=A0 =C2=A0 r
=C2= =A0 =C2=A0 =C2=A0 ~init:0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printf "a= cc : %d\n" acc; Async.return (acc + String.length s)
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ) in
=C2=A0 let _ : unit Defer= red.t =3D n =C2=A0>>| fun n -> printf "Chars read : %d" = n in
=C2=A0 (Ok () : uni= t Or_error.t)

=C2=A0= ```
then (for example), = on the first run I might see
```
acc : 0<= /div>
acc : 1118
acc : 7503
acc : 8780

```

=
and then on the second run I m= ight see
```
acc : 0
acc : 1118
acc : 4949
```
but never have I s= een "Chars read : 2700816".

I'= ;ve tried a bunch of different things:

=C2= =A0 - `Pipe.read_all`:
= =C2=A0 =C2=A0 =C2=A0```
= =C2=A0 =C2=A0 =C2=A0 =C2=A0let _ =3D
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Async.Pipe.read_all r >&g= t;|
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0fun q -> Queue.iter q ~f:(fun s -> printf "%s"= ; s) in
=C2=A0 =C2=A0 = =C2=A0 =C2=A0...

=C2= =A0 =C2=A0 =C2=A0```
=C2= =A0 =C2=A0 =C2=A0No output.
=C2=A0 =C2=A0 =C2=A0
=C2=A0 - `Pipe.drain_and_count`:
=C2=A0 =C2=A0 ```
=C2=A0 =C2=A0 =C2=A0 let _ =3D
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Async.Pipe.drain_and_count r >>= ;|
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 fun n -> printf "Count %d\n" n in
=C2=A0 =C2=A0 =C2=A0 ...

=C2=A0 =C2=A0 ```
=C2=A0 =C2=A0 No output.
=C2=A0=C2=A0
=C2=A0 - `Pipe.to_list`:
=C2=A0 =C2=A0 =C2=A0```
=C2=A0 =C2=A0 =C2=A0let _ =3D
=C2=A0 =C2=A0 =C2=A0 =C2=A0Async.Pipe.to_list r &= gt;>|
=C2=A0 =C2=A0 = =C2=A0 =C2=A0fun l -> printf "%s" (String.concat ~sep:"&q= uot; l) in
=C2=A0 =C2=A0= =C2=A0...

=C2=A0 = =C2=A0 =C2=A0```
=C2=A0 = =C2=A0 =C2=A0No output.
= =C2=A0=C2=A0
Also tried the obvious:

=C2= =A0 - `Cohttp_async.Body.to_string`:
=C2=A0 =C2=A0 ```
=C2=A0 =C2=A0 =C2=A0 let s =3D Cohttp_async.Body.to_string body in
=C2=A0 =C2=A0 =C2=A0 let = _ =3D s >>| fun =C2=A0s -> printf "%s" s in
=
=C2=A0 =C2=A0 =C2=A0 ...

=C2=A0 =C2=A0 ```
<= div>=C2=A0 =C2=A0 No output.
=

I'm new to this and obviously missing something fun= damental :) Is there a kind soul out there who can give me a hint on how to= proceed please?

Thanks!
<= span class=3D"gmail-m_4390981808640147649m_-8694811416117289099HOEnZb">

--
Shayne Fletcher





--
=
Shayne Fletcher
--001a11435db87a7daf0553a08dcf--