From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 50EC68018B for ; Thu, 6 Jul 2017 07:37:53 +0200 (CEST) Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=yminsky@janestreet.com; spf=Pass smtp.mailfrom=yminsky@janestreet.com; spf=None smtp.helo=postmaster@mxout3.mail.janestreet.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of yminsky@janestreet.com) identity=pra; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="yminsky@janestreet.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of yminsky@janestreet.com designates 38.105.200.229 as permitted sender) identity=mailfrom; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="yminsky@janestreet.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mxout3.mail.janestreet.com) identity=helo; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="postmaster@mxout3.mail.janestreet.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AORoyKB0sgP3IGTPCsmDT+DRfVm0co7zxezQtwd8Z?= =?us-ascii?q?seIeI/ad9pjvdHbS+e9qxAeQG96KtLQc06L/iOPJYSQ4+5GPsXQPItRndiQuro?= =?us-ascii?q?EopTEmG9OPEkbhLfTnPGQQFcVGU0J5rTngaRAGUMnxaEfPrXKs8DUcBgvwNRZv?= =?us-ascii?q?JuTyB4Xek9m72/q89pDXYQhEniaxba9vJxiqsAvdsdUbj5F/Iagr0BvJpXVIe+?= =?us-ascii?q?VSxWx2IF+Yggjx6MSt8pN96ipco/0u+dJOXqX8ZKQ4UKdXDC86PGAv5c3krgfM?= =?us-ascii?q?QA2S7XYBSGoWkx5IAw/Y7BHmW5r6ryX3uvZh1CScIMb7Vq4/Vyi84Kh3SR/okC?= =?us-ascii?q?YHOCA/8GHLkcx7kaZXrAu8qxBj34LYZYeYP+d8cKzAZ9MXXWhOXshRWSJPAY2y?= =?us-ascii?q?cpUBAPYaMOlCs4XwvUEDoQeiCQSuAu7k1z9GhmXx3a0/y+kuExvJ3BYnH9IIrX?= =?us-ascii?q?/aq9T1NKEJUeC2zKjD0DLObvNT2Tf87ojIaRMhoPGXULJya8XRz1cgFxjZjlWK?= =?us-ascii?q?t4PqIS+V2v4RvGic6uptTOSigHMppQF2pzig3MYsio/Ri4INy1DE7Tt2zJ4tJd?= =?us-ascii?q?25UE50f8SoEIdXty2CL4t2Q9siTH9ytCYg1LIJo4O0cDARyJQ73R7favqHc4eO?= =?us-ascii?q?7xn+V+iROS91iXxmdb6lmhq+7UitxvfiWsWozFpGtDRJnsfIu3wVyRDf9syKRu?= =?us-ascii?q?F+80qhwzqDyR3f5+JeLU02i6bWLYMqzKQqmZoJq0vDGzf7mEXog6+ScUUp4u2o?= =?us-ascii?q?6uvgYrX7up+ROZV4igTkPaQvnsyzG+Y4PRIIX2eF4uS8yKfj8lPnT7VWlPE5jq?= =?us-ascii?q?7ZsJXCKcQaoK62HRNV354t5hu8FTuqzcgUkHkdIF5bZB6Lk5LlN0/QLP38Ffu/?= =?us-ascii?q?hk6jkDZvx/DIJL3hBZDNI2DDkbfkfLZ97VVcyAkzzN1E5ZJbELcBL+zoWk/trt?= =?us-ascii?q?zXEAE2MxCuz+n7D9V905sSWXiTDa+BLKPSrViI6/ozLOaWYY8VvC/xK/wk5/71?= =?us-ascii?q?kX80gkQdfKms3ZsPcn+0BPVmI0ODYXrtmNgNC2kKvhBtBNDt3XuDWjlX7m2FeK?= =?us-ascii?q?Ax6jgMIg68RdPFR4uqhLWamiC5E5FfIGpPEHiIHGfpeYTCUPAJPnG8OMhkxw0N?= =?us-ascii?q?T7+nA6og0wqtpUeu2r9mKPHX6wUavJTuz8Rv6uDP0xo18GonXIymz2iRQjQszS?= =?us-ascii?q?szTDgs0fU6+BQlxw=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0D2AgACzF1ZfeXIaSZcHQEFAQsBGQYMh?= =?us-ascii?q?BGBEAeFH5pMiC6IKYc9J4V1AoMYB0IVAQEBAQEBAQEBAQESAQEJFghXgjMkAYJ?= =?us-ascii?q?AAQEBAQIBIwQZAQE3AQQLCQILDSoCAiEBEgEFARwGE4oXAw0IAw2SFJEaP4sfa?= =?us-ascii?q?4FsOoMHAQEFhC4Ng30BAQEBAQUBAQEBAQEBGQgSgQyCCYNMhQWCV0+EVxaCS5F?= =?us-ascii?q?ShV+HJDuPG4RqggyFSoNOI4ZXSIstN4dCFB+BFTWBLDEhCBsVSRoGgiyCOA8cG?= =?us-ascii?q?YFRcwGJAgEBAQ?= X-IPAS-Result: =?us-ascii?q?A0D2AgACzF1ZfeXIaSZcHQEFAQsBGQYMhBGBEAeFH5pMiC6?= =?us-ascii?q?IKYc9J4V1AoMYB0IVAQEBAQEBAQEBAQESAQEJFghXgjMkAYJAAQEBAQIBIwQZA?= =?us-ascii?q?QE3AQQLCQILDSoCAiEBEgEFARwGE4oXAw0IAw2SFJEaP4sfa4FsOoMHAQEFhC4?= =?us-ascii?q?Ng30BAQEBAQUBAQEBAQEBGQgSgQyCCYNMhQWCV0+EVxaCS5FShV+HJDuPG4Rqg?= =?us-ascii?q?gyFSoNOI4ZXSIstN4dCFB+BFTWBLDEhCBsVSRoGgiyCOA8cGYFRcwGJAgEBAQ?= X-IronPort-AV: E=Sophos;i="5.40,315,1496095200"; d="scan'208,217";a="282253806" Received: from mxout3.mail.janestreet.com ([38.105.200.229]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Jul 2017 07:37:51 +0200 Received: from [172.27.56.68] (helo=tot-qpr-mailcore1) by mxout3.mail.janestreet.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dSzU6-0001my-RT for caml-list@inria.fr; Thu, 06 Jul 2017 01:37:50 -0400 X-JS-Flow: external X-JS-Scanner-attachment: (ok) No attachments Received: by tot-qpr-mailcore1 with ocaml/mailcore/mailcore 1.0+97 (cbf81801a3e2) (envelope-from ) id BZXcyu-AkzbLp-aR; 2017-07-06 01:37:50.845371-04:00 Received: from mail-oi0-f72.google.com ([209.85.218.72]) by mxgoog1.mail.janestreet.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89) (envelope-from ) id 1dSzU6-0004zl-P3 for caml-list@inria.fr; Thu, 06 Jul 2017 01:37:50 -0400 Received: by mail-oi0-f72.google.com with SMTP id h143so794623oic.13 for ; Wed, 05 Jul 2017 22:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=d9O6faaiJpkrNzmzJb6CUYCHn/ahb8nCaqbI2N07xts=; b=I1771KjANhbaetnJyvuOcujBQeUhXrVnYr/qMvMFJ/E8iFcy5DVpsutjkF+Z9hHsbt ToYj6NMtNeWbh573xmENx8vnKDDT71IAbm8xsVspyywV1/sZ3q95wfWarGN7zPp/ABeD rxR1fFk+UL962zK9m+zL9Fy7shgQaPmeMryhk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=d9O6faaiJpkrNzmzJb6CUYCHn/ahb8nCaqbI2N07xts=; b=oBEepicPXDZyeb5GCZjAyxXH5Fql/JD6fAySCugJWKZmE3gV3IA0exvA3b9lVN/9V1 MYJjZvQzjNhZGe3wd1dX8h+JbYO9JKhSSey/XdhoZOfQYhitl/X5lxBqyJflturwFQv7 WJ2sNWA+br7SRiImcU96nzREFmr5WRmzh7tArv2XU5WTfZnk0KT7e/Wii1ow0RO9OCXy rXplEIOWrNEyqzW4qiqHbaQUzGjaXjk/P967NwMIgQLkfbwneiQRecmrc3gE55d52R8o YsoPLcL5e2ab51+6xM6RwkxsUUmVw5YrJJChOv+rCCcPC5Kn9rN9AgeVEA48ozLWK02g jYIw== X-Gm-Message-State: AKS2vOwFyNAARwdVSdCVCxNjO3lbmgc0hqD10UcvgAzFDNrvn3odBdzT uenuN8a+9FwiEyAusjkK4gn32pLE/R7ZnpWgXzVj/xTVkgkWcvrA4gtddt1LbbvAbMvlWxMs+So MthR8TPvekKhlV2Tw X-Received: by 10.202.206.8 with SMTP id e8mr26838501oig.45.1499319470006; Wed, 05 Jul 2017 22:37:50 -0700 (PDT) X-Received: by 10.202.206.8 with SMTP id e8mr26838491oig.45.1499319469723; Wed, 05 Jul 2017 22:37:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.235.82 with HTTP; Wed, 5 Jul 2017 22:37:29 -0700 (PDT) In-Reply-To: References: From:Yaron Minsky Date: Thu, 6 Jul 2017 08:37:29 +0300 Message-ID: To:=?UTF-8?Q?Nicol=C3=A1s_Ojeda_B=C3=A4r?= Cc:Shayne Fletcher , "caml-list@inria.fr users" Content-Type: multipart/alternative; boundary="001a113cbda265f8bf05539f846e" X-JS-Exim-Data-Received: 2017-07-06 01:37:50-0400 X-JS-Processed-by: mailcore Subject: Re: [Caml-list] [Async / Cohttp_async] Problem getting all of the data in a response body --001a113cbda265f8bf05539f846e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The fact that you're ignoring the result of Pipe.iter is concerning. That function should return a deferred that becomes determined once the iteration is complete. It's hard to say without having a bit more of the program, but I suspect the bug is related to that. y On Thu, Jul 6, 2017 at 8:23 AM, Nicol=C3=A1s Ojeda B=C3=A4r < nicolas.ojeda.bar@lexifi.com> wrote: > Hi Shayne, > > I am not very familiar with Async, but did you remember to run the > scheduler (Scheduler.go () or similar) ? > > Cheers, > Nicolas > > > On Thu, Jul 6, 2017 at 12:55 AM, Shayne Fletcher < > shayne.fletcher.50@gmail.com> wrote: > >> Can anyone help me out with this? >> >> In short, I'm doing a HTTP POST with a handler along the lines of: >> ``` >> fun ((resp : Cohttp_async.Response.t) >> , (body : Cohttp_async.Body.t)) : unit Or_error.t -> >> let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe bo= dy >> in >> let _ =3D >> Async.Pipe.iter >> r >> ~continue_on_error:true >> ~f:(fun s -> Async.return (printf "%s" s)) in >> (Ok () : unit Or_error.t) >> >> ``` >> >> Each time I invoke the program I get, more or less output (very >> occasionally none) but never all of it. The response header says it's >> "fixed 2700816" and indeed, if I replace the code above with >> ``` >> fun ((resp : Cohttp_async.Response.t) >> , (body : Cohttp_async.Body.t)) : unit Or_error.t -> >> let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.to_pipe body = in >> let n : int Deferred.t =3D >> Async.Pipe.fold >> r >> ~init:0 >> ~f:(fun acc s -> >> printf "acc : %d\n" acc; Async.return (acc + String.length s) >> ) in >> let _ : unit Deferred.t =3D n >>| fun n -> printf "Chars read : %d" n= in >> (Ok () : unit Or_error.t) >> >> ``` >> then (for example), on the first run I might see >> ``` >> acc : 0 >> acc : 1118 >> acc : 7503 >> acc : 8780 >> >> ``` >> >> and then on the second run I might see >> ``` >> acc : 0 >> acc : 1118 >> acc : 4949 >> >> ``` >> but never have I seen "Chars read : 2700816". >> >> I've tried a bunch of different things: >> >> - `Pipe.read_all`: >> ``` >> let _ =3D >> Async.Pipe.read_all r >>| >> fun q -> Queue.iter q ~f:(fun s -> printf "%s" s) in >> ... >> >> ``` >> No output. >> >> - `Pipe.drain_and_count`: >> ``` >> let _ =3D >> Async.Pipe.drain_and_count r >>| >> fun n -> printf "Count %d\n" n in >> ... >> >> ``` >> No output. >> >> - `Pipe.to_list`: >> ``` >> let _ =3D >> Async.Pipe.to_list r >>| >> fun l -> printf "%s" (String.concat ~sep:"" l) in >> ... >> >> ``` >> No output. >> >> Also tried the obvious: >> >> - `Cohttp_async.Body.to_string`: >> ``` >> let s =3D Cohttp_async.Body.to_string body in >> let _ =3D s >>| fun s -> printf "%s" s in >> ... >> >> ``` >> No output. >> >> I'm new to this and obviously missing something fundamental :) Is there a >> kind soul out there who can give me a hint on how to proceed please? >> >> Thanks! >> >> -- >> Shayne Fletcher >> > > --001a113cbda265f8bf05539f846e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The fact that you're ignoring the result of Pipe.iter = is concerning. That function should return a deferred that becomes determin= ed once the iteration is complete. It's hard to say without having a bi= t more of the program, but I suspect the bug is related to that.

y

On Thu, Jul 6, 2017 at 8:23 AM, Nicol=C3=A1s Ojeda B=C3=A4r <nicolas.ojeda.bar@lexifi.com> wrote:
Hi Shayne,

I am not very = familiar with Async, but did you remember to run the scheduler (Scheduler.g= o () or similar) ?

Cheers,
Nicolas
=


On Thu, Jul 6, 2017 at 12:55 AM= , Shayne Fletcher <shayne.fletcher.50@gmail.com> = wrote:
= Can anyone help me out with this?
=
In short, I'm doing a HTTP POST = with a handler along the lines of:
```
fun ((resp : Cohttp_async.Response.t= )
=C2= =A0 =C2=A0 =C2=A0let r : string Async.Pipe.Reader.t =3D Cohttp_async.Body.t= o_pipe body in
=C2=A0 =C2=A0 =C2=A0let _ =3D
=C2=A0 =C2=A0 =C2=A0 Async.Pip= e.iter
=C2=A0 =C2=A0 =C2=A0 =C2=A0 r
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ~continue_= on_error:true
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ~f:(fun s -> Async.return (pri= ntf "%s" s)) in
=C2=A0 =C2=A0 =C2=A0 (Ok () : unit Or_error.t)
```

Each time I invoke the program I get, more or less output (very = occasionally none) but never all of it.=C2=A0The response header says it's "fixed =C2=A0= 2700816" and indeed, if I replace the code above with
```
=
fun ((resp := Cohttp_async.Response.t)
=C2=A0 =C2=A0 , (body : Cohttp_async.Body.t)) : u= nit Or_error.t ->
=C2=A0 let r : string Async.Pipe.Reader.t =3D Cohttp_a= sync.Body.to_pipe body in
=C2=A0 let n : int Deferred.t =3D
=C2=A0 =C2=A0 A= sync.Pipe.fold
=C2=A0 =C2=A0 =C2=A0 r
=C2=A0 =C2=A0 =C2=A0 ~init:0
=C2=A0 = =C2=A0 =C2=A0 ~f:(fun acc s ->
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= printf "acc : %d\n" acc; Async.return (acc + String.length s)
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ) in
=C2=A0 let _ : unit Deferred.t =3D = n =C2=A0>>| fun n -> printf "Chars read : %d" n in
=C2= =A0 (Ok () : unit Or_error.t)

=C2=A0```
then (for example), on the firs= t run I might see
```
acc : 0
acc : 1118
acc : 7503
acc : 8780

```
and then on the second run I might see
```
acc : 0
acc : 1118
acc : 4949=

```
but never have I seen "Chars read : 2700816".

I'= ;ve tried a bunch of different things:

=C2=A0 - `Pipe.read_all`:
=C2= =A0 =C2=A0 =C2=A0```
=C2=A0 =C2=A0 =C2=A0 =C2=A0let _ =3D
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0Async.Pipe.read_all r >>|
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0fun q -> Queue.iter q ~f:(fun s -> printf "%s" s)= in
=C2=A0 =C2=A0 =C2=A0 =C2=A0...

=C2=A0 =C2=A0 =C2=A0```
=
=C2=A0 =C2= =A0 =C2=A0No output.
=C2=A0 =C2=A0 =C2=A0
=C2=A0 - `Pipe.drain_and_count`:<= /font>
= =C2=A0 =C2=A0 ```
=C2=A0 =C2=A0 =C2=A0 let _ =3D
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 Async.Pipe.drain_and_count r >>|
=C2=A0 =C2=A0 =C2=A0 =C2=A0 fun = n -> printf "Count %d\n" n in
=C2=A0 =C2=A0 =C2=A0 ...<= /div>

=C2= =A0 =C2=A0 ```
=C2=A0 =C2=A0 No output.
=C2=A0=C2=A0
=C2=A0 - `Pipe.to_list= `:
=C2=A0 =C2=A0 =C2=A0```
=C2=A0 =C2=A0 =C2=A0let _ =3D
=C2=A0 =C2=A0 =C2= =A0 =C2=A0Async.Pipe.to_list r >>|
=C2=A0 =C2=A0 =C2=A0 =C2=A0fun l -= > printf "%s" (String.concat ~sep:"" l) in
=C2=A0 = =C2=A0 =C2=A0...

=C2=A0 =C2=A0 =C2=A0```
=C2=A0 =C2=A0 =C2=A0No output.=
=C2=A0=C2=A0
Also tried the obvious:

=C2=A0 - `Cohttp_async.Body.t= o_string`:
=C2=A0 =C2=A0 ```
=C2=A0 =C2=A0 =C2=A0 let s =3D Cohttp_async.Bo= dy.to_string body in
=C2=A0 =C2=A0 =C2=A0 let _ =3D s >>| fun =C2=A0s= -> printf "%s" s in
= =C2=A0 =C2=A0 =C2=A0 ...

=C2=A0 =C2= =A0 ```
=C2=A0 =C2=A0 No output.
<= font face=3D"tahoma, sans-serif">
I'm new to this and obviously mis= sing something fundamental :) Is there a kind soul out there who can give m= e a hint on how to proceed please?

Thanks!

--
Shayne Fletcher


--001a113cbda265f8bf05539f846e--