Re: [Caml-list] [Async / Cohttp_async] Problem getting all of the data in a response body

[Yaron Minsky <yminsky@janestreet.com>]

[-- Attachment #1: Type: text/plain, Size: 3288 bytes --]

The fact that you're ignoring the result of Pipe.iter is concerning. That
function should return a deferred that becomes determined once the
iteration is complete. It's hard to say without having a bit more of the
program, but I suspect the bug is related to that.

y

On Thu, Jul 6, 2017 at 8:23 AM, Nicolás Ojeda Bär <
nicolas.ojeda.bar@lexifi.com> wrote:

> Hi Shayne,
>
> I am not very familiar with Async, but did you remember to run the
> scheduler (Scheduler.go () or similar) ?
>
> Cheers,
> Nicolas
>
>
> On Thu, Jul 6, 2017 at 12:55 AM, Shayne Fletcher <
> shayne.fletcher.50@gmail.com> wrote:
>
>> Can anyone help me out with this?
>>
>> In short, I'm doing a HTTP POST with a handler along the lines of:
>> ```
>> fun ((resp : Cohttp_async.Response.t)
>>     , (body : Cohttp_async.Body.t)) : unit Or_error.t ->
>>      let r : string Async.Pipe.Reader.t = Cohttp_async.Body.to_pipe body
>> in
>>      let _ =
>>       Async.Pipe.iter
>>         r
>>         ~continue_on_error:true
>>         ~f:(fun s -> Async.return (printf "%s" s)) in
>>       (Ok () : unit Or_error.t)
>>
>> ```
>>
>> Each time I invoke the program I get, more or less output (very
>> occasionally none) but never all of it. The response header says it's
>> "fixed  2700816" and indeed, if I replace the code above with
>> ```
>> fun ((resp : Cohttp_async.Response.t)
>>     , (body : Cohttp_async.Body.t)) : unit Or_error.t ->
>>   let r : string Async.Pipe.Reader.t = Cohttp_async.Body.to_pipe body in
>>   let n : int Deferred.t =
>>     Async.Pipe.fold
>>       r
>>       ~init:0
>>       ~f:(fun acc s ->
>>             printf "acc : %d\n" acc; Async.return (acc + String.length s)
>>           ) in
>>   let _ : unit Deferred.t = n  >>| fun n -> printf "Chars read : %d" n in
>>   (Ok () : unit Or_error.t)
>>
>>  ```
>> then (for example), on the first run I might see
>> ```
>> acc : 0
>> acc : 1118
>> acc : 7503
>> acc : 8780
>>
>> ```
>>
>> and then on the second run I might see
>> ```
>> acc : 0
>> acc : 1118
>> acc : 4949
>>
>> ```
>> but never have I seen "Chars read : 2700816".
>>
>> I've tried a bunch of different things:
>>
>>   - `Pipe.read_all`:
>>      ```
>>        let _ =
>>          Async.Pipe.read_all r >>|
>>          fun q -> Queue.iter q ~f:(fun s -> printf "%s" s) in
>>        ...
>>
>>      ```
>>      No output.
>>
>>   - `Pipe.drain_and_count`:
>>     ```
>>       let _ =
>>         Async.Pipe.drain_and_count r >>|
>>         fun n -> printf "Count %d\n" n in
>>       ...
>>
>>     ```
>>     No output.
>>
>>   - `Pipe.to_list`:
>>      ```
>>      let _ =
>>        Async.Pipe.to_list r >>|
>>        fun l -> printf "%s" (String.concat ~sep:"" l) in
>>      ...
>>
>>      ```
>>      No output.
>>
>> Also tried the obvious:
>>
>>   - `Cohttp_async.Body.to_string`:
>>     ```
>>       let s = Cohttp_async.Body.to_string body in
>>       let _ = s >>| fun  s -> printf "%s" s in
>>       ...
>>
>>     ```
>>     No output.
>>
>> I'm new to this and obviously missing something fundamental :) Is there a
>> kind soul out there who can give me a hint on how to proceed please?
>>
>> Thanks!
>>
>> --
>> Shayne Fletcher
>>
>
>

[-- Attachment #2: Type: text/html, Size: 10757 bytes --]