Re: [Caml-list] [Async / Cohttp_async] Problem getting all of the data in a response body

[Shayne Fletcher <shayne.fletcher.50@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4303 bytes --]

On Thu, Jul 6, 2017 at 1:37 AM, Yaron Minsky <yminsky@janestreet.com> wrote:

> The fact that you're ignoring the result of Pipe.iter is concerning. That
> function should return a deferred that becomes determined once the
> iteration is complete. It's hard to say without having a bit more of the
> program, but I suspect the bug is related to that.
>
> y
>
>
​Ah!​ Thank-you Yaron. Is something close to this schematic looking more
plausible?

let run : Command.t =
  let cmd_proc : (unit -> unit Deferred.Or_error.t) Param.t =
    let open Command.Let_syntax in
    [%map_open
       let uri = ...
       and ... in
       fun () ->
          Async_client.post ... uri >>|
          fun (_, body) ->
            let r = Cohttp_async.Body.to_pipe body in
            Async.Pipe.iter
              r
              ~continue_on_error:true
              ~f:(fun s -> Async.return (printf "%s" s)) >>|
            fun () -> Ok ()
    ] in
  Command.async_or_error' ~summary:"..." cmd_proc

let main : Command.t =
  Command.group ~summary:"..."
  [
    "run", run
  ]

let () = Command.run ~v main


On Thu, Jul 6, 2017 at 8:23 AM, Nicolás Ojeda Bär <
> nicolas.ojeda.bar@lexifi.com> wrote:
>
>> Hi Shayne,
>>
>> I am not very familiar with Async, but did you remember to run the
>> scheduler (Scheduler.go () or similar) ?
>>
>> Cheers,
>> Nicolas
>>
>>
>> On Thu, Jul 6, 2017 at 12:55 AM, Shayne Fletcher <
>> shayne.fletcher.50@gmail.com> wrote:
>>
>>> Can anyone help me out with this?
>>>
>>> In short, I'm doing a HTTP POST with a handler along the lines of:
>>> ```
>>> fun ((resp : Cohttp_async.Response.t)
>>>     , (body : Cohttp_async.Body.t)) : unit Or_error.t ->
>>>      let r : string Async.Pipe.Reader.t = Cohttp_async.Body.to_pipe body
>>> in
>>>      let _ =
>>>       Async.Pipe.iter
>>>         r
>>>         ~continue_on_error:true
>>>         ~f:(fun s -> Async.return (printf "%s" s)) in
>>>       (Ok () : unit Or_error.t)
>>>
>>> ```
>>>
>>> Each time I invoke the program I get, more or less output (very
>>> occasionally none) but never all of it. The response header says it's
>>> "fixed  2700816" and indeed, if I replace the code above with
>>> ```
>>> fun ((resp : Cohttp_async.Response.t)
>>>     , (body : Cohttp_async.Body.t)) : unit Or_error.t ->
>>>   let r : string Async.Pipe.Reader.t = Cohttp_async.Body.to_pipe body in
>>>   let n : int Deferred.t =
>>>     Async.Pipe.fold
>>>       r
>>>       ~init:0
>>>       ~f:(fun acc s ->
>>>             printf "acc : %d\n" acc; Async.return (acc + String.length s)
>>>           ) in
>>>   let _ : unit Deferred.t = n  >>| fun n -> printf "Chars read : %d" n in
>>>   (Ok () : unit Or_error.t)
>>>
>>>  ```
>>> then (for example), on the first run I might see
>>> ```
>>> acc : 0
>>> acc : 1118
>>> acc : 7503
>>> acc : 8780
>>>
>>> ```
>>>
>>> and then on the second run I might see
>>> ```
>>> acc : 0
>>> acc : 1118
>>> acc : 4949
>>>
>>> ```
>>> but never have I seen "Chars read : 2700816".
>>>
>>> I've tried a bunch of different things:
>>>
>>>   - `Pipe.read_all`:
>>>      ```
>>>        let _ =
>>>          Async.Pipe.read_all r >>|
>>>          fun q -> Queue.iter q ~f:(fun s -> printf "%s" s) in
>>>        ...
>>>
>>>      ```
>>>      No output.
>>>
>>>   - `Pipe.drain_and_count`:
>>>     ```
>>>       let _ =
>>>         Async.Pipe.drain_and_count r >>|
>>>         fun n -> printf "Count %d\n" n in
>>>       ...
>>>
>>>     ```
>>>     No output.
>>>
>>>   - `Pipe.to_list`:
>>>      ```
>>>      let _ =
>>>        Async.Pipe.to_list r >>|
>>>        fun l -> printf "%s" (String.concat ~sep:"" l) in
>>>      ...
>>>
>>>      ```
>>>      No output.
>>>
>>> Also tried the obvious:
>>>
>>>   - `Cohttp_async.Body.to_string`:
>>>     ```
>>>       let s = Cohttp_async.Body.to_string body in
>>>       let _ = s >>| fun  s -> printf "%s" s in
>>>       ...
>>>
>>>     ```
>>>     No output.
>>>
>>> I'm new to this and obviously missing something fundamental :) Is there
>>> a kind soul out there who can give me a hint on how to proceed please?
>>>
>>> Thanks!
>>>
>>> --
>>> Shayne Fletcher
>>>
>>
>>
>


-- 
Shayne Fletcher

[-- Attachment #2: Type: text/html, Size: 13019 bytes --]