From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id EDFC67EEF6 for ; Wed, 17 Jun 2015 16:57:54 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of kennethadammiller@gmail.com) identity=pra; client-ip=209.85.214.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of kennethadammiller@gmail.com designates 209.85.214.177 as permitted sender) identity=mailfrom; client-ip=209.85.214.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="kennethadammiller@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ob0-f177.google.com) identity=helo; client-ip=209.85.214.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="kennethadammiller@gmail.com"; x-sender="postmaster@mail-ob0-f177.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0B6AQAfioFVm7HWVdFcgkWBfgaDGMIlAoEvB0wBAQEBAQESAQEBAQEGCwsJIS6EIgEBAQMBEhEEGQEbHQEDAQsGAwILAwogAQkCAiEBAREBBQEcBhMIGod3AQMKCJZjkGs+MYs/gWuCeYtPChknDVeEagEBAQEBBQEBAQEBARYBBQ6LNoJNgW5LB4JogUMFk2iJZoFhgTSHE4hDgzuCERIjgQwJgi4DHIFuIjGBAySBIQEBAQ X-IPAS-Result: A0B6AQAfioFVm7HWVdFcgkWBfgaDGMIlAoEvB0wBAQEBAQESAQEBAQEGCwsJIS6EIgEBAQMBEhEEGQEbHQEDAQsGAwILAwogAQkCAiEBAREBBQEcBhMIGod3AQMKCJZjkGs+MYs/gWuCeYtPChknDVeEagEBAQEBBQEBAQEBARYBBQ6LNoJNgW5LB4JogUMFk2iJZoFhgTSHE4hDgzuCERIjgQwJgi4DHIFuIjGBAySBIQEBAQ X-IronPort-AV: E=Sophos;i="5.13,633,1427752800"; d="scan'208";a="165832328" Received: from mail-ob0-f177.google.com ([209.85.214.177]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 17 Jun 2015 16:57:53 +0200 Received: by obbgp2 with SMTP id gp2so34684012obb.2 for ; Wed, 17 Jun 2015 07:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4ZTKmyTVFx8DUWeLZw1x9lqkRFyJE5ywxHPabIqk074=; b=VvFwEuiaV4WQ2JxgsWNNIb/bT7GbW03AjdcVW0TWwANzn03KrQh7vWXlTDGf5Lc4+A +WEOsjR0nXwu6JoMxSVWX1EkHG11HsgX05qrmiL/ev8FMdaDJtXtRdf0bh+TLlJxQgo6 TGbj9RmxWEc8BxsWEQFyymUdFVofS4AvfrkCQxRjYhzToLqSZ7nSOm7xhyKWsoiSgZzf IvrTHn5e/QWZn5xPHRjXqZL9+bRtnLOrwxx9tJuHalTQ/THpoXhs+2hz+UsqSb2Svwq1 WTlfNVZNqXKJVNk6veKVx30S5LASwolXzaGVff5gcI6x27wfGta3swaGFzIqbgm0fqmv QoSw== MIME-Version: 1.0 X-Received: by 10.60.50.168 with SMTP id d8mr4984715oeo.41.1434553072316; Wed, 17 Jun 2015 07:57:52 -0700 (PDT) Received: by 10.202.191.8 with HTTP; Wed, 17 Jun 2015 07:57:52 -0700 (PDT) In-Reply-To: References: Date: Wed, 17 Jun 2015 10:57:52 -0400 Message-ID: From: Kenneth Adam Miller To: David House Cc: caml users Content-Type: multipart/alternative; boundary=001a11c30bf04979d30518b7ea08 Subject: Re: [Caml-list] Async Server not executing --001a11c30bf04979d30518b7ea08 Content-Type: text/plain; charset=UTF-8 I had to change it to make not weird, because I found that because writer.write wasn't sending the Eof like you said, and it wasn't causally correct so my code was hanging. I found that out when I got it working yesterday. My tests all run normally now! Thank you so so much. On Wed, Jun 17, 2015 at 4:09 AM, David House wrote: > Right, length-prefixing is another common trick. > > In fact, this is exactly what Writer.send does. Your code at the moment is > a little weird because it uses Reader.recv, whose documentation says "[recv > t] returns a string that was written with Writer.send", but you're not > using Writer.send. I think if you use those pairs of things then you get > the semantics you want. > > More generally, async has good support for reading chunks of data, which > might contain partial messages, and then putting those chunks together to > form full messages. See Unpack_sequence in async_extra, which works with > Unpack_buffer in core_kernel. > > On 16 June 2015 at 18:50, Kenneth Adam Miller > wrote: > >> Ok, I have one last question - I've gotten my unit tests to succeed >> properly, but I'm concerned. >> >> At one point in the Pipe documentation, it talked about how writes a >> recvs would guarantee progress by returning early if they had something at >> all. I just need some kind of guarantee of the semantics of a send and >> recv, so that if I send a string of length 4, I don't receive multiple >> fragments. >> >> On Tue, Jun 16, 2015 at 11:48 AM, Kenneth Adam Miller < >> kennethadammiller@gmail.com> wrote: >> >>> Ah. Well I think I can incorporate from that what I can, but my unit >>> tests need to be reflective of the use case I have. I'm sending protobuf >>> encoded messages between two processes on one machine-whatever the size of >>> the message, that's the size that should be received on the other end (I've >>> read about the returning partial bytes, I can't decode a part of a proto >>> message, it has to be the right size). >>> >>> I can't use write_line, I'll have to find a way to delimit the messages >>> based on size. I think I'll just prepend every message with the size that >>> it should expect, and then read a integer off the stream and then that many >>> bytes. >>> >>> On Tue, Jun 16, 2015 at 11:41 AM, David House >>> wrote: >>> >>>> Ah, now I read your code in more detail I think I see why. >>>> >>>> Reader.contents on the server side will block until eof. The client >>>> side sends some stuff, but does not close the writer, so the server never >>>> sees eof. (Recall that sockets are not like files: it's possible to read >>>> all of the available data right now, but not reach eof.) >>>> >>>> Network protocols normally have some explicit "this is the end of one >>>> message" marker, like a newline or something similar. Then the server just >>>> reads chunks until it sees that marker, at which point it can put the >>>> message together and to something with it. >>>> >>>> For example, you could use Writer.write_line on the client side and >>>> Reader.read_line on the server side. >>>> >>>> On 16 June 2015 at 16:36, Kenneth Adam Miller < >>>> kennethadammiller@gmail.com> wrote: >>>> >>>>> So, now I can get server received if I add that into the callback, but >>>>> at "writing shutdown to server" I don't see response received or even >>>>> something for Eof. >>>>> >>>>> On Tue, Jun 16, 2015 at 11:09 AM, David House >>>>> wrote: >>>>> >>>>>> The first thing to try is to make sure that everything is getting >>>>>> flushed. For temporary debugging messages I strongly recommend just using >>>>>> [Core.Std.eprintf "\n%!"]. >>>>>> >>>>>> On 16 June 2015 at 16:03, Kenneth Adam Miller < >>>>>> kennethadammiller@gmail.com> wrote: >>>>>> >>>>>>> I'm having trouble with OCaml Async. I wrote a small server with it, >>>>>>> and right now I'm trying to unit test that server. Here's my code for the >>>>>>> server: >>>>>>> >>>>>>> >>>>>>> let _main ()= >>>>>>> print_endline "Server running"; >>>>>>> let handler = print_endline in >>>>>>> let socket = Tcp.on_port 5554 in >>>>>>> let server = Tcp.Server.create socket (fun addr r w -> >>>>>>> (Reader.contents r) >>| handler; (Writer.write w "got it")) in >>>>>>> server >>>>>>> >>>>>>> >>>>>>> >>>>>>> In my unit test code I have: >>>>>>> >>>>>>> let test_shutdown test_ctxt = Thread_safe.block_on_async_exn (fun () >>>>>>> -> ( >>>>>>> print_endline "test_shutdown"; >>>>>>> let server = Server._main () in >>>>>>> server >>= fun server -> >>>>>>> let where = Tcp.to_host_and_port "127.0.0.1" 5554 in >>>>>>> Tcp.connect where >>= fun s -> >>>>>>> let socket, r, w = s in >>>>>>> ignore (Writer.write w "kill"); >>>>>>> ignore (Writer.flushed w); >>>>>>> (Reader.recv r >>> function >>>>>>> | `Ok result -> print_endline ("writing shutdown to >>>>>>> server" ^ result) >>>>>>> | `Eof -> ()); >>>>>>> return () >>>>>>> )); () >>>>>>> >>>>>>> >>>>>>> >>>>>>> I see test_shutdown and Server running, but not sign of "writing >>>>>>> shutdown to server" or even "got it"; why isn't my server or even any of >>>>>>> the connection executing? >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > --001a11c30bf04979d30518b7ea08 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I had to change it to make not weird, because I found that= because writer.write wasn't sending the Eof like you said, and it wasn= 't causally correct so my code was hanging. I found that out when I got= it working yesterday.

My tests all run normally now! Th= ank you so so much.

On Wed, Jun 17, 2015 at 4:09 AM, David House <dhouse@jan= estreet.com> wrote:
Right, length-prefixing is another common trick.

In fact, this is exactly what Writer.send does. Your code at the moment = is a little weird because it uses Reader.recv, whose documentation says &qu= ot;[recv t] returns a string that was written with Writer.send", but y= ou're not using Writer.send. I think if you use those pairs of things t= hen you get the semantics you want.

More generally= , async has good support for reading chunks of data, which might contain pa= rtial messages, and then putting those chunks together to form full message= s. See Unpack_sequence in async_extra, which works with Unpack_buffer in co= re_kernel.

On 16 June 2015 at 18:50, Kenn= eth Adam Miller <kennethadammiller@gmail.com> wrot= e:
Ok, I have one last q= uestion - I've gotten my unit tests to succeed properly, but I'm co= ncerned.

At one point in the Pipe documentation, it talk= ed about how writes a recvs would guarantee progress by returning early if = they had something at all. I just need some kind of guarantee of the semant= ics of a send and recv, so that if I send a string of length 4, I don't= receive multiple fragments.

On Tue, Jun 16, 2015 at 11:48 AM, Kenneth = Adam Miller <kennethadammiller@gmail.com> wrote:
Ah. Well I think I can in= corporate from that what I can, but my unit tests need to be reflective of = the use case I have. I'm sending protobuf encoded messages between two = processes on one machine-whatever the size of the message, that's the s= ize that should be received on the other end (I've read about the retur= ning partial bytes, I can't decode a part of a proto message, it has to= be the right size).=C2=A0

I can't use write_line, I= 'll have to find a way to delimit the messages based on size. I think I= 'll just prepend every message with the size that it should expect, and= then read a integer off the stream and then that many bytes.

On Tue, J= un 16, 2015 at 11:41 AM, David House <dhouse@janestreet.com> wrote:
Ah, now I re= ad your code in more detail I think I see why.

Reader.co= ntents on the server side will block until eof. The client side sends some = stuff, but does not close the writer, so the server never sees eof. (Recall= that sockets are not like files: it's possible to read all of the avai= lable data right now, but not reach eof.)

Network = protocols normally have some explicit "this is the end of one message&= quot; marker, like a newline or something similar. Then the server just rea= ds chunks until it sees that marker, at which point it can put the message = together and to something with it.

For example, yo= u could use Writer.write_line on the client side and Reader.read_line on th= e server side.

On 16 June 2015 at 16:36, Kenneth Adam Miller <kennethadammiller@gmail.com> wrote:
So, now I can get server received if I add t= hat into the callback, but at "writing shutdown to server" I don&= #39;t see response received or even something for Eof.

On Tue, Jun 16, 2015 a= t 11:09 AM, David House <dhouse@janestreet.com> wrote:
The first thing to try is= to make sure that everything is getting flushed. For temporary debugging m= essages I strongly recommend just using [Core.Std.eprintf "<message= >\n%!"].

On 16 June 2015 at 16:03, Kenneth Adam Miller <= kennethadammiller@gmail.com> wrote:
I'm having trouble with OCaml Async. I wrote a sma= ll server with it, and right now I'm trying to unit test that server. H= ere's my code for the server:


let _main=C2=A0()=3D
=C2=A0 print_endline=C2=A0"Server running";
=C2=A0 let handler=C2=A0=3D=C2=A0print_endline=C2=A0in
=C2=A0 let socket=C2=A0=3D= =C2=A0Tcp.on_port=C2=A05554=C2=A0in
=C2=A0 let server=C2=A0=3D=C2=A0Tcp.Server.create socket=C2=A0(= fun addr r w=C2=A0->=C2=A0 =C2=A0 =C2=A0=C2=A0(Reader.contents = r)=C2=A0>>|=C2=A0handler;=C2=A0(<= span style=3D"margin:0px;padding:0px;border:0px;vertical-align:baseline;col= or:rgb(102,0,102)">Writer.write w=C2=A0"got it"))=C2=A0in<= span style=3D"margin:0px;padding:0px;border:0px;vertical-align:baseline;col= or:rgb(0,0,0)">
=C2=A0 server



In my unit test code I have:

let test_shutdown test_ctxt=C2=A0= =3D=C2=A0Thread_safe.block_on_= async_exn=C2=A0(fun=C2=A0= ()=C2=A0->=C2=A0(
=C2=A0 =C2=A0 =C2=A0 print_endline=C2=A0"test_shutdown";<= span style=3D"margin:0px;padding:0px;border:0px;vertical-align:baseline;col= or:rgb(0,0,0)">
=C2=A0 =C2=A0 =C2=A0 let server=C2=A0=3D=C2=A0Server<= /span>._main=C2=A0()=C2=A0in
=C2=A0 =C2=A0 =C2=A0 server=C2=A0>>=3D=C2=A0fun server=C2=A0->
=C2=A0 =C2=A0 =C2=A0 let= =C2=A0where=C2=A0=3D=C2=A0Tcp.to_host_and_port=C2= =A0"127.0.0.1"=C2= =A05554=C2=A0in
=C2=A0 =C2=A0 =C2=A0=C2=A0= Tcp.c= onnect=C2=A0where=C2=A0>>=3D=C2=A0fun s=C2=A0<= /span>->=C2=A0
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 let socket,=C2=A0r,=C2=A0w=C2= =A0=3D=C2=A0s=C2=A0in
=C2=A0 =C2=A0 =C2=A0 =C2=A0= ignore=C2=A0(Writer<= /span>.write w=C2=A0"kill");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ignore=C2=A0Writer.flushed w);=C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0(Reader.rec= v r=C2=A0>>>=C2= =A0function
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0=C2=A0|=C2=A0`Ok result -> =C2=A0print_endline (= "writing shutdown to server" ^ result)
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 | `Eof=C2= =A0->=C2=A0());
=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return=C2=A0()
=C2=A0 =C2=A0=C2=A0));=C2=A0


I see= test_shutdown and Server running, but not sign of "writing shutdown t= o server" or even "got it"; why isn't my server or even = any of the connection executing?







--001a11c30bf04979d30518b7ea08--