[Caml-list] Beginner's question on Lwt usage

[Caml-list] Beginner's question on Lwt usage

[Raphael Bauduin]

[-- Attachment #1: Type: text/plain, Size: 1118 bytes --]

Hi,

After writing a small tcp server in ocaml, I wanted to make it work
asynchronously with Lwt.
I changed the function handling the accept on the socket to look like this:

let rec accept_connection sock:Unix.file_descr =
  let socket_thread = Lwt.return ( Unix.accept sock ) in
  let _ = Lwt.bind socket_thread
         ( fun (fd, caller) ->
             ignore(Lwt_io.printf "accepted\n%!");
             (*let _ = set_nonblock fd in*)
             Lwt.return (readall fd) >>= fun a ->
                 ignore(Lwt_io.printf "%s\n%!" a);
                 ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
                 Lwt.return 0;
         ); in
   accept_connection sock

The complete code is at http://pastie.org/10971189 .

But it doesn't seem to work. The problems are:
- nothing is printed on stdout
- when I open 2 connections, the second can only be closed after the first
has been closed.

In the mean time I've discovered Lwt_unix, but before looking at it, I'd
like to understand what's wrong in my current code. Can you spot errors in
the code? General advice is also welcome!

Thanks

Raphael

[-- Attachment #2: Type: text/html, Size: 1646 bytes --]

Re: [Caml-list] Beginner's question on Lwt usage

[Koen De Keyser]

[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]

You need to run your lwt code within Lwt_main.run

This starts the lwt scheduler. Otherwise your application just terminates
once it has created the lwt thread (which is a simple Ocaml value).

Koen

On Nov 27, 2016 18:30, "Raphael Bauduin" <rblists@gmail.com> wrote:

> Hi,
>
> After writing a small tcp server in ocaml, I wanted to make it work
> asynchronously with Lwt.
> I changed the function handling the accept on the socket to look like this:
>
> let rec accept_connection sock:Unix.file_descr =
>   let socket_thread = Lwt.return ( Unix.accept sock ) in
>   let _ = Lwt.bind socket_thread
>          ( fun (fd, caller) ->
>              ignore(Lwt_io.printf "accepted\n%!");
>              (*let _ = set_nonblock fd in*)
>              Lwt.return (readall fd) >>= fun a ->
>                  ignore(Lwt_io.printf "%s\n%!" a);
>                  ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
>                  Lwt.return 0;
>          ); in
>    accept_connection sock
>
> The complete code is at http://pastie.org/10971189 .
>
> But it doesn't seem to work. The problems are:
> - nothing is printed on stdout
> - when I open 2 connections, the second can only be closed after the first
> has been closed.
>
> In the mean time I've discovered Lwt_unix, but before looking at it, I'd
> like to understand what's wrong in my current code. Can you spot errors in
> the code? General advice is also welcome!
>
> Thanks
>
> Raphael
>
>
>

[-- Attachment #2: Type: text/html, Size: 2306 bytes --]

Re: [Caml-list] Beginner's question on Lwt usage

[Raphael Bauduin]

[-- Attachment #1: Type: text/plain, Size: 2924 bytes --]

Thanks for your answer!

I get the same result with the following code. Note that I can open 2
connections and that the closing of the connection works as expected with a
'close connection' pattern (with one line only with a dot followed by an
empty line). It is just that I cannot close the second connection while the
first stays open, the 'close connection' pattern is not triggering the
close.

let rec accept_connection sock:Unix.file_descr =
  let socket_thread = Lwt.return ( Unix.accept sock ) in
    let _ = Lwt.bind socket_thread
           ( fun (fd, caller) ->
               ignore(Lwt_io.printf "accepted\n%!");
               (*let _ = set_nonblock fd in*)
               Lwt.return (readall fd) >>= fun a ->
                   ignore(Lwt_io.printf "%s\n%!" a);
                   ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
                   Lwt.return 0;
           ); in
     accept_connection sock
let server port =
  let inet_addr = inet_addr_any in
  let sockaddr = ADDR_INET (inet_addr, port) in
  let domain = domain_of_sockaddr sockaddr in
  let sock:Unix.file_descr = socket domain SOCK_STREAM 0 in
  Unix.bind sock sockaddr;
  listen sock 10;
  Lwt_main.run ( Lwt.return (accept_connection sock) )


On Sun, Nov 27, 2016 at 6:41 PM, Koen De Keyser <koen.dekeyser@gmail.com>
wrote:

> You need to run your lwt code within Lwt_main.run
>
> This starts the lwt scheduler. Otherwise your application just terminates
> once it has created the lwt thread (which is a simple Ocaml value).
>
> Koen
>
> On Nov 27, 2016 18:30, "Raphael Bauduin" <rblists@gmail.com> wrote:
>
>> Hi,
>>
>> After writing a small tcp server in ocaml, I wanted to make it work
>> asynchronously with Lwt.
>> I changed the function handling the accept on the socket to look like
>> this:
>>
>> let rec accept_connection sock:Unix.file_descr =
>>   let socket_thread = Lwt.return ( Unix.accept sock ) in
>>   let _ = Lwt.bind socket_thread
>>          ( fun (fd, caller) ->
>>              ignore(Lwt_io.printf "accepted\n%!");
>>              (*let _ = set_nonblock fd in*)
>>              Lwt.return (readall fd) >>= fun a ->
>>                  ignore(Lwt_io.printf "%s\n%!" a);
>>                  ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
>>                  Lwt.return 0;
>>          ); in
>>    accept_connection sock
>>
>> The complete code is at http://pastie.org/10971189 .
>>
>> But it doesn't seem to work. The problems are:
>> - nothing is printed on stdout
>> - when I open 2 connections, the second can only be closed after the
>> first has been closed.
>>
>> In the mean time I've discovered Lwt_unix, but before looking at it, I'd
>> like to understand what's wrong in my current code. Can you spot errors in
>> the code? General advice is also welcome!
>>
>> Thanks
>>
>> Raphael
>>
>>
>>


-- 
Web database: http://www.myowndb.com
Free Software Developers Meeting: http://www.fosdem.org

[-- Attachment #2: Type: text/html, Size: 4746 bytes --]

Re: [Caml-list] Beginner's question on Lwt usage

[Anton Bachin]

[-- Attachment #1: Type: text/plain, Size: 5204 bytes --]

Hi Raphael,

The problem with not being able to close connection 2 while connection 1 is still being serviced is likely due to this code using system call functions from module Unix, rather than Lwt_unix (or other Lwt modules). Lwt doesn’t turn Unix module system calls, such as Unix.recv, into non-blocking ones – they can still block your whole program, even if you use Lwt elsewhere in it. Only the Lwt_unix system call functions are non-blocking. To get good results, you have to switch to using those when you switch to Lwt.

There are several other problems of a similar flavor. For example, Lwt.return (Unix.accept sock) doesn’t create a thread that runs Unix.accept in the background. What that actually does is: it blocks the whole process until Unix.accept completes. Then, it wraps the result of Unix.accept in a “already-finished” Lwt thread (admittedly, the existing Lwt docs are a bit vague on this). There is then no use in “binding” on that thread: it just takes out that result, and is equivalent to simply doing "let fd, caller = Unix.accept sock in”. The Lwt way would be

  open Lwt.Infix

  Lwt_unix.accept sock' >>= fun (fd, caller) ->
  …

or

  let%lwt fd, caller = Lwt_unix.accept sock’ in    (* if building with lwt.ppx *)
  ...

where sock’ is created by “Lwt_unix.of_unix_file_descr sock”, or better, directly by calling Lwt_unix.socket in the first place.

In general, Lwt threads don’t correspond to system threads, so you can’t just wrap existing code in Lwt.return and Lwt.bind, and expect it to become concurrent. Lwt threads correspond to promises (a.k.a. green threads, pick your term), so you have to use "promise-friendly" versions of everything.

If you’d like to discuss this, or your code, in some detail, I recommend #ocaml on freenode. I lurk there, as do many other Lwt users, and we can quickly go back and forth :)

Best,
Anton

> El nov 27, 2016, a las 12:59, Raphael Bauduin <rblists@gmail.com <mailto:rblists@gmail.com>> escribió:
> 
> Thanks for your answer!
> 
> I get the same result with the following code. Note that I can open 2 connections and that the closing of the connection works as expected with a 'close connection' pattern (with one line only with a dot followed by an empty line). It is just that I cannot close the second connection while the first stays open, the 'close connection' pattern is not triggering the close.
> 
> let rec accept_connection sock:Unix.file_descr =
>   let socket_thread = Lwt.return ( Unix.accept sock ) in
>     let _ = Lwt.bind socket_thread
>            ( fun (fd, caller) ->
>                ignore(Lwt_io.printf "accepted\n%!");
>                (*let _ = set_nonblock fd in*)
>                Lwt.return (readall fd) >>= fun a ->
>                    ignore(Lwt_io.printf "%s\n%!" a);
>                    ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
>                    Lwt.return 0;
>            ); in
>      accept_connection sock
> let server port =
>   let inet_addr = inet_addr_any in
>   let sockaddr = ADDR_INET (inet_addr, port) in
>   let domain = domain_of_sockaddr sockaddr in
>   let sock:Unix.file_descr = socket domain SOCK_STREAM 0 in
>   Unix.bind sock sockaddr;
>   listen sock 10;
>   Lwt_main.run ( Lwt.return (accept_connection sock) )
> 
> 
> On Sun, Nov 27, 2016 at 6:41 PM, Koen De Keyser <koen.dekeyser@gmail.com <mailto:koen.dekeyser@gmail.com>> wrote:
> You need to run your lwt code within Lwt_main.run 
> 
> This starts the lwt scheduler. Otherwise your application just terminates once it has created the lwt thread (which is a simple Ocaml value). 
> 
> Koen
> 
> On Nov 27, 2016 18:30, "Raphael Bauduin" <rblists@gmail.com <mailto:rblists@gmail.com>> wrote:
> Hi,
> 
> After writing a small tcp server in ocaml, I wanted to make it work asynchronously with Lwt.
> I changed the function handling the accept on the socket to look like this:
> 
> let rec accept_connection sock:Unix.file_descr =
>   let socket_thread = Lwt.return ( Unix.accept sock ) in
>   let _ = Lwt.bind socket_thread
>          ( fun (fd, caller) ->
>              ignore(Lwt_io.printf "accepted\n%!");
>              (*let _ = set_nonblock fd in*)
>              Lwt.return (readall fd) >>= fun a ->
>                  ignore(Lwt_io.printf "%s\n%!" a);
>                  ignore(Lwt.async (fun () -> Lwt.return (close fd) ) );
>                  Lwt.return 0;
>          ); in
>    accept_connection sock
> 
> The complete code is at http://pastie.org/10971189 <http://pastie.org/10971189> .
> 
> But it doesn't seem to work. The problems are:
> - nothing is printed on stdout
> - when I open 2 connections, the second can only be closed after the first has been closed.
> 
> In the mean time I've discovered Lwt_unix, but before looking at it, I'd like to understand what's wrong in my current code. Can you spot errors in the code? General advice is also welcome!
> 
> Thanks
> 
> Raphael
> 
> 
> 
> 
> 
> -- 
> Web database: http://www.myowndb.com <http://www.myowndb.com/>
> Free Software Developers Meeting: http://www.fosdem.org <http://www.fosdem.org/>

[-- Attachment #2: Type: text/html, Size: 9199 bytes --]