[Caml-list] Compressing stream in Async.Tcp.Server

[Caml-list] Compressing stream in Async.Tcp.Server

[Paolo Donadeo]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

Hi all, I'm writing a simple client/server application using Async.Rpc. The
skeletons of the client and server is essentially derived from the examples
available on line.

Since the messages are highly compressible, I would like to use ZLib (e.g.
Cryptokit) but I can't figure out how.

Any ideas?

CLIENT:
*    Tcp.with_connection*
*      (Tcp.to_host_and_port host port)*
*      ~timeout:(sec 1.)*
*      (fun _ r w ->*
*        (* COMPRESSION HERE, HOW? *)*
*        Rpc.Connection.create r w ~connection_state:Pervasives.ignore*
*        >>= function*
*        | Error exn -> raise exn*
*        | Ok conn   -> begin*
*            let msg = {*
*              Rpc_protocol.scada_uuid = conf.Config_manager.scada_uuid;*
*              measures;*
*            } in*
*            Rpc.Rpc.dispatch Rpc_protocol.export_data_rpc conn msg >>=
function*
*            | Ok response -> return response*
*            | Error e -> begin*
*                error ~tags "SERVER SIDE EXCEPTION";*
*                error ~tags "    %s" (Error.to_string_hum e);*
*                return false*
*              end*
*          end*
*      )*


SERVER:
*  Tcp.Server.create*
*    ~on_handler_error:(`Call (fun _ exn -> Log.Global.sexp (Exn.sexp_of_t
exn)))*
*    (Tcp.on_port port)*
*    (fun _addr r w ->*
*      (* COMPRESSION HERE, HOW? *)*
*      Rpc.Connection.server_with_close r w*
*        ~connection_state:(fun c ->*
*          {*
*            db_thread = db_thread;*
*          }*
*        )*
*        ~on_handshake_error:(*
*          `Call (fun exn -> Log.Global.sexp (Exn.sexp_of_t exn); return
()))*
*        ~implementations)*


-- 
*Paolo*

[-- Attachment #2: Type: text/html, Size: 4371 bytes --]

Re: [Caml-list] Compressing stream in Async.Tcp.Server

[Yaron Minsky]

Looping in some folk who work on Async-RPC.

This is a good question. I think there isn't an especially nice
solution for this, but one can write an alternate transport satisfying
this interface:

https://github.com/janestreet/async_rpc_kernel/blob/master/src/transport_intf.ml

that does the compression, which should be enough. But this is not
nicely pre-packaged functionality. It would be nice to have that, and
PRs are certainly welcome...

y

On Thu, Nov 24, 2016 at 12:19 PM, Paolo Donadeo <p.donadeo@gmail.com> wrote:
> Hi all, I'm writing a simple client/server application using Async.Rpc. The
> skeletons of the client and server is essentially derived from the examples
> available on line.
>
> Since the messages are highly compressible, I would like to use ZLib (e.g.
> Cryptokit) but I can't figure out how.
>
> Any ideas?
>
> CLIENT:
>     Tcp.with_connection
>       (Tcp.to_host_and_port host port)
>       ~timeout:(sec 1.)
>       (fun _ r w ->
>         (* COMPRESSION HERE, HOW? *)
>         Rpc.Connection.create r w ~connection_state:Pervasives.ignore
>         >>= function
>         | Error exn -> raise exn
>         | Ok conn   -> begin
>             let msg = {
>               Rpc_protocol.scada_uuid = conf.Config_manager.scada_uuid;
>               measures;
>             } in
>             Rpc.Rpc.dispatch Rpc_protocol.export_data_rpc conn msg >>=
> function
>             | Ok response -> return response
>             | Error e -> begin
>                 error ~tags "SERVER SIDE EXCEPTION";
>                 error ~tags "    %s" (Error.to_string_hum e);
>                 return false
>               end
>           end
>       )
>
>
> SERVER:
>   Tcp.Server.create
>     ~on_handler_error:(`Call (fun _ exn -> Log.Global.sexp (Exn.sexp_of_t
> exn)))
>     (Tcp.on_port port)
>     (fun _addr r w ->
>       (* COMPRESSION HERE, HOW? *)
>       Rpc.Connection.server_with_close r w
>         ~connection_state:(fun c ->
>           {
>             db_thread = db_thread;
>           }
>         )
>         ~on_handshake_error:(
>           `Call (fun exn -> Log.Global.sexp (Exn.sexp_of_t exn); return ()))
>         ~implementations)
>
>
> --
> Paolo

Re: [Caml-list] Compressing stream in Async.Tcp.Server

[Francois BERENGER]

On 11/24/2016 06:19 PM, Paolo Donadeo wrote:
> Hi all, I'm writing a simple client/server application using Async.Rpc.
> The skeletons of the client and server is essentially derived from the
> examples available on line.
> 
> Since the messages are highly compressible, I would like to use ZLib
> (e.g. Cryptokit) but I can't figure out how.

I have some using-cryptokit compression example in there (function zlib):

https://github.com/UnixJunkie/daft/blob/master/src/socket_wrapper.ml

> Any ideas?
> 
> CLIENT:
> *    Tcp.with_connection*
> *      (Tcp.to_host_and_port host port)*
> *      ~timeout:(sec 1.)*
> *      (fun _ r w ->*
> *        (* COMPRESSION HERE, HOW? *)*
> *        Rpc.Connection.create r w ~connection_state:Pervasives.ignore*
> *        >>= function*
> *        | Error exn -> raise exn*
> *        | Ok conn   -> begin*
> *            let msg = {*
> *              Rpc_protocol.scada_uuid = conf.Config_manager.scada_uuid;*
> *              measures;*
> *            } in*
> *            Rpc.Rpc.dispatch Rpc_protocol.export_data_rpc conn msg >>=
> function*
> *            | Ok response -> return response*
> *            | Error e -> begin*
> *                error ~tags "SERVER SIDE EXCEPTION";*
> *                error ~tags "    %s" (Error.to_string_hum e);*
> *                return false*
> *              end*
> *          end*
> *      )*
> 
> 
> SERVER:
> *  Tcp.Server.create*
> *    ~on_handler_error:(`Call (fun _ exn -> Log.Global.sexp
> (Exn.sexp_of_t exn)))*
> *    (Tcp.on_port port)*
> *    (fun _addr r w ->*
> *      (* COMPRESSION HERE, HOW? *)*
> *      Rpc.Connection.server_with_close r w*
> *        ~connection_state:(fun c ->*
> *          {*
> *            db_thread = db_thread;*
> *          }*
> *        )*
> *        ~on_handshake_error:(*
> *          `Call (fun exn -> Log.Global.sexp (Exn.sexp_of_t exn); return
> ()))*
> *        ~implementations)*
> 
> 
> -- 
> /Paolo/

-- 
Regards,
Francois.
"When in doubt, use more types"