[Caml-list] Who was working on ocaml bindings for zeromq?

[Caml-list] Who was working on ocaml bindings for zeromq?

[Goswin von Brederlow]

Hi,

last year someone here was working on bindings for zeromq.

Is that still ongoing and how is it going? Does it support zeromq 4.0
with CURVE?

MfG
	Goswin

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Daniel Bünzli]

Le lundi, 13 janvier 2014 à 16:48, Goswin von Brederlow a écrit :

> Hi,
>  
> last year someone here was working on bindings for zeromq.
There's at least this:

> opam list --all | grep zmq | cut -d " " -f 1 | xargs opam info

             package: lwt-zmq
             version: 1.0-beta4
          repository: default
        upstream-url: https://github.com/hcarty/lwt-zmq/archive/v1.0-beta4.tar.gz
       upstream-kind: http
   upstream-checksum: 29338d125a545daf45df9e3d7631d01d
            homepage: https://github.com/hcarty/lwt-zmq
             depends: ocamlfind & lwt & ocaml-zmq
   installed-version:  
  available-versions: 1.0-beta3, 1.0-beta4
         description: Lwt-friendly wrapping around ZeroMQ sockets

             package: ocaml-zmq
             version: 0
          repository: default
        upstream-url: https://github.com/pdhborges/ocaml-zmq/tarball/master
       upstream-kind: http
   upstream-checksum: 8e845370b99630c2a84cf4495480522e
            homepage: https://github.com/pdhborges/ocaml-zmq
             depends: ocamlfind & ounit & uint
   installed-version:  
   available-version: 0
         description: OCaml bindings for ZMQ 2.1



Daniel

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Goswin von Brederlow]

On Mon, Jan 13, 2014 at 04:58:05PM +0100, Daniel Bünzli wrote:
> 
> 
> Le lundi, 13 janvier 2014 à 16:48, Goswin von Brederlow a écrit :
> 
> > Hi,
> >  
> > last year someone here was working on bindings for zeromq.
> There's at least this:
> 
> > opam list --all | grep zmq | cut -d " " -f 1 | xargs opam info
> 
>              package: ocaml-zmq
>              version: 0
>           repository: default
>         upstream-url: https://github.com/pdhborges/ocaml-zmq/tarball/master
>        upstream-kind: http
>    upstream-checksum: 8e845370b99630c2a84cf4495480522e
>             homepage: https://github.com/pdhborges/ocaml-zmq
>              depends: ocamlfind & ounit & uint
>    installed-version:  
>    available-version: 0
>          description: OCaml bindings for ZMQ 2.1
> 
> 
> 
> Daniel

Thanks, those look outdated but good. Combining that with the
libsodium bindings David Sheets pointed out and updating to ZMQ 4.0
should do the trick.

I should start using opam.

MfG
	Goswin

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Anders Peter Fugmann]

Hi,

At Issuu we are activly maintaing a fork of pdhborges ocaml-zmq 
bindings. We have updated it to support version 3.2 and added new 
features such as socket event listening.

I do not know how much work it would require to update that to version 
4.0, but I would expect it to be rather strait forward.

You can find the github fork here:
https://github.com/issuu/ocaml-zmq

Regards
Anders Fugmann



On 16/01/14 09:35, Goswin von Brederlow wrote:
> On Mon, Jan 13, 2014 at 04:58:05PM +0100, Daniel Bünzli wrote:
>>
>>
>> Le lundi, 13 janvier 2014 à 16:48, Goswin von Brederlow a écrit :
>>
>>> Hi,
>>>
>>> last year someone here was working on bindings for zeromq.
>> There's at least this:
>>
>>> opam list --all | grep zmq | cut -d " " -f 1 | xargs opam info
>>
>>               package: ocaml-zmq
>>               version: 0
>>            repository: default
>>          upstream-url: https://github.com/pdhborges/ocaml-zmq/tarball/master
>>         upstream-kind: http
>>     upstream-checksum: 8e845370b99630c2a84cf4495480522e
>>              homepage: https://github.com/pdhborges/ocaml-zmq
>>               depends: ocamlfind & ounit & uint
>>     installed-version:
>>     available-version: 0
>>           description: OCaml bindings for ZMQ 2.1
>>
>>
>>
>> Daniel
>
> Thanks, those look outdated but good. Combining that with the
> libsodium bindings David Sheets pointed out and updating to ZMQ 4.0
> should do the trick.
>
> I should start using opam.
>
> MfG
> 	Goswin
>

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Goswin von Brederlow]

On Thu, Jan 16, 2014 at 10:04:04PM +0100, Anders Peter Fugmann wrote:
> Hi,
> 
> At Issuu we are activly maintaing a fork of pdhborges ocaml-zmq
> bindings. We have updated it to support version 3.2 and added new
> features such as socket event listening.
> 
> I do not know how much work it would require to update that to
> version 4.0, but I would expect it to be rather strait forward.
> 
> You can find the github fork here:
> https://github.com/issuu/ocaml-zmq
> 
> Regards
> Anders Fugmann
> 
> 
> 
> On 16/01/14 09:35, Goswin von Brederlow wrote:
> >On Mon, Jan 13, 2014 at 04:58:05PM +0100, Daniel Bünzli wrote:
> >>
> >>
> >>Le lundi, 13 janvier 2014 à 16:48, Goswin von Brederlow a écrit :
> >>
> >>>Hi,
> >>>
> >>>last year someone here was working on bindings for zeromq.
> >>There's at least this:
> >>
> >>>opam list --all | grep zmq | cut -d " " -f 1 | xargs opam info
> >>
> >>              package: ocaml-zmq
> >>              version: 0
> >>           repository: default
> >>         upstream-url: https://github.com/pdhborges/ocaml-zmq/tarball/master
> >>        upstream-kind: http
> >>    upstream-checksum: 8e845370b99630c2a84cf4495480522e
> >>             homepage: https://github.com/pdhborges/ocaml-zmq
> >>              depends: ocamlfind & ounit & uint
> >>    installed-version:
> >>    available-version: 0
> >>          description: OCaml bindings for ZMQ 2.1
> >>
> >>
> >>
> >>Daniel
> >
> >Thanks, those look outdated but good. Combining that with the
> >libsodium bindings David Sheets pointed out and updating to ZMQ 4.0
> >should do the trick.
> >
> >I should start using opam.
> >
> >MfG
> >	Goswin

Does that mean pdhborges is unresponsive to pull requests?

So far I've added support for STREAM sockets to his ocaml-zmq, fixed a
bug in get_byte_socketoption and added a workaround for sending zero
sized strings (stable libzmq gives an EFAULT in zmq_msg_close() with
them). Trivial changes so far. Adding CURVE suport will be a lot more
work. And since I don't need that right now ...


I've been considering interface changes though. Or additions.

1) Track zmq resources with the Gc

Currently you can create a context and sockets and simply forget about
them. The sockets will never be closed and the context never destroyed.
Or you can close/destroy them and still use them.

I would like to add a finalizer to each and a "valid" flag. On
close/destroy the "valid" flag gets set to false. Trying to use them
then would raise an exception. The finalizer would also check the
"valid" flag and if true would print a warning to stderr and close the
socket or destroy the context. This isn't ment as replacement for
manually closing/destroying. Just a safeguard in case you forget.

2) send should have ?no_block and ?more (similar for recv)

I don't like the sndoption variant. Why not simply have 2 optional
bools in send? And lets get rid of the negation too:

val send : ?block:bool -> ?more:bool -> 'a 'a t -> string -> unit

let send ?(block=true) ?(more=false) sock str ) ...

Having them boot also is easier to type. No need to use the module
path.

3) sockets have phantom types, use them

You can't recv from a PUSH socket or send to a SUB socket. The socket
type is already polymorphic with a phantom type. Use that in send/recv
to limit the kinds of sockets that can be used.

4) Polling is ugly

Creating a mask, polling and then figuring out which socket had what
event is complex and every user will have to do that. It would be
better to have another interface with a callback or returning a list
of sockets and events that occured or something.

5) OO interface

I've considered an interface with objects. The usage would then look
something like this:

let ctx = new ZMQ.ctx
let sock = ctx#socket ZMQ.REQ
sock#send "Hello";
Printf.printf "%s\n" sock#recv;
sock#close;
ctx#destroy

Different socket kinds could have different send/recv signatures. For
example ROUTER/DEALER sockets start with routing frames:

val send_msg : ?block:bool -> string list -> string list -> unit

The class would then send the strings with seperaring 0 message and
handle the ZMQ_SNDMORE flag internally.

The OO interface would be simply on top of the existing one. I don't
see why we can't have both.

MfG
	Goswin

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Anders Peter Fugmann]

Hi Goswin,

Please see my reply below. For completeness your original mail is also 
included in full at the end of the mail.

On 18/01/14 12:33, Goswin von Brederlow wrote:
> Does that mean pdhborges is unresponsive to pull requests?
I guess that he is just busy. He did merge the first set of changes to 
support zmq 3.2, but has yet to merge a set of updates / extenstions to 
the API.

> 1) Track zmq resources with the Gc
Sounds like a nice addition.

> 2) send should have ?no_block and ?more (similar for recv)
We just implemented the exact same interface changes a week ago.

> 3) sockets have phantom types, use them
Agreed.

> 4) Polling is ugly
Indeed this could be made simpler.

> 5) OO interface
I guess OO is a matter of personal taste, but I agree that one does not 
exclude the other.

Pull requests will be most welcome.

Regards
Anders Fugmann


On 18/01/14 12:33, Goswin von Brederlow wrote:
> On Thu, Jan 16, 2014 at 10:04:04PM +0100, Anders Peter Fugmann wrote:
>> Hi,
>>
>> At Issuu we are activly maintaing a fork of pdhborges ocaml-zmq
>> bindings. We have updated it to support version 3.2 and added new
>> features such as socket event listening.
>>
>> I do not know how much work it would require to update that to
>> version 4.0, but I would expect it to be rather strait forward.
>>
>> You can find the github fork here:
>> https://github.com/issuu/ocaml-zmq
>>
>> Regards
>> Anders Fugmann
>>

> Does that mean pdhborges is unresponsive to pull requests?
>
> So far I've added support for STREAM sockets to his ocaml-zmq, fixed a
> bug in get_byte_socketoption and added a workaround for sending zero
> sized strings (stable libzmq gives an EFAULT in zmq_msg_close() with
> them). Trivial changes so far. Adding CURVE suport will be a lot more
> work. And since I don't need that right now ...
>
>
> I've been considering interface changes though. Or additions.
>
> 1) Track zmq resources with the Gc
>
> Currently you can create a context and sockets and simply forget about
> them. The sockets will never be closed and the context never destroyed.
> Or you can close/destroy them and still use them.
>
> I would like to add a finalizer to each and a "valid" flag. On
> close/destroy the "valid" flag gets set to false. Trying to use them
> then would raise an exception. The finalizer would also check the
> "valid" flag and if true would print a warning to stderr and close the
> socket or destroy the context. This isn't ment as replacement for
> manually closing/destroying. Just a safeguard in case you forget.
>
> 2) send should have ?no_block and ?more (similar for recv)
>
> I don't like the sndoption variant. Why not simply have 2 optional
> bools in send? And lets get rid of the negation too:
>
> val send : ?block:bool -> ?more:bool -> 'a 'a t -> string -> unit
>
> let send ?(block=true) ?(more=false) sock str ) ...
>
> Having them boot also is easier to type. No need to use the module
> path.
>
> 3) sockets have phantom types, use them
>
> You can't recv from a PUSH socket or send to a SUB socket. The socket
> type is already polymorphic with a phantom type. Use that in send/recv
> to limit the kinds of sockets that can be used.
>
> 4) Polling is ugly
>
> Creating a mask, polling and then figuring out which socket had what
> event is complex and every user will have to do that. It would be
> better to have another interface with a callback or returning a list
> of sockets and events that occured or something.
>
> 5) OO interface
>
> I've considered an interface with objects. The usage would then look
> something like this:
>
> let ctx = new ZMQ.ctx
> let sock = ctx#socket ZMQ.REQ
> sock#send "Hello";
> Printf.printf "%s\n" sock#recv;
> sock#close;
> ctx#destroy
>
> Different socket kinds could have different send/recv signatures. For
> example ROUTER/DEALER sockets start with routing frames:
>
> val send_msg : ?block:bool -> string list -> string list -> unit
>
> The class would then send the strings with seperaring 0 message and
> handle the ZMQ_SNDMORE flag internally.
>
> The OO interface would be simply on top of the existing one. I don't
> see why we can't have both.
>
> MfG
> 	Goswin
>

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Goswin von Brederlow]

On Sun, Jan 19, 2014 at 06:16:29PM +0100, Anders Peter Fugmann wrote:
> Hi Goswin,
> 
> Please see my reply below. For completeness your original mail is
> also included in full at the end of the mail.
> 
> On 18/01/14 12:33, Goswin von Brederlow wrote:
> >3) sockets have phantom types, use them
> Agreed.
> 
> >4) Polling is ugly
> Indeed this could be made simpler.

Yesterday I actually run into a problem with this. Creating a socket
gave me a odd type (from memory):

val stream: _[> `Push | `Pull | `Pub | `Sub | `Stream] ZMQ.Socket.t
val dealer: _[> `Push | `Pull | `Pub | `Sub | `Dealer] ZMQ.Socket.t

A few problems with that:

1) A stream socket should be just `Stream.

2) Note the _ there. The type is not polymorphic and gives an error
that it can't be unified or something. Don't have the exact error in
memory.

I think there is something wrong with the way the socket types are
specified that cause this. First time I ever got that kind of error.

3) Polling takes an ('a ZMQ.Socket.t, ZMQ.Socket.event) array. Trying
to poll a stream and dealer socket causes type errors.

I think phantom typed sockets are incompatible with putting them into
an array outside the module. This needs an interface where you have a
poller instance and can add sockets to it one at a time. Internally
the phantom type is known to be "phantom" and the sockets can be put
into an array, hashtbl or list.

I didn't have time yet to look into those problems. Currently I used
Obj.magic to make it work because I wanted to work on the application
instead of on zmq. But that certainly isn't a solution.


So in summary: We need a new polling interface (or phantom typing).
Not just because I don't like it but because it doesn't work for
combining different socket types.

MfG
	Goswin

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Gabriel Scherer]

> 2) Note the _ there. The type is not polymorphic and gives an error
> that it can't be unified or something. Don't have the exact error in
> memory.
>
> I think there is something wrong with the way the socket types are
> specified that cause this. First time I ever got that kind of error.

This smells like value restriction kicking in because your avstract
type is not marked as covariant, as explained for example here:
  http://www.reddit.com/r/ocaml/comments/t8g2e/pdf_relaxing_the_value_restriction_by_jacques/c4kf68s

Note that there is a risk of bad interaction between:
- adding a covariance annotation to your phantom type
- using polymorphic variants for you phantom type parameter
- switching from a purely-phantom to an internally-safe GADT implementation

Covariance (or any other annotation) is always accepted for a
purely-phantom implementation, but would often be rejected by the
type-checker for a corresponding GADT implementation (and would be
unsound anymay with many modes of use of polymorphic variants for that
purpose). Switching to a GADT implementation is often an excellent
idea in the life of a library using phantom types (I have no idea
whether that is the case here), so it may be best to avoid variance
claims and just add a unit parameter to those values that hit typing
problems.


On Mon, Jan 20, 2014 at 10:12 AM, Goswin von Brederlow
<goswin-v-b@web.de> wrote:
> On Sun, Jan 19, 2014 at 06:16:29PM +0100, Anders Peter Fugmann wrote:
>> Hi Goswin,
>>
>> Please see my reply below. For completeness your original mail is
>> also included in full at the end of the mail.
>>
>> On 18/01/14 12:33, Goswin von Brederlow wrote:
>> >3) sockets have phantom types, use them
>> Agreed.
>>
>> >4) Polling is ugly
>> Indeed this could be made simpler.
>
> Yesterday I actually run into a problem with this. Creating a socket
> gave me a odd type (from memory):
>
> val stream: _[> `Push | `Pull | `Pub | `Sub | `Stream] ZMQ.Socket.t
> val dealer: _[> `Push | `Pull | `Pub | `Sub | `Dealer] ZMQ.Socket.t
>
> A few problems with that:
>
> 1) A stream socket should be just `Stream.
>
> 2) Note the _ there. The type is not polymorphic and gives an error
> that it can't be unified or something. Don't have the exact error in
> memory.
>
> I think there is something wrong with the way the socket types are
> specified that cause this. First time I ever got that kind of error.
>
> 3) Polling takes an ('a ZMQ.Socket.t, ZMQ.Socket.event) array. Trying
> to poll a stream and dealer socket causes type errors.
>
> I think phantom typed sockets are incompatible with putting them into
> an array outside the module. This needs an interface where you have a
> poller instance and can add sockets to it one at a time. Internally
> the phantom type is known to be "phantom" and the sockets can be put
> into an array, hashtbl or list.
>
> I didn't have time yet to look into those problems. Currently I used
> Obj.magic to make it work because I wanted to work on the application
> instead of on zmq. But that certainly isn't a solution.
>
>
> So in summary: We need a new polling interface (or phantom typing).
> Not just because I don't like it but because it doesn't work for
> combining different socket types.
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[Anders Peter Fugmann]

Hi Goswin,

I cannot reproduce the type problems when mixing different socket types, 
including dealer socket type, in the polling interface.

Can you reproduce the same problems against the Issuu fork of ocaml-zmq, 
and post an example that exposes the problem?

Regards
Anders Fugmann


On 20/01/14 10:12, Goswin von Brederlow wrote:
> Yesterday I actually run into a problem with this. Creating a socket
> gave me a odd type (from memory):
>
> val stream: _[> `Push | `Pull | `Pub | `Sub | `Stream] ZMQ.Socket.t
> val dealer: _[> `Push | `Pull | `Pub | `Sub | `Dealer] ZMQ.Socket.t
>
> A few problems with that:
>
> 1) A stream socket should be just `Stream.
>
> 2) Note the _ there. The type is not polymorphic and gives an error
> that it can't be unified or something. Don't have the exact error in
> memory.
>
> I think there is something wrong with the way the socket types are
> specified that cause this. First time I ever got that kind of error.
>
> 3) Polling takes an ('a ZMQ.Socket.t, ZMQ.Socket.event) array. Trying
> to poll a stream and dealer socket causes type errors.
>
> I think phantom typed sockets are incompatible with putting them into
> an array outside the module. This needs an interface where you have a
> poller instance and can add sockets to it one at a time. Internally
> the phantom type is known to be "phantom" and the sockets can be put
> into an array, hashtbl or list.
>
> I didn't have time yet to look into those problems. Currently I used
> Obj.magic to make it work because I wanted to work on the application
> instead of on zmq. But that certainly isn't a solution.
>
>
> So in summary: We need a new polling interface (or phantom typing).
> Not just because I don't like it but because it doesn't work for
> combining different socket types.
>
> MfG
> 	Goswin
>

Re: [Caml-list] Who was working on ocaml bindings for zeromq?

[David Sheets]

On Mon, Jan 13, 2014 at 3:48 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Hi,
>
> last year someone here was working on bindings for zeromq.
>
> Is that still ongoing and how is it going? Does it support zeromq 4.0
> with CURVE?

<https://github.com/fmp88> was working on
<https://github.com/fmp88/ocaml-zeromq> and
<https://github.com/fmp88/ocaml-czmq> but I don't know much about
their status.

Also, I have <https://github.com/dsheets/ocaml-sodium> which is
sufficient to implement DNSCurve.

These all use ctypes which should be getting stub generation Real Soon Now.

Hope this helps,

David

> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs