[Caml-list] Tensorflow bindings for OCaml

[Caml-list] Tensorflow bindings for OCaml

[Milo Davis]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

Are there Tensorflow bindings for OCaml?  I've looked around the internet
and can't find any, but I'd like to double check before trying to build my
own.  If not, does anyone have any suggestions for an alternative framework
for neural networks in OCaml?

Thanks,
Milo

[-- Attachment #2: Type: text/html, Size: 339 bytes --]

Re: [Caml-list] Tensorflow bindings for OCaml

[Francois Berenger]

On 03/10/2016 03:19 AM, Milo Davis wrote:
> Are there Tensorflow bindings for OCaml?  I've looked around the
> internet and can't find any, but I'd like to double check before trying
> to build my own.  If not, does anyone have any suggestions for an
> alternative framework for neural networks in OCaml?

Recently, I was looking for some Kohonen Self Organizing Map (SOM)
library in OCaml and I didn't find one.

In opam, it looks like there is nothing related to neural networks
or I missed it.
This smells like a bad news (as in "nothing mature out there").

It looks like I will have to use R the day I really need to use SOM maps.

Here is what I found however:

https://code.google.com/archive/p/ocaml-onnt/

http://cvs.savannah.gnu.org/viewvc/neurocaml/src/?root=neurocaml

Article 71 in the OCaml Journal (not free, unfortunately):

http://www.ffconsultancy.com/products/ocaml_journal/index.html

An interesting e-mail but with dead links:

http://caml.inria.fr/pub/ml-archives/caml-list/1997\/10/e3746cfab892fc757d0d8abcbcc13420.fr.html

If we had at least some bindings in opam to a high quality neural 
networks library, that would be useful to several people ...

-- 
Regards,
Francois.
"When in doubt, use more types"

Re: [Caml-list] Tensorflow bindings for OCaml

[Milo Davis]

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

Okay.  I'll see what needs to be done to wrap the C++ code.  Google
recommends using Swig.  Does anyone have any advice about using it to wrap
C++?

On Thu, Mar 10, 2016 at 10:05 AM, Francois Berenger <
francois.berenger@inria.fr> wrote:

> On 03/10/2016 03:19 AM, Milo Davis wrote:
>
>> Are there Tensorflow bindings for OCaml?  I've looked around the
>> internet and can't find any, but I'd like to double check before trying
>> to build my own.  If not, does anyone have any suggestions for an
>> alternative framework for neural networks in OCaml?
>>
>
> Recently, I was looking for some Kohonen Self Organizing Map (SOM)
> library in OCaml and I didn't find one.
>
> In opam, it looks like there is nothing related to neural networks
> or I missed it.
> This smells like a bad news (as in "nothing mature out there").
>
> It looks like I will have to use R the day I really need to use SOM maps.
>
> Here is what I found however:
>
> https://code.google.com/archive/p/ocaml-onnt/
>
> http://cvs.savannah.gnu.org/viewvc/neurocaml/src/?root=neurocaml
>
> Article 71 in the OCaml Journal (not free, unfortunately):
>
> http://www.ffconsultancy.com/products/ocaml_journal/index.html
>
> An interesting e-mail but with dead links:
>
>
> http://caml.inria.fr/pub/ml-archives/caml-list/1997\/10/e3746cfab892fc757d0d8abcbcc13420.fr.html
>
> If we had at least some bindings in opam to a high quality neural networks
> library, that would be useful to several people ...
>
> --
> Regards,
> Francois.
> "When in doubt, use more types"
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3162 bytes --]

Re: [Caml-list] Tensorflow bindings for OCaml

[Travis]

[-- Attachment #1: Type: text/plain, Size: 2262 bytes --]

I've wanted the same and have kicked around the idea of binding to mxnet
via ctypes.

Here is mxnet's c api if you're curious:
https://github.com/dmlc/mxnet/blob/7773549f5a96eb844d34a3c7814610d4ca26c994/include/mxnet/c_api.h

You might also have a look at
https://medium.com/jim-fleming/loading-a-tensorflow-graph-with-the-c-api-4caaff88463f#.2cxtvz6gd



On Thu, Mar 10, 2016 at 11:43 AM, Milo Davis <davis.mil@husky.neu.edu>
wrote:

> Okay.  I'll see what needs to be done to wrap the C++ code.  Google
> recommends using Swig.  Does anyone have any advice about using it to wrap
> C++?
>
> On Thu, Mar 10, 2016 at 10:05 AM, Francois Berenger <
> francois.berenger@inria.fr> wrote:
>
>> On 03/10/2016 03:19 AM, Milo Davis wrote:
>>
>>> Are there Tensorflow bindings for OCaml?  I've looked around the
>>> internet and can't find any, but I'd like to double check before trying
>>> to build my own.  If not, does anyone have any suggestions for an
>>> alternative framework for neural networks in OCaml?
>>>
>>
>> Recently, I was looking for some Kohonen Self Organizing Map (SOM)
>> library in OCaml and I didn't find one.
>>
>> In opam, it looks like there is nothing related to neural networks
>> or I missed it.
>> This smells like a bad news (as in "nothing mature out there").
>>
>> It looks like I will have to use R the day I really need to use SOM maps.
>>
>> Here is what I found however:
>>
>> https://code.google.com/archive/p/ocaml-onnt/
>>
>> http://cvs.savannah.gnu.org/viewvc/neurocaml/src/?root=neurocaml
>>
>> Article 71 in the OCaml Journal (not free, unfortunately):
>>
>> http://www.ffconsultancy.com/products/ocaml_journal/index.html
>>
>> An interesting e-mail but with dead links:
>>
>>
>> http://caml.inria.fr/pub/ml-archives/caml-list/1997\/10/e3746cfab892fc757d0d8abcbcc13420.fr.html
>>
>> If we had at least some bindings in opam to a high quality neural
>> networks library, that would be useful to several people ...
>>
>> --
>> Regards,
>> Francois.
>> "When in doubt, use more types"
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>
>

[-- Attachment #2: Type: text/html, Size: 4235 bytes --]

Re: [Caml-list] Tensorflow bindings for OCaml

[Jesper Louis Andersen]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

On Thu, Mar 10, 2016 at 6:43 PM, Milo Davis <davis.mil@husky.neu.edu> wrote:

> Okay.  I'll see what needs to be done to wrap the C++ code.  Google
> recommends using Swig.


If memory serves, you only need to support around 4-5 function calls in
order to load graphs and use them in sessions. The graph itself can be
built in python, and the interface looks much like a tensored variant of
Janes St.'s incremental library.

I'd definitely start by getting that part working first, and then add the
remaining functions if need be. But I wouldn't make that a priority. If you
do succeed however, I need a better language than Python for some TF-work :)


-- 
J.

[-- Attachment #2: Type: text/html, Size: 1107 bytes --]

Re: [Caml-list] Tensorflow bindings for OCaml

[Laurent Mazare]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

We've started putting together some TensorFlow bindings for OCaml. It's
very much a work in progress so there are some rough edges: some operators
are not supported, the api is likely to change, you may even get some
segfaults, etc. However it can already be used to train a convolutional
neural network to recognize digits with 98% accuracy.
Here are a couple highlights:
- The code for most operators is automatically generated from the
TensorFlow protobuf specs.
- The supported optimizers are gradient descent, momentum, and the adam
optimizer.
- The gradient backpropagation graph is generated within the OCaml wrapper.
We need to register the gradients for each operator manually, this has
already been done for ~30 operators.
- The api should be pretty straightforward, as an example here is a simple
linear classifier for the MNIST dataset:
https://github.com/LaurentMazare/tensorflow-ocaml/blob/master/examples/mnist/mnist_linear.ml

Some installation instructions can be found on the main page of the project.
https://github.com/LaurentMazare/tensorflow-ocaml
Installation is quite tricky for now as one has to compile or extract
libtensorflow.so but we're hoping to improve this soon.

Any feedback would be very welcome.

(this has been cross-posted to the tensorflow-discuss group)

On Thu, Mar 10, 2016 at 7:58 PM, Jesper Louis Andersen <
jesper.louis.andersen@gmail.com> wrote:

>
> On Thu, Mar 10, 2016 at 6:43 PM, Milo Davis <davis.mil@husky.neu.edu>
> wrote:
>
>> Okay.  I'll see what needs to be done to wrap the C++ code.  Google
>> recommends using Swig.
>
>
> If memory serves, you only need to support around 4-5 function calls in
> order to load graphs and use them in sessions. The graph itself can be
> built in python, and the interface looks much like a tensored variant of
> Janes St.'s incremental library.
>
> I'd definitely start by getting that part working first, and then add the
> remaining functions if need be. But I wouldn't make that a priority. If you
> do succeed however, I need a better language than Python for some TF-work :)
>
>
> --
> J.
>

[-- Attachment #2: Type: text/html, Size: 3235 bytes --]

Re: [Caml-list] Tensorflow bindings for OCaml

[Jesper Louis Andersen]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On Tue, Mar 29, 2016 at 6:01 PM, Laurent Mazare <laurent.mazare@gmail.com>
wrote:

> We've started putting together some TensorFlow bindings for OCaml.
>

Excellent news!

I've built most of an Erlang/OCaml layer so you can run OCaml programs as
computation engines as were they Erlang port programs. Most base-types have
been verified through QuickCheck, though I still need to improve the code a
bit with some refactoring.

If you pipeline messages, the processing roundtrip is around 1.2 million
requests per second on my machine, which is quite fine in most cases. The
overhead of sending the request to OCaml is around 1000 nanos. And this is
without optimization I might add. The overhead is about the same if you
move the computation to a dirty scheduler inside the Erlang node, so if you
can work with the pipelining overhead, it is not too shabby. Of course the
number is worse if you actually do some computation on the OCaml side, but
that is the purpose of it all. At 1000 nanos or less, the feasibility of
computing in OCaml just went up.

Tensorflow in OCaml was next on my TODO list for these kinds of things, but
I would much rather use these bindings than writing my own :)

-- 
J.

[-- Attachment #2: Type: text/html, Size: 1822 bytes --]