[Caml-list] whither portability?

[Caml-list] whither portability?

[Raoul Duke]

I would dearly love to be able to use OCaml everywhere. But the last I
knew, it was not easy to use it on Android & iOS. Is that a solved
problem now? And I do not mean somebody did it once and checked it in,
I mean that it is in constant use, to avoid bit rot and "it worked on
my machine" standard problems when people say something works. ;-)

Re: [Caml-list] whither portability?

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 1468 bytes --]

Am Freitag, den 25.09.2015, 14:13 -0700 schrieb Raoul Duke:
> I would dearly love to be able to use OCaml everywhere. But the last I
> knew, it was not easy to use it on Android & iOS. Is that a solved
> problem now? And I do not mean somebody did it once and checked it in,
> I mean that it is in constant use, to avoid bit rot and "it worked on
> my machine" standard problems when people say something works. ;-)

I was working the past months on iOS support (see
http://caml.inria.fr/mantis/view.php?id=6986), for both 32 bit and 64
bit. This is very close to being solved. The goal is to include it into
the main distribution, and there were very positive signals from the
OCaml team. There are already users of this patch. I am still working on
getting the test suite running which is not easy in a cross-compile
environment.

You are very welcome to testing the patch out and provide feedback. This
is fairly important for this platform as it is quite special and usual
testing strategies do not work well (because of cross-compilation, e.g.
you cannot do a bootstrap cycle).

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] whither portability?

[Raoul Duke]

Exciting to hear! Thank you and your contributors for this work. I
know there are ways people can be parochial, but I have never been
able to fathom how the core OCaml team can ignore mobile so actively,
it seems. It is to sigh, oy veh.

Throwing out random ideas as a potential some-day user of such a thing:

* Getting anything working as simply as possible would be a nice first
step, even if it isn't the most performant. I.e. bytecode. It seems
Apple has loosened up on that a bit over the years? And I wonder if
LLVM bytecode would be no problem with them now.

* FFI will be essential. It is one thing to have 2+2 working on
mobile, it is another to be able to get any real app work done. (In
the past I have had some negative experiences with OCaml and FFI for
OpenGL so I am always a little worried about trying to reignite my
relationship with OCaml.)

* Ideally it has to integrate well with the platform toolchains,
including the preferred IDEs. E.g. if there's no good installer, if I
have to do a lot of work to build things, etc., then I'm way less
likely to even try to use it. I know that is work most people do not
like to do. I say that empirically.

* If you can find some commerical users and support, that could help
drive some of the above issues further faster (although of course the
vector along which they proceed is subjective).

Re: [Caml-list] whither portability?

[whitequark]

I will offer a few comments as the author of opam-android[1].

On 2015-09-26 15:48, Raoul Duke wrote:
> Exciting to hear! Thank you and your contributors for this work. I
> know there are ways people can be parochial, but I have never been
> able to fathom how the core OCaml team can ignore mobile so actively,
> it seems. It is to sigh, oy veh.

No one is being parochial or "actively" ignoring mobile. The OCaml core
team consists of unpaid volunteers. I suggest that next time, in place
of a snide remark, you contribute some code and/or money.

That being said...

> 
> Throwing out random ideas as a potential some-day user of such a thing:
> 
> * Getting anything working as simply as possible would be a nice first
> step, even if it isn't the most performant. I.e. bytecode. It seems
> Apple has loosened up on that a bit over the years? And I wonder if
> LLVM bytecode would be no problem with them now.

Both ocamlc and ocamlopt work for iOS and Android; there is not really
a good reason to use the bytecode compiler for deployment to mobile.

There is no such thing as "LLVM bytecode" and LLVM bitcode (sic)
is merely an intermediate format for native executable code. Shipping
LLVM bitcode offers no conceptual improvement over shipping
machine code and in any case, there is neither an LLVM backend
for ocamlopt nor much need for such a backend.

> 
> * FFI will be essential. It is one thing to have 2+2 working on
> mobile, it is another to be able to get any real app work done. (In
> the past I have had some negative experiences with OCaml and FFI for
> OpenGL so I am always a little worried about trying to reignite my
> relationship with OCaml.)

Definitely. ocaml-ctypes[2] in cstubs mode allows bidirectional
integration of OCaml code with C code, i.e. OCaml can call into C
and C can call into OCaml. Daniel Bünzli's tgls OpenGL bindings
can be made to compile in cstubs mode, but this is not quite
upstream yet[2].

> 
> * Ideally it has to integrate well with the platform toolchains,
> including the preferred IDEs. E.g. if there's no good installer, if I
> have to do a lot of work to build things, etc., then I'm way less
> likely to even try to use it. I know that is work most people do not
> like to do. I say that empirically.

With opam-android, the workflow is exactly identical to building any
regular OCaml project, with the difference that the final binary 
produced
is a .o or .so file that you can use as any other C library in
your Android project; the Android cross-toolchain is just another switch
in opam.

I don't know how it works for iOS but it is almost certainly possible
to achieve the same level of convenience.

[1]: https://github.com/whitequark/opam-android
[2]: https://github.com/dbuenzli/tgls/pull/14

-- 
whitequark

Re: [Caml-list] whither portability?

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

Am Samstag, den 26.09.2015, 18:59 +0300 schrieb whitequark:
> With opam-android, the workflow is exactly identical to building any
> regular OCaml project, with the difference that the final binary 
> produced
> is a .o or .so file that you can use as any other C library in
> your Android project; the Android cross-toolchain is just another switch
> in opam.
> 
> I don't know how it works for iOS but it is almost certainly possible
> to achieve the same level of convenience.

On iOS you just use ocamlopt -output-obj to get a .o and link this into
your app together with libasmrun.a (apps cannot use shared libraries on
iOS so creating an .so is pointless). You can also create a standalone
executable, but this is only useful for jailbroken devices. 

Note that even bytecode works on the device. Maybe useful for a toploop
on the go?

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] whither portability?

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 4204 bytes --]

Am Samstag, den 26.09.2015, 05:48 -0700 schrieb Raoul Duke:
> Exciting to hear! Thank you and your contributors for this work. I
> know there are ways people can be parochial, but I have never been
> able to fathom how the core OCaml team can ignore mobile so actively,
> it seems. It is to sigh, oy veh.

I wouldn't say it was "actively" ignored. It is a matter of resources,
and - to some agree - also a matter of how important you think support
for mobile is. And finally, the devices got first recently so powerful
that it makes a lot of sense to target mobile/desktop/server in similar
ways, in particular with the same dev languages. This is an important
niche for OCaml because the choice of languages on mobile OS is somewhat
restricted.

> * Getting anything working as simply as possible would be a nice first
> step, even if it isn't the most performant. I.e. bytecode. It seems
> Apple has loosened up on that a bit over the years? And I wonder if
> LLVM bytecode would be no problem with them now.

As whitequark already said there is no need to back up to using
bytecode. Both 32 and 64 bit ARM works (the latter is now required for
Apple's appstore). There might be here and there uses for bytecode,
though, e.g. if you need to compile code on the fly.

> * FFI will be essential. It is one thing to have 2+2 working on
> mobile, it is another to be able to get any real app work done.

The FFI works in the normal way. You just need to ensure that you also
cross-compile the wrapped library for the mobile device. If you compile
C files with ocamlc/ocamlopt as driver it just works (and all tools
basing on that work, e.g. ocamlbuild). If you invoke the C compiler
directly you need to ensure to pass the right flags for cross-compiling
(e.g. -arch arm64).

I guess the biggest missing piece here is getting wrappers for the
typical platform libraries.

> * Ideally it has to integrate well with the platform toolchains,
> including the preferred IDEs. E.g. if there's no good installer, if I
> have to do a lot of work to build things, etc., then I'm way less
> likely to even try to use it. I know that is work most people do not
> like to do. I say that empirically.

For the iOS support we are not yet that far.

> * If you can find some commerical users and support, that could help
> drive some of the above issues further faster (although of course the
> vector along which they proceed is subjective).

This is not as you think. The iOS port is actually driven by commercial
users, which has two big advantages: developers have time, and there are
plenty of testing opportunities. These users, however, don't have any
problems with the FFI or need an IDE. This is just a matter of using
development resources well.

Let me just explain for what I need the iOS port. The company I'm
working for developed a query compiler in OCaml for a new database
system. This database backs a UI, and there is also a mobile version of
it. Getting the compiler to mobile is fairly essential for the whole
success of the product. Now, a compiler doesn't need much of the FFI as
it only transforms some data (a query string into bytecode, an int
array). External libs are not really needed. The UI is built with the
platform tools (I think it is still in Objective C), and we deliver the
query compiler simply as a normal library with a C header file to the UI
team.

I guess this is fairly typical of commercial users. There is some highly
complex library written in OCaml providing some functionality you don't
want to duplicate in lower-level languages like Swift (sorry,..., a
matter of perspective). The UI is better to develop with the platform
tools, though (and keep in mind that there is usually a separate UI team
not knowing anything about OCaml).

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] whither portability?

[Raoul Duke]

> No one is being parochial or "actively" ignoring mobile. The OCaml core
> team consists of unpaid volunteers. I suggest that next time, in place
> of a snide remark, you contribute some code and/or money.

i would contribute money. not much, but some. i'm not super rich. :-)

but, no idea where one could do that.

doubt most people would have known even if it were possible.

googlign returns bad broken history about this e.g.
https://github.com/ocaml/opam/issues/326

Re: [Caml-list] whither portability?

[Oliver Bandel]

Zitat von whitequark <whitequark@whitequark.org> (Sat, 26 Sep 2015  
18:59:50 +0300)

> I will offer a few comments as the author of opam-android[1].
>
> On 2015-09-26 15:48, Raoul Duke wrote:
>> Exciting to hear! Thank you and your contributors for this work. I
>> know there are ways people can be parochial, but I have never been
>> able to fathom how the core OCaml team can ignore mobile so actively,
>> it seems. It is to sigh, oy veh.
>
> No one is being parochial or "actively" ignoring mobile. The OCaml core
> team consists of unpaid volunteers.
[...]


Oh, really?
Are you sure?

I thought, they have jobs at INRIA, and can put at least part of their paid
time into OCaml-development!


Ciao,
    Oliver

Re: [Caml-list] whither portability?

[Yotam Barnoy]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

Oh man. You do realize sarcasm doesn't travel on the Internet, right?
Please avoid sarcastic comments before you start a flame war.

On Sat, Sep 26, 2015 at 8:20 PM, Oliver Bandel <oliver@first.in-berlin.de>
wrote:

>
> Zitat von whitequark <whitequark@whitequark.org> (Sat, 26 Sep 2015
> 18:59:50 +0300)
>
> I will offer a few comments as the author of opam-android[1].
>>
>> On 2015-09-26 15:48, Raoul Duke wrote:
>>
>>> Exciting to hear! Thank you and your contributors for this work. I
>>> know there are ways people can be parochial, but I have never been
>>> able to fathom how the core OCaml team can ignore mobile so actively,
>>> it seems. It is to sigh, oy veh.
>>>
>>
>> No one is being parochial or "actively" ignoring mobile. The OCaml core
>> team consists of unpaid volunteers.
>>
> [...]
>
>
> Oh, really?
> Are you sure?
>
> I thought, they have jobs at INRIA, and can put at least part of their paid
> time into OCaml-development!
>
>
> Ciao,
>    Oliver
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 2253 bytes --]

Re: [Caml-list] whither portability?

[Spiros Eliopoulos]

Raoul, your sarcasm may have passed on the Elm list, but as your lawyer I advise you to keep it to a minimum around here. Nobody wants to hear it.

-Spiros E.

i sent this from my Phone.

On Sep 26, 2015, at 7:29 PM, Raoul Duke <raould@gmail.com> wrote:

>> No one is being parochial or "actively" ignoring mobile. The OCaml core
>> team consists of unpaid volunteers. I suggest that next time, in place
>> of a snide remark, you contribute some code and/or money.
> 
> i would contribute money. not much, but some. i'm not super rich. :-)
> 
> but, no idea where one could do that.
> 
> doubt most people would have known even if it were possible.
> 
> googlign returns bad broken history about this e.g.
> https://github.com/ocaml/opam/issues/326
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] whither portability?

[Raoul Duke]

Sarcasm about what? I would donate money, now that people mention it.
I have no idea where I or anybody else would successfully do that,
though. I did search for it.

Re: [Caml-list] whither portability?

[Raoul Duke]

> Both ocamlc and ocamlopt work for iOS and Android; there is not really
> a good reason to use the bytecode compiler for deployment to mobile.
>
> There is no such thing as "LLVM bytecode" and LLVM bitcode (sic)
> is merely an intermediate format for native executable code. Shipping
> LLVM bitcode offers no conceptual improvement over shipping
> machine code and in any case, there is neither an LLVM backend
> for ocamlopt nor much need for such a backend.

Yes, "bitcode" is the right term. It is now the format Apple wants. So
I dunno why people would actively discount it here.

Re: [Caml-list] whither portability?

[Oliver Bandel]

What is your problem, man?


Zitat von Yotam Barnoy <yotambarnoy@gmail.com> (Sat, 26 Sep 2015  
20:27:19 -0400)

> Oh man. You do realize sarcasm doesn't travel on the Internet, right?
> Please avoid sarcastic comments before you start a flame war.
>
> On Sat, Sep 26, 2015 at 8:20 PM, Oliver Bandel <oliver@first.in-berlin.de>
> wrote:
>
>>
>> Zitat von whitequark <whitequark@whitequark.org> (Sat, 26 Sep 2015
>> 18:59:50 +0300)
>>
>> I will offer a few comments as the author of opam-android[1].
>>>
>>> On 2015-09-26 15:48, Raoul Duke wrote:
>>>
>>>> Exciting to hear! Thank you and your contributors for this work. I
>>>> know there are ways people can be parochial, but I have never been
>>>> able to fathom how the core OCaml team can ignore mobile so actively,
>>>> it seems. It is to sigh, oy veh.
>>>>
>>>
>>> No one is being parochial or "actively" ignoring mobile. The OCaml core
>>> team consists of unpaid volunteers.
>>>
>> [...]
>>
>>
>> Oh, really?
>> Are you sure?
>>
>> I thought, they have jobs at INRIA, and can put at least part of their paid
>> time into OCaml-development!
>>
>>
>> Ciao,
>>    Oliver
>>
>>
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] whither portability?

[whitequark]

On 2015-09-27 03:55, Raoul Duke wrote:
>> Both ocamlc and ocamlopt work for iOS and Android; there is not really
>> a good reason to use the bytecode compiler for deployment to mobile.
>> 
>> There is no such thing as "LLVM bytecode" and LLVM bitcode (sic)
>> is merely an intermediate format for native executable code. Shipping
>> LLVM bitcode offers no conceptual improvement over shipping
>> machine code and in any case, there is neither an LLVM backend
>> for ocamlopt nor much need for such a backend.
> 
> Yes, "bitcode" is the right term. It is now the format Apple wants. So
> I dunno why people would actively discount it here.

Right now, LLVM IR (of which the bitcode is a serialization) is not
flexible enough to express the invariants and metadata required
for the OCaml GC to function[1]. Even if someone released the necessary,
highly nontrivial changes to both LLVM and OCaml today, it would take
many months for them to be reviewed, merged & propagated into Apple's
LLVM fork--all for a small improvement in a select few numeric workloads
that can benefit from using the instruction selector tailored for
the particular device, and a substantial reduction in being able to
debug your code[2].

[1]: 
https://github.com/whitequark/ocaml-llvm-ng/issues/1#issuecomment-141844579
[2]: 
https://medium.com/@FredericJacobs/why-i-m-not-enabling-bitcode-f35cd8fbfcc5

-- 
whitequark

Re: [Caml-list] whither portability?

[Raoul Duke]

> Right now, LLVM IR (of which the bitcode is a serialization) is not
> flexible enough to express the invariants and metadata required
> for the OCaml GC to function[1]. Even if someone released the necessary,
> highly nontrivial changes to both LLVM and OCaml today, it would take
> many months for them to be reviewed, merged & propagated into Apple's
> LLVM fork--all for a small improvement in a select few numeric workloads
> that can benefit from using the instruction selector tailored for
> the particular device, and a substantial reduction in being able to
> debug your code[2].

Thanks for the details. I am (only) a little bit aware of the history
of LLVM vs. GCs, so I can believe what you report. I would much prefer
to be able to debug, as well.

(Given Apple, I wouldn't be surprised if an a year they required
bitcode, however.)

Re: [Caml-list] whither portability?

[whitequark]

On 2015-09-27 20:19, Raoul Duke wrote:
>> Right now, LLVM IR (of which the bitcode is a serialization) is not
>> flexible enough to express the invariants and metadata required
>> for the OCaml GC to function[1]. Even if someone released the 
>> necessary,
>> highly nontrivial changes to both LLVM and OCaml today, it would take
>> many months for them to be reviewed, merged & propagated into Apple's
>> LLVM fork--all for a small improvement in a select few numeric 
>> workloads
>> that can benefit from using the instruction selector tailored for
>> the particular device, and a substantial reduction in being able to
>> debug your code[2].
> 
> Thanks for the details. I am (only) a little bit aware of the history
> of LLVM vs. GCs, so I can believe what you report. I would much prefer
> to be able to debug, as well.
> 
> (Given Apple, I wouldn't be surprised if an a year they required
> bitcode, however.)

It is already required for Apple Watch. Given that bitcode allows to
include inline assembly, I wonder if one could just take OCaml's output,
put it in one giant module-level inline assembly block, and ship it like
that.

-- 
whitequark