[Caml-list] Need advice for a mobile application server

[Caml-list] Need advice for a mobile application server

[Eray Ozkural]

Hello there,

I am looking to consider ocaml for writing a multi-threaded application server 
which involves data structures of medium complexity (to be specific geometric 
structures). Do you think ocaml is a good bet for this purpose, or should I 
stick with C++? (in which I know how to write SMP code :)

Will it be easy and efficient for me to implement imperative data structures 
which should be locked on update ops, but will not need such on queries (yes, 
I know it sounds naughty but it should be possible)? Or are there well known 
obstacles to performing such feats using our beloved ocaml?

Your advice will be much appreciated.

Regards,

-- 
Eray Ozkural (exa) <erayo@cs.bilkent.edu.tr>
Comp. Sci. Dept., Bilkent University, Ankara  KDE Project: http://www.kde.org
www: http://www.cs.bilkent.edu.tr/~erayo  Malfunction: http://mp3.com/ariza
GPG public key fingerprint: 360C 852F 88B0 A745 F31B  EA0F 7C07 AE16 874D 539C

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[skaller]

On Thu, 2003-11-20 at 05:03, Eray Ozkural wrote:
> Hello there,
> 
> I am looking to consider ocaml for writing a multi-threaded application server 
> which involves data structures of medium complexity (to be specific geometric 
> structures). Do you think ocaml is a good bet for this purpose, or should I 
> stick with C++? (in which I know how to write SMP code :)
> 
> Will it be easy and efficient for me to implement imperative data structures 
> which should be locked on update ops, but will not need such on queries (yes, 
> I know it sounds naughty but it should be possible)? Or are there well known 
> obstacles to performing such feats using our beloved ocaml?

Executive Summary [for lazy reader ..]: Use Ocaml.

I have written some multi-threaded client/server and peer2peer
Caml code for simulating a telephone network. For this purpose
I used module Event which is very good (once you get
the hang of it ..)

To create servers I modified the source code
of the Ocaml client/server function Unix.establish_server.
This function creates processes rather than threads,
and so I had to adapt it to threads which is quite easy.

My code was all native code using Posix threads,
it ran on both Linux and Solaris BUT be warned
that at least at that time (several years ago now)
Linux cannot create threads, only processes,
and consequently all threads are detached.
In the case of an error, the main thread can die
but the other threads can keep running ..
[this doesn't happen on Solaris and violates Posix,
I don't know whether this is fixed yet in Linux ..
in any case it has nothing to do with the choice
of Ocaml ..]

As usual I think Ocaml is superior *especially*
for complex tasks where abstractions with strong
guarrantees are highly desirable for reasoning
about software.

The only caveat I know of is that Ocaml
sometimes uses global locks (the gc can't
run concurrently with allocations etc)
and if you create C bindings for Ocaml you
have to be careful to make sure you manage
the exclusion correctly or you might end 
up with a deadlock.


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[Richard Jones]

Actually I don't think it was ever as bad as you made it out to be,
but certainly this is fixed[1] in new versions of Linux. See various
papers about this here:

http://people.redhat.com/drepper/

Rich.

[1] Of course the original LinuxThreads being written by Xavier Leroy,
so I wouldn't want to say it was broken ...

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
"I wish more software used text based configuration files!"
 -- A Windows NT user, quoted on Slashdot.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[Xavier Leroy]

This is getting off-topic, but:

> [1] Of course the original LinuxThreads being written by Xavier Leroy,
> so I wouldn't want to say it was broken ...

You can safely say LinuxThreads is slightly borken, in that lack of
thread support in the Linux kernel (until recently) prevented full
compliance with the POSIX standard.  These quirks affected some fairly
obscure corners of the POSIX spec, though.  In particular, the earlier
comment that "LinuxThreads creates all threads detached" is not
accurate: thread creation and joining was implemented correctly.

Back to the original question: writing a multithreaded server in OCaml
is definitely feasible.  The only limitation to be aware of is that
OCaml threads do not offer parallelism, just concurrent execution.
What this means is that if there are several processors on your
machine, only one can execute OCaml code at any time, but other
threads (and therefore possibly other processors) can do I/O or C
computations in parallel.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[Eray Ozkural]

Hello Xavier,

On Thursday 20 November 2003 11:27, Xavier Leroy wrote:
> You can safely say LinuxThreads is slightly borken, in that lack of
> thread support in the Linux kernel (until recently) prevented full
> compliance with the POSIX standard.  These quirks affected some fairly
> obscure corners of the POSIX spec, though.  In particular, the earlier
> comment that "LinuxThreads creates all threads detached" is not
> accurate: thread creation and joining was implemented correctly.
>

I suppose. I bet I have seen some strange quirks which somehow made a thread 
wrapper library in C++ I had written incompatible across linux and solaris. I 
guess I had thought Solaris kernel got it right, but I ended up not having 
understood the reasons, at least not from the user documentation.

> Back to the original question: writing a multithreaded server in OCaml
> is definitely feasible.  The only limitation to be aware of is that
> OCaml threads do not offer parallelism, just concurrent execution.
> What this means is that if there are several processors on your
> machine, only one can execute OCaml code at any time, but other
> threads (and therefore possibly other processors) can do I/O or C
> computations in parallel.

Thanks for the info.

I couldn't quite figure out why only one proc. can execute ocaml code yet, but 
I can dig into the documentation :)

Regards,

-- 
Eray Ozkural (exa) <erayo@cs.bilkent.edu.tr>
Comp. Sci. Dept., Bilkent University, Ankara  KDE Project: http://www.kde.org
www: http://www.cs.bilkent.edu.tr/~erayo  Malfunction: http://mp3.com/ariza
GPG public key fingerprint: 360C 852F 88B0 A745 F31B  EA0F 7C07 AE16 874D 539C

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[Basile Starynkevitch]

On Thu, Nov 20, 2003 at 02:08:06PM +0200, Eray Ozkural wrote:

Eray> On Thursday 20 November 2003 11:27, Xavier Leroy wrote: [...]

Xavier>> Back to the original question: writing a multithreaded server in OCaml
Xavier>> is definitely feasible.  The only limitation to be aware of is that
Xavier>> OCaml threads do not offer parallelism, just concurrent execution.
Xavier>> What this means is that if there are several processors on your
Xavier>> machine, only one can execute OCaml code at any time, but other
Xavier>> threads (and therefore possibly other processors) can do I/O or C
Xavier>> computations in parallel.


Eray> I couldn't quite figure out why only one proc. can execute ocaml
Eray> code yet, but I can dig into the documentation :)

Mostly, because of the garbage collector. The Ocaml GC is very
efficient, and writing a generational [half-] copying *multi-threaded*
GC is quite difficult (as usual, the devil is in the
details). Actually, coding an efficient multithreaded GC with write
barriers using Posix threads is very tricky (if not impossible,
notably for synchronising minor GCs on every thread.) - for example
you'll probably need to access thread local storage on each
allocation, which (in portable C code) means calling
pthread_getspecific every time.


Regarding multi-threading in Ocaml, I think that it is much less
useful in Ocaml than say in Java or C. In my experience (on my
previous job) I started to code a multi-threaded application (the
POESIA monitor), and prefered to switch to a single-threaded design,
taking into advantage the functional abilities of Ocaml (i.e. mentally
replacing threads by sets of small correlated tasks, each represented
as functional values, ie closures, which means coding in a
continuation passing style). For instance the Unixqueue library (from
the Equeue software, used with Netclient) is single-threaded but very
useful to handle several "concurrent" communication channels
-e.g. HTTP or RSH connections.

Regards.
-- 
Basile STARYNKEVITCH -- basile dot starynkevitch at inria dot fr
Project cristal.inria.fr - phone +33 1 3963 5197 - mobile 6 8501 2359
http://cristal.inria.fr/~starynke --- all opinions are only mine 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[Basile Starynkevitch]

On Thu, Nov 20, 2003 at 04:16:34PM +0000, Richard Jones wrote:
> On Thu, Nov 20, 2003 at 01:49:50PM +0100, Basile Starynkevitch wrote:
Basile>> Regarding multi-threading in Ocaml, I think that it is much less
Basile>> useful in Ocaml than say in Java or C. In my experience (on my
Basile>> previous job) I started to code a multi-threaded application (the
Basile>> POESIA monitor), and prefered to switch to a single-threaded design,
Basile>> taking into advantage the functional abilities of Ocaml (i.e. mentally
Basile>> replacing threads by sets of small correlated tasks, each represented
Basile>> as functional values, ie closures, which means coding in a
Basile>> continuation passing style).

Richard> It's interesting that you use this technique. If I get what you mean
Richard> then you're talking about a Reactor pattern (correct me if I've missed
Richard> the point).

Very probably yes, but I am not sure to exactly know what the Reactor
pattern is, even if I probably guess it.

Richard> How do you find handling state in such a model? I once tried to write
Richard> a simple HTTP server using an event-driven model, but it became a real
Richard> pain handling partial state, such as when you have only received part
Richard> of the HTTP headers.  Nevertheless this is how some webservers are
Richard> actually written, notably Boa.

And also the squid HTTP proxy (coded in C, so understanding it is not
straightforward - I believe it could be much easier to recode it in
Ocaml, using the same algorithms, and coding with lots of small
closures representing the intermediate state handling)

In the Poesia ICAP server part (the poesia monitor is an ICAP server
and an HTTP client), the partial state is explicitly kept as such.
Parsing the ICAP protocol (syntactically similar to HTTP - see
www.i-cap.org or RFC3507) is done by expliciting partial states. See
icap.ml & icapsrc.ml source files in Poesia monitor

Richard> Also, I'm writing a server program myself in OCaml, and it uses an
Richard> event-driven model. Do you know of any freeware code for OCaml which
Richard> implements the Reactor pattern?

The Poesia monitor is open-source (or free) software, under GPL
license (see http://www.gnu.org/philosophy/free-sw.html and
http://www.gnu.org/copyleft/gpl.html for more). its CVS repository is
under sourceforge. The monitor is under
http://cvs.sourceforge.net/viewcvs.py/poesia/PoesiaSoft/PoesiaMonIcap/
It uses only open-source Ocaml libraries. The HTTP client is the
Http_client module from Netclient (version >= 0.90) library. The
reactor is (as I understand it) in the Unixqueue module from the
Equeue library (version >= 1.2.3). The Ocamlnet (version >= 0.95)
library is also used.  See the README file and Configure.sh script
from PoesiaMonIcap

So if opensource software is freeware code for you (for me it is not;
opensource != freeware, and actually opensource > freeware) then you
can find lot of opensource ocaml code illustrating these
tricks. However, in a more usual but restrictive sense of freeware
code, I don't know of any Ocaml freeware code actually. But I
definitely don't want to start any offtopic flame war about such
terminological and philosophical issues on the Caml list, so if you
want to discuss these issues with me please reply in private.

In my previous job at CEA, I initiated Poesia which is partly funded
by the European Commission, and I believe that its opensource
character was the major reason (in additional to technical
innovations) it got funded.

FWIW, I do continue to work (mostly bug fixes) on Poesia on my spare
time at home. Of course this is unrelated to my current work at INRIA.

-- 
Basile STARYNKEVITCH -- basile dot starynkevitch at inria dot fr
Project cristal.inria.fr - phone +33 1 3963 5197 - mobile 6 8501 2359
http://cristal.inria.fr/~starynke --- all opinions are only mine 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Need advice for a mobile application server

[skaller]

On Thu, 2003-11-20 at 23:49, Basile Starynkevitch wrote:
> On Thu, Nov 20, 2003 at 02:08:06PM +0200, Eray Ozkural wrote:

> Regarding multi-threading in Ocaml, I think that it is much less
> useful in Ocaml than say in Java or C. 

I think that's also application dependent.

For the telephone network I was simulating,
threading was a vital aid to thinking
about asynchronous behaviour. 

Unusual perhaps .. but the end product
was not intended to be used, not even once.
Instead the strong static typing and nice
concepts in the Event module were used to help
develop a model which one could believe was
at least well formed (due to static type checking).
[I did this to try to understand what the ^&*%$#
the telecommunications Standard was on about:]


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners