[Caml-list] High level (asynchronous) networking library?

[Caml-list] High level (asynchronous) networking library?

[Jeremy Fincher]

I've been learning O'Caml for the past little bit, and after not finding any 
high level asynchronous networking libraries (that is, libraries/modules for 
writing high performance network servers), I figured I would write one, 
since it seemed a non-trivial problem that wasn't too far out of my league.

Anyway, I've been looking at possible ways to write one.  In Python, I've 
made quite a bit of use of the "asyncore" module 
(http://www.nightmare.org/medusa/index.html), which is a very production 
proven (it's the asynchronous networking engine used in Zope) and simple 
framework for writing high performance asynchronous servers.  It seems that 
implementing asyncore in O'Caml would be fairly straightforward, so I've 
been putting it off a bit while I search for a possibly better solution.

Someone has written "asyncore turned inside out" using Stackless Python, an 
implementation of Python that has first-class continuations) using 
continuations to implement the apparent multi-tasking.  It apparently (I've 
not used it myself) makes the asynchronous networking part of the coding a 
little more transparent to the user than asyncore does.  I'm curious if 
something similar could be implemented in O'Caml with closures, and if it 
could achieve similar user-transparency.

Does anyone have any comments on any of the above options?  Are there other 
options I'm not considering which could be better?  Are there any 
implementations of what I want already?  What I'm looking for is as much 
abstraction away from the low-level socket/select loops as is reasonable for 
efficiency's sake.

Thanks,
Jeremy
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] High level (asynchronous) networking library?

[Markus Mottl]

On Thu, 12 Jul 2001, Jeremy Fincher wrote:
> I've been learning O'Caml for the past little bit, and after not
> finding any high level asynchronous networking libraries (that is,
> libraries/modules for writing high performance network servers),
> I figured I would write one, since it seemed a non-trivial problem
> that wasn't too far out of my league.

Have you taken a look at the Ensemble-library? -

  http://www.cs.cornell.edu/Info/Projects/Ensemble

It may not be exactly what you want (I guess you want TCP/IP asynchronous
networking), but it surely contains tons of leading-edge networking code
that can be reused for other purposes.

Short description:

  For an application builder, Ensemble provides a library of
  protocols that can be used for quickly building complex distributed
  applications. An application registers 10 or so event handlers with
  Ensemble, and then the Ensemble protocols handle the details of reliably
  sending and receiving messages, transferring state, implementing
  security, detecting failures, and managing reconfigurations in the
  system.

And it's hellish fast for sure:

  With specially optimized protocols, Ensemble can be around 10%
  slower than raw UDP socket communication.

Regards,
Markus Mottl

-- 
Markus Mottl                                             markus@oefai.at
Austrian Research Institute
for Artificial Intelligence                  http://www.oefai.at/~markus
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] High level (asynchronous) networking library?

[Jeremy Fincher]

> Have you taken a look at the Ensemble-library? -
>
>   http://www.cs.cornell.edu/Info/Projects/Ensemble
>
> It may not be exactly what you want (I guess you want TCP/IP asynchronous
> networking), but it surely contains tons of leading-edge networking code
> that can be reused for other purposes.

I've seen Ensemble, but it seems much more oriented toward peer-to-peer
group communication than toward the type of client-server model I'm hoping
that a high-level asynchronous library would make easier to implement.  If
Ensemble does provide the necessary tools for writing standard TCP network
servers, then by all means, someone correct me, because it'd be great to be
able to integrate servers and group communication in the same package (and
it could make some of the stuff I hope to implement go much more easily :))

Jeremy
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] High level (asynchronous) networking library?

[Gerd Stolpmann]

On Thu, 12 Jul 2001, you wrote:
>I've been learning O'Caml for the past little bit, and after not finding any 
>high level asynchronous networking libraries (that is, libraries/modules for 
>writing high performance network servers), I figured I would write one, 
>since it seemed a non-trivial problem that wasn't too far out of my league.

You may have a look at Equeue:
http://www.ocaml-programming.de/packages/documentation/equeue/manual/

It implements an event queue, and you can attach "event sources" (such as file
descriptors) and event handlers. Because it is a queue, it is not very fast if
there are many events at the same time.

You may also look at my RPC implementation:
http://test.ocaml-programming.de/packages/documentation/rpc/

It bases on Equeue, and is an excellent example.

>Anyway, I've been looking at possible ways to write one.  In Python, I've 
>made quite a bit of use of the "asyncore" module 
>(http://www.nightmare.org/medusa/index.html), which is a very production 
>proven (it's the asynchronous networking engine used in Zope) and simple 
>framework for writing high performance asynchronous servers.  It seems that 
>implementing asyncore in O'Caml would be fairly straightforward, so I've 
>been putting it off a bit while I search for a possibly better solution.

The link does not work.

>Someone has written "asyncore turned inside out" using Stackless Python, an 
>implementation of Python that has first-class continuations) using 
>continuations to implement the apparent multi-tasking.  It apparently (I've 
>not used it myself) makes the asynchronous networking part of the coding a 
>little more transparent to the user than asyncore does.  I'm curious if 
>something similar could be implemented in O'Caml with closures, and if it 
>could achieve similar user-transparency.

You could do something like

let task1 arg cont =
  let before_call () = ... in
  let after_call () = ... in
  before_call();
  task2 arg' (fun () -> cont(); after_call())

(task2 in a similar way). "cont" is the function that continues. The problem is
that this scheme needs much memory, as every task switch allocates a new
closure. When the closure is evaluated, some stack space is needed, too, so
after some time there will be lots of half-evaluated closures, and a stack
overflow is very likely.

>Does anyone have any comments on any of the above options?  Are there other 
>options I'm not considering which could be better?  Are there any 
>implementations of what I want already?  What I'm looking for is as much 
>abstraction away from the low-level socket/select loops as is reasonable for 
>efficiency's sake.

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 45             
64293 Darmstadt     EMail:   gerd@gerd-stolpmann.de
Germany                     
----------------------------------------------------------------------------
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] High level (asynchronous) networking library?

[John Max Skaller]

Jeremy Fincher wrote:

> Someone has written "asyncore turned inside out" using Stackless Python, an
> implementation of Python that has first-class continuations) using
> continuations to implement the apparent multi-tasking.  It apparently (I've
> not used it myself) makes the asynchronous networking part of the coding a
> little more transparent to the user than asyncore does.  I'm curious if
> something similar could be implemented in O'Caml with closures, and if it
> could achieve similar user-transparency.

	If you want to do this with an Ocaml like language,
consider Felix, which I'm developing at present. It uses heap
allocated stack frames, like Stackless Python, and operates
by continuation passing. Unlike Stackless, it generates native
code (via C++), and functions still use the machine stack
So it should be much faster than Stackless, while supporting
a strongly typed language with first class functions.

-- 
John (Max) Skaller, mailto:skaller@maxtal.com.au 
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
New generation programming language Felix  http://felix.sourceforge.net
Literate Programming tool Interscript     
http://Interscript.sourceforge.net
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] High level (asynchronous) networking library?

[Ohad Rodeh]

Jeremy,
  If you're interested in writing TCP based servers, than Ensemble
won't be much help to you. The standard OCaml Unix library is quite
sufficient for these purposes. Ensemble comes into its own when you
want to write fault-tolerant group-communication style servers. 
  
  All the best, 
	Ohad.

On Thu, 12 Jul 2001, Jeremy Fincher wrote:

> > Have you taken a look at the Ensemble-library? -
> >
> > http://www.cs.cornell.edu/Info/Projects/Ensemble
> >
> > It may not be exactly what you want (I guess you want TCP/IP asynchronous
> > networking), but it surely contains tons of leading-edge networking code
> > that can be reused for other purposes.
> 
> I've seen Ensemble, but it seems much more oriented toward peer-to-peer
> group communication than toward the type of client-server model I'm hoping
> that a high-level asynchronous library would make easier to implement.If
> Ensemble does provide the necessary tools for writing standard TCP network
> servers, then by all means, someone correct me, because it'd be great to be
> able to integrate servers and group communication in the same package (and
> it could make some of the stuff I hope to implement go much more easily :))
> 
> Jeremy
> -------------------
> Bug reports: http://caml.inria.fr/bin/caml-bugsFAQ: http://caml.inria.fr/FAQ/
> To unsubscribe, mail caml-list-request@inria.frArchives: http://caml.inria.fr
> 

-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr