ocamlnet and kernel poll

ocamlnet and kernel poll

[Joel Reymont]

Suppose I want to build a server that runs on top of ocamlnet and  
handles 10k+ connections.

ocamlnet seems to use select exclusively.

Any suggestions on how to add kernel poll? Is this possible even?

	Thanks, Joel

--
wagerlabs.com

Re: ocamlnet and kernel poll

[Sylvain Le Gall]

On 11-09-2008, Joel Reymont <joelr1@gmail.com> wrote:
> Suppose I want to build a server that runs on top of ocamlnet and  
> handles 10k+ connections.
>
> ocamlnet seems to use select exclusively.
>
> Any suggestions on how to add kernel poll? Is this possible even?
>

I am not sure about what kind of server you want to create, but for this
high number of connection, I think "fork"ing a little bit will help you
to have more connection and is to my mind more secure et al.

Handling more than 10k connection in a single executable is not a
"general" good idea... But maybe in your case, you have no choice.

Regards,
Sylvain Le Gall

Re: [Caml-list] ocamlnet and kernel poll

[Gerd Stolpmann]

Am Donnerstag, den 11.09.2008, 13:43 +0100 schrieb Joel Reymont:
> Suppose I want to build a server that runs on top of ocamlnet and  
> handles 10k+ connections.
> 
> ocamlnet seems to use select exclusively.

There's also netplex in ocamlnet, which is a kind of fork framework. I
don't know which protocol you want to use. For instance, you can arrange
that netplex starts 100 processes and every process handles 100
connections.

> Any suggestions on how to add kernel poll? Is this possible even?

I'm in the middle of changing ocamlnet here, so we can have more
flexibility for choosing the kind of event polling. Look at the svn
version: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/

There's a lot of new stuff in the src/netsys directory. In the future,
all polling will be based on the pollset class type
(src/netsys/netsys_pollset.mli). There is currently an implementation
that uses the poll() syscall (src/netsys/netsys_pollset_posix.mli) and
another one for Win32 (src/netsys/netsys_pollset_win32.mli - limited to
sockets and named pipes for now).

As this is a class type you can also go ahead, and implement it for
every kernel mechanism you like, and just you your class instead of the
classes provided by ocamlnet. I'll add some more mechanisms later (any
help is welcome).

You can turn these pollsets into event_system by using
src/equeue/unixqueue2.mli. This is not very much tested, however, and
there is no support for multi-threading yet in this module. Many
ocamlnet modules allow to inject whatever event_system you like to have
(should be all modules in the future).

All this is experimental for now. I have tested it only with small
programs, not with large production systems. But maybe it's an option
for you to use the svn version.

Gerd



> 
> 	Thanks, Joel
> 
> --
> wagerlabs.com
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------

Re: [Caml-list] ocamlnet and kernel poll

[Markus Mottl]

On Thu, Sep 11, 2008 at 8:43 AM, Joel Reymont <joelr1@gmail.com> wrote:
> Suppose I want to build a server that runs on top of ocamlnet and handles
> 10k+ connections.
>
> ocamlnet seems to use select exclusively.
>
> Any suggestions on how to add kernel poll? Is this possible even?

The Core-library that we developed at Jane Street, and which is also
in Godi, contains a module "Linux_ext", which has a fully-featured
interface to epoll (besides lots of other Linux-specific goodies).

There you just call "Epoll.create" to get a file descriptor on which
to listen for I/O-events.  Then you only need to add other file
descriptors you want to monitor with "Epoll.add", specifying flags for
the kind of events you want to wait for.  "Epoll.modify" and
"Epoll.del" modify event flags of and remove monitored descriptors
respectively.  "Epoll.wait" allows you to wait for received events
(similar to "select").

Note, however, that "select" is usually more efficient for small (=
tens) numbers of descriptors!

Regards,
Markus

-- 
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com

Re: [Caml-list] ocamlnet and kernel poll

[Mattias Engdegård]

>Suppose I want to build a server that runs on top of ocamlnet and  
>handles 10k+ connections.
>
>ocamlnet seems to use select exclusively.
>
>Any suggestions on how to add kernel poll? Is this possible even?

There is an ocaml wrapper for libevent: http://www.xs4all.nl/~mmzeeman/ocaml/

With that many connections, you may actually benefit from multiple
processors, which in the OCaml world usually means multiple processes.

There's nothing wrong with serving myriads of connections from a
single process, especially when using a strongly typed language, but
parallelism can be useful. On the other hand, one process per
connection may be inefficient as well - a hybrid N:M solution is
probably best.

Re: [Caml-list] ocamlnet and kernel poll

[Markus Mottl]

On Thu, Sep 11, 2008 at 10:37 AM, Mattias Engdegård
<mattias@virtutech.se> wrote:
> There's nothing wrong with serving myriads of connections from a
> single process, especially when using a strongly typed language, but
> parallelism can be useful. On the other hand, one process per
> connection may be inefficient as well - a hybrid N:M solution is
> probably best.

Note that if your application is I/O-intensive, you can already
exploit parallelism within one process.  Use bigstrings as buffers and
perform I/O outside of the OCaml-lock if releasing it (depends on
I/O-size) is more efficient.  That way you can have many threads
performing I/O-system calls simultaneously.

Regards,
Markus

-- 
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com

Re: [Caml-list] ocamlnet and kernel poll

[Joel Reymont]

On Sep 11, 2008, at 4:32 PM, Markus Mottl wrote:

> Use bigstrings as buffers and
> perform I/O outside of the OCaml-lock if releasing it (depends on
> I/O-size) is more efficient.

Are you suggesting using multiple OS threads within a single process?

When exactly do you release the lock and how do you perform IO outside  
of it?

How do you determine whether to release the lock or not?

	Thanks, Joel

--
wagerlabs.com

Re: ocamlnet and kernel poll

[Sylvain Le Gall]

On 11-09-2008, Joel Reymont <joelr1@gmail.com> wrote:
>
> On Sep 11, 2008, at 4:32 PM, Markus Mottl wrote:
>
>> Use bigstrings as buffers and
>> perform I/O outside of the OCaml-lock if releasing it (depends on
>> I/O-size) is more efficient.
>
> Are you suggesting using multiple OS threads within a single process?
>
> When exactly do you release the lock and how do you perform IO outside  
> of it?
>
> How do you determine whether to release the lock or not?
>

I tink Markus is talking about the lock of the OCaml GC. When you do a
potentially long syscall (like select, read...) the GC lock is released
letting other OCaml thread running. 

Look at enter_blocking_section()/leave_blocking_section() in C part of
OCaml source.

You will still have 1 OCaml thread running but many other thread in C
call can also be running.

In particular, it means that OCaml thread can be used for standard
"non-blocking" syscall replacement.

Regards,
Sylvain Le Gall

Re: [Caml-list] ocamlnet and kernel poll

[Markus Mottl]

On Thu, Sep 11, 2008 at 11:43 AM, Joel Reymont <joelr1@gmail.com> wrote:
> Are you suggesting using multiple OS threads within a single process?

Yes.

> When exactly do you release the lock and how do you perform IO outside of
> it?
>
> How do you determine whether to release the lock or not?

Look at our Core-library.  It contains a module "Bigstring", which
provides many efficient I/O-functions for those.  Look at
"bigstring_stubs.c", where you'll see how these functions are
implemented in C-land, e.g. when the lock gets released, etc.

There are functions like e.g. "read_assume_nonblocking", which one can
use if one knows for sure that a read cannot block (e.g. after a
"select").  Then the function determines whether it's worth/necessary
releasing the lock.

If, for example, the read is performed into a bigstring which happens
to be a memory-mapped file, then the lock will always be released,
because if the memory page happens to be on disk you might otherwise
experience latency spikes when it's being paged in, since all other
threads would temporarily freeze.  Same for very large I/O-operations:
there the lock would be released, too, to exploit parallelism, improve
throughput, and avoid latency spikes.

Regards,
Markus

-- 
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com

Re: [Caml-list] ocamlnet and kernel poll

[Rich Neswold]

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

On Thu, Sep 11, 2008 at 11:31 AM, Markus Mottl <markus.mottl@gmail.com>wrote:

> Look at our Core-library.  It contains a module "Bigstring", which
> provides many efficient I/O-functions for those.  Look at
> "bigstring_stubs.c", where you'll see how these functions are
> implemented in C-land, e.g. when the lock gets released, etc.
>

I'm new to Ocaml and trying to get up to speed. Sorry, but I can't seem to
find the "bigstring" module in the Core library.

    http://caml.inria.fr/pub/docs/manual-ocaml/manual033.html

nor is it in Pervasives (mentioned in the Core-library page):

    http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html

Where do I find documentation for the bigstring module? Thanks!

-- 
Rich

LOI: https://www.google.com/reader/shared/00900594587109808626

[-- Attachment #2: Type: text/html, Size: 1446 bytes --]

Re: [Caml-list] ocamlnet and kernel poll

[Gabriel Kerneis]

Hi,

On Wed, Sep 24, 2008 at 10:05:54AM -0500, Rich Neswold wrote:
> On Thu, Sep 11, 2008 at 11:31 AM, Markus Mottl <markus.mottl@gmail.com>wrote:
> 
> > Look at our Core-library.  It contains a module "Bigstring", which
> > provides many efficient I/O-functions for those.  Look at
> > "bigstring_stubs.c", where you'll see how these functions are
> > implemented in C-land, e.g. when the lock gets released, etc.
> 
> I'm new to Ocaml and trying to get up to speed. Sorry, but I can't seem to
> find the "bigstring" module in the Core library.

"Our" means "Jane Street's" in this context:

http://ocaml.janestcapital.com/?q=node/13

Regards,
-- 
Gabriel Kerneis

Re: [Caml-list] ocamlnet and kernel poll

[Rich Neswold]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On Wed, Sep 24, 2008 at 10:43 AM, Gabriel Kerneis <kerneis@enst.fr> wrote:

> On Wed, Sep 24, 2008 at 10:05:54AM -0500, Rich Neswold wrote:
> > I'm new to Ocaml and trying to get up to speed. Sorry, but I can't seem
> to
> > find the "bigstring" module in the Core library.
>
> "Our" means "Jane Street's" in this context:
>
> http://ocaml.janestcapital.com/?q=node/13
>
>
Thanks for the clarification!

-- 
Rich

LOI: https://www.google.com/reader/shared/00900594587109808626

[-- Attachment #2: Type: text/html, Size: 1009 bytes --]

Re: [Caml-list] ocamlnet and kernel poll

[Markus Mottl]

2008/9/24 Rich Neswold <rich.neswold@gmail.com>:
> On Thu, Sep 11, 2008 at 11:31 AM, Markus Mottl <markus.mottl@gmail.com>
> wrote:
>>
>> Look at our Core-library.  It contains a module "Bigstring", which
>> provides many efficient I/O-functions for those.  Look at
>> "bigstring_stubs.c", where you'll see how these functions are
>> implemented in C-land, e.g. when the lock gets released, etc.
>
> I'm new to Ocaml and trying to get up to speed. Sorry, but I can't seem to
> find the "bigstring" module in the Core library.

"Core" doesn't refer to any standard library distributed with OCaml
here.  "Core" is a library distributed by Jane Street Capital:

  http://ocaml.janestcapital.com/?q=node/13

Regards,
Markus

-- 
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com