[Caml-list] announcing iox-1.00 (beta 1)

[Caml-list] announcing iox-1.00 (beta 1)

[james woodyatt]

everyone--

Okay, so I wasn't completely forthcoming.  While I do make the rent as a 
practitioner, it is true that I indulge in a little amateur software 
research from time to time.

This message announces the availability in source code of my "Iox" 
library, which is a framework for concurrent, single-threaded Internet 
application services.  It's released under the 2-clause BSD-style 
"as-is" license, but I might consider waiving even those terms for 
peanuts, so you shouldn't have any worries.

It's the first result of my encounter with Objective Caml, and I'm 
probably more pleased with it than I should be.  You can download the 
distribution directly from either of these locators:

	http://www.wetware.com/jhw/src/iox-1.00b1.tar.gz
	http://www.wetware.com/jhw/src/iox-1.00b1.tar.bz2

Alternatively, you may browse the source directly:

	http://www.wetware.com/jhw/src/iox-1.00b1/

There is documentation generated with the [excellent] ocamldoc tool:

	http://www.wetware.com/jhw/src/iox-1.00b1/doc/index.html

Sometime in May this year, I will post a version of this library with 
appropriate patches applied to fix bugs (assuming some are found, which 
is probably safe to assume).  Some time later, I plan to release another 
version which has improvements in the feature set.

Please direct constructive comments or helpful tips to me.  If you find 
a bug, please help me out by also finding a patch that fixes it.

The remainder of this message is a reprint of the README file from the 
distribution.

-----

NAME

     Iox -- A Framework for Concurrent, Single-threaded Network 
Applications


SYNOPSIS

     The Iox library is a foundation layer for concurrent, single-threaded
     network application servers.  It comprises the following subsystems:

     + Modular event loop kernel.
     + Functional buffer-chained messages.
     + Flow-control mechanisms.
     + I/O reactor (for multiplexing network events).
     + Abstractions for reactive socket I/O.

     See the accompanying INSTALL file, for instructions on integrating.


INTRODUCTION

     For most network applications, a serializing design (forked or multi-
     threaded) performs adequately and scales sufficiently well enough 
that the
     extra difficulty of implementing a concurrent, single-threaded 
design is
     not worth the effort.  Some applications-- in particular, application
     services which manage a set of common resources on behalf of many
     simultaneous users-- can benefit substantially from a concurrent, 
single-
     threaded design.  This library is intended to help in the 
implementation
     of such designs.

     The common element in concurrent, single-threaded network application
     services is a top-level event loop.  At the start of the loop, the 
program
     waits for operating system notification of I/O operations that may 
proceed
     on one or more of its communication channels.  When the program 
receives
     this notification, it performs all possible I/O operations that do 
not
     require the process to block, then it re-enters the start of the 
loop.
     The loop is terminated only when all possible I/O operations are 
performed,
     and no more operations are pending in the operating system.

     To support this programming paradigm, the Iox library offers a 
modular
     top-level event loop, allowing small reuseable components to be 
combined
     into larger complex applications.  Also, a flow control mechanism 
(which
     borrows some fundamental concepts from the architecture of the 
STREAMS
     environment in Unix SVR3) is introduced and used to implement 
reactive
     programming interfaces to the BSD sockets transport I/O layer.


EXAMPLES

     The following annotated source code implements a simple loopback 
test of
     the Iox library.  It establishes a loopback TCP connection and 
transfers
     a large string of octets through the socket.  It shows how the flow 
control
     mechanism is used with the modular event loop.

     open Iox_kernel	(* the modular event loop *)
     open Iox_circuit	(* reactive interfaces to BSD sockets *)

     (* the component defining the main event loop module *)
     module C : Component_sig = struct

         (* need to run with the Iox_circuit event loop module *)
         let depend = [ Iox_circuit.ring ]

         (* the core of the component *)
         type core_t = {
             kernel_: Loop.t;			(* event loop *)
             teststring_: string;		(* string to transfer *)
             listener_fd_: Iox_socket.TCP.t;	(* TCP listener socket *)
             listener_src_: TCP.listener;	(* reactor for TCP listen *)
         }

         (* the internet address of localhost *)
         let loopback_addr = Unix.inet_addr_of_string "127.0.0.1"

         (* the maximum receive segment size *)
         let rss = 999

         (* the maximum transmit segment size *)
         let tss = 640

         (* constants for constructing the message to transport *)
         let msgpattern = "all work and no play makes jack a dull boy. "
         let msgsize = 100000

         (* a mutable flag indicating success *)
         let complete = ref false

         (* function to convert Unix.Unix_error exceptions into Failure *)
         let catch_exn_ x =
             try raise x with
             | Unix.Unix_error (code, fname, _) ->
                 failwith begin
                     Printf.sprintf "error in %s: %s" fname
                     (Unix.error_message code)
                 end

         (* the class defining the receiver object.  initialized with a 
core
            and a TCP socket *)
         class receiver c fd =
             let k = c.kernel_ in
             let p = TCP.protocol ~k ~x:catch_exn_ ~rss fd in
             object(self)
                 inherit [STREAM.io_t, STREAM.io_t] Iox_protocol.client ~k
                     as super

                 (* the message received so far... *)
                 val mutable message_ = Iox_message.empty

                 (* receive events from the TCP circuit agent. *)
                 method private rx_put =
                     function
                     | `Data m ->
                         message_ <- Iox_message.join message_ m;
                     | `Release ->
                         if (Iox_message.size message_) <> msgsize then
                             failwith "receiver: bad message size!";
                         let s = Iox_message.contents message_ in
                         if (String.length s) <> msgsize then
                             failwith "message: contents not same size!";
                         if s <> c.teststring_ then
                             failwith "message: contents not equal!";
                         super#tx_recycle

                 (* transmit release events to the TCP circuit agent. *)
                 method private tx_cycle =
                     match super#tx_canput with
                     | None ->
                         failwith "receiver#tx_cycle can't put"
                     | Some put ->
                         put `Release;
                         super#detach;
                         Iox_socket.close fd

                 (* init: cancel the listener, attach the TCP agent and 
unblock
                    the receive flow from the TCP agent. *)
                 initializer
                     c.listener_src_#cancel;
                     super#attach p;
                     super#rx_unblock
             end

         (* the class defining the transmitter object.  initialized with 
a core
            and a TCP socket *)
         class transmitter c fd =
             let k = c.kernel_ in
             let p = TCP.protocol ~k ~x:catch_exn_ ~rss fd in
             let implemented_ = false in
             object(self)
                 inherit [STREAM.io_t, STREAM.io_t] Iox_protocol.client ~k
                     as super

                 (* the remainder of the message to transmit still... *)
                 val mutable message_ =
                     let b = Iox_message.buffer_of_string c.teststring_ in
                     Iox_message.create [ b ]

                 (* receive release events from the TCP circuit agent. *)
                 method private rx_put =
                     function
                     | `Data _ ->
                         failwith "transmitter#rx_put unexpected input"
                     | `Release ->
                         super#detach;
                         Iox_socket.close fd;
                         complete := true

                 (* transmit events to the TCP circuit agent. *)
                 method private tx_cycle =
                     match super#tx_canput with
                     | None ->
                         self#tx_recycle
                     | Some put ->
                         if message_ == Iox_message.empty then
                             put `Release
                         else begin
                             let m =
                                 if tss < Iox_message.size message_ then 
begin
                                     let m0, m1 =
                                         Iox_message.split message_ tss
                                     in
                                     message_ <- m1;
                                     m0
                                 end
                                 else begin
                                     let m = message_ in
                                     message_ <- Iox_message.empty;
                                     m
                                 end
                             in
                             assert (m != Iox_message.empty);
                             assert (Iox_message.size m > 0);
                             put (`Data m);
                             self#tx_cycle
                         end

                 (* init: attach the TCP circuit agent, unblock the 
receive
                    flow, initiate the transmit flow. *)
                 initializer
                     super#attach p;
                     super#rx_unblock;
                     super#tx_recycle
             end

         (* initialize the core of the test *)
         let init k =
             (* create the test string. *)
             let s =
                 let b = Buffer.create msgsize in
                 while (Buffer.length b) < msgsize do
                     Buffer.add_string b msgpattern
                 done;
                 String.sub (Buffer.contents b) 0 msgsize
             in
             (* create the TCP listener socket *)
             let lfd = Iox_socket.TCP.create () in
             (* bind the TCP listener socket to an ephemeral loopback 
port *)
             let laddr =
                 Iox_socket.TCP.Provider.A.to_sockaddr (loopback_addr, 0)
             in
             Iox_socket.bind lfd laddr;
             (* get the actual address of the listener socket *)
             let laddr =
                 Iox_socket.TCP.Provider.A.of_sockaddr
                     (Iox_socket.getsockname lfd)
             in
             (* listen for one connection on the listener socket *)
             Iox_socket.listen lfd 1;
             (* create a new TCP listener object *)
             let lsrc = new TCP.listener ~k ~x:catch_exn_ lfd in
             (* construct the core record *)
             let c = {
                 kernel_ = k;
                 teststring_ = s;
                 listener_fd_ = lfd;
                 listener_src_ = lsrc;
             } in
             (* construct a sink for the listener that creates receiver 
objects
                with each accepted socket, and start the listener on the 
flow.
             *)
             let lflow =
                 Iox_flow.create ~f:(fun fd -> ignore (new receiver c fd))
             in
             lsrc#start lflow;
             (* create a new TCP initiator object that constructs 
transmitter
                objects with successful connections. *)
             let connect =
                 function
                 | `Connect fd -> ignore (new transmitter c fd)
                 | `Error e -> catch_exn_ e
             in
             let _ = new TCP.initiator ~k ~f:connect laddr in
             (* return the core *)
             c

         (* the finalization function for the test.  closes the listener 
socket
            and fails if the test was not successful. *)
         let fini c =
             Iox_socket.close c.listener_fd_;
             if not !complete then failwith "not complete at fini"
     end

     (* create the event loop module *)
     include Component(C)

     (* run the event loop *)
     Loop.run ring


BUGS

     There is no programmer's guide.  Though, there is documentation 
generated
     with Maxence Guesdon's excellent 'ocamldoc' tool.

     On multi-CPU hardware architectures, the event loop needs to be 
implemented
     by a pool of N threads, one for each CPU.

     Threads should be cheap enough that this library is never worth the 
hassle.


LICENSE

     See the accompanying LICENSE file for the details.  It's the 2-clause
     BSD-style open-source license.


AUTHOR

     j h woodyatt <jhw@wetware.com>


ACKNOWLEDGEMENTS
     <insert verbiage here>

--
j h woodyatt <jhw@wetware.com>
"somebody has to do something, and it's just incredibly
pathetic that it has to be us." --jerry garcia

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] announcing iox-1.00 (beta 2)

[james woodyatt]

On Tuesday, March 26, 2002, at 03:02 PM, james woodyatt wrote:
>
> This message announces the availability in source code of my "Iox" 
> library, which is a framework for concurrent, single-threaded Internet 
> application services.

I hate when this happens.

It turns out that the code I posted doesn't build (let alone install) 
the native version of the code properly.  I've posted an update that 
seems to fix it.  The only source files that changed are the Makefile 
and the META file.  I've added a ChangeLog file.

The new distributions are here:

	http://www.wetware.com/jhw/src/iox-1.00b2.tar.gz
	http://www.wetware.com/jhw/src/iox-1.00b2.tar.bz2

I've also moved the locators for browsing the source and the autodocs:

	http://www.wetware.com/jhw/src/iox/
	http://www.wetware.com/jhw/src/iox/doc/index.html

And, by the way, if you're building this on Mac OS X (or a BSD variant 
with a similar issue), the native version won't link until you use 
ranlib on the .a file in $PREFIX/ocaml/site-lib/iox/iox.a after you 
install it.

I think this might be because ocamlfind install doesn't know to run 
ranlib on the .a file after it moves its absolute path.


--
j h woodyatt <jhw@wetware.com>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners