From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id UAA19024; Tue, 17 Jul 2001 20:50:41 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id UAA19012 for ; Tue, 17 Jul 2001 20:50:40 +0200 (MET DST) Received: from templestowe.rem.cmu.edu ([207.233.206.52]) by concorde.inria.fr (8.11.1/8.10.0) with ESMTP id f6HIodb13985 for ; Tue, 17 Jul 2001 20:50:39 +0200 (MET DST) Received: from aheitner by templestowe.rem.cmu.edu with local (Exim 3.22 #1 (Debian)) id 15MZvQ-0006K1-00; Tue, 17 Jul 2001 14:50:28 -0400 Date: Tue, 17 Jul 2001 14:50:28 -0400 From: Ari Heitner To: John Max Skaller Cc: web-caml@quatramaran.ens.fr, caml-list@inria.fr Subject: Re: [Caml-list] Web Development with OCaml Message-ID: <20010717145028.A24276@andrew.cmu.edu> References: <20010713160428.A9419@pauillac.inria.fr> <20010715140351.B20044@andrew.cmu.edu> <3B546572.FFE73964@maxtal.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.17i In-Reply-To: <3B546572.FFE73964@maxtal.com.au>; from skaller@maxtal.com.au on Wed, Jul 18, 2001 at 02:18:58AM +1000 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk On Wed, Jul 18, 2001 at 02:18:58AM +1000, John Max Skaller wrote: > Ari Heitner wrote: > > The sane thing to do seems to be to have only a few more processes than > > CPUs, and drive those processes has hard as possible with async io. There's > > no strong reason to make either your httpd or your database process even > > multithreaded... > > Poor OS design is the problem. The correct way to service > HTTP requests is to have a queue of pairs: > (user-id, request) > and an object for each user. One then checks the user-id of the > oldest message in the queue, and calls the resume method > of the object for that user, passing the request. > > There are two big problems here. The first is that writing > event driven code is very hard. Felix solves that problem. > [...] > The second problem is the design of operating > system schedulers. It is necessary to do TCP/IP > on a single channel. Most OS's cannot do this. > > So, if there is anyone that knows enough TCP/IP to provide > the required logic from a RAW (connectionless) socket, > then there is a solution. I believe FreeBSD provides > such sockets. The TCP layer must still be implemented. > There may be a way to hack Linux to intercept the > construction of messages per socket (and steal the > message away, and put it into the message queue). Wow. You're a mind reader :) This is how I've been wanting to do communication for some time. My background for this is actually videogames -- something I started doing in high school in a DOS extender. Similar situation -- you've got a render pipeline (initially a hand-coded software render, later a hardware card), representing a serial resource. And you've got a bunch of concrete conceptual models to render, each of which can be broken into little parts. What you don't have is overhead ;) since we wrote everything ourselves. Our homebrew OS had non-premptive multiprocessing via an event-driven model -- a straightforward minimalist RTOS. 120 times a second you process necessary physics. The rest of your spare time, you activate render events, which give every game object an equal shot at rendering a pass (think "sending out a packet"). And of course it worked fine on top of Windows or Linux or whatever -- since the render pipeline is handled at a very low level, traditionally (if it was like TCP/IP, it would be like having GL replaced by a scenegraph system). > > It is possible that Felix can be modified/enhanced to > generate Ocaml instead of C++: the key requirement > is for classes with a resume method. [The biggest > technical problem is that efficient implementation > of control structures requires a goto which Caml > doesn't have, but there is a slightly less efficient > workaround already available] I've looked at Felix a tiny bit, but not enough to know how it interprets event-driven stuff...I never felt uncomfortable programming event-driven stuff in C++, but then, I literally grew up on it :) you just have to keep a good tag on how how long any even takes :) ... I never seriously looked at implementing any of this -- it's way off topic for my normal stuff. But surely Linux or the BSDs have some kind of raw ethernet interface? I mean, eth0 is a character device, right? Except of course for the weird magical distinction between network devices, which exist in their own magical TCP/IP-land-namespace, and all other devices (serial ports, usb ports, parallel ports, hard drives, floppy drives, smoke signals, hamsters with laser pointers) all have /dev/ entries and can presumably be accessed in raw mode (oh wait, "smoke signals" are a networking device). I suspect half the problem is the evils of TCP/IP ("a hoagie with many many layers"). But then I never took 15-411 Networks, I took 15-412 OS. So aside from the fact I know every packet has a lot of bytes worth of headers around the payload, I have no real clue how hard it would be, or what libs exist, for assembling packets on a raw interface. ... Our scheme in our video game engine was simple: all objects can register for ticks (120 times a second) and renders (whenever there's free resources). Everyone plays nice. Easy enough to do in o'caml, given the right libs. ... But doesn't this sound a lot like what's happening in kernel space anyhow? I mean, you've got a queue of packets to send (skb_buf's, in Linux, iirc). Those include the actual data you've produced, which can even be scatter-gathered to avoid copying data -- it's just a collection of pages to send directly to the card. And the kernel has what we descreibed above -- when the card generates a RTS interrupt, the driver pulls more packets off the queue. So what's the hangup? I'm no longer sure :) Is it just the expense of having zillions of threads/processes managing all these connections, and context switching expensively to get tiny bits of work done? You can't easily get away w/o at least a process or thread for your DB work, right, because the database needs fore each query are complicated and it would be *really* hard to manage that in an event-oriented way. So even if your webserver was a single thread, or even entirely in kernel space, you spend a lot on managing the assebly of data from your DB -- which has to be done (at least) a thread per conceptual request? Or does Felix do that nicely too? Or did I totally miss the point? [or am i just rambling?] ari ------------------- Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr