From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id BDAC7BCAC for ; Wed, 25 May 2005 00:14:47 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by concorde.inria.fr (8.13.0/8.13.0) with ESMTP id j4OMElYV020436 for ; Wed, 25 May 2005 00:14:47 +0200 Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id AAA13802 for ; Wed, 25 May 2005 00:14:46 +0200 (MET DST) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.193]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id j4OMEjuo028252 for ; Wed, 25 May 2005 00:14:46 +0200 Received: by wproxy.gmail.com with SMTP id 69so366945wri for ; Tue, 24 May 2005 15:14:45 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=CD3g6j+qjikROSc4bJ/PXjism/1FeSm/yf0pKY8A6b86s7SChBu7bjEeMpZchW07p2Oi03bcxJ57NOMqRinmGIarq46ZELoqkm+dkwVcXObwiH3qhhpLt55rQyRHUpewTaGyKsaaCei4ILy4OrLu1wcNJvPvTDVWmfp7tnnDqXQ= Received: by 10.54.8.41 with SMTP id 41mr798494wrh; Tue, 24 May 2005 15:14:45 -0700 (PDT) Received: by 10.54.2.21 with HTTP; Tue, 24 May 2005 15:14:45 -0700 (PDT) Message-ID: <891bd33905052415145bb1818f@mail.gmail.com> Date: Tue, 24 May 2005 18:14:45 -0400 From: Yaron Minsky Reply-To: yminsky@cs.cornell.edu To: Caml Mailing List Subject: Efficient I/O with threads Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Miltered: at concorde with ID 4293A757.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at nez-perce with ID 4293A756.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; yaron:01 minsky:01 yminsky:01 threads:01 ocaml:01 outputting:01 sexp:01 sexp:01 parsing:01 marshalling:01 buffer:01 buffer:01 yaron:01 wire:98 functions:01 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on yquem.inria.fr X-Spam-Status: No, score=0.0 required=5.0 tests=RCVD_BY_IP autolearn=disabled version=3.0.2 X-Spam-Level: We've been running into some interesting problems building highly efficient I/O routines in threaded code in ocaml, and I'm curious if anyone else has some thoughts on this. The basic problem seems to be that the locking and unlocking of the IO channels seems to take a large fraction of the execution time. A little bit of background first. The data type we're outputting is basically a simple s-expression, with the following type: type sexp =3D Atom of string | List of sexp list We write out an s-expression by writing a tag-byte to determine whether the s-expression is an atom or a string. If the s-expression is an atom, we then write a 4-byte int, which is the length of the string, and then the string. If the s-expression is a list, we write an atom which is the number of s-expression that are contained, and then write those s-expressions. It's very easy to write parsing and marshalling for this type of wire protocol, but that code turns out to be quite inefficient, because you end up making too many calls to the input and output functions, and each one of those calls requires releasing and acquiring locks. I just can't think of a clean way of implementing a reader for this kind of protocol. (a writer could be done by writing stuff to a buffer first, and then writing the whole buffer out at the socket at once.) Any thoughts? Yaron