From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id 8DA77BBAF for ; Mon, 21 Dec 2009 23:50:39 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AuEAAO6JL0vRVdzVkGdsb2JhbACECpcEPwEBAQEJCQwHEwOrdoFbhGeIYwECAwWBKoItUgSBZYlL X-IronPort-AV: E=Sophos;i="4.47,433,1257116400"; d="scan'208";a="39146170" Received: from mail-fx0-f213.google.com ([209.85.220.213]) by mail2-smtp-roc.national.inria.fr with ESMTP; 21 Dec 2009 23:50:39 +0100 Received: by fxm5 with SMTP id 5so5726440fxm.8 for ; Mon, 21 Dec 2009 14:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=wetIo2YtTs2OMODLSLlg1jUgoKUGzzKQ69i6pvBQAs4=; b=JJW2qMbgEzwPmkopoyqkixiNtah+UjWgr4vLKZeJ3JSz/+RvnwZ9lr8fzTC6ajFC6g QprrROBAriRtLJUkuA9qImOlpYbYzSjaWtvLag5MoPHyJ0wQQG6rgYkH0Ch/gbYNAqCF dgO+cy4L9DLJq1FizIDudcgH1rq/UWp36nxuA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=d2Ba60bZ0amkC5hmBALQaIJ3XccR/VhNR7/gh0XQhi3CzlE4Oiy9hTc9tWPZWVjjxY pnvUzA4WnDWngG2/hxRX/qSYrcz8wYoC2itvNqAqrdxpBzVJPKiVNj0zT2EPIAxtLvSi nWNlW+CTMtdTGS81envfVGF8FkMMDLJQ/ZHJQ= MIME-Version: 1.0 Sender: rigtorp@gmail.com Received: by 10.223.5.87 with SMTP id 23mr241183fau.87.1261435836224; Mon, 21 Dec 2009 14:50:36 -0800 (PST) In-Reply-To: <891bd3390912200547i67c3852dv1c91900018fdea9b@mail.gmail.com> References: <4B2D2BC1.6020204@msu.edu> <200912200443.57698.jon@ffconsultancy.com> <891bd3390912200547i67c3852dv1c91900018fdea9b@mail.gmail.com> Date: Mon, 21 Dec 2009 23:50:36 +0100 X-Google-Sender-Auth: a487f0d48552f0c4 Message-ID: Subject: Re: [***SPAM*** Score/Req: 10.1/8.0] Re: [***SPAM*** Score/Req: 10.1/8.0] Re: [Caml-list] Re: OCaml is broken From: Erik Rigtorp To: yminsky Cc: caml-list Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam: no; 0.00; ocaml:01 yaron:01 minsky:01 yminsky:01 ocaml:01 runtime:01 runtimes:01 off-list:01 sockets:01 loopback:01 runtime:01 req:98 req:98 20,:98 2009:98 On Sun, Dec 20, 2009 at 14:47, Yaron Minsky wrote: > On Sun, Dec 20, 2009 at 7:21 AM, Erik Rigtorp wrote: >> >> The first step for OCaml would be to be able to run multiple >> communicating instances of the runtime bound to one core each in one >> process and have them communicate via lock free queues. > > We've done some experiments in this direction at Jane Street. =C2=A0On Li= nux, > we've been able to get fast enough IPC channels for our purposes that > slamming things into the same memory space has not in the end been > necessary. =C2=A0(There is I agree some pain associated with running mult= iple > runtimes in the same process. =C2=A0If you're interested, contact me off-= list and > I can try to get you some of the details of what we ran into.) > But have you tried using shared-memory segments for communicating between > different processes? =C2=A0You say the latencies are too high, but do you= have > any measurements you could share? Have you tried queues using shared memo= ry > segments, in particular? =C2=A0Inter-thread communication has latency as = well, > and the performance issues depend on lots of things, OS and hardware > platform included. =C2=A0It would help in understanding the tradeoffs. Some IPC Benchmarks, Solaris 10 on a quad core Intel Core2 Duo. The benchmarks are running on a cpuset with 1 core. I measure the time from sending in one process until the other process receives the message. So a context switch and the message passing is included in the measurements. Max/Min/Avg * Pipes: 28205/5973/6259 * Unix domain sockets: 44256/7748/8153 * SYSv message queues: 19197/5895/6173 * Posix message queues: 37399/10965/11303 * TCP on loopback: 29017/7471/7885 So the latency is roughly 10=C2=B5s for all these solutions. That latency is pretty high and would be several times the processing time of the message itself. I haven't tried using shm for IPC yet. I'll try see if I can do something with this lock-free queue: http://www.liblfds.org/. > As we go to higher-and-higher numbers of cores, I suspect that > message-passing solutions are likely to scale better than shared memory, = so > I'm not so sure that OCaml is on the wrong path here. =C2=A0I think that = most of I agree, but the message passing must be cheap and the creation and scheduling of lightweight threads must be fast. In order to do that the runtime needs to handle the communication via shared memory between OS threads or processes, binding OS processes and threads to cores and know about different cores different memory latencies and so on.