From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 3A33CBB84 for ; Tue, 15 Jul 2008 20:04:21 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Al0CAKyDfEjU4366gWdsb2JhbACBWpBeAQEQIAOcfQE X-IronPort-AV: E=Sophos;i="4.30,367,1212357600"; d="scan'208";a="15136383" Received: from moutng.kundenserver.de ([212.227.126.186]) by mail3-smtp-sop.national.inria.fr with ESMTP; 15 Jul 2008 20:04:13 +0200 Received: from gate.lan.gerd-stolpmann.de (dslb-084-059-097-221.pools.arcor-ip.net [84.59.97.221]) by mrelayeu.kundenserver.de (node=mrelayeu7) with ESMTP (Nemesis) id 0ML2xA-1KIosP2tOy-00033u; Tue, 15 Jul 2008 20:04:00 +0200 Received: from [192.168.0.32] (fw.lan.gerd-stolpmann.de [192.168.1.1]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id 4EDC6C08C; Tue, 15 Jul 2008 20:03:49 +0200 (CEST) Subject: Re: [Caml-list] Re: thousands of CPU cores From: Gerd Stolpmann To: Kuba Ober Cc: caml-list@yquem.inria.fr In-Reply-To: <200807151157.22397.ober.14@osu.edu> References: <1215724745.24773.37.camel@flake.lan.gerd-stolpmann.de> <200807151157.22397.ober.14@osu.edu> Content-Type: text/plain Date: Tue, 15 Jul 2008 20:03:48 +0200 Message-Id: <1216145029.16617.42.camel@flake.lan.gerd-stolpmann.de> Mime-Version: 1.0 X-Mailer: Evolution 2.12.1 Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX1+9vMeXup821KFsnOVd+XW7sTNDe5URK6QgelJ edK1boQpQ2zwWkijYm8GM5b+aV9VUXVnz9lfG+ZuNwHaBAuk2L 3CJb0ykifCbpVQAZ/T4o+YmFTaXxn0y X-Spam: no; 0.00; gerd:01 stolpmann:01 gerd:01 stolpmann:01 donnerstag:01 donnerstag:01 ocaml:01 stdin:01 stdout:01 mingw:01 sockets:01 syscalls:01 orthogonal:01 syscalls:01 unices:01 Am Dienstag, den 15.07.2008, 11:57 -0400 schrieb Kuba Ober: > On Thursday 10 July 2008, Gerd Stolpmann wrote: > > Am Donnerstag, den 10.07.2008, 21:02 +0000 schrieb Sylvain Le Gall: > > > On 10-07-2008, Gerd Stolpmann wrote: > > > > Am Donnerstag, den 10.07.2008, 20:07 +0000 schrieb Sylvain Le Gall: > > > >> On 10-07-2008, Gerd Stolpmann wrote: > > > >> > In Ocaml you can exploit multi-core currently only by using > > > >> > multi-processing parallel programs that communicate over message > > > >> > passing (and only on Unix). Actually, it's an excellent language for > > > >> > this style. > > > >> > > > >> Why only on Unix ? > > > > > > > > No fork() on Windows. And emulating its effects is hard. > > > > > > open_process + stdin/stdout should do the trick... at least i think so. > > > > After having ported godi to mingw I am not sure whether this works at > > all. The point is that you usually want to inherit OS resources to the > > child process (e.g. sockets). The CreateProcess Win32 call > > (http://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx) mentions > > that you can inherit handles, but I would be careful with the > > information given in MSDN. Often it works only as far as the presented > > examples. Windows isn't written for multi-processing, and its syscalls > > aren't as orthogonal as in Unix-type systems. > > Windows syscalls are quite reasonable IMHO, if a tad undocumented. ReactOS > folk have done a great job of reimplementing most of them, and there isn't > anything mucho broken about those. In fact, I'd posit that Windows native > syscalls expose some functionality that's traditionally unavailable on unices > and requires hacks to achieve (usually via executable code injection). Just > look at what Wine folks had to do in order to emulate some win32 (not even > native!) API on Linux: a lot of hard work for what amounts to a single API > call. This of course works both ways, and on Windows, while a fork() > implementation is simple, it AFAIK requires a custom loader or some other > ingenuity to work. Sure, both systems follow different philosophies. > > Furthermore, it looks like a pain in the ass - often you want to run > > some initialization code, and without fork() you have to run it as often > > as you start processes. > > On Windows, there's the native API, which is then used by the win32 subsystem > and posix subsystems to do the job. Native API allows fork() implementations > mostly on par with what you get on Unices. MS has a posix subsystem on which > fork() performs in the same ballpark as fork() on linux, and make Cygwin's > fork() look bad like it deserves. About the only good thing about Cygwin's > fork() is that it works on win9x. Well, there's now SFU for Windows (but only for XP Professional and Windows 2003, not for XP Home and Vista, AFAIK). That's a cool solution when you want to run Win32 and POSIX programs on the same system, and maybe an alternative to using virtualization. But it is nothing for developing consumer programs on Windows. Btw, has something tried to compile O'Caml on SFU? It's a 230M free download. There seems to be gcc and lots of GNU stuff, too (yes, it's from MS...). > > Also, Windows is just a bad platform for event-based programs, and you > > want to do it to some extent (e.g. for watching all your child > > processes). Only for socket handles there is a select() call. For all > > other types of handles you cannot find out in advance whether the > > operation would block or not. > > This is misinformation at best, FUD at worst. I'm no Microsoft fanboy, > but the reality is quite opposite to what you claim. Windows has quite > robust asynchronous I/O support. No, this is not misinformation, this is the result of digging deeply into the Win32 API for an attempt to port Ocamlnet to Win32 (which will finally happen to some degree). There's overlapped I/O, but the difficulty is that you have to start an operation before you can watch asynchronously for its completion. There is no way to check in advance for that (and that was my claim). Also, there is a quite small limit for the number of resources you can watch at the same time (I think 32 or 64). Look at what Cygwin has done. Basically, they start helper threads for emulating select(). For some cases, there is no real select() support, e.g. the output side of pipes is always considered as writable. Only the input side can be watched. Of course, these difficulties result from porting Unix libraries to Windows. You can say with some right: If you want to program in the event-based way on Windows, you must do it the Windows style. Sure, we are running into the same problems as with fork() - different philosophies make it problematic to write portable programs. What's quite interesting is that the Win32 APIs are less powerful in these areas (process creation, watching events) than the Unix counterpart. That's my whole claim. If I had to develop programs only for Windows, I'd do it multi-threaded because Win32 is much better there. Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------