Re: [Caml-list] Re: thousands of CPU cores

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Dienstag, den 15.07.2008, 11:57 -0400 schrieb Kuba Ober:
> On Thursday 10 July 2008, Gerd Stolpmann wrote:
> > Am Donnerstag, den 10.07.2008, 21:02 +0000 schrieb Sylvain Le Gall:
> > > On 10-07-2008, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
> > > > Am Donnerstag, den 10.07.2008, 20:07 +0000 schrieb Sylvain Le Gall:
> > > >> On 10-07-2008, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
> > > >> > In Ocaml you can exploit multi-core currently only by using
> > > >> > multi-processing parallel programs that communicate over message
> > > >> > passing (and only on Unix). Actually, it's an excellent language for
> > > >> > this style.
> > > >>
> > > >> Why only on Unix ?
> > > >
> > > > No fork() on Windows. And emulating its effects is hard.
> > >
> > > open_process + stdin/stdout should do the trick... at least i think so.
> >
> > After having ported godi to mingw I am not sure whether this works at
> > all. The point is that you usually want to inherit OS resources to the
> > child process (e.g. sockets). The CreateProcess Win32 call
> > (http://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx) mentions
> > that you can inherit handles, but I would be careful with the
> > information given in MSDN. Often it works only as far as the presented
> > examples. Windows isn't written for multi-processing, and its syscalls
> > aren't as orthogonal as in Unix-type systems.
> 
> Windows syscalls are quite reasonable IMHO, if a tad undocumented. ReactOS
> folk have done a great job of reimplementing most of them, and there isn't
> anything mucho broken about those. In fact, I'd posit that Windows native
> syscalls expose some functionality that's traditionally unavailable on unices
> and requires hacks to achieve (usually via executable code injection). Just
> look at what Wine folks had to do in order to emulate some win32 (not even
> native!) API on Linux: a lot of hard work for what amounts to a single API
> call. This of course works both ways, and on Windows, while a fork()
> implementation is simple, it AFAIK requires a custom loader or some other
> ingenuity to work.

Sure, both systems follow different philosophies.

> > Furthermore, it looks like a pain in the ass - often you want to run
> > some initialization code, and without fork() you have to run it as often
> > as you start processes.
> 
> On Windows, there's the native API, which is then used by the win32 subsystem
> and posix subsystems to do the job. Native API allows fork() implementations
> mostly on par with what you get on Unices. MS has a posix subsystem on which
> fork() performs in the same ballpark as fork() on linux, and make Cygwin's
> fork() look bad like it deserves. About the only good thing about Cygwin's 
> fork() is that it works on win9x.

Well, there's now SFU for Windows (but only for XP Professional and
Windows 2003, not for XP Home and Vista, AFAIK). That's a cool solution
when you want to run Win32 and POSIX programs on the same system, and
maybe an alternative to using virtualization. But it is nothing for
developing consumer programs on Windows.

Btw, has something tried to compile O'Caml on SFU? It's a 230M free
download. There seems to be gcc and lots of GNU stuff, too (yes, it's
from MS...).

> > Also, Windows is just a bad platform for event-based programs, and you
> > want to do it to some extent (e.g. for watching all your child
> > processes). Only for socket handles there is a select() call. For all
> > other types of handles you cannot find out in advance whether the
> > operation would block or not.
> 
> This is misinformation at best, FUD at worst. I'm no Microsoft fanboy,
> but the reality is quite opposite to what you claim. Windows has quite
> robust asynchronous I/O support.

No, this is not misinformation, this is the result of digging deeply
into the Win32 API for an attempt to port Ocamlnet to Win32 (which will
finally happen to some degree). There's overlapped I/O, but the
difficulty is that you have to start an operation before you can watch
asynchronously for its completion. There is no way to check in advance
for that (and that was my claim). Also, there is a quite small limit for
the number of resources you can watch at the same time (I think 32 or
64).

Look at what Cygwin has done. Basically, they start helper threads for
emulating select(). For some cases, there is no real select() support,
e.g. the output side of pipes is always considered as writable. Only the
input side can be watched.

Of course, these difficulties result from porting Unix libraries to
Windows. You can say with some right: If you want to program in the
event-based way on Windows, you must do it the Windows style. Sure, we
are running into the same problems as with fork() - different
philosophies make it problematic to write portable programs. What's
quite interesting is that the Win32 APIs are less powerful in these
areas (process creation, watching events) than the Unix counterpart.
That's my whole claim. If I had to develop programs only for Windows,
I'd do it multi-threaded because Win32 is much better there.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------