From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4354602C.7060102@lanl.gov> Date: Mon, 17 Oct 2005 20:38:36 -0600 From: Ronald G Minnich User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050317) MIME-Version: 1.0 To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu> Subject: Re: [9fans] xcpu note References: <4353D498.2060008@lanl.gov> <3e1162e60510171441k6999dc37i389557af60f6ea14@mail.gmail.com> In-Reply-To: <3e1162e60510171441k6999dc37i389557af60f6ea14@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Topicbox-Message-UUID: 9b6ac01a-ead0-11e9-9d60-3106f5b1d025 David Leimbach wrote: > Clustermatic is pretty cool, I think it's what was installed on one of > the other clusters I used at LANL as a contractor at the time. I > recall a companion tool for bproc to request nodes, sort of an ad-hoc > scheduler. I had to integrate support for this in our MPI's start up > that I was testing on that machine. the simple scheduler, bjs, was written by erik hendriks (now at Google, sigh) and was rock-solid. It ran on one cluster, unattended, scheduling 128 2-cpu nodes with a very diverse job mix, for one year. It was a great piece of software. It was far faster, and far more reliable, than any scheduler we have ever seen, then or now. In one test, we ran about 20,000 jobs through it on about an hour, on a 1024-node cluster, just to test. Note that it could probably have scheduled a lot more jobs, but the run-time of the job was non-zero. No other scheduler we have used comes close to this kind of performance. Scheduler overhead was basically insignificant. > > I'm curious to see how this all fits together with xcpu, if there is > such a resource allocation setup needed etc. we're going to take bjs and have it schedule nodes to give to users. Note one thing we are going to do with xcpu: attach nodes to a user's desktop machine, rather than make users log in to the cluster. So users will get interactive clusters that look like they own them. This will, we hope, kill batch mode. Plan 9 ideas make this possible. It's going to be a big change, one we hope users will like. If you look at how most clusters are used today, they closely resemble the batch world of the 1960s. It is actually kind of shocking. I downloaded a JCL manual a year or two ago, and compared what JCL did to what people wanted batch schedulers for clusters to do, and the correspondance was a little depressing. The Data General ad said it best: "Batch is a bitch". Oh yeah, if anyone has a copy of that ad (Google does not), i'd like it in .pdf :-) It appeared in the late 70s IIRC. ron p.s. go ahead, google JCL, and you can find very recent manuals on how to use it. I will be happy to post the JCL for "sort + copy" if anyone wants to see it.