[9fans] Process distribution?

[9fans] Process distribution?

[tt.gustavsson]

if you have say 5 cpu servers in your network, how do you choose wich server to run
applications in? Would you logically dedicate them to different types of tasks, like
compilation server,  graphics processing, ... , or dedicate them to different groups
of people, the programmers in the building across the street, and the programmers in
the same building... ?

It is an elegant distributed computing environment, and I am curious of how to
use it effectively. The idea seems too not be intended in this environment,
and my idea was to deal with two different issues:

1. many terminals connecting to a couple of cpu servers, were you would not actively
choose which cpu to use for certain programs.

2. Somehow use a number of cpu servers for raw cpu processing (seti and such...)

(I just realized that the second issue really does not apply to my idea).

Anyway, I will set up an old dual Ppro as my first CPU server, interestingley enough.


Cheers

/Tony G

Re: [9fans] Process distribution?

[a]

// if you have say 5 cpu servers in your network, how do you
// choose wich server to run applications in?

when i've been in groups with multiple cpu servers, we tended
to have one or two dedicated to services - mail, web, and the
services specific to our project - and split the remainder by
people. we only did that latter part very roughly, though:
telling people "you guys use server foo, you guys use bar.".
we never did access controll stuff, and if i logged into foo
and found a co-worker pounding on it, i'd check out bar.
ア

Re: [9fans] Process distribution?

[YAMANASHI Takeshi]

> i forgetfirewall right now): does cpu preserve stdout and stderr?

cpu doesn't.

This sometimes bites me while using cpu in a pipeline:
	% cat /dev/window | cpu -c topng | page
--

[9fans] Process distribution?

[tt.gustavsson]

I have an idea, don't know though if it is in line with
the intentions of the plan 9 project, but anyway.

If when starting processes on my terminal the processes
would be started randomly to a number of CPU servers,
instead of manually bind to a CPU whenever I would start a
process/program. Could this work as a load balancing idea.

Functionality on the CPU servers to stop responding to CPU
binding requests when they would be overloaded, and accept
when the load drops. I read something about an effort to
make a program/process "scheduler" that would be based on
load of the CPU servers, but when enough CPU servers would be
setup, and enough terminals starting processes, wouldn't
this balance the CPU servers to a reasonable degree?

Just a though.

/Tony G, Sweden.

Re: [9fans] Process distribution?

[a]

i suspect a random distribution of processes would yield worse
results than you expect. you really want a scheduler of sorts
that'll find the most appropriate system to run your proc on.

i think the process dispatch stuff you're looking for can all
be done in user mode, with just an app that essentially calls
cpu and does a few namespace ops. the slightly trickier part
(still not *too* hard) is choosing who to dispatch to, but i
think that can be managed properly by having the participating
cpu servers provide their own status information and have a
dispatch daemon on the client poll them when an app needs to
be dispatched. you could even define a way for the app to
specify its resource requirements to the dispatch daemon.

i really should look at the plan 9 and inferno grid stuff.

i forget\0 (and drawterm doesn't work from behind my work
firewall right now): does cpu preserve stdout and stderr?

things get tricky - downright *hard* - when you start
looking for a clean way to do process *migration*.
ア

Re: [9fans] Process distribution?

[Fco.J.Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

We do something similar for Plan B, and are in the
process of backporting that to Plan 9. See
http://lsub.org/ls/export/bman/4/proc.html

The man page is out of date but that doesn't matter here.

Regarding process migration, please don't, ;-)

[-- Attachment #2: Type: message/rfc822, Size: 3148 bytes --]

From: a@9srv.net
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Process distribution?
Date: Fri, 21 May 2004 09:24:08 -0400
Message-ID: <71b00c64409f2dda50adeea8d576ab4f@9srv.net>

i suspect a random distribution of processes would yield worse
results than you expect. you really want a scheduler of sorts
that'll find the most appropriate system to run your proc on.

i think the process dispatch stuff you're looking for can all
be done in user mode, with just an app that essentially calls
cpu and does a few namespace ops. the slightly trickier part
(still not *too* hard) is choosing who to dispatch to, but i
think that can be managed properly by having the participating
cpu servers provide their own status information and have a
dispatch daemon on the client poll them when an app needs to
be dispatched. you could even define a way for the app to
specify its resource requirements to the dispatch daemon.

i really should look at the plan 9 and inferno grid stuff.

i forget\0 (and drawterm doesn't work from behind my work
firewall right now): does cpu preserve stdout and stderr?

things get tricky - downright *hard* - when you start
looking for a clean way to do process *migration*.
ア

Re: [9fans] Process distribution?

[boyd, rounin]

> Regarding process migration, please don't, ;-)

    From: <9fans-bounces+boyd=insultant.net@cse.psu.edu>
    To: 9fans@cse.psu.edu
    Subject: Re: [9fans] Process distribution?
    Illegal-Object: Syntax error in From: address found on
ams.ftl.affinity.com:
     From: Fco.J.Ballesteros<nemo@lsub.org>
         ^-missing end of mailbox

please change the above to:

    From "Fco.J.Ballesteros" <nemo@lsub.org>

or remove the dots.

Re: [9fans] Process distribution?

[Fco. J. Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 30 bytes --]

thanks for the warning.
done.

[-- Attachment #2: Type: message/rfc822, Size: 2917 bytes --]

From: "boyd, rounin" <boyd@insultant.net>
To: "Fans of the OS Plan 9 from Bell Labs" <9fans@cse.psu.edu>
Subject: Re: [9fans] Process distribution?
Date: Fri, 21 May 2004 17:18:55 +0200
Message-ID: <01a401c43f46$efab7b70$207e7d50@SOMA>

> Regarding process migration, please don't, ;-)

    From: <9fans-bounces+boyd=insultant.net@cse.psu.edu>
    To: 9fans@cse.psu.edu
    Subject: Re: [9fans] Process distribution?
    Illegal-Object: Syntax error in From: address found on
ams.ftl.affinity.com:
     From: Fco.J.Ballesteros<nemo@lsub.org>
         ^-missing end of mailbox

please change the above to:

    From "Fco.J.Ballesteros" <nemo@lsub.org>

or remove the dot s.

Re: [9fans] Process distribution?

[boyd, rounin]

> thanks for the warning.

we _aim_ to please ;)

Re: [9fans] Process distribution?

[ron minnich]

On Fri, 21 May 2004, Fco.J.Ballesteros wrote:

> Regarding process migration, please don't, ;-)

yep. System-level migration is a bad idea.

ron

Re: [9fans] Process distribution?

[boyd, rounin]

> Functionality on the CPU servers to stop responding to CPU
> binding requests when they would be overloaded, and accept
> when the load drops.

i think you need virtual cpu servers.  cpu <name> binds stuff
to a group of CPUs and they work out the scheduling.

'natch some CPUs have are multiprocessor, eliminating
the process migration problem.

621109-4138

Re: [9fans] Process distribution?

[Geoff Collyer]

I think you want a little more control over where programs run.
The canonical examples are that editors are better suited to
terminals and compilers to CPU servers.  CPU servers are not
necessarily all equivalent, even within an installation.

The cpu command takes a second or two to start the named
command running (depending upon the complexity of your profile)
so it's often simpler to just run `cpu' to create a window
running an rc on a CPU server, or run `cpu -c rio' to run a
window system there.

I have rarely felt slowed by a CPU server being overloaded,
so I'm not sure that there's a real problem here.