On Tue, Mar 3, 2009 at 3:52 AM, hugo rivera
<uair00@gmail.com> wrote:
Hi,
this is not really a plan 9 question, but since you are the wisest
guys I know I am hoping that you can help me.
You see, I have to launch many tasks running in parallel (~5000) in a
cluster running linux. Each of the task performs some astronomical
calculations and I am not pretty sure if using fork is the best answer
here.
First of all, all the programming is done in python and c, and since
we are using os.fork() python facility I think that it is somehow
related to the underlying c fork (well, I really do not know much of
forks in linux, the few things I do know about forks and threads I got
them from Francisco Ballesteros' "Introduction to operating system
abstractions").
My knowledge on this subject is about 8 or 9 years old, so check with your local Python guru....
The last I'd heard about Python's threading is that it was cooperative only, and that you couldn't get real parallelism out of it. It serves as a means to organize your program in a concurrent manner.
In other words no two threads run at the same time in Python, even if you're on a multi-core system, due to something they call a "Global Interpreter Lock".
The point here is if I should use forks or threads to deal with the job at hand?
I heard that there are some problems if you fork too many processes (I
am not sure how many are too many) so I am thinking to use threads.
I know some basic differences between threads and forks, but I am not
aware of the details of the implementation (probably I will never be).
Finally, if this is a question that does not belong to the plan 9
mailing list, please let me know and I'll shut up.
Saludos
I think you need to understand the system limits, which is something you can look up for yourself. Also you should understand what kind of runtime model threads in the language you're using actually implements.
Those rules basically apply to any system.
--
Hugo