From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: <3e1162e60903030719v141b41e9ma5fd98c73d8b0e7c@mail.gmail.com> References: <138575260903030352s623807d7p5a3075b1f7a591f6@mail.gmail.com> <3e1162e60903030719v141b41e9ma5fd98c73d8b0e7c@mail.gmail.com> Date: Tue, 3 Mar 2009 16:32:32 +0100 Message-ID: <5d375e920903030732i12f95d0ft8c3ddf579fbeed9f@mail.gmail.com> From: Uriel To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [9fans] threads vs forks Topicbox-Message-UUID: ac7235d8-ead4-11e9-9d60-3106f5b1d025 Python 'threads' are the same pthreads turds all other lunix junk uses. The only difference is that the interpreter itself is not threadsafe, so they have a global lock which means threads suck even more than usual. Forking a python interpreter is a *bad* idea, because python's start up takes billions of years. This has nothing to do with the merits of fork, and all with how much python sucks. There is Stackless Python, which has proper CSP threads/procs and channels, very similar to limbo. http://www.stackless.com/ But that is too sane for the mainline python folks obviously, so they stick to the pthrereads turds, ... My advice: unless you can use Stackless, stay as far away as you can from any concurrent python stuff. (And don't get me started on twisted and their event based hacks). Oh, and as I mentioned in another thread, in my experience if you are going to fork, make sure you compile statically, dynamic linking is almost as evil as pthreads. But this is lunix, so what do you expect? uriel On Tue, Mar 3, 2009 at 4:19 PM, David Leimbach wrote: > > > On Tue, Mar 3, 2009 at 3:52 AM, hugo rivera wrote: >> >> Hi, >> this is not really a plan 9 question, but since you are the wisest >> guys I know I am hoping that you can help me. >> You see, I have to launch many tasks running in parallel (~5000) in a >> cluster running linux. Each of the task performs some astronomical >> calculations and I am not pretty sure if using fork is the best answer >> here. >> First of all, all the programming is done in python and c, and since >> we are using os.fork() python facility I think that it is somehow >> related to the underlying c fork (well, I really do not know much of >> forks in linux, the few things I do know about forks and threads I got >> them from Francisco Ballesteros' "Introduction to operating system >> abstractions"). > > My=C2=A0knowledge=C2=A0on=C2=A0this=C2=A0subject=C2=A0is=C2=A0about=C2=A0= 8=C2=A0or=C2=A09=C2=A0years=C2=A0old,=C2=A0so=C2=A0check=C2=A0with=C2=A0you= r=C2=A0local=C2=A0Python=C2=A0guru.... > The last I'd heard about Python's threading is that it was cooperative on= ly, > and that you couldn't get real parallelism out of it. =C2=A0It serves as = a means > to organize your program in a concurrent manner. > In other words no two threads run at the same time in Python, even if you= 're > on a multi-core system, due to something they call a "Global Interpreter > Lock". > >> >> The point here is if I should use forks or threads to deal with the job = at >> hand? >> I heard that there are some problems if you fork too many processes (I >> am not sure how many are too many) so I am thinking to use threads. >> I know some basic differences between threads and forks, but I am not >> aware of the details of the implementation (probably I will never be). >> Finally, if this is a question that does not belong to the plan 9 >> mailing list, please let me know and I'll shut up. >> Saludos > > I think you need to understand the system limits, which is something you = can > look up for yourself. =C2=A0Also you should understand what kind of runti= me model > threads in the language you're using actually implements. > Those rules basically apply to any system. > >> >> -- >> Hugo >> > >