* [9fans] Barrelfish @ 2009-10-14 19:09 Tim Newsham 2009-10-14 19:54 ` Roman Shaposhnik 2009-10-15 18:28 ` Christopher Nielsen 0 siblings, 2 replies; 37+ messages in thread From: Tim Newsham @ 2009-10-14 19:09 UTC (permalink / raw) To: 9fans Rethinking multi-core systems as distributed heterogeneous systems. Thoughts? http://www.sigops.org/sosp/sosp09/papers/baumann-sosp09.pdf Tim Newsham http://www.thenewsh.com/~newsham/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 19:09 [9fans] Barrelfish Tim Newsham @ 2009-10-14 19:54 ` Roman Shaposhnik 2009-10-14 21:21 ` Tim Newsham 2009-10-14 21:36 ` [9fans] Barrelfish Eric Van Hensbergen 2009-10-15 18:28 ` Christopher Nielsen 1 sibling, 2 replies; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-14 19:54 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Oct 14, 2009 at 12:09 PM, Tim Newsham <newsham@lava.net> wrote: > Rethinking multi-core systems as distributed heterogeneous > systems. Thoughts? Somehow this feels related to the work that came out of Berkeley a year or so ago. I'm still not convinced what is the benefits of multiple kernels. If you are managing a couple of 100s of cores a single kernel would do just fine, once the industry is ready for a couple dozen of thousands PUs -- the kernel is most likely to be dispensed with anyway. Did you find any ideas there particularly engaging? Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 19:54 ` Roman Shaposhnik @ 2009-10-14 21:21 ` Tim Newsham 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) ` (2 more replies) 2009-10-14 21:36 ` [9fans] Barrelfish Eric Van Hensbergen 1 sibling, 3 replies; 37+ messages in thread From: Tim Newsham @ 2009-10-14 21:21 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > Somehow this feels related to the work that came out of Berkeley a year > or so ago. I'm still not convinced what is the benefits of multiple > kernels. If you are managing a couple of 100s of cores a single kernel > would do just fine, once the industry is ready for a couple dozen of > thousands PUs -- the kernel is most likely to be dispensed with anyway. I'm not familiar with the berkeley work. > Did you find any ideas there particularly engaging? I'm still digesting it. My first thoughts were that if my pc is a distributed heterogeneous computer, what lessons it can borrow from earlier work on distributed heterogeneous computing (ie. plan9). I found the discussion on cache coherency, message passing and optimization to be enlightening. The fact that you may want to organize your core OS quite a bit differently depending on which model cpus in the same family you use is kind of scary. The mention that "... the overhead of cache coherence restricts the ability to scale up to even 80 cores" is also eye openeing. If we're at aprox 8 cores today, thats only 5 yrs away (if we double cores every 1.5 yrs). > Roman. Tim Newsham http://www.thenewsh.com/~newsham/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:21 ` Tim Newsham @ 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2009-10-14 21:42 ` Noah Evans 2009-10-15 1:03 ` David Leimbach 2009-10-15 1:50 ` Roman Shaposhnik 2 siblings, 1 reply; 37+ messages in thread From: Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2009-10-14 21:33 UTC (permalink / raw) To: 9fans > I'm not familiar with the berkeley work. Me either. Any chance of some references to this? ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2009-10-14 21:42 ` Noah Evans 2009-10-14 21:45 ` erik quanstrom 2009-10-14 22:10 ` Eric Van Hensbergen 0 siblings, 2 replies; 37+ messages in thread From: Noah Evans @ 2009-10-14 21:42 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs http://ramp.eecs.berkeley.edu/ Tim: Andrew Baumann is aware of Plan 9 but their approach is quite a bit different. They are consciously avoiding the networking issue as well(they've been asked to extend their messaging model to the network and have actively said they're not interested). On Wed, Oct 14, 2009 at 11:33 PM, Lyndon Nerenberg (VE6BBM/VE7TFX) <lyndon@orthanc.ca> wrote: >> I'm not familiar with the berkeley work. > > Me either. Any chance of some references to this? > > > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:42 ` Noah Evans @ 2009-10-14 21:45 ` erik quanstrom 2009-10-14 21:57 ` Noah Evans 2009-10-14 22:10 ` Eric Van Hensbergen 1 sibling, 1 reply; 37+ messages in thread From: erik quanstrom @ 2009-10-14 21:45 UTC (permalink / raw) To: 9fans > http://ramp.eecs.berkeley.edu/ > > Tim: Andrew Baumann is aware of Plan 9 but their approach is quite a > bit different. They are consciously avoiding the networking issue as > well(they've been asked to extend their messaging model to the network > and have actively said they're not interested). every interconnect is a network. sometimes we don't admit it. - erik ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:45 ` erik quanstrom @ 2009-10-14 21:57 ` Noah Evans 0 siblings, 0 replies; 37+ messages in thread From: Noah Evans @ 2009-10-14 21:57 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Have you read the paper? I don't think you understand the difference in scope or goals here. On Wed, Oct 14, 2009 at 11:45 PM, erik quanstrom <quanstro@coraid.com> wrote: >> http://ramp.eecs.berkeley.edu/ >> >> Tim: Andrew Baumann is aware of Plan 9 but their approach is quite a >> bit different. They are consciously avoiding the networking issue as >> well(they've been asked to extend their messaging model to the network >> and have actively said they're not interested). > > every interconnect is a network. sometimes we don't admit it. > > - erik > > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:42 ` Noah Evans 2009-10-14 21:45 ` erik quanstrom @ 2009-10-14 22:10 ` Eric Van Hensbergen 2009-10-14 22:21 ` Noah Evans 1 sibling, 1 reply; 37+ messages in thread From: Eric Van Hensbergen @ 2009-10-14 22:10 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Oct 14, 2009, at 3:42 PM, Noah Evans wrote: > http://ramp.eecs.berkeley.edu/ > > Tim: Andrew Baumann is aware of Plan 9 but their approach is quite a > bit different. They are consciously avoiding the networking issue as > well(they've been asked to extend their messaging model to the network > and have actively said they're not interested). > While they may not be interested in implementing a network messaging model, they don't oppose it. Jonathan and I talked with Andrew about porting Barrelfish to Blue Gene yesterday to test some of their scalability claims at large scale. -eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 22:10 ` Eric Van Hensbergen @ 2009-10-14 22:21 ` Noah Evans 0 siblings, 0 replies; 37+ messages in thread From: Noah Evans @ 2009-10-14 22:21 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Do want. On Thu, Oct 15, 2009 at 12:10 AM, Eric Van Hensbergen <ericvh@gmail.com> wrote: > On Oct 14, 2009, at 3:42 PM, Noah Evans wrote: > >> http://ramp.eecs.berkeley.edu/ >> >> Tim: Andrew Baumann is aware of Plan 9 but their approach is quite a >> bit different. They are consciously avoiding the networking issue as >> well(they've been asked to extend their messaging model to the network >> and have actively said they're not interested). >> > > While they may not be interested in implementing a network messaging model, > they don't oppose it. Jonathan and I talked with Andrew about porting > Barrelfish to Blue Gene yesterday to test some of their scalability claims > at large scale. > > -eric > > > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:21 ` Tim Newsham 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2009-10-15 1:03 ` David Leimbach 2009-10-15 1:50 ` Roman Shaposhnik 2 siblings, 0 replies; 37+ messages in thread From: David Leimbach @ 2009-10-15 1:03 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1074 bytes --] > > > > Did you find any ideas there particularly engaging? >> > > I'm still digesting it. My first thoughts were that if my pc is a > distributed heterogeneous computer, what lessons it can borrow from earlier > work on distributed heterogeneous computing (ie. plan9). > > I found the discussion on cache coherency, message passing and optimization > to be enlightening. The fact that you may want to > organize your core OS quite a bit differently depending on which > model cpus in the same family you use is kind of scary. > > The mention that "... the overhead of cache coherence restricts the ability > to scale up to even 80 cores" is also eye openeing. If we're at aprox 8 > cores today, thats only 5 yrs away (if we double cores every > 1.5 yrs). > I personally thought the use of DSLs built on Haskell was rather clever, but the other discoveries are the sort of feedback I suspect our CPU vendors aren't going to think about on their own somehow :-) > > Roman. >> > > Tim Newsham > http://www.thenewsh.com/~newsham/ > > [-- Attachment #2: Type: text/html, Size: 1788 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:21 ` Tim Newsham 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2009-10-15 1:03 ` David Leimbach @ 2009-10-15 1:50 ` Roman Shaposhnik 2009-10-15 2:12 ` Eric Van Hensbergen 2009-10-15 10:53 ` Sam Watkins 2 siblings, 2 replies; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-15 1:50 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Oct 14, 2009 at 2:21 PM, Tim Newsham <newsham@lava.net> wrote: > I'm not familiar with the berkeley work. Sorry I can't readily find the paper (the URL is somewhere on IMAP @Sun :-() But it got presented at the Birkeley ParLab overview given to us by Dave Patterson. They were talking thin hypervisors and that sort of stuff. More details here: http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-23.pdf (page 10) but still no original paper in sight... > I'm still digesting it. My first thoughts were that if my pc is a > distributed heterogeneous computer It may very well be that, but why would you want to manage that complexity? Your GPU is already heavily "multicore", yet you don't see it (and you really don't want to see it!) > The mention that "... the overhead of cache coherence restricts the ability > to scale up to even 80 cores" is also eye openeing. If we're at aprox 8 > cores today, thats only 5 yrs away (if we double cores every > 1.5 yrs). A couple of years ago we had a Plan9 summit @Google campus and Ken was there. I still remember the question he asked me: what exactly would you make all those core do on your desktop? Frankly, I still don't have a good answer. Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 1:50 ` Roman Shaposhnik @ 2009-10-15 2:12 ` Eric Van Hensbergen 2009-10-15 10:53 ` Sam Watkins 1 sibling, 0 replies; 37+ messages in thread From: Eric Van Hensbergen @ 2009-10-15 2:12 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Oct 14, 2009, at 7:50 PM, Roman Shaposhnik wrote: > On Wed, Oct 14, 2009 at 2:21 PM, Tim Newsham <newsham@lava.net> wrote: >> I'm not familiar with the berkeley work. > > Sorry I can't readily find the paper (the URL is somewhere on IMAP > @Sun :-() > But it got presented at the Birkeley ParLab overview given to us by > Dave Patterson. > They were talking thin hypervisors and that sort of stuff. More > details here: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-23.pdf > (page 10) but still no original paper in sight... You may be thinking about the Tesselation work from Berkley ParLab (http://parlab.eecs.berkeley.edu/publication/221 ) -eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 1:50 ` Roman Shaposhnik 2009-10-15 2:12 ` Eric Van Hensbergen @ 2009-10-15 10:53 ` Sam Watkins 2009-10-15 11:50 ` Richard Miller ` (3 more replies) 1 sibling, 4 replies; 37+ messages in thread From: Sam Watkins @ 2009-10-15 10:53 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Oct 14, 2009 at 06:50:28PM -0700, Roman Shaposhnik wrote: > > The mention that "... the overhead of cache coherence restricts the ability > > to scale up to even 80 cores" is also eye openeing. If we're at aprox 8 > > cores today, thats only 5 yrs away (if we double cores every > > 1.5 yrs). Sharing the memory between processes is a stupid approach to multi-processing / multi-threading. Modern popular computer architecture and software design is fairly much uniformly stupid. > A couple of years ago we had a Plan9 summit @Google campus and Ken was > there. I still remember the question he asked me: what exactly would you make > all those core do on your desktop? It's easy to write good code that will take advantage of arbitrarily many processors to run faster / smoother, if you have a proper language for the task. With respect to Ken, Bill Gates said something along the lines of "who would need more than 640K?". There is a vast range of applications that cannot be managed in real time using existing single-core technology. Sam ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 10:53 ` Sam Watkins @ 2009-10-15 11:50 ` Richard Miller 2009-10-15 12:00 ` W B Hacker 2009-10-16 17:03 ` Sam Watkins 2009-10-15 11:56 ` Josh Wood ` (2 subsequent siblings) 3 siblings, 2 replies; 37+ messages in thread From: Richard Miller @ 2009-10-15 11:50 UTC (permalink / raw) To: 9fans > It's easy to write good code that will take advantage of arbitrarily many > processors to run faster / smoother, if you have a proper language for the > task. ... and if you can find a way around Amdahl's law (qv). ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 11:50 ` Richard Miller @ 2009-10-15 12:00 ` W B Hacker 2009-10-16 17:03 ` Sam Watkins 1 sibling, 0 replies; 37+ messages in thread From: W B Hacker @ 2009-10-15 12:00 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Richard Miller wrote: >> It's easy to write good code that will take advantage of arbitrarily many >> processors to run faster / smoother, if you have a proper language for the >> task. > > ... and if you can find a way around Amdahl's law (qv). > > > http://www.cis.temple.edu/~shi/docs/amdahl/amdahl.html ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 11:50 ` Richard Miller 2009-10-15 12:00 ` W B Hacker @ 2009-10-16 17:03 ` Sam Watkins 2009-10-16 18:17 ` ron minnich 2009-10-17 12:42 ` Roman Shaposhnik 1 sibling, 2 replies; 37+ messages in thread From: Sam Watkins @ 2009-10-16 17:03 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Oct 15, 2009 at 12:50:48PM +0100, Richard Miller wrote: > > It's easy to write good code that will take advantage of arbitrarily many > > processors to run faster / smoother, if you have a proper language for the > > task. > > ... and if you can find a way around Amdahl's law (qv). "The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program." So it would only be a problem supposing that a significant part of the program is unparallelizable. I can think of many many tasks where "Amdahl's law" is not going to be a problem at all, for a properly designed system. For example if I had a thousand processors I might raytrace complex scenes for an animated game at 100 fps, or do complex dsp over a 2 hour audio track in one millisecond. I suppose most difficult/interesting tasks can be parallelized effectively. Seems that Amdahl's law is a minor issue. Of course if you are trying to run old-fashioned sequential programs on a parallel machine you will not benefit. You would need to rewrite them. Sam ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-16 17:03 ` Sam Watkins @ 2009-10-16 18:17 ` ron minnich 2009-10-16 18:39 ` Wes Kussmaul 2009-10-17 12:42 ` Roman Shaposhnik 1 sibling, 1 reply; 37+ messages in thread From: ron minnich @ 2009-10-16 18:17 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Oct 16, 2009 at 10:03 AM, Sam Watkins <sam@nipl.net> wrote: > So it would only be a problem supposing that a significant part of the program > is unparallelizable. I can think of many many tasks where "Amdahl's law" is > not going to be a problem at all, for a properly designed system. > > For example if I had a thousand processors I might raytrace complex scenes for > an animated game at 100 fps, or do complex dsp over a 2 hour audio track in one > millisecond. Yes, if you had that few processors, it might seem easy. It gets somewhat harder when you have, say, 128,000. Insignificant bits of code that were not even visible suddenly dominate the time. ron ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-16 18:17 ` ron minnich @ 2009-10-16 18:39 ` Wes Kussmaul 0 siblings, 0 replies; 37+ messages in thread From: Wes Kussmaul @ 2009-10-16 18:39 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs ron minnich wrote: > Insignificant > bits of code that were not even visible suddenly dominate the time. Reminds me of some project development teams. Maybe Marvin Minsky was on to something. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-16 17:03 ` Sam Watkins 2009-10-16 18:17 ` ron minnich @ 2009-10-17 12:42 ` Roman Shaposhnik 1 sibling, 0 replies; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-17 12:42 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Oct 16, 2009 at 10:03 AM, Sam Watkins <sam@nipl.net> wrote: > On Thu, Oct 15, 2009 at 12:50:48PM +0100, Richard Miller wrote: >> > It's easy to write good code that will take advantage of arbitrarily many >> > processors to run faster / smoother, if you have a proper language for the >> > task. >> >> ... and if you can find a way around Amdahl's law (qv). > > "The speedup of a program using multiple processors in parallel computing is > limited by the time needed for the sequential fraction of the program." > > So it would only be a problem supposing that a significant part of the program > is unparallelizable. I can think of many many tasks where "Amdahl's law" is > not going to be a problem at all, for a properly designed system. Lets do a little math, shall we? Better yet, lets graph it: http://en.wikipedia.org/wiki/File:AmdahlsLaw.svg Now, do you see what's on the right side of X axis? That's right 65536 cores. Pause and appreciate the measeleness of speedup... Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 10:53 ` Sam Watkins 2009-10-15 11:50 ` Richard Miller @ 2009-10-15 11:56 ` Josh Wood 2009-10-15 13:11 ` hiro 2009-10-18 1:15 ` Roman Shaposhnik 3 siblings, 0 replies; 37+ messages in thread From: Josh Wood @ 2009-10-15 11:56 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Oct 15, 2009, at 3:53 AM, Sam Watkins wrote: > With respect to Ken, Bill Gates said something along the lines of "who > would need more than 640K?" With respect to Ken, from Roman's report, you only know that he asked a question. Roman was the one without an answer, and no one echoed Gates's arbitrary limit. -Josh ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 10:53 ` Sam Watkins 2009-10-15 11:50 ` Richard Miller 2009-10-15 11:56 ` Josh Wood @ 2009-10-15 13:11 ` hiro 2009-10-15 15:05 ` David Leimbach 2009-10-18 1:15 ` Roman Shaposhnik 3 siblings, 1 reply; 37+ messages in thread From: hiro @ 2009-10-15 13:11 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > There is a vast range of applications that cannot > be managed in real time using existing single-core technology. I'm sorry to interrupt your discussion, but what is real time? ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 13:11 ` hiro @ 2009-10-15 15:05 ` David Leimbach 0 siblings, 0 replies; 37+ messages in thread From: David Leimbach @ 2009-10-15 15:05 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 605 bytes --] On Thu, Oct 15, 2009 at 6:11 AM, hiro <23hiro@googlemail.com> wrote: > > There is a vast range of applications that cannot > > be managed in real time using existing single-core technology. > > I'm sorry to interrupt your discussion, but what is real time? > > Real time just means "fast enough to work properly". You can throw all kinds of other crap on top of that and say things about scheduling requirements and timeslices within which a process must complete, and duty cycles, but those are just things to look at to figure out if your system is "fast enough to work properly" Dave [-- Attachment #2: Type: text/html, Size: 932 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 10:53 ` Sam Watkins ` (2 preceding siblings ...) 2009-10-15 13:11 ` hiro @ 2009-10-18 1:15 ` Roman Shaposhnik 2009-10-18 3:15 ` Bakul Shah 3 siblings, 1 reply; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-18 1:15 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Oct 15, 2009 at 10:53 AM, Sam Watkins <sam@nipl.net> wrote: > On Wed, Oct 14, 2009 at 06:50:28PM -0700, Roman Shaposhnik wrote: >> > The mention that "... the overhead of cache coherence restricts the ability >> > to scale up to even 80 cores" is also eye openeing. If we're at aprox 8 >> > cores today, thats only 5 yrs away (if we double cores every >> > 1.5 yrs). > > Sharing the memory between processes is a stupid approach to multi-processing / > multi-threading. Modern popular computer architecture and software design is > fairly much uniformly stupid. It is. But what's your proposal on code sharing? All those PC registers belonging to different cores have to point somewhere. Is that somewhere is not shared memory the code has to be put there for every single core, right? Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-18 1:15 ` Roman Shaposhnik @ 2009-10-18 3:15 ` Bakul Shah [not found] ` <e763acc10910180606q1312ff7cw9a465d6af39c0fbe@mail.gmail.com> 0 siblings, 1 reply; 37+ messages in thread From: Bakul Shah @ 2009-10-18 3:15 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sun, 18 Oct 2009 01:15:45 -0000 Roman Shaposhnik <roman@shaposhnik.org> wrote: > On Thu, Oct 15, 2009 at 10:53 AM, Sam Watkins <sam@nipl.net> wrote: > > On Wed, Oct 14, 2009 at 06:50:28PM -0700, Roman Shaposhnik wrote: > >> > The mention that "... the overhead of cache coherence restricts the ab= > ility > >> > to scale up to even 80 cores" is also eye openeing. If we're at aprox = > 8 > >> > cores today, thats only 5 yrs away (if we double cores every > >> > 1.5 yrs). > > > > Sharing the memory between processes is a stupid approach to multi-proces= > sing / > > multi-threading. =A0Modern popular computer architecture and software des= > ign is > > fairly much uniformly stupid. > It is. But what's your proposal on code sharing? All those PC > registers belonging to > different cores have to point somewhere. Is that somewhere is not shared me= > mory > the code has to be put there for every single core, right? Different technoglogies/techniques make sense at different levels of scaling and at different points in time so sharing memory is not necessarily stupid -- unless one thinks that any compromise (to produce usable solutions in a realistic time frame) is stupid. At the hardware level we do have message passing between a processor and the memory controller -- this is exactly the same as talking to a shared server and has the same issues of scaling etc. If you have very few clients, a single shared server is indeed a cost effective solution. When you absolutely have to share state, somebody has to mediate access to the shared state and you can't get around the fact that it's going to cost you. But if you know something about the patterns of sharing, you can get away from a single shared memory & increase concurrency. A simple example is a h/w fifo (to connect producer/consumer but you also gave up some flexibility). As the number of processors increases on a device, sharing state between neighbors will be increasingly cheaper compared any global sharing. Even if you use message passing, messages between near neighbors will be far cheaper than between processors in different neighboorhoods. So switching to message passing is not going to fix things; you have to worry about placement as well (just like in h/w design). ^ permalink raw reply [flat|nested] 37+ messages in thread
[parent not found: <e763acc10910180606q1312ff7cw9a465d6af39c0fbe@mail.gmail.com>]
* Re: [9fans] Barrelfish [not found] ` <e763acc10910180606q1312ff7cw9a465d6af39c0fbe@mail.gmail.com> @ 2009-10-18 13:22 ` Roman Shaposhnik 2009-10-18 19:18 ` Bakul Shah 0 siblings, 1 reply; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-18 13:22 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sun, Oct 18, 2009 at 6:06 AM, Roman Shaposhnik <shaposhnik@gmail.com> wrote >> It is. But what's your proposal on code sharing? All those PC >> registers belonging to >> different cores have to point somewhere. Is that somewhere is not shared me= >> mory >> the code has to be put there for every single core, right? > > At the hardware level we do have message passing between a > processor and the memory controller -- this is exactly the > same as talking to a shared server and has the same issues of > scaling etc. If you have very few clients, a single shared > server is indeed a cost effective solution. I guess I'm not following. My question to OP was strictly about code sharing. Basically were do the cores get instructions from if not from shared memory. Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-18 13:22 ` Roman Shaposhnik @ 2009-10-18 19:18 ` Bakul Shah 2009-10-18 20:12 ` ron minnich 0 siblings, 1 reply; 37+ messages in thread From: Bakul Shah @ 2009-10-18 19:18 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sun, 18 Oct 2009 06:22:33 PDT Roman Shaposhnik <roman@shaposhnik.org> wrote: > On Sun, Oct 18, 2009 at 6:06 AM, Roman Shaposhnik <shaposhnik@gmail.com> wrot > e > >> It is. But what's your proposal on code sharing? All those PC > >> registers belonging to > >> different cores have to point somewhere. Is that somewhere is not shared m > e= > >> mory > >> the code has to be put there for every single core, right? > > > > At the hardware level we do have message passing between a > > processor and the memory controller -- this is exactly the > > same as talking to a shared server and has the same issues of > > scaling etc. If you have very few clients, a single shared > > server is indeed a cost effective solution. > > I guess I'm not following. My question to OP was strictly about > code sharing. Basically were do the cores get instructions from > if not from shared memory. Sorry, I should've done a better job of editing. I was really responding to the OP's point that sharing memory between processes is a stupid approach. My point was that "sharing memory" is just a low level programming interface (implemented by message passing in h/w) and it makes sense at some scale. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-18 19:18 ` Bakul Shah @ 2009-10-18 20:12 ` ron minnich 2009-10-20 0:04 ` [9fans] Parallelism is over a barrel(fish)? Lyndon Nerenberg (VE6BBM/VE7TFX) 0 siblings, 1 reply; 37+ messages in thread From: ron minnich @ 2009-10-18 20:12 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Since we seem to be having the parallel programming discussion again please look at this: https://asc.llnl.gov/sequoia/benchmarks/ The summaries are interesting. ron ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Parallelism is over a barrel(fish)? 2009-10-18 20:12 ` ron minnich @ 2009-10-20 0:04 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2009-10-20 1:11 ` W B Hacker 0 siblings, 1 reply; 37+ messages in thread From: Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2009-10-20 0:04 UTC (permalink / raw) To: 9fans >From last week's ACM Technews ... Why Desktop Multiprocessing Has Speed Limits Computerworld (10/05/09) Vol. 43, No. 30, P. 24; Wood, Lamont Despite the mainstreaming of multicore processors for desktops, not every desktop application can be rewritten for multicore frameworks, which means some bottlenecks will persist. "If you have a task that cannot be parallelized and you are currently on a plateau of performance in a single-processor environment, you will not see that task getting significantly faster in the future," says analyst Tom Halfhill. Adobe Systems' Russell Williams points out that performance does not scale linearly even with parallelization on account of memory bandwidth issues and delays dictated by interprocessor communications. Analyst Jim Turley says that, overall, consumer operating systems "don't do anything smart" with multicore architecture. "We have to reinvent computing, and get away from the fundamental premises we inherited from von Neumann," says Microsoft technical fellow Burton Smith. "He assumed one instruction would be executed at a time, and we are no longer even maintaining the appearance of one instruction at a time." Analyst Rob Enderle notes that most applications will operate on only a single core, which means that the benefits of a multicore architecture only come when multiple applications are run. "What we'd all like is a magic compiler that takes yesterday's source code and spreads it across multiple cores, and that is just not happening," says Turley. Despite the performance issues, vendors prefer multicore processors because they can facilitate a higher level of power efficiency. "Using multiple cores will let us get more performance while staying within the power envelope," says Acer's Glenn Jystad. http://www.computerworld.com/s/article/342870/The_Desktop_Traffic_Jam?intsrc=print_latest ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Parallelism is over a barrel(fish)? 2009-10-20 0:04 ` [9fans] Parallelism is over a barrel(fish)? Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2009-10-20 1:11 ` W B Hacker 0 siblings, 0 replies; 37+ messages in thread From: W B Hacker @ 2009-10-20 1:11 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Lyndon Nerenberg (VE6BBM/VE7TFX) wrote: >>From last week's ACM Technews ... > > Why Desktop Multiprocessing Has Speed Limits > Computerworld (10/05/09) Vol. 43, No. 30, P. 24; Wood, Lamont > > Despite the mainstreaming of multicore processors for desktops, not > every desktop application can be rewritten for multicore frameworks, > which means some bottlenecks will persist. "If you have a task that > cannot be parallelized and you are currently on a plateau of > performance in a single-processor environment, you will not see that > task getting significantly faster in the future," says analyst Tom > Halfhill. Adobe Systems' Russell Williams points out that performance > does not scale linearly even with parallelization on account of memory > bandwidth issues and delays dictated by interprocessor communications. > Analyst Jim Turley says that, overall, consumer operating systems > "don't do anything smart" with multicore architecture. "We have to > reinvent computing, and get away from the fundamental premises we > inherited from von Neumann," says Microsoft technical fellow Burton > Smith. "He assumed one instruction would be executed at a time, and > we are no longer even maintaining the appearance of one instruction at > a time." Analyst Rob Enderle notes that most applications will operate > on only a single core, which means that the benefits of a multicore > architecture only come when multiple applications are run. "What we'd > all like is a magic compiler that takes yesterday's source code and > spreads it across multiple cores, and that is just not happening," > says Turley. Despite the performance issues, vendors prefer multicore > processors because they can facilitate a higher level of power > efficiency. "Using multiple cores will let us get more performance > while staying within the power envelope," says Acer's Glenn Jystad. > > http://www.computerworld.com/s/article/342870/The_Desktop_Traffic_Jam?intsrc=print_latest > > > 'The usual <talking through their anatomy> suspects'.... But they miss the point. Work will always be foudn for 'faster' devices, but the majority of the 'needed' benefit has been accomplished until entirely new challenges surface. Computer games and digital video special-effects are just candy. Even a dual-core allows moving the overhead, housekeeping, I/O, interrupt servicing, et al out of the way of a single-core-bound application. OS/2 Hybrid Multi-Procesing - even with unmodified Win 3X era apps. Beyond that it matters little. Given a 'decent' (not magical)[1] OS, and environment, the apps that actually *matter* to 99% of the population are more than fast enough on the likes of a VIA C6 --> nano/Geode/Atom [2], embedded Ppc [3], or even an ARMish single-core [4]- with or without DSP etc. on-substrate. Faster storage and networks now matter far more than faster local CPU. The ratio of these 'goodies', and their benefits to the population in general to the count of supercomputers [5] and near-real-time video-stream processors [6] is - and will remain - extremely lopsided in favor of the small 'appliance'. Those hyping multi-multi core for the consumer 'PC" market are locked ino an obsolete behaviour pattern. Lower power consumption, smaller form-factor, better display and input interface faster networking is where the need lies. Nothing yet shipped can match the effectiveness of an experienced Wife or Personal Assistant (human) at the other end of an ordinary phone line when (s)he has *anticipated* your needs and called you *before* you recognized the need yourself. Code THAT into silicon, teach it to cook, and you still have a lousy bed-partner... Bill Hacker [1] Anything not horribly wasteful (eg - not Windows), such as Plan9, any *BSD, the leaner Linux (Vector/Slackware), Haiku - all make a more than fast enough desktop on any single-core of 700 MHz or better, even if dragging X-Windows and the like around as a boat-anchor. [2] Laptops amd Netbooks [3] Embedded high-end. Game boxen, Ford and other motor cars [4] A large percentage of PDA's and telecoms handhelds [5] Devilishly hard to substitute for, SETI et al notwithstanding, but needed in relatively small numbers vs, for example, a mobile phone or automobile fuel/pollution reduction system. [6] Given the preponderance of dreck spewed from television and cinema, civilization could well be better-off if all such devices on the planet went on a long holiday and humans returned to actually paying attention to one another. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 19:54 ` Roman Shaposhnik 2009-10-14 21:21 ` Tim Newsham @ 2009-10-14 21:36 ` Eric Van Hensbergen 2009-10-15 2:05 ` Roman Shaposhnik 1 sibling, 1 reply; 37+ messages in thread From: Eric Van Hensbergen @ 2009-10-14 21:36 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs And how does one deal with heterogeneous cores and complex on chip interconnect topologies? Barrelfish also gas a nice benefit in that it could span coherence domains. There's no real evdence that single kernels do well with hundreds of real cores (as opposed to hw threads) - in fact most of the data I've seen is to the contrary. Sent from my iPhone On Oct 14, 2009, at 1:54 PM, Roman Shaposhnik <roman@shaposhnik.org> wrote: > On Wed, Oct 14, 2009 at 12:09 PM, Tim Newsham <newsham@lava.net> > wrote: >> Rethinking multi-core systems as distributed heterogeneous >> systems. Thoughts? > > Somehow this feels related to the work that came out of Berkeley a > year > or so ago. I'm still not convinced what is the benefits of multiple > kernels. If you are managing a couple of 100s of cores a single kernel > would do just fine, once the industry is ready for a couple dozen of > thousands PUs -- the kernel is most likely to be dispensed with > anyway. > > Did you find any ideas there particularly engaging? > > Thanks, > Roman. > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 21:36 ` [9fans] Barrelfish Eric Van Hensbergen @ 2009-10-15 2:05 ` Roman Shaposhnik 2009-10-15 2:17 ` Eric Van Hensbergen 0 siblings, 1 reply; 37+ messages in thread From: Roman Shaposhnik @ 2009-10-15 2:05 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > And how does one deal with heterogeneous cores and complex on chip > interconnect topologies? Good question. Do they have to be heterogeneous? My oppinion is that the future of big multicore will be more Cell-like. > There's no real evdence that single kernels do well with hundreds of real > cores (as opposed to hw threads) - in fact most of the data I've seen is to > the contrary. Agreed. But then, again, you don't really want a kernel for anything but message passing in such an architecture (the other function of the kernel -- multiplexing I/O is only needed on selected few cores) at which point it really becomes a misnomer to even call it a kernel -- a thin hypervisor perhaps... Thanks, Roman. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 2:05 ` Roman Shaposhnik @ 2009-10-15 2:17 ` Eric Van Hensbergen 2009-10-15 3:32 ` Tim Newsham 0 siblings, 1 reply; 37+ messages in thread From: Eric Van Hensbergen @ 2009-10-15 2:17 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Oct 14, 2009, at 8:05 PM, Roman Shaposhnik wrote: >> And how does one deal with heterogeneous cores and complex on chip >> interconnect topologies? > > Good question. Do they have to be heterogeneous? My oppinion is that > the > future of big multicore will be more Cell-like. > They don't have to be, but that is part of both the multikernel and satellite kernel vision. >> There's no real evdence that single kernels do well with hundreds >> of real >> cores (as opposed to hw threads) - in fact most of the data I've >> seen is to >> the contrary. > > Agreed. But then, again, you don't really want a kernel for anything > but message > passing in such an architecture (the other function of the kernel -- > multiplexing > I/O is only needed on selected few cores) at which point it really > becomes a > misnomer to even call it a kernel -- a thin hypervisor perhaps... > If you look at the core of Barrelfish, you'll see that this is essentially what they are doing -- essentially using an extremely small microkernel (like L4) that's very efficient at various forms of message passing. That's the only thing that is duplicated on the various cores. The services themselves can be distributed and/or replicated as appropriate (although their approach favors replication) -- it all depends on the characteristics of the workload. -eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 2:17 ` Eric Van Hensbergen @ 2009-10-15 3:32 ` Tim Newsham 2009-10-15 3:59 ` Eric Van Hensbergen 0 siblings, 1 reply; 37+ messages in thread From: Tim Newsham @ 2009-10-15 3:32 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > If you look at the core of Barrelfish, you'll see that this is essentially > what they are doing -- essentially using an extremely small microkernel (like > L4) that's very > efficient at various forms of message passing. That's the only thing that is > duplicated on the various cores. The services themselves can be distributed > and/or replicated as appropriate (although their approach favors replication) > -- it all depends on the characteristics of the workload. it sounds like the kernel (L4-like, supposedly tuned to the specific hardware) and the "monitor" (userland, portable) are shared, from the paper. Btw, they have the source code up for free (http://www.barrelfish.org/release_20090914.html) which I supposed could be used to more definitively answer these questions with some effort... > -eric Tim Newsham http://www.thenewsh.com/~newsham/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 3:32 ` Tim Newsham @ 2009-10-15 3:59 ` Eric Van Hensbergen 2009-10-15 17:39 ` Tim Newsham 0 siblings, 1 reply; 37+ messages in thread From: Eric Van Hensbergen @ 2009-10-15 3:59 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Oct 14, 2009, at 9:32 PM, Tim Newsham wrote: >> If you look at the core of Barrelfish, you'll see that this is >> essentially what they are doing -- essentially using an extremely >> small microkernel (like L4) that's very >> efficient at various forms of message passing. That's the only >> thing that is duplicated on the various cores. The services >> themselves can be distributed >> and/or replicated as appropriate (although their approach favors >> replication) -- it all depends on the characteristics of the >> workload. > > it sounds like the kernel (L4-like, supposedly tuned to the specific > hardware) and the "monitor" (userland, portable) are shared, from > the paper. I'm confused what you mean by "shared". The monitor is replicated on every core as it is responsible for coordination amongst the cores - some things are replicated while others are coordinated. They do choose to replicate most things as part of their core scalability argument, in an effort to reduce lock contention to centralized resources. (from section 4.4): On each core, replicated data structures, such as memory alloca- tion tables and address space mappings, are kept globally consistent by means of an agreement protocol run by the monitors. Application requests that access global state are handled by the monitors, which mediate access to remote copies of state. -eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 3:59 ` Eric Van Hensbergen @ 2009-10-15 17:39 ` Tim Newsham 0 siblings, 0 replies; 37+ messages in thread From: Tim Newsham @ 2009-10-15 17:39 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs >> it sounds like the kernel (L4-like, supposedly tuned to the specific >> hardware) and the "monitor" (userland, portable) are shared, from >> the paper. > > I'm confused what you mean by "shared". ugh, I completely botched that.. I meant "replicated" not "shared". > -eric Tim Newsham http://www.thenewsh.com/~newsham/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-14 19:09 [9fans] Barrelfish Tim Newsham 2009-10-14 19:54 ` Roman Shaposhnik @ 2009-10-15 18:28 ` Christopher Nielsen 2009-10-15 18:55 ` W B Hacker 1 sibling, 1 reply; 37+ messages in thread From: Christopher Nielsen @ 2009-10-15 18:28 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I think this is an interesting approach. There are several interesting ideas being pursued here. The focus of the discussion has been on the multikernel approach, which I think has merit. Something that has not been discussed here is the wide use of DSLs for systems programming, and using haskell to write a framework for rapidly developing and proving correctness of DSLs. This is just as significant as the multikernel ideas. I downloaded the source, built the system, and will be playing with it. Thoughts? On Wed, Oct 14, 2009 at 12:09, Tim Newsham <newsham@lava.net> wrote: > Rethinking multi-core systems as distributed heterogeneous > systems. Thoughts? > > http://www.sigops.org/sosp/sosp09/papers/baumann-sosp09.pdf > > Tim Newsham > http://www.thenewsh.com/~newsham/ > > -- Christopher Nielsen "They who can give up essential liberty for temporary safety, deserve neither liberty nor safety." --Benjamin Franklin ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [9fans] Barrelfish 2009-10-15 18:28 ` Christopher Nielsen @ 2009-10-15 18:55 ` W B Hacker 0 siblings, 0 replies; 37+ messages in thread From: W B Hacker @ 2009-10-15 18:55 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Christopher Nielsen wrote: > I think this is an interesting approach. > > There are several interesting ideas being pursued here. The focus of > the discussion has been on the multikernel approach, which I think has > merit. > > Something that has not been discussed here is the wide use of DSLs for > systems programming, and using haskell to write a framework for > rapidly developing and proving correctness of DSLs. This is just as > significant as the multikernel ideas. > > I downloaded the source, built the system, and will be playing with it. > > Thoughts? Their 'plan' for security needs a recce as well. Message-passing-based 'creatures' - kernel-level or otherwise - have their own challenges in this regard (Windows, to name one bad example). Likewise, though I've only just started looking at it, if 'Haiku' even *has* a security model, I am (still, yet) blissfuly unaware of it... Bill Hacker > > On Wed, Oct 14, 2009 at 12:09, Tim Newsham <newsham@lava.net> wrote: >> Rethinking multi-core systems as distributed heterogeneous >> systems. Thoughts? >> >> http://www.sigops.org/sosp/sosp09/papers/baumann-sosp09.pdf >> >> Tim Newsham >> http://www.thenewsh.com/~newsham/ >> >> > > > ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2009-10-20 1:11 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-10-14 19:09 [9fans] Barrelfish Tim Newsham 2009-10-14 19:54 ` Roman Shaposhnik 2009-10-14 21:21 ` Tim Newsham 2009-10-14 21:33 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2009-10-14 21:42 ` Noah Evans 2009-10-14 21:45 ` erik quanstrom 2009-10-14 21:57 ` Noah Evans 2009-10-14 22:10 ` Eric Van Hensbergen 2009-10-14 22:21 ` Noah Evans 2009-10-15 1:03 ` David Leimbach 2009-10-15 1:50 ` Roman Shaposhnik 2009-10-15 2:12 ` Eric Van Hensbergen 2009-10-15 10:53 ` Sam Watkins 2009-10-15 11:50 ` Richard Miller 2009-10-15 12:00 ` W B Hacker 2009-10-16 17:03 ` Sam Watkins 2009-10-16 18:17 ` ron minnich 2009-10-16 18:39 ` Wes Kussmaul 2009-10-17 12:42 ` Roman Shaposhnik 2009-10-15 11:56 ` Josh Wood 2009-10-15 13:11 ` hiro 2009-10-15 15:05 ` David Leimbach 2009-10-18 1:15 ` Roman Shaposhnik 2009-10-18 3:15 ` Bakul Shah [not found] ` <e763acc10910180606q1312ff7cw9a465d6af39c0fbe@mail.gmail.com> 2009-10-18 13:22 ` Roman Shaposhnik 2009-10-18 19:18 ` Bakul Shah 2009-10-18 20:12 ` ron minnich 2009-10-20 0:04 ` [9fans] Parallelism is over a barrel(fish)? Lyndon Nerenberg (VE6BBM/VE7TFX) 2009-10-20 1:11 ` W B Hacker 2009-10-14 21:36 ` [9fans] Barrelfish Eric Van Hensbergen 2009-10-15 2:05 ` Roman Shaposhnik 2009-10-15 2:17 ` Eric Van Hensbergen 2009-10-15 3:32 ` Tim Newsham 2009-10-15 3:59 ` Eric Van Hensbergen 2009-10-15 17:39 ` Tim Newsham 2009-10-15 18:28 ` Christopher Nielsen 2009-10-15 18:55 ` W B Hacker
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).