[9fans] parallel systems

[9fans] parallel systems

[Sam Watkins]

I think my main points were good.

  * can parallelize by duplicating subsystems / divide and conquer
  * can parallelize by pipelining, even down to the arithmetic level
  * latency is limited by Ahmdal's law, potential throughput should not be
  * multi-tasking can potentially use close to the full power of a system

A factory is a parallel system.  A car factory can come close to fully
utilizing thousands of human and robot workers.

I think well-designed parallel systems can efficiently solve many laborious
computing problems.  Invocations of Ahmdal have not convinced me otherwise.

Sam

Re: [9fans] parallel systems

[andrey mirtchovski]

> A factory is a parallel system.  A car factory can come close to fully
> utilizing thousands of human and robot workers.

as long as we're using wrong analogies, keep in mind that:

- a car factory can also come to a standstill if one or more resources
arrive at a rate slower than they're being consumed at;
- a factory's line can be brought to a standstill if one of its
elements breaks;
- a factory's workers need to be scheduled to work in shifts and you
must always have enough workers for each critical position to keep the
line going;
- a factory 's line is at least as slow as its slowest worker

and so on.

last but not least:

- if all the workers at a car factory came to work at the same time
they wouldn't be able to get through the door.

Re: [9fans] parallel systems

[Jack Norton]

Sam Watkins wrote:
> I think my main points were good.
>
>   * can parallelize by duplicating subsystems / divide and conquer
>   * can parallelize by pipelining, even down to the arithmetic level
>   * latency is limited by Ahmdal's law, potential throughput should not be
>   * multi-tasking can potentially use close to the full power of a system
>
> A factory is a parallel system.  A car factory can come close to fully
> utilizing thousands of human and robot workers.
>
> I think well-designed parallel systems can efficiently solve many laborious
> computing problems.  Invocations of Ahmdal have not convinced me otherwise.
>
> Sam
>
>
I would say a factory is heavily pipelined.  Although if the jobs aren't
big enough, the workers are underutilized.  The mediators (supervisors)
that keep said workers efficiently running are paid more than the
workers, and it can be deduced that their job is more critical
overall.   Hmm what does that say about parallel computer systems?
Maybe you should have a shop foreman design a parallel system.   I
thought I would respond as I work at a small company and we build all
our parts in our factory.  I regularly deal with such parallel system
latencies...

also, one more thought: near 100% factory utilization only occurs when
the assembly steps (pipeline) and division of assembly (divide and
conquer) is tailored for the exact product (task/instruction/process) to
be made.  There is no such thing as a 100% utilized general purpose
factory.  At least not from what I have seen.

-Jack

Re: [9fans] parallel systems

[Sam Watkins]

> - a factory's line can be brought to a standstill if one of its
> elements breaks;

one would hope that software elements do not break so much

> - a factory 's line is at least as slow as its slowest worker

a slow part of the line can be split / duplicated to use multiple workers

> - if all the workers at a car factory came to work at the same time
> they wouldn't be able to get through the door.

and yet people do come to work, car factories do exist, and they are obviously
more powerful and efficient than a whole lot of people building individually.

> if the jobs aren't big enough, the workers are underutilized.

That's fine, you can switch off processing units that aren't needed, or use
them for another task.  Software systems are much more flexible than factories.

> The mediators (supervisors) that keep said workers efficiently running are
> paid more than the workers, and it can be deduced that their job is more
> critical overall.

Yes, a parallel system might need significant resources dedicated to organizing
and optimizing the rest of the system.

> also, one more thought: near 100% factory utilization only occurs when the
> assembly steps (pipeline) and division of assembly (divide and conquer) is
> tailored for the exact product (task/instruction/process) to be made.

Yes, it is much more difficult to reconfigure a factory than a software system.
It is easy to configure a software system for a specific task, it may even be
reconfigured it at run time.

Sam

Re: [9fans] parallel systems

[W B Hacker]

Sam Watkins wrote:
>> - a factory's line can be brought to a standstill if one of its
>> elements breaks;
>
> one would hope that software elements do not break so much
>
>> - a factory 's line is at least as slow as its slowest worker
>
> a slow part of the line can be split / duplicated to use multiple workers
>
>> - if all the workers at a car factory came to work at the same time
>> they wouldn't be able to get through the door.
>
> and yet people do come to work, car factories do exist, and they are obviously
> more powerful and efficient than a whole lot of people building individually.
>
>> if the jobs aren't big enough, the workers are underutilized.
>
> That's fine, you can switch off processing units that aren't needed, or use
> them for another task.  Software systems are much more flexible than factories.
>
>> The mediators (supervisors) that keep said workers efficiently running are
>> paid more than the workers, and it can be deduced that their job is more
>> critical overall.
>
> Yes, a parallel system might need significant resources dedicated to organizing
> and optimizing the rest of the system.
>
>> also, one more thought: near 100% factory utilization only occurs when the
>> assembly steps (pipeline) and division of assembly (divide and conquer) is
>> tailored for the exact product (task/instruction/process) to be made.
>
> Yes, it is much more difficult to reconfigure a factory than a software system.
> It is easy to configure a software system for a specific task, it may even be
> reconfigured it at run time.
>
> Sam
>
>

You forget how far our best systems - capable, running flat-out, of emulating
only a short time-slice of the brain of a rodent -  lag 'nature'.

One mouse.

Let alone our our own trillion-bit-plus equivalent massively-parallel brain.
Issued one-per to each of those factory workers, BTW - and arguably utilized to
better effect on average than has been the case right here of late...

If paralellism actually scaled as well as you wish .. 'God' would be the sole
ant-queen in the known universe, she would have faster-than-light travel...

And she never would have permitted wasps to evolve - let alone dinosaurs or humans..

Didn't happen that way.

Instead, ants learned the limitations of parallelism millions of years ago, and
decided to wait for humans to evolve so we can build starships *for* them.

Clever folks, ants are. You think we are smsrter, just compare your tax bill
with theirs.

;-)

Bill

Re: [9fans] parallel systems

[Latchesar Ionkov]

How many parallel systems you have impelemented?

Thanks,
    Lucho

On Wed, Oct 28, 2009 at 1:08 PM, Sam Watkins <sam@nipl.net> wrote:
> I think my main points were good.
>
>  * can parallelize by duplicating subsystems / divide and conquer
>  * can parallelize by pipelining, even down to the arithmetic level
>  * latency is limited by Ahmdal's law, potential throughput should not be
>  * multi-tasking can potentially use close to the full power of a system
>
> A factory is a parallel system.  A car factory can come close to fully
> utilizing thousands of human and robot workers.
>
> I think well-designed parallel systems can efficiently solve many laborious
> computing problems.  Invocations of Ahmdal have not convinced me otherwise.
>
> Sam
>
>

Re: [9fans] parallel systems

[ron minnich]

The issue here is that all the things you are saying can be (and have
been) measured. They can be quantified. There are variations in just
how much parallelism is possible depending on the application or even
the type of application.

This type of discussion, absent some sort of quantification, belongs
more in the Pointy Headed Boss domain than this mailling list. It's
akin to arguing about power without knowing what terms like volts and
amps mean.

In other words, the discussion to date has had zero semantic content.

ron

Re: [9fans] parallel systems

[erik quanstrom]

On Wed Oct 28 15:09:54 EDT 2009, sam@nipl.net wrote:
> I think my main points were good.
>
>   * can parallelize by duplicating subsystems / divide and conquer
>   * can parallelize by pipelining, even down to the arithmetic level
>   * latency is limited by Ahmdal's law, potential throughput should not be
>   * multi-tasking can potentially use close to the full power of a system
>
> A factory is a parallel system.  A car factory can come close to fully
> utilizing thousands of human and robot workers.
>
> I think well-designed parallel systems can efficiently solve many laborious
> computing problems.  Invocations of Ahmdal have not convinced me otherwise.

i think this thread is not useful.  your arguments aren't
convincing to the rest of us and vice versa.  let's agree
to disagree.

- erik