[9fans] GCC/G++: some stress testing

[9fans] GCC/G++: some stress testing

[lucio]

Before I apply some serious effort to bring P9GCC in line with the
latest release, I'd like to convince myself that the effort is worth
it.  I'm keen to catch two birds with one stone: (a) make sure that
version 3.0 is sufficiently functional and (b) determine how useful it
really is.

Please will anybody who has a Plan 9 objective that can only be
attained using GCC/G++ please drop me a line to let me know briefly
what it is?  If the whole exercise gets a lot of support, I'll happily
set up more infrastructure to deal with it (wiki, blog, remote access,
whatever Bell Labs would rather not do themselves).

++L

PS: I prefer if you use the 9fans list, I may miss your mail if I
haven't already have entered your sender address in my whitelist.  Use
your discretion.

Re: [9fans] GCC/G++: some stress testing

[Eric Van Hensbergen]

On Fri, Feb 29, 2008 at 10:55 PM,  <lucio@proxima.alt.za> wrote:
>
>  Please will anybody who has a Plan 9 objective that can only be
>  attained using GCC/G++ please drop me a line to let me know briefly
>  what it is?  If the whole exercise gets a lot of support, I'll happily
>  set up more infrastructure to deal with it (wiki, blog, remote access,
>  whatever Bell Labs would rather not do themselves).
>

It will no doubt be useful to us folks doing work for the gov't.  They
DOE has lots of apps written for GCC or Fortran -- while there may be
other methods of accommodating these applications, having them "just
work" with GCC (particularly if the GCC fortran could be part of the
port) would help us a lot.  It could also serve as a baseline for
performance/efficiency comparisons with other methodologies such as
linuxemu, etc.

Similarly, I've been working with other folks at potentially using
Plan 9 (for instance with the RAMP project) -- they'd be much happier
if they knew they could compile apps with GCC.

               -eric

Re: [9fans] GCC/G++: some stress testing

[lucio]

> It will no doubt be useful to us folks doing work for the gov't.  They
> DOE has lots of apps written for GCC or Fortran -- while there may be
> other methods of accommodating these applications, having them "just
> work" with GCC (particularly if the GCC fortran could be part of the
> port) would help us a lot.  It could also serve as a baseline for
> performance/efficiency comparisons with other methodologies such as
> linuxemu, etc.
>
My interpretation of what you're saying is that I ought to wrap up
what I have in a format that can be installed successfully and more or
less effortlessly, then present myself as the "maintainer" and get on
with tracking the more recent releases.

Unfortunately, there's quite a bit of effort required catching up with
the missing documentation that would make the project more readily
supported, namely the binary formats dhog implemented to match GCC's
different function interfacing.

In fact, the whole exercise is quite vast and lack of documentation
(or understanding) makes it even larger.  But we have a starting point
and I would like to see some use of what we have before embarking on
an even bigger task.  If we can prove that the foundations are solid
(GCC does compile itself, which is no small achievement), we may be
able to draw some attention and funding from potential users.  My time
is free, when available, but more skilful resources may need to be
paid for.

> Similarly, I've been working with other folks at potentially using
> Plan 9 (for instance with the RAMP project) -- they'd be much happier
> if they knew they could compile apps with GCC.

Well, Lyndon's suggestion seems useful, to go the ansi2knr route
wherever possible and, my response, use CFront where C++ does not
exceed its capabilities.  GCC/G++ then becomes the last resort.

But nn my opinion GCC does not really address the problem you mention.
The Auto* tools are a much bigger problem where "they'd be much
happier if they knew they could compile apps with GCC".  I'm sure that
translates into "if they could just type './configure; make install'".
No one I know about is figuring that one out.

But that's precisely why I posted, because someone out there _may_
need just GCC/G++ (fortran and Java included) and I would very much
like to know.  I can find ./configure examples by the bushelful, they
are no good in my situation.

++L

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Feb-29, at 22:02 , Eric Van Hensbergen wrote:

> It will no doubt be useful to us folks doing work for the gov't.  They
> DOE has lots of apps written for GCC or Fortran -- while there may be
> other methods of accommodating these applications, having them "just
> work" with GCC (particularly if the GCC fortran could be part of the
> port) would help us a lot.  It could also serve as a baseline for
> performance/efficiency comparisons with other methodologies such as
> linuxemu, etc.

But none of this code will "just work" on Plan 9 (especially the
Fortran code), so what's the point?

--lyndon

Re: [9fans] GCC/G++: some stress testing

[lucio]

> But none of this code will "just work" on Plan 9 (especially the
> Fortran code), so what's the point?

That is of course true.  Thing is, until one moves along, many of
these "obvious" truths will not be revealed to all interested parties,
nor will alternatives be identified.  I really would like to see a
team of competent persons address this issue as carefully as it
deserves.  Consider that this has become a non-issue in the world of
Wintel/GNU-Linux by blocking any alternative development paths,
including Plan 9's only slight eccentricities.

++L

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Feb-29, at 22:52 , lucio@proxima.alt.za wrote:

> onsider that this has become a non-issue in the world of
> Wintel/GNU-Linux by blocking any alternative development paths,
> including Plan 9's only slight eccentricities.

But these are the stampeding herd of lemmings that discover, at the
last minute, they aren't supposed to (and can't) dive over the cliff
of popular myth. Do you want to become their path of last resort?

--lyndon (applying duct tape to the doors and windows)

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Fri, Feb 29, 2008 at 10:39 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:

>
>  But none of this code will "just work" on Plan 9 (especially the
>  Fortran code), so what's the point?

Why do you say that?

ron

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

http://www.mpqc.org/

ron

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Feb-29, at 23:11 , ron minnich wrote:

>> But none of this code will "just work" on Plan 9 (especially the
>> Fortran code), so what's the point?
>
> Why do you say that?

The lack of a F95 compiler in /bin?  (If you have one in house, that's
cheating.)

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Feb-29, at 23:12 , ron minnich wrote:

> http://www.mpqc.org/

> Platforms
>
>     * Unix Workstations (Intel/Linux, RS/6000, SGI/IRIX)
>     * Symmetric multi-processors (Intel/Linux, SGI/IRIX)
>     * Massively parallel (IBM SP, Intel Paragon)
>
> Back to top.
> Implementation
>
>     * C++ with a few C and FORTRAN 77 functions
>     * Object-oriented designed throughout

Where are the Plan 9 bits? (Seriously.)

Re: [9fans] GCC/G++: some stress testing

[lucio]

>> onsider that this has become a non-issue in the world of
>> Wintel/GNU-Linux by blocking any alternative development paths,
>> including Plan 9's only slight eccentricities.
>
> But these are the stampeding herd of lemmings that discover, at the
> last minute, they aren't supposed to (and can't) dive over the cliff
> of popular myth. Do you want to become their path of last resort?

I'm assuming that eventually reason prevails.  It would be sad if at
that point there were no options left.

++L

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Feb-29, at 23:42 , lucio@proxima.alt.za wrote:

> I'm assuming that eventually reason prevails.  It would be sad if at
> that point there were no options left.

There is a Canadian federal election coming up ...

I'm feeling very depressed.

Re: [9fans] GCC/G++: some stress testing

[lucio]

> On Fri, Feb 29, 2008 at 10:39 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
>
>>
>>  But none of this code will "just work" on Plan 9 (especially the
>>  Fortran code), so what's the point?
>
> Why do you say that?
>
> ron

Looking at GCC, there's plenty more effort required before the full
suite of compilers (don't forget ADA's in there, too) is ready for
prime time under Plan 9.  But it does seem that in going there, it is
possible to feed back to GNU how to avoid the more obvious pitfalls
(Auto* tools when the compiler in fact defines the environment almost
entirely) and, reason prevailing, this might lead to a different
approach.

One way or another, eventually the current flood of software has to
undergo some quality control and at that point it would be good if
there were principles by which to "measure" such quality.  Looking the
other way isn't going to be helpful and we're all caught up in it, so
those of us with opinions and knowledge may need to contribute.

++L

PS: I still haven't a single offer of software to stress test GCC 3.0,
nevermind the assistance I'm bound to need to make use of the C++
features.

Re: [9fans] GCC/G++: some stress testing

[hiro]

>  One way or another, eventually the current flood of software has to
>  undergo some quality control and at that point it would be good if
>  there were principles by which to "measure" such quality.

Perhaps you should look at this page:
http://www.gnu.org/software/reliability.html
"It is no fluke that the GNU utilities are so reliable. There are good
reasons why free software tends to be of high quality."

We don't need any measurement for free software, because everything
will get well if you just license your software under the GPL.

You must keep ini mind, that even "Cancer Clinic Relies on Free
Software!" It must be all good!

--
hiro

[9fans] intellect?

[Lyndon Nerenberg]

On 2008-Mar-1, at 03:13 , hiro wrote:

> Perhaps you should look at this page:

Perhaps you should read this book (among others):

http://www.powells.com/biblio/0-201-03669-X?&PID=30607

Re: [9fans] GCC/G++: some stress testing

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 28 bytes --]

can it compile a working pm?

[-- Attachment #2: Type: message/rfc822, Size: 3404 bytes --]

From: lucio@proxima.alt.za
To: 9fans@cse.psu.edu
Subject: [9fans] GCC/G++: some stress testing
Date: Sat, 1 Mar 2008 06:55:06 +0200
Message-ID: <df122de4adc50d9cf99138a6f8e566d9@proxima.alt.za>

Before I apply some serious effort to bring P9GCC in line with the
latest release, I'd like to convince myself that the effort is worth
it.  I'm keen to catch two birds with one stone: (a) make sure that
version 3.0 is sufficiently functional and (b) determine how useful it
really is.

Please will anybody who has a Plan 9 objective that can only be
attained using GCC/G++ please drop me a line to let me know briefly
what it is?  If the whole exercise gets a lot of support, I'll happily
set up more infrastructure to deal with it (wiki, blog, remote access,
whatever Bell Labs would rather not do themselves).

++L

PS: I prefer if you use the 9fans list, I may miss your mail if I
haven't already have entered your sender address in my whitelist.  Use
your discretion.

Re: [9fans] intellect?

[Lyndon Nerenberg]

On 2008-Mar-1, at 03:47 , Lyndon Nerenberg wrote:

> Perhaps you should read this book (among others):

And I should go crawl in the hot tub. I'll shut up now. It's been a
long day.

Re: [9fans] intellect?

[Charles Forsyth]

i think hiro was being sarcastic

Re: [9fans] GCC/G++: some stress testing

[lucio]

> can it compile a working pm?

I'll try.

++L

Re: [9fans] intellect?

[hiro]

http://en.wikipedia.org/wiki/Irony, eh.

>Perhaps you should read this book (among others)
Oh, you are a good man;)

--
hiro

Re: [9fans] intellect?

[hiro]

His books are controlling quality more than we could ever do.
The books take over the brains of its readers. Furthermore a lot of
them will end up here on this list.

Re: [9fans] GCC/G++: some stress testing

[lejatorn]

Hello,

At work, most of the users need a fortran compiler (although almost
none of them actually use gfortran, they prefer ifort) and some of them
do parallel computation so they need MPI. If I could have at least
those two items thanks to P9GCC, maybe I could convince some of them to
work on the plan9 servers I'm slowly setting up there.

As for me, I'd be pretty happy if I could have a bittorrent client
(especially libtorrent/rtorrent, written in c++) on plan9 so it'd be
rather nice if your P9GCC could achieve building that. But yeah, that
one relies on auto*, configure, etc..

Mathieu.

On Sat, Mar 01, 2008 at 06:55:06AM +0200, lucio@proxima.alt.za wrote:
> Before I apply some serious effort to bring P9GCC in line with the
> latest release, I'd like to convince myself that the effort is worth
> it.  I'm keen to catch two birds with one stone: (a) make sure that
> version 3.0 is sufficiently functional and (b) determine how useful it
> really is.
>
> Please will anybody who has a Plan 9 objective that can only be
> attained using GCC/G++ please drop me a line to let me know briefly
> what it is?  If the whole exercise gets a lot of support, I'll happily
> set up more infrastructure to deal with it (wiki, blog, remote access,
> whatever Bell Labs would rather not do themselves).
>
> ++L
>
> PS: I prefer if you use the 9fans list, I may miss your mail if I
> haven't already have entered your sender address in my whitelist.  Use
> your discretion.
>

--
GPG key on subkeys.pgp.net:

KeyID:	| Fingerprint:
683DE5F3 | 4324 5818 39AA 9545 95C6 09AF B0A4 DFEA 683D E5F3
--

Re: [9fans] GCC/G++: some stress testing

[Pietro Gagliardi]

Why not just port Version 7 f77 and Version 7 Ratfor?

On Mar 1, 2008, at 9:45 AM, lejatorn@gmail.com wrote:

> Hello,
>
> At work, most of the users need a fortran compiler (although almost
> none of them actually use gfortran, they prefer ifort) and some of
> them
> do parallel computation so they need MPI. If I could have at least
> those two items thanks to P9GCC, maybe I could convince some of
> them to
> work on the plan9 servers I'm slowly setting up there.
>
> As for me, I'd be pretty happy if I could have a bittorrent client
> (especially libtorrent/rtorrent, written in c++) on plan9 so it'd be
> rather nice if your P9GCC could achieve building that. But yeah, that
> one relies on auto*, configure, etc..
>
> Mathieu.
>
> On Sat, Mar 01, 2008 at 06:55:06AM +0200, lucio@proxima.alt.za wrote:
>> Before I apply some serious effort to bring P9GCC in line with the
>> latest release, I'd like to convince myself that the effort is worth
>> it.  I'm keen to catch two birds with one stone: (a) make sure that
>> version 3.0 is sufficiently functional and (b) determine how
>> useful it
>> really is.
>>
>> Please will anybody who has a Plan 9 objective that can only be
>> attained using GCC/G++ please drop me a line to let me know briefly
>> what it is?  If the whole exercise gets a lot of support, I'll
>> happily
>> set up more infrastructure to deal with it (wiki, blog, remote
>> access,
>> whatever Bell Labs would rather not do themselves).
>>
>> ++L
>>
>> PS: I prefer if you use the 9fans list, I may miss your mail if I
>> haven't already have entered your sender address in my whitelist.
>> Use
>> your discretion.
>>
>
> --
> GPG key on subkeys.pgp.net:
>
> KeyID:	| Fingerprint:
> 683DE5F3 | 4324 5818 39AA 9545 95C6 09AF B0A4 DFEA 683D E5F3
> --

Re: [9fans] GCC/G++: some stress testing

[lucio]

> At work, most of the users need a fortran compiler (although almost
> none of them actually use gfortran, they prefer ifort) and some of them
> do parallel computation so they need MPI. If I could have at least
> those two items thanks to P9GCC, maybe I could convince some of them to
> work on the plan9 servers I'm slowly setting up there.
>
I don't have any hasty plans for Fortran, but it seems to be in
greater demand than C++.  We'll see how things pan out.  I am a little
concerned that potential users need more than a compiler invocation to
make things work, specially on the graphics side and maybe I ought to
wave that flag rather frantically.  Still, one obstacle out of the way
may encourage others to address the next ones.

> As for me, I'd be pretty happy if I could have a bittorrent client
> (especially libtorrent/rtorrent, written in c++) on plan9 so it'd be
> rather nice if your P9GCC could achieve building that. But yeah, that
> one relies on auto*, configure, etc..

Let me emphasise that the auto* stuff is nowhere near the stumbling
block it's made out to be.  Benavento (I hope I'm not pointing fingers
at the wrong person right now - no way to check) and I have different
techniques to address this, but we both have done a good deal of
porting auto* dependent stuff to APE with the help of moderately
simple mkfiles.  Then again, I stumbled with Graphviz version 2, sadly.

Graphics, networking and multithreading are much bigger issues to
resolve.  So your bittorrent client may be difficult to port and damn
easy to redevelop.  Any chance you may give it a try?

++L

Re: [9fans] GCC/G++: some stress testing

[lucio]

> Why not just port Version 7 f77 and Version 7 Ratfor?

Sounds like an idea.  Where do I find the source code?  Mind you, it's
been tens of years since I programmed in Fortran IV, it's going to be
hard for me to do any testing.

++L

Re: [9fans] GCC/G++: some stress testing

[Eric Van Hensbergen]

On Sat, Mar 1, 2008 at 12:39 AM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
>
>  On 2008-Feb-29, at 22:02 , Eric Van Hensbergen wrote:
>
>  > It will no doubt be useful to us folks doing work for the gov't.  They
>  > DOE has lots of apps written for GCC or Fortran -- while there may be
>  > other methods of accommodating these applications, having them "just
>  > work" with GCC (particularly if the GCC fortran could be part of the
>  > port) would help us a lot.  It could also serve as a baseline for
>  > performance/efficiency comparisons with other methodologies such as
>  > linuxemu, etc.
>
>  But none of this code will "just work" on Plan 9 (especially the
>  Fortran code), so what's the point?
>

We are well aware of the peeling onion effect - it is just a step.
Many of the HPC apps will "just work" with a minimum of fuss, others
will require a considerable bit of fuss.

To add to Ron's MPQC example, I'll just through out the HPCC benchmark
suite: http://icl.cs.utk.edu/hpcc/

              -eric

Re: [9fans] GCC/G++: some stress testing

[Eric Van Hensbergen]

On Sat, Mar 1, 2008 at 9:02 AM,  <lucio@proxima.alt.za> wrote:
>
>  Graphics, networking and multithreading are much bigger issues to
>  resolve.  So your bittorrent client may be difficult to port and damn
>  easy to redevelop.  Any chance you may give it a try?
>

Networking and multithreading are going to be more important than
graphics to us -- but as you said earlier, one step at a time.  We'll
be working towards this stuff as well as part of the FastOS program so
hopefully we'll be able to help provide some of these pieces.  IBM has
already done a good amount w.r.t. supporting POSIX networking APIs
using Inferno devip as a backend, I imagine we can apply many of these
techniques to APE or other accomodation platforms.

           -eric

Re: [9fans] GCC/G++: some stress testing

[Pietro Gagliardi]

Caldera (now SCO) released the source code a while ago. It has since
been mirrored. The direct links to the f77 and Ratfor are:

http://minnie.tuhs.org/UnixTree/V7/usr/src/cmd/f77
http://minnie.tuhs.org/UnixTree/V7/usr/src/cmd/ratfor

I can get started with it later today.

On Mar 1, 2008, at 10:17 AM, lucio@proxima.alt.za wrote:

>> Why not just port Version 7 f77 and Version 7 Ratfor?
>
> Sounds like an idea.  Where do I find the source code?  Mind you, it's
> been tens of years since I programmed in Fortran IV, it's going to be
> hard for me to do any testing.
>
> ++L
>

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Fri, Feb 29, 2008 at 11:29 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
>
>  On 2008-Feb-29, at 23:11 , ron minnich wrote:
>
>  >> But none of this code will "just work" on Plan 9 (especially the
>  >> Fortran code), so what's the point?
>  >
>  > Why do you say that?
>
>  The lack of a F95 compiler in /bin?  (If you have one in house, that's
>  cheating.)
>

The comment was "especially the Fortran code", but also saying "none
of this code".

So my question stands: why is it that *none* of this code would work?

ron

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Sat, Mar 1, 2008 at 7:17 AM,  <lucio@proxima.alt.za> wrote:
> > Why not just port Version 7 f77 and Version 7 Ratfor?
>
>  Sounds like an idea.  Where do I find the source code?  Mind you, it's
>  been tens of years since I programmed in Fortran IV, it's going to be
>  hard for me to do any testing.
>

very litlle f77 left in my world, maybe somebody else has some.

Ratfor? Surely you must be joking.

ron

Re: [9fans] GCC/G++: some stress testing

[Bruce Ellis]

There is a lot of G code that really essentially is only portable to
linux (or close, e.g. BSDs).  There is other code that works nearly
everywhere that has a GCC.  The "why bother" pessimism is best
reserved for more suitable occasions.  I'm really glad when APE
allows me to compile legacy code and would be equally pleased
when on the odd occaison that GCC was needed, available and
kind to me.

brucee

On Sun, Mar 2, 2008 at 3:40 AM, ron minnich <rminnich@gmail.com> wrote:
> On Fri, Feb 29, 2008 at 11:29 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
> >
> >  On 2008-Feb-29, at 23:11 , ron minnich wrote:
> >
> >  >> But none of this code will "just work" on Plan 9 (especially the
> >  >> Fortran code), so what's the point?
> >  >
> >  > Why do you say that?
> >
> >  The lack of a F95 compiler in /bin?  (If you have one in house, that's
> >  cheating.)
> >
>
> The comment was "especially the Fortran code", but also saying "none
> of this code".
>
> So my question stands: why is it that *none* of this code would work?
>
> ron
>

Re: [9fans] GCC/G++: some stress testing

[Bruce Ellis]

Also note that neither F77 nor ratfor produced particularly good code.
They did, however, work.  Both attributes are required by the Fortran
community.

If the GCC stuff provides this service and you want to do the work
then I won't throw stones.

brucee

On Sun, Mar 2, 2008 at 3:41 AM, ron minnich <rminnich@gmail.com> wrote:
> On Sat, Mar 1, 2008 at 7:17 AM,  <lucio@proxima.alt.za> wrote:
> > > Why not just port Version 7 f77 and Version 7 Ratfor?
> >
> >  Sounds like an idea.  Where do I find the source code?  Mind you, it's
> >  been tens of years since I programmed in Fortran IV, it's going to be
> >  hard for me to do any testing.
> >
>
> very litlle f77 left in my world, maybe somebody else has some.
>
> Ratfor? Surely you must be joking.
>
> ron
>

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Sat, Mar 1, 2008 at 8:53 AM, Bruce Ellis <bruce.ellis@gmail.com> wrote:
> Also note that neither F77 nor ratfor produced particularly good code.
>  They did, however, work.  Both attributes are required by the Fortran
>  community.

ratfor was a preprocessor written in fortran, for fortran, which we
don't have, and gave us lots of
useful tools like, um, ed, and, uh, grep, uh, nroff and, uh, and ...
oh we have those already right?

I don't get the point of ratfor at all. It was originally to give
non-Unix people a taste of Unix.

Don't get me wrong, I used and loved those tools, but that was a generation ago.

But it's dead now, and the reasons for its existence don't apply to Plan 9.

We've got enough to do without any more Lazarus projects.

ron

Re: [9fans] GCC/G++: some stress testing

[lejatorn]

On Sat, Mar 01, 2008 at 05:02:59PM +0200, lucio@proxima.alt.za wrote:

> > As for me, I'd be pretty happy if I could have a bittorrent client
> > (especially libtorrent/rtorrent, written in c++) on plan9 so it'd be
> > rather nice if your P9GCC could achieve building that. But yeah, that
> > one relies on auto*, configure, etc..
>
> Let me emphasise that the auto* stuff is nowhere near the stumbling
> block it's made out to be.  Benavento (I hope I'm not pointing fingers
> at the wrong person right now - no way to check) and I have different
> techniques to address this, but we both have done a good deal of
> porting auto* dependent stuff to APE with the help of moderately
> simple mkfiles.  Then again, I stumbled with Graphviz version 2, sadly.
>
> Graphics, networking and multithreading are much bigger issues to
> resolve.  So your bittorrent client may be difficult to port and damn
> easy to redevelop.  Any chance you may give it a try?

Well if there was one project I'd try to code for, that would probably
be it. I even thought about proposing myself as a student for gsoc
with this idea in mind (although I seem to recall one condition is to be
a student irl, which is not my case anymore). However I'm pretty sure I
would not be able to commit enough time to it, so it would be kindof
worthless to start with it and never get to finish it properly. Besides,
I have not coded seriously in a long time, so I'm probably not the right
candidate to write something that doesn't suck atm.

Mathieu.

--
GPG key on subkeys.pgp.net:

KeyID:	| Fingerprint:
683DE5F3 | 4324 5818 39AA 9545 95C6 09AF B0A4 DFEA 683D E5F3
--

Re: [9fans] GCC/G++: some stress testing

[don bailey]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If GNU was so reliable we wouldn't see the C compiler generate random
opcodes for architectures we use at my work. And that's *with* the 4x
toolchain.

I think we've all had enough software evangelism. Everyone has bugs. GNU
is absolutely no exception.

D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFHyZ1WyWX0NBMJYAcRAkfYAKCZ4tXZlv6Q8GoiJ1fYSdViUMOf0QCeLS1j
itTtT5VQs1lyfmjq1++uRtQ=
=w2e0
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[erik quanstrom]

> If GNU was so reliable we wouldn't see the C compiler generate random
> opcodes for architectures we use at my work. And that's *with* the 4x
> toolchain.
>
> I think we've all had enough software evangelism. Everyone has bugs. GNU
> is absolutely no exception.

they do, with complete reliability, break new things in new releases.

- erik

Re: [9fans] GCC/G++: some stress testing

[lucio]

> If GNU was so reliable we wouldn't see the C compiler generate random
> opcodes for architectures we use at my work. And that's *with* the 4x
> toolchain.

I'm not sure if I read you correctly, but all I'm looking for is some
confidence that P9GCC is worth pursuing.  I can't use the supplied
regression tests because they rely on expect and in any event they
only prove that things work, not that there are needs out there that
P9GCC actually satisfies.

++L

Re: [9fans] GCC/G++: some stress testing

[geoff]

Ron, I think you're confusing ratfor with software tools.  There was a
ratfor implementation in C on Unix, and I wrote ratfor when I had to
use Fortran, and others did too, independent of the software tools
effort.  The point of Ratfor was to make Fortran bearable; I can't
imagine writing bare Fortran any more.

I compiled ratfor and f2c on my home Plan 9 systems a while ago and
they just worked.  I don't remember any porting effort.  I might have
had to change a path name in a makefile.

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

There are a ton of biology tools written in rather simple c++. Those
people are willing to look at p9 if we have two things:
-g++
-python

thanks

ron

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Sat, Mar 1, 2008 at 12:29 PM,  <geoff@plan9.bell-labs.com> wrote:
> Ron, I think you're confusing ratfor with software tools.  There was a
>  ratfor implementation in C on Unix, and I wrote ratfor when I had to
>  use Fortran, and others did too, independent of the software tools
>  effort.  The point of Ratfor was to make Fortran bearable; I can't
>  imagine writing bare Fortran any more.

You're right. My mistake. No more stones. Sorry brucee.

ron

Re: [9fans] GCC/G++: some stress testing

[Bruce Ellis]

I mentioned to skip that you were mistaken but it seemed like
an honest mistake so we had brunch.  Where is all the abuse
gone anyway, or are my filters working?

brucee

On Sun, Mar 2, 2008 at 8:36 AM, ron minnich <rminnich@gmail.com> wrote:
> On Sat, Mar 1, 2008 at 12:29 PM,  <geoff@plan9.bell-labs.com> wrote:
> > Ron, I think you're confusing ratfor with software tools.  There was a
> >  ratfor implementation in C on Unix, and I wrote ratfor when I had to
> >  use Fortran, and others did too, independent of the software tools
> >  effort.  The point of Ratfor was to make Fortran bearable; I can't
> >  imagine writing bare Fortran any more.
>
> You're right. My mistake. No more stones. Sorry brucee.
>
> ron
>

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have a project on the go that would make an awesome plan9 platform.

Except that all our users code in C++.  The complicated kind, with
templates and hideous metaprogramming.  And can even show good reason
to do so.

Makes me weep.

Paul

On Mar 1, 2008, at 1:36 PM, ron minnich wrote:

> There are a ton of biology tools written in rather simple c++. Those
> people are willing to look at p9 if we have two things:
> -g++
> -python
>
> thanks
>
> ron

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFHyf3ApJeHo/Fbu1wRAuyOAKCvc0TnvBfcxd3qBvI1CetHHFqJPQCgkROU
B0P9e8lJdECTriryAsB3ut0=
=J9rR
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Sat, Mar 1, 2008 at 5:07 PM, Paul Lalonde <plalonde@telus.net> wrote:

>  Except that all our users code in C++.  The complicated kind, with
>  templates and hideous metaprogramming.  And can even show good reason
>  to do so.

welcome to my life.

Lots of holes in the walls around here, roughly the size of my head.

ron

Re: [9fans] GCC/G++: some stress testing

[Bruce Ellis]

i hope you tell them repeatedly that it makes you want to eat your own vomit.

life is too short too play with badd technology, particularly if you
are trying to make money. (Badd is a TM of Badd Attitude, girlie
dancers of mine you wish you'd seen.)

brucee

On Sun, Mar 2, 2008 at 3:28 PM, ron minnich <rminnich@gmail.com> wrote:
> On Sat, Mar 1, 2008 at 5:07 PM, Paul Lalonde <plalonde@telus.net> wrote:
>
> >  Except that all our users code in C++.  The complicated kind, with
> >  templates and hideous metaprogramming.  And can even show good reason
> >  to do so.
>
> welcome to my life.
>
> Lots of holes in the walls around here, roughly the size of my head.
>
> ron
>

Re: [9fans] GCC/G++: some stress testing

[Charles Forsyth]

> Except that all our users code in C++.  The complicated kind, with
> templates and hideous metaprogramming.  And can even show good reason
> to do so.

C++ is probably the wrong language for the application, then;
it should appear straightforward.

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Mar 2, 2008, at 3:12 AM, Charles Forsyth wrote:
> C++ is probably the wrong language for the application, then;
> it should appear straightforward.

Almost certainly.  And so is C.   Programming many-core shared-cache
machines in languages with global state and aliasing is just plain
wrong, in the same way that programming in assembly instead of C is
wrong.  Add a highly heterogeneous real-time task mix on top of that,
and you're in for a world of poor cache performance and deadlocks,
which could be avoided by better choices of implementation language.

One of the interesting things about copy-on-write no-aliasing
languages is that the overhead they used to represent is quickly
starting to disappear in the gaps between cache misses.  It's amazing
how much bookkeeping you can handle while waiting a thousand cycles
for some memory to become local; the no-aliasing rules even make that
kind of scheduling possible through a good match against the cache
model.

Programming for the memory hierarchy is way more important than
optimizing CPU clocks anymore (though that winds up still having a
place in some compute kernels).  I wish our programming languages
reflected that change in perspective.

Paul

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFHytQGpJeHo/Fbu1wRAhepAKCzjr3LIMoDK/RfaG1cVCHXguLdpQCbBS8I
r8XFPG2F/vn2cWtHkQoss6c=
=+7dG
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[sqweek]

On Sun, Mar 2, 2008 at 12:02 AM,  <lucio@proxima.alt.za> wrote:
>  Still, one obstacle out of the way
>  may encourage others to address the next ones.

 How close is linuxemu to being able to run gcc?
-sqweek

Re: [9fans] GCC/G++: some stress testing

[lucio]

>  How close is linuxemu to being able to run gcc?

I'm not sure how much it will help, given that one presumably wants to
produce code that runs on Plan 9, not Linux.  That said, I should
think that linux-native gcc is probably already supported by linuxemu.

++L

Re: [9fans] GCC/G++: some stress testing

[erik quanstrom]

> Almost certainly.  And so is C.   Programming many-core shared-cache
> machines in languages with global state and aliasing is just plain
> wrong, in the same way that programming in assembly instead of C is
> wrong.  Add a highly heterogeneous real-time task mix on top of that,
> and you're in for a world of poor cache performance and deadlocks,
> which could be avoided by better choices of implementation language.

i don't understand this argument.  are you saying that csp doesn't work
in c?  or are you saying that csp has caching problems that some other
languages solve?

also, could you define what you mean by "shared cache" a bit more.
would you consider an intel quad core cpu to be a "shared cache"
machine, since the two l2 caches sit on the same fsb?

> Programming for the memory hierarchy is way more important than
> optimizing CPU clocks anymore (though that winds up still having a
> place in some compute kernels).  I wish our programming languages
> reflected that change in perspective.

what do you mean by "programming for the memory heirarchy"?

- erik

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

CSP doesn't scale very well to hundreds of simultaneously executing
threads (my claim, not, as far as I've found yet, anyone else's).  It
is very well suited to a small number of threads that need to
communicate, and as a model of concurrency for tasks with few points
of contact.  For performance, the channel locks become a bottleneck
as the number of cores scale up.  As far as expressiveness, there are
still issues with composability and correctness as the number of
threads interacting increases.  Yes, you at least get local stacks,
but the work seems to get exponentially harder as the number of
systems in the simulation (um, game engine) increases.

By shared cache I mean any number of caches that are kept coherent at
the hardware level without serializing instructions.

Programming the memory hierarchy is really a specific instance of
programming for masking latency.  This goes well beyond inserting
prefetches in an optimization stage, presenting itself as problem
decompositions that keep the current working set in cache (at L3, L2,
or L1 granularity, depending), while simultaneously avoiding having
multiple processors chewing on the same data (which leads to vast
amounts of cache synchronization bus traffic).  Successful algorithms
in this space work on small bundles of data that either get flushed
back to memory uncached (to keep more cache for streaming in), or in
small bundles that can be passed from compute kernel to compute
kernel cheaply.  Having language structures to help with these
decompositions and caching decisions is a great help - that's one of
the reasons why functional programming keeps rearing its head in this
space.  Without aliasing and global (serializing) state it's much
easier to analyze the program and chose how to break up the
computation into kernels that can be streamed, pipelined, or
otherwise separated to allow better cache utilization and parallelism.

Currently, the best performing programs I know for exploiting the
memory hierarchy are C and C++ programs written in a  "kernel
composition" kind of model that the language supports poorly.  You
can do it, but it feels more like coding in assembly than like
expressing your algorithms.  Much of the template metaprogramming is
about taking measures of cache spaces and data sets and turning out
code (statically) tuned to those sizes.  There's a huge opportunity
for a JIT language to allow better adaptability to changing data sets
and numbers of active threads.

Paul

On 2-Mar-08, at 10:59 AM, erik quanstrom wrote:

>
>> Almost certainly.  And so is C.   Programming many-core shared-cache
>> machines in languages with global state and aliasing is just plain
>> wrong, in the same way that programming in assembly instead of C is
>> wrong.  Add a highly heterogeneous real-time task mix on top of that,
>> and you're in for a world of poor cache performance and deadlocks,
>> which could be avoided by better choices of implementation language.
>
> i don't understand this argument.  are you saying that csp doesn't
> work
> in c?  or are you saying that csp has caching problems that some other
> languages solve?
>
> also, could you define what you mean by "shared cache" a bit more.
> would you consider an intel quad core cpu to be a "shared cache"
> machine, since the two l2 caches sit on the same fsb?
>
>> Programming for the memory hierarchy is way more important than
>> optimizing CPU clocks anymore (though that winds up still having a
>> place in some compute kernels).  I wish our programming languages
>> reflected that change in perspective.
>
> what do you mean by "programming for the memory heirarchy"?
>
> - erik
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFHyw9XpJeHo/Fbu1wRAgNJAJ9pCFh0kixsaCir2fGKXBZhXTXsDQCfROva
LKnBfk+TaRKNrih36OBexbA=
=Mhdg
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 88 bytes --]

i could run gcc and compile hello world with it, but did no
further testing.

cinap

[-- Attachment #2: Type: message/rfc822, Size: 4067 bytes --]

From: sqweek <sqweek@gmail.com>
To: "Fans of the OS Plan 9 from Bell Labs" <9fans@cse.psu.edu>
Subject: Re: [9fans] GCC/G++: some stress testing
Date: Mon, 3 Mar 2008 02:57:13 +0900
Message-ID: <140e7ec30803020957g49af1f61g2fbc2094050fb55c@mail.gmail.com>

On Sun, Mar 2, 2008 at 12:02 AM,  <lucio@proxima.alt.za> wrote:
>  Still, one obstacle out of the way
>  may encourage others to address the next ones.

 How close is linuxemu to being able to run gcc?
-sqweek

Re: [9fans] GCC/G++: some stress testing

[Philippe Anel]

Paul Lalonde wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> CSP doesn't scale very well to hundreds of simultaneously executing
> threads (my claim, not, as far as I've found yet, anyone else's).  It
> is very well suited to a small number of threads that need to
> communicate, and as a model of concurrency for tasks with few points
> of contact.  For performance, the channel locks become a bottleneck as
> the number of cores scale up.  As far as expressiveness, there are
> still issues with composability and correctness as the number of
> threads interacting increases.  Yes, you at least get local stacks,
> but the work seems to get exponentially harder as the number of
> systems in the simulation (um, game engine) increases.
Interesting.
I agree with you, taking care about memory hierarchy is becoming very
important. Especially if you think about the upcoming NUMAcc systems
(Opterons are already there though).
But the fact is doesn't scale well is not about CSP itself, but the way
it has been implemented.
If CSP system itself takes care about memory hierarchy and uses no
synchronisation (using IPI to send message to another core by example),
CSP scales very well.
Of course IPI mechanism requires a switch to kernel mode which costs a
lot. But this is necessary only if the destination thread is running on
another core, and I don't think latency is very important in algorigthms
requiring a lot of cpus.
What do you think ?

Phil;

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Sun, Mar 2, 2008 at 2:00 PM, Philippe Anel <xigh@free.fr> wrote:

>  If CSP system itself takes care about memory hierarchy and uses no
>  synchronisation (using IPI to send message to another core by example),
>  CSP scales very well.

Is this something you have measured or is this conjecture?

>  Of course IPI mechanism requires a switch to kernel mode which costs a
>  lot. But this is necessary only if the destination thread is running on
>  another core, and I don't think latency is very important in algorigthms
>  requiring a lot of cpus.

same question.

For a look at an interesting library that scaled well on a 1024-node
SMP at NASA Ame's, by Jim Taft.
Short form: use shared memory for IPC, not data sharing.

he's done very well this way.

ron

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Mar 2, 2008, at 2:00 PM, Philippe Anel wrote:
> I agree with you, taking care about memory hierarchy is becoming
> very important. Especially if you think about the upcoming NUMAcc
> systems (Opterons are already there though).
> But the fact is doesn't scale well is not about CSP itself, but the
> way it has been implemented.
> If CSP system itself takes care about memory hierarchy and uses no
> synchronisation (using IPI to send message to another core by
> example), CSP scales very well.
> Of course IPI mechanism requires a switch to kernel mode which
> costs a lot. But this is necessary only if the destination thread
> is running on another core, and I don't think latency is very
> important in algorigthms requiring a lot of cpus.

Latency is quite important in the application domain I have to
target: the target is to produce a new image every 60th of a second,
including all the simulation effort to get there.  In addition, we
have user input which needs to be processed, and usually network
delays to worry about as well.  Every bit of latency between user
input and display breaks the illusion of control.  And though TVs are
getting better, it's not atypical to see 4-6 frames of latency
introduced by the display subsystem, once you've finished generating
a frame buffer.

I don't know what you mean by "CSP system itself takes care about
memory hierarchy".  Do you mean that the CSP implementation does
something about it, or do you mean that the code using the CSP
approach takes care of it?

IPI isn't free either - apart from the OS switch, it generates bus
traffic that competes with the cache coherence protocols and memory
traffic; in a well designed compute kernel that saturates both
compute and bandwidth the latency hiccups so introduced can propagate
really badly.

Paul

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFHy2+MpJeHo/Fbu1wRAgyUAKDdSB8B1vKRt8dpNA0MoT+3jnV63wCdGtNP
6FVzjgBJIkvy37rVNlmbE7Q=
=RnvR
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[Roman V. Shaposhnik]

On Sun, 2008-03-02 at 12:34 -0800, Paul Lalonde wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> CSP doesn't scale very well to hundreds of simultaneously executing
> threads (my claim, not, as far as I've found yet, anyone else's).

I take it that you really do mean "simultaneous". As in: you actually
have hundreds of cores available on the same system. I'm actually
quite curious to find out what's the highest number of cores available
of a single shared memory systems that you, or anybody else has
experienced in practice so far (mine would be 32 == 8 CPUs x 4 cores AMD
Barcelona)? Now even more important question is -- what are the
expectations for, this number in 5 years from now?

> Programming the memory hierarchy is really a specific instance of
> programming for masking latency.  This goes well beyond inserting
> prefetches in an optimization stage,

In fact the more I think about it, the more it seems like having
a direct way of manipulating L1/L2 caches would be more of a benefit
than a curse at this point. Prefetches are nothing but a patchwork
over the fundamental need for programming memory hierarchy in an
efficient way. But, I guess, there's more to it on the hardware side
than just crafting additional opcodes.

> presenting itself as problem
> decompositions that keep the current working set in cache (at L3, L2,
> or L1 granularity, depending), while simultaneously avoiding having
> multiple processors chewing on the same data (which leads to vast
> amounts of cache synchronization bus traffic).  Successful algorithms
> in this space work on small bundles of data that either get flushed
> back to memory uncached (to keep more cache for streaming in), or in
> small bundles that can be passed from compute kernel to compute
> kernel cheaply.  Having language structures to help with these
> decompositions and caching decisions is a great help - that's one of
> the reasons why functional programming keeps rearing its head in this
> space.  Without aliasing and global (serializing) state it's much
> easier to analyze the program and chose how to break up the
> computation into kernels that can be streamed, pipelined, or
> otherwise separated to allow better cache utilization and parallelism.

Very well said, indeed! I can't agree more, especially on the issue
of functional programming being very well suited for parallelism.
In fact, here at Sun we've had a curious little experiment run by
Tim Bray:
   http://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder
with a very unexpected result as to what would be the ideal
language for what seems to be an embarrassingly parallel problem.

> Currently, the best performing programs I know for exploiting the
> memory hierarchy are C and C++ programs written in a  "kernel
> composition" kind of model that the language supports poorly.  You
> can do it, but it feels more like coding in assembly than like
> expressing your algorithms.  Much of the template metaprogramming is
> about taking measures of cache spaces and data sets and turning out
> code (statically) tuned to those sizes.  There's a huge opportunity
> for a JIT language to allow better adaptability to changing data sets
> and numbers of active threads.

These techniques help. No doubt about it. Still, I don't think that we
have much of choice when the # of compute units hits hundreds of
thousands -- at that level shared state  and side effects propagated
through it will be just too much to stomach. We clearly need a very
different way of thinking about programming and also the way of
expressing our ideas. None of the mainstream languages seem to fit
the bill. And as far as I can tell this makes it one of the hottest
research areas in the industry. Everybody wants to hit that jackpot
by inventing C for the parallel world.

Thanks,
Roman.

P.S. We had Don Knuth lecturing at Sun Labs a couple of days ago and he
gave us all a real scare when he officially proclaimed that he could NOT
see any way of how his reasoning about computation and algorithms
could be applied to parallelism. In fact, in his latest volume he has
an explicit disclaimer to that effect.

Re: [9fans] GCC/G++: some stress testing

[Philippe Anel]

Ron, I thought Paul was talking about cache coherent system on which a
high-contention lock can become a huge problem.
Although the work did by Jim Taft on the NASA project looks very
interesting (and if you have pointers to papers about locking primitive
on such system, I would appreciate), it seems this system is memory
coherent, not cache coherent (coherency maintained by SGI NUMALink
interconnect fabric).
And I agree with you. I also think (global) shared memory for IPC is
more efficient than passing copied data across the nodes, and I suppose
several papers tend to confirm this is the case: today's interconnect
fabrics are lot of faster than memory memory access.
My conjecture (I only have access to a simple dual core machines) is
about locking primitive used in CSP (and IPC), I mean libthread which is
based on rendezvous system call (which does use locking primitives
9/proc.c:sysrendezvous() ). I think this is the only reason why CSP
would not scale well.
Regarding my (other) conjecture about IPI, please read my answer to Paul.

Phil;

>> If CSP system itself takes care about memory hierarchy and uses no
>>  synchronisation (using IPI to send message to another core by example),
>>  CSP scales very well.
>>
>
> Is this something you have measured or is this conjecture?
>

>>  Of course IPI mechanism requires a switch to kernel mode which costs a
>>  lot. But this is necessary only if the destination thread is running on
>>  another core, and I don't think latency is very important in algorigthms
>>  requiring a lot of cpus.
>>
>
> same question.
>
> For a look at an interesting library that scaled well on a 1024-node
> SMP at NASA Ame's, by Jim Taft.
> Short form: use shared memory for IPC, not data sharing.
>
> he's done very well this way.
>
> ron
>
>
>

Re: [9fans] GCC/G++: some stress testing

[Philippe Anel]

Please note I'm not an expert in this domain. I am only interested in
this area, and have only read a few papers.
It is interesting to talk with you about this 'real world' problems.
> Latency is quite important in the application domain I have to target:
> the target is to produce a new image every 60th of a second, including
> all the simulation effort to get there.  In addition, we have user
> input which needs to be processed, and usually network delays to worry
> about as well.  Every bit of latency between user input and display
> breaks the illusion of control.  And though TVs are getting better,
> it's not atypical to see 4-6 frames of latency introduced by the
> display subsystem, once you've finished generating a frame buffer.
So, does this mean the latency is only required by the I/O system of
your program ? If so, maybe I'm wrong, what you need is to be able to
interrupt working cores and I'm afraid libthread doesn't help here.
If not and your algorithm requires (a lot of) fast IPC, maybe this is
the reason why it doesn't scale well ?
>
> I don't know what you mean by "CSP system itself takes care about
> memory hierarchy".  Do you mean that the CSP implementation does
> something about it, or do you mean that the code using the CSP
> approach takes care of it?
Both :)
I agree with you about the fact programming for the memory hierarchy is
way more important than optimizing CPU clocks.
But I also think synchronization primitives used in CSP systems are the
main reason why CSP programs do not scale well (excepted bad designed
algorithm of course).
I meant that a different CSP implementation, based on different
synchronisation primitive (IPI), can help here.
>
> IPI isn't free either - apart from the OS switch, it generates bus
> traffic that competes with the cache coherence protocols and memory
> traffic; in a well designed compute kernel that saturates both compute
> and bandwidth the latency hiccups so introduced can propagate really
> badly.
>
This is very interesting. For sure IPI is not free. But I thought the
bus traffic generated by IPI was less important than cache coherence
protocols such as MESI, mainly because it is a one way message.
I think now IPI are sent through the system bus (local APIC used to talk
through a separate bus), so I agree with you about the fact it can
saturate the bandwidth. But I wonder if locking primitive are not worse.
It would be interesting to test this.

Phil;

Re: [9fans] GCC/G++: some stress testing

[erik quanstrom]

> In fact the more I think about it, the more it seems like having
> a direct way of manipulating L1/L2 caches would be more of a benefit
> than a curse at this point. Prefetches are nothing but a patchwork
> over the fundamental need for programming memory hierarchy in an
> efficient way. But, I guess, there's more to it on the hardware side
> than just crafting additional opcodes.
>

really?  to out-predict the cache hardware, you have to have pretty
complete knowlege of everything running on all cores and be pretty
good at guessing what will want scheduling next.  not to mention,
you'd need to keep close tabs on which memory is cachable/wc/etc.
maybe system management mode makes too much of an impression
on me.  but on an intel system, there's on way to prevent the cpu/bios/ipmi
from issuing an smm interrupt anytime it pleases and taking over
your hardware.  my conclusion is we don't have as much control
over the hardware as we think we do.

> > decompositions that keep the current working set in cache (at L3, L2,
> > or L1 granularity, depending), while simultaneously avoiding having
> > multiple processors chewing on the same data (which leads to vast
> > amounts of cache synchronization bus traffic).  Successful algorithms
> > in this space work on small bundles of data that either get flushed
> > back to memory uncached (to keep more cache for streaming in), or in
> > small bundles that can be passed from compute kernel to compute
> > kernel cheaply.  Having language structures to help with these
> > decompositions and caching decisions is a great help - that's one of
> > the reasons why functional programming keeps rearing its head in this
> > space.  Without aliasing and global (serializing) state it's much
> > easier to analyze the program and chose how to break up the
> > computation into kernels that can be streamed, pipelined, or
> > otherwise separated to allow better cache utilization and parallelism.

aren't these arguments for networked rather than shared memory
multiprocessors?

- erik

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Mar 3, 2008, at 4:49 PM, erik quanstrom wrote:
> really?  to out-predict the cache hardware, you have to have pretty
> complete knowlege of everything running on all cores and be pretty
> good at guessing what will want scheduling next.  not to mention,
> you'd need to keep close tabs on which memory is cachable/wc/etc.

On the flip side, ignoring the cache leads to algorithms whose
working sets can't fit in cache, wasting a considerable amount of
processing to cache misses.  Being able to parameterize your
algorithms to work comfortably in one or two ways of your cache can
bring *huge* performance improvements without dropping to the
traditional assembly.  I'm arguing that being aware of the caches
lets the system better schedule your work because you aren't
preventing it from doing something smart.

>
> aren't these arguments for networked rather than shared memory
> multiprocessors?

Yes.  Although I work for a company that prides itself on its cache
coherence know-how, I'm very much a believer in networked
multiprocessors, even on a chip.   I like Cell better than Opteron,
for example.  They are harder to program up front, however, which
causes difficulties in adoption.  Flip-side, once you've overcome
your startup hurdles the networked model seems to provide more
predictable performance management.

Paul

>
> - erik

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFHzLEdpJeHo/Fbu1wRAkNNAKC54Me95evWld4cUlUb0Wd9NXQF7QCfZ9zn
QZ5kn6JLfK3EXocNz+plF4c=
=xSzm
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[Paul Lalonde]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Mar 3, 2008, at 1:12 AM, Philippe Anel wrote:
>>
> So, does this mean the latency is only required by the I/O system
> of your program ? If so, maybe I'm wrong, what you need is to be
> able to interrupt working cores and I'm afraid libthread doesn't
> help here.
> If not and your algorithm requires (a lot of) fast IPC, maybe this
> is the reason why it doesn't scale well ?

No, the whole simulation has to run in the low-latency space - it's a
video game and its rendering engine, which are generally highly
heterogeneous workload.  And that heterogeneity means that there are
many points of contact between various subsystems.  And the (semi-)
real-time constraint means that you can't just scale the problem up
to cover overhead costs.

>>
>> I don't know what you mean by "CSP system itself takes care about
>> memory hierarchy".  Do you mean that the CSP implementation does
>> something about it, or do you mean that the code using the CSP
>> approach takes care of it?
> Both :)
> I agree with you about the fact programming for the memory
> hierarchy is way more important than optimizing CPU clocks.
> But I also think synchronization primitives used in CSP systems are
> the main reason why CSP programs do not scale well (excepted bad
> designed algorithm of course).
> I meant that a different CSP implementation, based on different
> synchronisation primitive (IPI), can help here.

I'm more interested just now in working with lock-free algorithms;
I've not made any good measurements of how badly our kernels would
hit channels as the number of threads increases.  Perhaps some could
be mitigated through a better channel implementation.

>>
>> IPI isn't free either - apart from the OS switch, it generates bus
>> traffic that competes with the cache coherence protocols and
>> memory traffic; in a well designed compute kernel that saturates
>> both compute and bandwidth the latency hiccups so introduced can
>> propagate really badly.
>>
> This is very interesting. For sure IPI is not free. But I thought
> the bus traffic generated by IPI was less important than cache
> coherence protocols such as MESI, mainly because it is a one way
> message.

It depends immensely on the hardware implementation of your IPI.  If
you wind up having to pay for MESI as well, then the advantage
becomes less.

> I think now IPI are sent through the system bus (local APIC used to
> talk through a separate bus), so I agree with you about the fact it
> can saturate the bandwidth. But I wonder if locking primitive are
> not worse. It would be interesting to test this.

Agreed!

Paul

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFHzLSSpJeHo/Fbu1wRAkv/AKDKK4fuuWyYCqXv4JqbWWj+RXQd0wCfSFoS
b9E6X/a13bg6AzUGT5dLSqU=
=ppoF
-----END PGP SIGNATURE-----

Re: [9fans] GCC/G++: some stress testing

[erik quanstrom]

> Yes.  Although I work for a company that prides itself on its cache
> coherence know-how, I'm very much a believer in networked
> multiprocessors, even on a chip.   I like Cell better than Opteron,
> for example.  They are harder to program up front, however, which
> causes difficulties in adoption.  Flip-side, once you've overcome
> your startup hurdles the networked model seems to provide more
> predictable performance management.

tell me about it.  a certain (nameless) vendor makes a pcie ethernet
chipset with its descriptor rings in system memory, not pci space.
it's bizarre watching the performance vs. the number of buffers loaded
into the ring between head ptr updates.  slight tweeks to the algorithm
can result in 35% performance differences.

suprisingly, another (also nameless) vendor makes a similar chipset with
rings in pci space.  this chipset has very stable performance in the face of
tuning of the reloading loop.  this chip performs just as well as the former
though each 32-bit write to the ring buffer results in a round trip over
the pcie bus to the card.

- erik

Re: [9fans] GCC/G++: some stress testing

[Lyndon Nerenberg]

On 2008-Mar-1, at 08:41 , ron minnich wrote:

> very litlle f77 left in my world, maybe somebody else has some.

And also in response to Pietro's comments ...

I have lots of dusty but still valid F77 code I use for antenna and RF
circuit design (i.e. NEC and SPICE). Yes, there are newer versions of
this stuff in C, but none of the C code fixes any bugs I'm not aware
of or cannot otherwise deal with. Or more importantly, my models are
not already aware of and compensate for.

And then there is Dungeon.  Netlib's f2c + libraries are ready to go
with Plan 9, given the correct compiler defines. Building a colossal
cave out of old card decks does not require outsourcing to the
dwarves ;-)

--lyndon

Re: [9fans] GCC/G++: some stress testing

[Paweł Lasek]

On Mon, Mar 3, 2008 at 4:31 AM, Roman V. Shaposhnik <rvs@sun.com> wrote:
> On Sun, 2008-03-02 at 12:34 -0800, Paul Lalonde wrote:
>  > -----BEGIN PGP SIGNED MESSAGE-----
>  > Hash: SHA1
>  >
>  > CSP doesn't scale very well to hundreds of simultaneously executing
>  > threads (my claim, not, as far as I've found yet, anyone else's).
>
>  I take it that you really do mean "simultaneous". As in: you actually
>  have hundreds of cores available on the same system. I'm actually
>  quite curious to find out what's the highest number of cores available
>  of a single shared memory systems that you, or anybody else has
>  experienced in practice so far (mine would be 32 == 8 CPUs x 4 cores AMD
>  Barcelona)? Now even more important question is -- what are the
>  expectations for, this number in 5 years from now?
>

Actually, with parts available from AMD you can directly mesh up to 64
sockets, each with (currently) 4 cores, 8 core cpu announced (as MCP
in the beginning). And there were available methods for routing HT
traffic with number of sockets nearing thousands or tens of thousands.
Dunno if they used it directly with cache coherency protocol though.

--
Paul Lasek

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Tue, Mar 4, 2008 at 2:57 AM, Paweł Lasek <pawel.lasek@gmail.com> wrote:
>And there were available methods for routing HT
>  traffic with number of sockets nearing thousands or tens of thousands.

as in this: http://www.cray.com/products/xt4/

>  Dunno if they used it directly with cache coherency protocol though.

no, it does not.

Plan 9 port in progress ... soon we hope.

ron

Re: [9fans] GCC/G++: some stress testing

[Philippe Anel]

> no, it does not.
>
> Plan 9 port in progress ... soon we hope.
>
> ron
>

Are you working on this port Ron ?
Are you planning to have several kernels or just one ?

Phil;

Re: [9fans] GCC/G++: some stress testing

[Roman V. Shaposhnik]

On Tue, 2008-03-04 at 11:57 +0100, Paweł Lasek wrote:
> >  I take it that you really do mean "simultaneous". As in: you actually
> >  have hundreds of cores available on the same system. I'm actually
> >  quite curious to find out what's the highest number of cores available
> >  of a single shared memory systems that you, or anybody else has
> >  experienced in practice so far (mine would be 32 == 8 CPUs x 4 cores AMD
> >  Barcelona)? Now even more important question is -- what are the
> >  expectations for, this number in 5 years from now?
> >
> 
> Actually, with parts available from AMD you can directly mesh up to 64
> sockets, each with (currently) 4 cores, 8 core cpu announced (as MCP
> in the beginning). And there were available methods for routing HT
> traffic with number of sockets nearing thousands or tens of thousands.
> Dunno if they used it directly with cache coherency protocol though.

Understood. Although, what I meant was a practical hands-on experience.
So that the war stories can be told. Otherwise, on the SPARC side, I'd
be able to claim 128 in Sun's M9000.

Thanks,
Roman.

Re: [9fans] GCC/G++: some stress testing

[ron minnich]

On Tue, Mar 4, 2008 at 7:55 AM, Philippe Anel <xigh@free.fr> wrote:

>  Are you working on this port Ron ?

soon.

I just realized today that, for the part, linuxemu may save our neck
on the XT4. That's because there are proprietary programs that need to
run, and they are only linux programs.

So I owe Russ and cinap ...

ron

Re: [9fans] GCC/G++: some stress testing

[lucio]

Forsyth asks:
> can it compile a working pm?

Not without some adjustments to the stuff in
/n/sources/extra/mpm/mpm.bundle, but it does eventually compile it.

The binary runs, but someone else may want to test it:

# ls -l pm
total 1000
-rwxrwxr-x 1 lucio lucio 430935 Mar  7 07:15 pm

Any offers?

++L

PS: The time shown is totally wrong.

PPS: Why is "typename" a G++ reserved word?

Re: [9fans] GCC/G++: some stress testing

[Russ Cox]

> PPS: Why is "typename" a G++ reserved word?

[now safely off-topic; sorry]

Because it is a C++ reserved word.
It gets used in contexts (templates, in particular)
where the parser needs extra help deciding that
a name is going to be used as a type.

Russ

Re: [9fans] GCC/G++: some stress testing

[Pietro Gagliardi]

There was a time when you said
	template <typename x, typename y>
instead of
	template <class x, class y>
(Was it TC++PL 2nd edition?) The former is still allowed.

It is also used when the context that a variable declaration is in is
ambiguous, such as this: http://publib.boulder.ibm.com/infocenter/
pseries/v5r3/index.jsp?topic=/com.ibm.xlcpp8a.doc/language/ref/
keyword_typename.htm


On Mar 7, 2008, at 5:21 AM, Russ Cox wrote:

>> PPS: Why is "typename" a G++ reserved word?
>
> [now safely off-topic; sorry]
>
> Because it is a C++ reserved word.
> It gets used in contexts (templates, in particular)
> where the parser needs extra help deciding that
> a name is going to be used as a type.
>
> Russ
>

Re: [9fans] GCC/G++: some stress testing

[lucio]

> Because it is a C++ reserved word.
> It gets used in contexts (templates, in particular)
> where the parser needs extra help deciding that
> a name is going to be used as a type.

I used to like C++.  Back in the days when David Bright built an
entire compiler for MS-Dos himself (Zortech C++) and it actually
worked extremely reliably.

It's been uphill (not even downhill) ever since.  Even the phrase
"creeping featurism" isn't adequate to describe the complexity C++
seems to be subjected to.

I really find it hard to believe that the best intellects in computing
are incapable of stemming the tide.  I appreciate that biological
evolution seems to thrive on serendipitous change, but I fail to see
that human cognition can cope with a similar lack of direction.  Yet,
that seems to me to be the trend in "bazaar" programming.

As for P9GCC, its scope fairly exceeds (this) one person's
capabilities.  Just looking at "binutils" and "bfd", without having
the slightest idea what their real intent might be and how to apply it
to P9GCC, then finding wonderful snippets such as

> The documentation on using BFD is scanty and may be occasionally
> incorrect.  Pointers to documentation problems, or an entirely
> rewritten manual, would be appreciated.
>
> There is some BFD internals documentation in doc/bfdint.texi which may
> help programmers who want to modify BFD.

is enough to depress me until the end of the decade.  Nevermind having
to use my NetBSD server to read it and needing to learn a whole new
documentation reading package to do it.

++L

Re: [9fans] GCC/G++: some stress testing

[Skip Tavakkolian]

i don't remember typename and checked my copy of the Bjorne's design
and evolution of c++ and it's not there either.  apparently it evovled
some more.

you know you're in trouble when you have to add keywords to help the
compiler (not the programmer)

> There was a time when you said
> 	template <typename x, typename y>
> instead of
> 	template <class x, class y>
> (Was it TC++PL 2nd edition?) The former is still allowed.
>
> It is also used when the context that a variable declaration is in is
> ambiguous, such as this: http://publib.boulder.ibm.com/infocenter/
> pseries/v5r3/index.jsp?topic=/com.ibm.xlcpp8a.doc/language/ref/
> keyword_typename.htm

Re: [9fans] GCC/G++: some stress testing

[lucio]

> i don't remember typename and checked my copy of the Bjorne's design
> and evolution of c++ and it's not there either.  apparently it evovled
> some more.

Pity it didn't notice that it was already estinct.  Biological
evolution at least has some checks and balances, computing evolution
instead seems to favour the grossest.

++L

Re: [9fans] GCC/G++: some stress testing

[Skip Tavakkolian]

> Just out of curiosity, what is the Plan 9 executable format?

rtfm(1)

Re: [9fans] GCC/G++: some stress testing

[erik quanstrom]

> Just out of curiosity, what is the Plan 9 executable format?

it is documented in a.out(6).

- erik

Re: [9fans] GCC/G++: some stress testing

[Pietro Gagliardi]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

On Mar 9, 2008, at 1:53 PM, Aharon Robbins wrote:

>> capabilities.  Just looking at "binutils" and "bfd", without having
>> the slightest idea what their real intent might be and how to
>> apply it
>> to P9GCC, then finding wonderful snippets such as
>
> Binutils are the things for dealing with binaries: as, ld, nm are
> the primary ones. They're built on top of BFD (Binary File Descriptor)
> which is essentially an object-oriented way of looking at object
> file formats; the tools can be configured to be cross platform
> and even convert an object file in say, a.out format to COFF or
> from COFF to ELF, and so on.

Just out of curiosity, what is the Plan 9 executable format?


[-- Attachment #2: Type: text/html, Size: 2171 bytes --]

Re: [9fans] GCC/G++: some stress testing

[Bruce Ellis]

and take it all in.  gcc texi is larger than the kenc source.

i know which is easier to read.

brucee

On Mon, Mar 10, 2008 at 4:53 AM, Aharon Robbins <arnold@skeeve.com> wrote:
> > I really find it hard to believe that the best intellects in computing
> > are incapable of stemming the tide.
>
> I'm not sure they're aware it's an issue (CS professor and research types).
>
> > As for P9GCC, its scope fairly exceeds (this) one person's
> > capabilities.  Just looking at "binutils" and "bfd", without having
> > the slightest idea what their real intent might be and how to apply it
> > to P9GCC, then finding wonderful snippets such as
>
> Binutils are the things for dealing with binaries: as, ld, nm are
> the primary ones. They're built on top of BFD (Binary File Descriptor)
> which is essentially an object-oriented way of looking at object
> file formats; the tools can be configured to be cross platform
> and even convert an object file in say, a.out format to COFF or
> from COFF to ELF, and so on.
>
> GDB uses the same framework so that it can be cross-platform
> debugger.
>
> I suspect you need to teach the binutils ('as') how to write the Plan 9
> object file format, but it may be enough to teach GCC how to write
> Plan 9 x86 assembler and then assemble it.
>
> > Nevermind having
> > to use my NetBSD server to read it and needing to learn a whole new
> > documentation reading package to do it.
>
> The 'info' reader is nothing to write home about, although the Texinfo
> documentation language is one of the best markup languages I've ever
> used. (I've written whole books in troff, Docbook SGML / XML and Texinfo,
> and I'd take Texinfo any day.)
>
> The best way to read Texinfo doc is to run
>
>        makeinfo --no-split --hmtl file.texi
>
> which produces one gignatic HTML file from the Texinfo source, and
> then use a web browser to read it.
>
> Another alternative is to use texi2pdf and then use a PDF viewer,
> but I suspect that HTML will meet your needs the best.
>
> Hope this helps,
>
> Arnold
>

Re: [9fans] GCC/G++: some stress testing

[Aharon Robbins]

> I really find it hard to believe that the best intellects in computing
> are incapable of stemming the tide.

I'm not sure they're aware it's an issue (CS professor and research types).

> As for P9GCC, its scope fairly exceeds (this) one person's
> capabilities.  Just looking at "binutils" and "bfd", without having
> the slightest idea what their real intent might be and how to apply it
> to P9GCC, then finding wonderful snippets such as

Binutils are the things for dealing with binaries: as, ld, nm are
the primary ones. They're built on top of BFD (Binary File Descriptor)
which is essentially an object-oriented way of looking at object
file formats; the tools can be configured to be cross platform
and even convert an object file in say, a.out format to COFF or
from COFF to ELF, and so on.

GDB uses the same framework so that it can be cross-platform
debugger.

I suspect you need to teach the binutils ('as') how to write the Plan 9
object file format, but it may be enough to teach GCC how to write
Plan 9 x86 assembler and then assemble it.

> Nevermind having
> to use my NetBSD server to read it and needing to learn a whole new
> documentation reading package to do it.

The 'info' reader is nothing to write home about, although the Texinfo
documentation language is one of the best markup languages I've ever
used. (I've written whole books in troff, Docbook SGML / XML and Texinfo,
and I'd take Texinfo any day.)

The best way to read Texinfo doc is to run

	makeinfo --no-split --hmtl file.texi

which produces one gignatic HTML file from the Texinfo source, and
then use a web browser to read it.

Another alternative is to use texi2pdf and then use a PDF viewer,
but I suspect that HTML will meet your needs the best.

Hope this helps,

Arnold