[9fans] "ridiculous benchmarks"-r-us

[9fans] "ridiculous benchmarks"-r-us

[mirtchov]

[-- Attachment #1: Type: text/plain, Size: 3357 bytes --]

Rog at vitanuova already caught how stupid the following article is
(linked to from slashdot):

	http://osnews.com/story.php?news_id=5602

It took me much longer -- until I looked at their "C" code (term used
very loosely) trying to compile it on Plan 9...  There are multiple
gems in it including ++ on a double, mid-function variable definitions
(C-code, right?), log10(0.0), impossible to look at I/O, etc.  In
fact, I'm attaching the benchmark for your viewing pleasure :)

Anyway, the exercise took very little time and here are some "results"
showing how good Plan 9's compiler are (where's a Bushnell when you
need one?), it would be interesting to see how the long arithmetic can be improved:

pcc (code untouched except for mid-function variable definitions moved to top):

	Int arithmetic elapsed time: 17540 ms with intMax of 1000000000
	 i: 1000000001
	 intResult: 1
	Double arithmetic elapsed time: 65500 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
	 i: 11000000000.000000
	 doubleResult: 10011632717.495501
	Long arithmetic elapsed time: 262540 ms with longMin I64d, longMax I64d
	 i: I64d
	 longResult: I64d
	Trig elapsed time: 25070 ms with max of 10000000
	 i: 10000000.000000
	 sine: 0.990665
	 cosine: -0.136322
	 tangent: -7.267119
	 logarithm: 7.000000
	 squareRoot: 3162.277502
	[crashes during the last test while reading back the file it just wrote]

gcc (code untouched, no -O):

	Int arithmetic elapsed time: 23770 ms with intMax of 1000000000
	 i: 1000000001
	 intResult: 1
	Double arithmetic elapsed time: 56040 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
	 i: 11000000000.000000
	 doubleResult: 10011632717.495501
	Long arithmetic elapsed time: 53500 ms with longMin I64d, longMax I64d
	 i: I64d
	 longResult: I64d
	Trig elapsed time: 30680 ms with max of 10000000
	 i: 10000000.000000
	 sine: 0.990665
	 cosine: -0.136322
	 tangent: -7.267119
	 logarithm: 7.000000
	 squareRoot: 3162.277502
	[crashes during the last test while reading back the file it just wrote]

8c (modified to use nsec() for timing, printf -> print, fixed log10(0.0) crash, fix %I64d in print()):

	Int arithmetic elapsed time: 17517 ms with intMax of 1000000000
	 i: 1000000001
	 intResult: 1
	Double arithmetic elapsed time: 22525 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
	 i: 11000000000.000000
	 doubleResult: 10011632717.495500
	Long arithmetic elapsed time: 565067 ms with longMin 10000000000, longMax 11000000000
	 i: 11000000000
	 longResult: 776627965
	Trig elapsed time: 25881 ms with max of 10000000
	 i: 10000000.100000
	 sine: 0.972106
	 cosine: -0.234542
	 tangent: -4.144701
	 logarithm: 7.000000
	 squareRoot: 3162.277518


It is important to note that the integer performance of 8c is
comparable to gcc 3.3's -O[23] running on Plan 9 and Linux
respectively.  The double arithmetic lags by about 20%.  The I/O is
comparable (2x slower when going over a 100mbit link, no caching).

Profiling the long long code to see where it looses all that time may
be interesting.

All computation performed on a 8-way 700mhz Xeon, only 1 cpu used :)

Sorry for wasting your time -- just wanted to see how good 8c was and
got caught in all the other crap. It's Friday after all... :)

Andrey

[-- Attachment #2: Benchmark.c --]
[-- Type: text/plain, Size: 4494 bytes --]

/* All code Copyright 2004 Christopher W. Cowell-Shah */

#include <time.h>
#include <math.h>
#include <stdio.h>

double intArithmetic(int);
double doubleArithmetic(double, double);
double longArithmetic(long long int, long long int);
double trig(double);
double io(int);

int main()
{
	int intMax =            1000000000; // 1B
	double doubleMin =      10000000000.0; // 10B
	double doubleMax =      11000000000.0; // 11B
	long long int longMin = 10000000000LL; // 10B
	long long int longMax = 11000000000LL; // 11B
	double trigMax =        10000000; // 10M
	int ioMax =             1000000; // 1M


	printf("Start C benchmark\n");

	long intArithmeticTime = (long)intArithmetic(intMax);
	long doubleArithmeticTime = (long)doubleArithmetic(doubleMin, doubleMax);
	long longArithmeticTime = (long)longArithmetic(longMin, longMax);
	long trigTime = (long)trig(trigMax);
	long ioTime = (long)io(ioMax);
	long totalTime = intArithmeticTime + doubleArithmeticTime + longArithmeticTime + trigTime + ioTime;
	printf("Total elapsed time: %d ms\n", totalTime);

	printf("Stop C benchmark\n");
	return 0;
}


double intArithmetic(int intMax)
{
	double elapsedTime;
	clock_t stopTime;
	clock_t startTime = clock();

	int intResult = 1;
	int i = 1;
	while (i < intMax)
	{
		intResult -= i++;
		intResult += i++;
		intResult *= i++;
		intResult /= i++;
	}

	stopTime = clock();
	elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / (double) 1000.0);
	printf("Int arithmetic elapsed time: %1.0f ms with intMax of %ld\n", elapsedTime, intMax);
	printf(" i: %d\n", i);
	printf(" intResult: %d\n", intResult);
	return elapsedTime;
}


double doubleArithmetic(double doubleMin, double doubleMax)
{
	double elapsedTime;
	clock_t stopTime;
	clock_t startTime = clock();

	double doubleResult = doubleMin;
	double i = doubleMin;
	while (i < doubleMax)
	{
		doubleResult -= i++;
		doubleResult += i++;
		doubleResult *= i++;
		doubleResult /= i++;
	}

	stopTime = clock();
	elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / (double) 1000.0);
	printf("Double arithmetic elapsed time: %1.0f ms with doubleMin %f, doubleMax %f\n", elapsedTime, doubleMin, doubleMax);
	printf(" i: %f\n", i);
	printf(" doubleResult: %f\n", doubleResult);
	return elapsedTime;
}


double longArithmetic(long long int longMin, long long int longMax)
{
	double elapsedTime;
	clock_t stopTime;
	clock_t startTime = clock();

	long long longResult = longMin;
	long long i = longMin;
	while (i < longMax)
	{
		longResult -= i++;
		longResult += i++;
		longResult *= i++;
		longResult /= i++;
	}

	stopTime = clock();
	elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / (double) 1000.0);
	printf("Long arithmetic elapsed time: %1.0f ms with longMin %I64d, longMax %I64d\n", elapsedTime, longMin, longMax);
	printf(" i: %I64d\n", i);
	printf(" longResult: %I64d\n", longResult);
	return elapsedTime;
}


double trig(double trigMax)
{
	double elapsedTime;
	clock_t stopTime;
	clock_t startTime = clock();

	double sine;
	double cosine;
	double tangent;
	double logarithm;
	double squareRoot;

	double i = 0.0;
	while (i < trigMax)
	{
		sine = sin(i);
		cosine = cos(i);
		tangent = tan(i);
		logarithm = log10(i);
		squareRoot = sqrt(i);
		i++;
	}

	stopTime = clock();
	elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / (double) 1000.0);
	printf("Trig elapsed time: %1.0f ms with max of %1.0f\n", elapsedTime, trigMax);
	printf(" i: %f\n", i);
	printf(" sine: %f\n", sine);
	printf(" cosine: %f\n", cosine);
	printf(" tangent: %f\n", tangent);
	printf(" logarithm: %f\n", logarithm);
	printf(" squareRoot: %f\n", squareRoot);
	return elapsedTime;
}


double io(int ioMax)
{
	double elapsedTime;
	clock_t stopTime;
	clock_t startTime = clock();

	FILE *stream;
	stream = fopen("C:\\TestGcc.txt", "w");
	int i = 0;
	while (i++ < ioMax)
	{
		fputs("abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz1234567890abcdefgh\n", stream);
	}
	fclose(stream);

	char readLine[100];
	stream = fopen("C:\\TestGcc.txt", "r");
	i = 0;
	while (i++ < ioMax)
	{
		fgets(readLine, 100, stream);
	}
	fclose(stream);

	stopTime = clock();
	elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / (double) 1000.0);
	printf("I/O elapsed time: %1.0f ms with max of %d\n", elapsedTime, ioMax);
	printf(" last line: %s", readLine);

	return elapsedTime;
}

Re: [9fans] "ridiculous benchmarks"-r-us

[ron minnich]

what's funny is that the guy may have violated his m$ licensing by
reporting performance info on a microsoft product.

Yes, they really have that in many of their new licenses ...

ron

Re: [9fans] "ridiculous benchmarks"-r-us

[Icarus Sparry]

On 2004-01-09, mirtchov@cpsc.ucalgary.ca <mirtchov@cpsc.ucalgary.ca> wrote:
> It took me much longer -- until I looked at their "C" code (term used
> very loosely) trying to compile it on Plan 9...  There are multiple
> gems in it including ++ on a double, mid-function variable definitions
> (C-code, right?)

C99 added mid-function variable definitions.

Re: [9fans] "ridiculous benchmarks"-r-us

[mirtchov]

> C99 added mid-function variable definitions.

APE doesn't have it:

	/tmp/Benchmark.c:26[stdin:384] syntax error, last name: long

I'd rather go with APE than with gcc ;)

Re: [9fans] "ridiculous benchmarks"-r-us

[Brantley Coile]

[-- Attachment #1: Type: text/plain, Size: 191 bytes --]

What possible rationale did they have for adding nested procedures to C?
Did they call them mid-functions in the standard?  Why rename a concept
just because it's 47 years old!

 Brantley

[-- Attachment #2: Type: message/rfc822, Size: 2326 bytes --]

From: Icarus Sparry <usenet@icarus.freeuk.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] "ridiculous benchmarks"-r-us
Date: Fri, 9 Jan 2004 18:19:11 GMT
Message-ID: <slrnbvtrgl.h6p.usenet@icarus.freeuk.com>

On 2004-01-09, mirtchov@cpsc.ucalgary.ca <mirtchov@cpsc.ucalgary.ca> wrote:
> It took me much longer -- until I looked at their "C" code (term used
> very loosely) trying to compile it on Plan 9...  There are multiple
> gems in it including ++ on a double, mid-function variable definitions
> (C-code, right?)

C99 added mid-function variable definitions.

Re: [9fans] "ridiculous benchmarks"-r-us

[Roman Shaposhnick]

On Fri, Jan 09, 2004 at 10:42:49AM -0700, mirtchov@cpsc.ucalgary.ca wrote:
> Rog at vitanuova already caught how stupid the following article is
> (linked to from slashdot):
>
> 	http://osnews.com/story.php?news_id=5602
>
> It took me much longer -- until I looked at their "C" code (term used
> very loosely) trying to compile it on Plan 9...  There are multiple
> gems in it including ++ on a double, mid-function variable definitions
> (C-code, right?), log10(0.0), impossible to look at I/O, etc.  In
> fact, I'm attaching the benchmark for your viewing pleasure :)

  Benchmarking nowdays, is a real pain in the kazoo, if you know what I
  mean. Benchmarking compiler itself is no way of demonstrating performance
  of the real application where heavily templated library like libm or
  even libc can make all the difference. The trends I've seen seem to be
  quite amusing when compilers are made to recognize parts of the AST
  (including calls to stuff like sqrt and such) and replace them with
  the predefine assembler template. For evert supported combination
  of arch. switches. Painful.

  The interesting part, of course is when we start considering JITs like
  HotSpot for example. Just recently I was quite amused to see what
  JDK 1.4 did for the simplest fibonacci. Quite educational, I might add.
  Somehow, I feel, that for the best performance possible one simply *has*
  consider execution environment with JIT-modifiable code. No way around it.

Thanks,
Roman.

Re: [9fans] "ridiculous benchmarks"-r-us

[Tristan Seligmann]

On Fri, Jan 09, 2004 at 13:52:55 -0500, Brantley Coile wrote:
> What possible rationale did they have for adding nested procedures to C?
> Did they call them mid-functions in the standard?  Why rename a concept
> just because it's 47 years old!

I think he means that C99 allows you to have variable definitions at any
point in a block, instead of forcing you to place them at the beginning
of the block.

mithrandi

Re: [9fans] "ridiculous benchmarks"-r-us

[Sam]

> I'd rather go with APE than with gcc ;)

A really fair default considering:

http://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html#C%20Extensions

With all they put in you'd think plan 9's extensions would
have been consumed by now.  Must be the license.

Sam

Re: [9fans] "ridiculous benchmarks"-r-us

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 257 bytes --]

worse, i've had gcc fanatics work themselves up into a terrible
state about how presumptious it was for Plan 9 C to add its own extensions.
since i've historically been troubled more by bugs in gcc than
bugs in Plan 9's C, i've not been much impressed.

[-- Attachment #2: Type: message/rfc822, Size: 2103 bytes --]

From: Sam <sah@softcardsystems.com>
To: <9fans@cse.psu.edu>
Subject: Re: [9fans] "ridiculous benchmarks"-r-us
Date: Fri, 9 Jan 2004 15:25:26 -0500 (EST)
Message-ID: <Pine.LNX.4.30.0401091521480.4947-100000@athena>

> I'd rather go with APE than with gcc ;)

A really fair default considering:

http://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html#C%20Extensions

With all they put in you'd think plan 9's extensions would
have been consumed by now.  Must be the license.

Sam

Re: [9fans] "ridiculous benchmarks"-r-us

[Sam]

> worse, i've had gcc fanatics work themselves up into a terrible
> state about how presumptious it was for Plan 9 C to add its own extensions.
> since i've historically been troubled more by bugs in gcc than
> bugs in Plan 9's C, i've not been much impressed.
>

And darnit'all if I didn't look closely enough.  Already in:

http://gcc.gnu.org/onlinedocs/gcc/Compound-Literals.html#Compound%20Literals
http://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html#Unnamed%20Fields

Re: [9fans] "ridiculous benchmarks"-r-us

[Brantley Coile]

This is sort of off topic, but sort of on topic.  Is the following
legal C89 (or C99) code?

#ifdef notdef
	Why won't this work?
#endif

The gcc balks at the single quote.  Clearly he's scaning for tokens
inside of the ifdef/endif lines.  Should he?

 Brantley

Re: [9fans] "ridiculous benchmarks"-r-us

[mirtchov]

> worse, i've had gcc fanatics work themselves up into a terrible
> state about how presumptious it was for Plan 9 C to add its own extensions.
> since i've historically been troubled more by bugs in gcc than
> bugs in Plan 9's C, i've not been much impressed.

The last sentence of this page is indicative:

	http://www.gnome.org/projects/ORBit2/orbit-docs/orbit/x166.html

quote:

	If ORBit2 2.7 does not compile with some commercial C
	compilers, this is because it uses a number of gcc extensions
	which are not supported elsewhere.  you can always download
	gcc and install that for it to work.

What's puzzling is why ORBit is such an important program that is
can't live without the gcc extensions?

Re: [9fans] "ridiculous benchmarks"-r-us

[Atanas Bachvaroff]

[-- Attachment #1: Type: TEXT/plain, Size: 4516 bytes --]

On  9 Jan, mirtchov@cpsc.ucalgary.ca wrote:
> Rog at vitanuova already caught how stupid the following article is
> (linked to from slashdot):
>
> 	http://osnews.com/story.php?news_id=5602
>
> It took me much longer -- until I looked at their "C" code (term used
> very loosely) trying to compile it on Plan 9...  There are multiple
> gems in it including ++ on a double, mid-function variable definitions
> (C-code, right?), log10(0.0), impossible to look at I/O, etc.  In
> fact, I'm attaching the benchmark for your viewing pleasure :)
>
> Anyway, the exercise took very little time and here are some "results"
> showing how good Plan 9's compiler are (where's a Bushnell when you
> need one?), it would be interesting to see how the long arithmetic can be improved:
>
> pcc (code untouched except for mid-function variable definitions moved to top):
>
> 	Int arithmetic elapsed time: 17540 ms with intMax of 1000000000
> 	 i: 1000000001
> 	 intResult: 1
> 	Double arithmetic elapsed time: 65500 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
> 	 i: 11000000000.000000
> 	 doubleResult: 10011632717.495501
> 	Long arithmetic elapsed time: 262540 ms with longMin I64d, longMax I64d
> 	 i: I64d
> 	 longResult: I64d
> 	Trig elapsed time: 25070 ms with max of 10000000
> 	 i: 10000000.000000
> 	 sine: 0.990665
> 	 cosine: -0.136322
> 	 tangent: -7.267119
> 	 logarithm: 7.000000
> 	 squareRoot: 3162.277502
> 	[crashes during the last test while reading back the file it just wrote]
>
> gcc (code untouched, no -O):
>
> 	Int arithmetic elapsed time: 23770 ms with intMax of 1000000000
> 	 i: 1000000001
> 	 intResult: 1
> 	Double arithmetic elapsed time: 56040 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
> 	 i: 11000000000.000000
> 	 doubleResult: 10011632717.495501
> 	Long arithmetic elapsed time: 53500 ms with longMin I64d, longMax I64d
> 	 i: I64d
> 	 longResult: I64d
> 	Trig elapsed time: 30680 ms with max of 10000000
> 	 i: 10000000.000000
> 	 sine: 0.990665
> 	 cosine: -0.136322
> 	 tangent: -7.267119
> 	 logarithm: 7.000000
> 	 squareRoot: 3162.277502
> 	[crashes during the last test while reading back the file it just wrote]
>
> 8c (modified to use nsec() for timing, printf -> print, fixed log10(0.0) crash, fix %I64d in print()):
>
> 	Int arithmetic elapsed time: 17517 ms with intMax of 1000000000
> 	 i: 1000000001
> 	 intResult: 1
> 	Double arithmetic elapsed time: 22525 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
> 	 i: 11000000000.000000
> 	 doubleResult: 10011632717.495500
> 	Long arithmetic elapsed time: 565067 ms with longMin 10000000000, longMax 11000000000
> 	 i: 11000000000
> 	 longResult: 776627965
> 	Trig elapsed time: 25881 ms with max of 10000000
> 	 i: 10000000.100000
> 	 sine: 0.972106
> 	 cosine: -0.234542
> 	 tangent: -4.144701
> 	 logarithm: 7.000000
> 	 squareRoot: 3162.277518
>
>
> It is important to note that the integer performance of 8c is
> comparable to gcc 3.3's -O[23] running on Plan 9 and Linux
> respectively.  The double arithmetic lags by about 20%.  The I/O is
> comparable (2x slower when going over a 100mbit link, no caching).
>
> Profiling the long long code to see where it looses all that time may
> be interesting.
>
> All computation performed on a 8-way 700mhz Xeon, only 1 cpu used :)
>
> Sorry for wasting your time -- just wanted to see how good 8c was and
> got caught in all the other crap. It's Friday after all... :)
>
> Andrey

That's what I get on SGI Octane R12k/270MHz with the MIPSpro 7.3 C++
compiler as my MIPSpr C compiler is too much standard...

Start C benchmark
Int arithmetic elapsed time: 32870 ms with intMax of 1000000000
 i: 1000000001
 intResult: 1
Double arithmetic elapsed time: 21700 ms with doubleMin 10000000000.000000, doubleMax 11000000000.000000
 i: 11000000000.000000
 doubleResult: 10011632717.495501
Long arithmetic elapsed time: 72920 ms with longMin I64d, longMax I64d
 i: I64d
 longResult: I64d
Trig elapsed time: 5730 ms with max of 10000000
 i: 10000000.000000
 sine: 0.990665
 cosine: -0.136322
 tangent: -7.267119
 logarithm: 7.000000
 squareRoot: 3162.277502
I/O elapsed time: 3540 ms with max of 1000000
 last line: abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz1234567890abcdefgh
Total elapsed time: 136760 ms
Stop C benchmark
--
BEATVSHOMOQVIINVENITSAPIENTIAMETQVIAFFLVITPRVDENTIALIVRILVCIFERIVS

[-- Attachment #2: Type: APPLICATION/pgp-signature, Size: 256 bytes --]

Re: [9fans] "ridiculous benchmarks"-r-us

[Dennis Ritchie]

 > This is sort of off topic, but sort of on topic.  Is the following
 > legal C89 (or C99) code?

 > #ifdef notdef
 > 	Why won't this work?
 > #endif

Not really legal (Ken's off-brand cpp).

#ifdef ... #endif wants tokens, and the ' is unmatched.

Try it with /bin/cpp .

	Dennis

Re: [9fans] "ridiculous benchmarks"-r-us

[boyd, rounin]

any number between 1 and 9 is a pint.

Re: [9fans] 'ridiculous benchmarks'-r-us

[Joel Salomon]

> I'd rather go with APE than with gcc ;)

One monkey vs. infinite number of monkeys?

--Joel