Re: [Caml-list] STM support in OCaml

[Brian Hurt <bhurt@spnz.org>]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1608 bytes --]



On Thu, 9 Mar 2006, skaller wrote:

> Ahem. Now try that on an AMDx2 (dual core). The cost goes through
> the roof if one process has a thread on each core. Because each
> core has its own cache and both caches have to be flushed/
> synchronised. And those caches are BIG!

Love to.  Wanna buy me the box?  :-}  Seriously- my code is attached, if 
someone wants to run it on other boxes and post the results, feel free. 
It's GNU-C/x86 specific, as I'm using GNU C's inline assembler and the 
rdtsc instruction to get accurate cycle counts.

As to the cache comment: the whole caches don't have to be flushed, just 
the line the mutex is on.  Which makes it approximately the cost of a 
cache miss- that's a good approximation of the cost of getting an 
uncontended lock.

>
> I have no idea if Linux, for example, running SMP kernel,
> is smart enough to know if a mutex is shared between two
> processing units or not: AFAIK Linux doesn't support
> interprocess mutex. Windows does. Be interesting to
> compare.

It doesn't look like the mutex software is even going into the kernel. 
I don't think the Linux kernel even knows the mutex *exists*, let alone 
what threads are competing for it.  On the x86, at least, lock 
instructions are not priveledged.

>
> As mentioned before the only data I have at the moment
> is a two thread counter increment experiment on a dual
> CPU G5 box, where the speed up from 2 CPUs vs 1 was
> a factor of 15 .. times SLOWER.

If you're ping-ponging a cache line between two CPUs (and the AMD dual 
cores count as two CPUs), then I can easily beleive that.

So?

Brian

[-- Attachment #2: Type: TEXT/X-CSRC, Size: 1847 bytes --]

#include <stdio.h>
#include <pthread.h>
#include <semaphore.h>

#if !defined(__GNUC__) && !defined(__i386__)
#error This code only works with GCC/i386.
#endif

/* The reason this only works under GNU C and the x86 is we're using the
 * rdtsc instruction.
 */
static inline unsigned long long rdtsc() {
	unsigned long long rval;

	asm volatile ("rdtsc" : "=A" (rval));
	return rval;
}

static sem_t waiting_thread_semaphore;
static pthread_mutex_t mutex;

void * waiting_thread_func(void * param __attribute__((unused))) {
	sem_wait(&waiting_thread_semaphore);
	return NULL;
}

int main(void) {
	int i;
	pthread_t waiting_thread;
	void * trash;
	unsigned long long start, stop, time, min;


	/* Create a thread to force us to actually do multi-threaded work */
	sem_init(&waiting_thread_semaphore, 1, 0);
	pthread_create(&waiting_thread, NULL, waiting_thread_func, NULL);

	pthread_mutex_init(&mutex, NULL);

	/* Time how long a rdtsc takes- we do this ten times and take the 
	 * cheapest run.
	 */
	min = ~0ull;
	for (i = 0; i < 10; ++i) {
		start = rdtsc();
		stop = rdtsc();
		time = stop - start;
		if (time < min) {
			min = time;
		}
	}
	printf("Minimum time for a rdtsc instruction (in clocks): %llu\n", min);

	/* Now time how long the pair of mutex lock + unlock take */
	min = ~0ull;
	for (i = 0; i < 10; ++i) {
		start = rdtsc();
		pthread_mutex_lock(&mutex);
		pthread_mutex_unlock(&mutex);
		stop = rdtsc();
		time = stop - start;
		if (time < min) {
			min = time;
		}
	}
	printf("Minimum time for a mutex lock+unlock + rdtsc (in clocks): %llu\n", min);

	/* Clean up the waiting thread we spawned just to be multithreaded. */
	sem_post(&waiting_thread_semaphore);
	pthread_join(waiting_thread, &trash);
	sem_destroy(&waiting_thread_semaphore);
	return 0;
}