Re: [Caml-list] Asynchronous IO programming in OCaml

[Goswin von Brederlow <goswin-v-b@web.de>]

Jérémie Dimino <jeremie@dimino.org> writes:

> On Mon, Oct 25, 2010 at 11:34:41AM -0400, Yaron Minsky wrote:
>>    I don't quite understand how this whole benchmark holds together.  Could
>>    you post the C code?  I don't understand the differences between (1), (2)
>>    and (3) well enough to explain where the factor of 100 comes in.
>
> Yes. Here is the code of the first program:
>
> ,----
> | #include <sys/types.h>
> | #include <sys/stat.h>
> | #include <fcntl.h>
> | #include <unistd.h>
> | 
> | int main()
> | {
> |   int fd = open("data", O_RDONLY);
> |   char buffer[4096];
> | 
> |   while (read(fd, buffer, 4096) > 0);
> | 
> |   close(fd);
> | 
> |   return 0;
> | }
> `----

Obvious example so nothing to comment. :)

> the code of the second:
>
> ,----
> | #include <sys/types.h>
> | #include <sys/stat.h>
> | #include <fcntl.h>
> | #include <unistd.h>
> | #include <pthread.h>
> | 
> | int fd;
> | char buffer[4096];
> | int done = 0;
> | 
> | void *callback(void* data)
> | {
> |   int count = read(fd, buffer, 4096);
> |   if (count == 0) done = 1;
> |   return NULL;
> | }
> | 
> | int main()
> | {
> |   fd = open("data", O_RDONLY);
> | 
> |   while (!done) {
> |     pthread_t thread;
> |     pthread_create(&thread, NULL, callback, NULL);
> |     pthread_join(thread, NULL);
> |   }
> | 
> |   close(fd);
> | 
> |   return 0;
> | }
> `----

You aren't doing any multithreading. You are creating a thread and
waiting for the thread to finish its read before strating a second.
There are never ever 2 reads running in parallel. So all you do is add
thread creation and destruction for every read to your first example.

You should start multiple threads and let them read from different
offsets (use pread) and only once they are all started join them all
again.

> and the third:
>
> ,----
> | #include <sys/types.h>
> | #include <sys/stat.h>
> | #include <fcntl.h>
> | #include <unistd.h>
> | #include <pthread.h>
> | 
> | int fd;
> | char buffer[4096];
> | int done = 0;
> | pthread_cond_t start = PTHREAD_COND_INITIALIZER;
> | pthread_cond_t stop = PTHREAD_COND_INITIALIZER;
> | pthread_mutex_t start_mutex = PTHREAD_MUTEX_INITIALIZER;
> | pthread_mutex_t stop_mutex = PTHREAD_MUTEX_INITIALIZER;
> | 
> | void *callback(void* data)
> | {
> |   while (!done) {
> |     pthread_cond_wait(&start, &start_mutex);
> | 
> |     int count = read(fd, buffer, 4096);
> |     if (count == 0) done = 1;
> | 
> |     pthread_mutex_lock(&stop_mutex);
> |     pthread_cond_signal(&stop);
> |     pthread_mutex_unlock(&stop_mutex);
> |   }
> |   return NULL;
> | }
> | 
> | int main()
> | {
> |   fd = open("data", O_RDONLY);
> | 
> |   pthread_cond_init(&start, NULL);
> |   pthread_cond_init(&stop, NULL);
> | 
> |   pthread_mutex_lock(&start_mutex);
> |   pthread_mutex_lock(&stop_mutex);
> | 
> |   pthread_t thread;
> |   pthread_create(&thread, NULL, callback, NULL);
> | 
> |   while (!done) {
> |     pthread_mutex_lock(&start_mutex);
> |     pthread_cond_signal(&start);
> |     pthread_mutex_unlock(&start_mutex);
> | 
> |     pthread_cond_wait(&stop, &stop_mutex);
> |   }
> | 
> |   pthread_join(thread, NULL);
> |   close(fd);
> | 
> |   return 0;
> | }
> `----

Again no parallelism at all. Instead of thread creation and destruction
you now add mutexes between the reads. Again the only expected result is
a slowdown.

You should create X threads at the start and then repeadately give them
work (an offset to read). Only when they are all busy you wait for
one of them to become idle again.

> Jérémie


So far you have failed to do asynchronous IO. You examples all wait for
the read to complete making it synchronous and then threads are
obviously slower.

Also sequential reads from a file should result in sequential reads from
the disk (unless the filesystem fragments the file a lot or you created
it that way). That is the ideal situation for the disks and kernels
read-ahead. One advantage of doing many read/writes asynchronous is that
the kernel can reorder the requests and potentially merge
requests. Unless you crafted your input file to be fragmented you won't
see this effect in a test with sequential reads. And that effect would
be the only thing making multithreaded reads faster in a test like the
above.

MfG
        Goswin