Question about ocaml threads and TLS (on linux)

Question about ocaml threads and TLS (on linux)

[Goswin von Brederlow]

Hi,

I'm having a little problem for my libfuse-ocaml bindings for the
threaded interface. For those that don't want to read all of the mail my
question is:

Will every ocaml thread have its own thread-local-storage in the C stubs?


I have the following calling sequence:

User ocaml code   | Fuse C stub              | libfuse code
------------------+--------------------------+-------------
Fuse.process fs   > 'process stub'           |
                  | enter_blocking_section() |
                  | char *buf = malloc(size) |
                  | fuse_session_process()   >
                  |                          < ops->write(buf+off)
                  | 'write stub'             |
                  | leave_blocking_section() |
                  | a = caml_ba_alloc_dims() |
                  < caml_callback(...,a,...) |
my_ref := a       >                          |
                  | enter_blocking_section() >
                  |                          < callback done
                  | 'process stub'           |
                  | free(buf)                |
                  < leave_blocking_section() |
Fuse.process done |


The 'process stub' allocates a buffer and frees it at the end, which is
usualy fine. Except in the case of a write callback where the buffer is
passed back to ocaml as Bigarray. If the Bigarray is copied, like above,
then the ocaml code still has a reference to the data at the point the
'process stub' wants to free it.

To solve that problem I need the write callback to signal that the
buffer was passed to ocaml and is now under GC control. The buffer must
not be free()ed by the 'process stub'. The libfuse API does not provide
for this so I have to somehow communicate between 'process stub' and
'write stub' around the libfuse code.


Possible solution:
------------------

__thread char *buf = NULL;

value ocaml_fuse_process(...) {
  buf = malloc(size);
  fuse_session_process()
  if (buf != NULL) free(buf);
}

void write_callback(...) {
  a = caml_ba_alloc_dims(...);
  buf = NULL;
}


This way ocaml_fuse_process will allocate a new buffer whenever it
doesn't have one and the write_callback will take over the buffer and
give it to the GC.


Now my question is: Does that work? Is it safe? Will every ocaml thread
have its own thread-local-storage buf?

Currently I'm only interested in supporting Linux. If it is safe there
that is enough.

MfG
        Goswin

Re: [Caml-list] Question about ocaml threads and TLS (on linux)

[Philippe Wang]

Hi,

I'm not sure I understand (though I've read the whole text), but maybe
this will answer your question:
On Linux, OCaml threads (with the native compiler "ocamlopt") are
implemented with POSIX threads (in C), so when your OCaml thread runs
the C stub, it's the same as if you were running the C stub in some C
thread.
When you are in a section declared as a blocking section, a collection
can be triggered concurrently in another thread and so the heap must
not be accessed neither for reading or writing, that's all.

Using __thread recent feature should also work if you manage to
compile everything correctly. Notably, we use it in some places in
ocaml4multicore (a patch to ocaml's runtime library to allow parallel
threads). However, I don't know how __thread is handled by the
compiler... I mean : it there a pointer for buf in every thread or
only in those that use it?

I hope my answer isn't useless!

Cheers,

-- 
Philippe Wang
   mail@philippewang.info



On Wed, Feb 24, 2010 at 10:00 PM, Goswin von Brederlow
<goswin-v-b@web.de> wrote:
> Hi,
>
> I'm having a little problem for my libfuse-ocaml bindings for the
> threaded interface. For those that don't want to read all of the mail my
> question is:
>
> Will every ocaml thread have its own thread-local-storage in the C stubs?
>
>
> I have the following calling sequence:
>
> User ocaml code   | Fuse C stub              | libfuse code
> ------------------+--------------------------+-------------
> Fuse.process fs   > 'process stub'           |
>                  | enter_blocking_section() |
>                  | char *buf = malloc(size) |
>                  | fuse_session_process()   >
>                  |                          < ops->write(buf+off)
>                  | 'write stub'             |
>                  | leave_blocking_section() |
>                  | a = caml_ba_alloc_dims() |
>                  < caml_callback(...,a,...) |
> my_ref := a       >                          |
>                  | enter_blocking_section() >
>                  |                          < callback done
>                  | 'process stub'           |
>                  | free(buf)                |
>                  < leave_blocking_section() |
> Fuse.process done |
>
>
> The 'process stub' allocates a buffer and frees it at the end, which is
> usualy fine. Except in the case of a write callback where the buffer is
> passed back to ocaml as Bigarray. If the Bigarray is copied, like above,
> then the ocaml code still has a reference to the data at the point the
> 'process stub' wants to free it.
>
> To solve that problem I need the write callback to signal that the
> buffer was passed to ocaml and is now under GC control. The buffer must
> not be free()ed by the 'process stub'. The libfuse API does not provide
> for this so I have to somehow communicate between 'process stub' and
> 'write stub' around the libfuse code.
>
>
> Possible solution:
> ------------------
>
> __thread char *buf = NULL;
>
> value ocaml_fuse_process(...) {
>  buf = malloc(size);
>  fuse_session_process()
>  if (buf != NULL) free(buf);
> }
>
> void write_callback(...) {
>  a = caml_ba_alloc_dims(...);
>  buf = NULL;
> }
>
>
> This way ocaml_fuse_process will allocate a new buffer whenever it
> doesn't have one and the write_callback will take over the buffer and
> give it to the GC.
>
>
> Now my question is: Does that work? Is it safe? Will every ocaml thread
> have its own thread-local-storage buf?
>
> Currently I'm only interested in supporting Linux. If it is safe there
> that is enough.
>
> MfG
>        Goswin

Re: [Caml-list] Question about ocaml threads and TLS (on linux)

[Goswin von Brederlow]

Philippe Wang <philippe.wang.lists@gmail.com> writes:

> Hi,
>
> I'm not sure I understand (though I've read the whole text), but maybe
> this will answer your question:
> On Linux, OCaml threads (with the native compiler "ocamlopt") are
> implemented with POSIX threads (in C), so when your OCaml thread runs
> the C stub, it's the same as if you were running the C stub in some C
> thread.

How about bytecode?

> When you are in a section declared as a blocking section, a collection
> can be triggered concurrently in another thread and so the heap must
> not be accessed neither for reading or writing, that's all.
>
> Using __thread recent feature should also work if you manage to
> compile everything correctly. Notably, we use it in some places in
> ocaml4multicore (a patch to ocaml's runtime library to allow parallel
> threads). However, I don't know how __thread is handled by the
> compiler... I mean : it there a pointer for buf in every thread or
> only in those that use it?

Only in those that use it. But that will be multiple threads. In those
that don't use it 4/8 bytes will be wasted.

> I hope my answer isn't useless!
>
> Cheers,

MfG
        Goswin