caml-list - the Caml user's mailing list
 help / color / mirror / Atom feed
* [Caml-list] C, threads, callbacks, and corrupted local_roots
@ 2019-06-20 18:03 rixed
  2019-06-21  7:30 ` Sébastien Hinderer
  0 siblings, 1 reply; 6+ messages in thread
From: rixed @ 2019-06-20 18:03 UTC (permalink / raw)
  To: caml-list

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

Hello !

I have a programs which entry point is in C++ and that spawn a few threads, some of which are calling back some OCaml code. 
My understanding is that for this to work, one must only follow those rules (in addition to the usual "how to leave in peace with the GC" rules:

1. call caml_startup(argv) before any OCaml code or any other function from the OCaml runtime is run;

2. for each thread that will eventually run some OCaml code (anything that could call the GC (and also, possibly, do anything with signals)), call caml_c_thread_register before;

3. before a thread terminates and another thread runs the OCaml GC, call (from that terminating thread) caml_c_thread_unregister (it is not clear exactly why to be honest, as a finished thread should have no root left)

What is a bit unclear is how thread registering and caml_startup are supposed to deal with the giant lock. From testing and
reading the code, it seems that caml_c_thread_(un)register expects the giant lock to be released, which caml_startup does not do, so calling caml_release_runtime_system right after caml_startup seems in order (and therefore reacquiring it before calling back to OCaml).

Neither is it clear to me how to declare local roots in the function registering the thread (more specifically: should caml_c_thread_register be called first, then a new block created and CAMLparam and friends be called now that the current root list is initialized?); But in the exemple below it does not matter as there are no values requiring such protection.

Yet, whatever I try, I get a crash in the Gc when it's scaninng the roots in caml_do_local_roots, with a corrupted local_root pointer that is suspiciously small (few hundred bytes, frequently but not always just 0xfff). I am not certain about those values though, as I'm not using a debug runtime (opam had such debug variants for a couple of old ocaml versions but not any longer it seems)

Thankfully, the attached minimal example dies in the exact same circumstances.

Do anyone know what I am doing wrong?

[-- Attachment #2: Makefile --]
[-- Type: application/octet-stream, Size: 424 bytes --]

OCAMLLIB = $(shell ocamlfind ocamlc -where)
CFLAGS += -g -O0 -Wall -W -fPIC -fstandalone-debug
CPPFLAGS += -I $(OCAMLLIB)
LDFLAGS += -L$(OCAMLLIB) -lunix -lcamlstr -lthreads -lasmrun_shared

all: b

b: a.o b.o

a.cmx: a.ml
	ocamlfind ocamlopt -thread -package threads -c $< -o $@

a.o: a.cmx
	ocamlfind ocamlopt -thread -package threads -linkpkg $< -output-obj -o a.tmp.o
	mv a.tmp.o $@

clean:
	$(RM) *.o *.cmx *.cmi b *.s

[-- Attachment #3: a.ml --]
[-- Type: application/octet-stream, Size: 217 bytes --]

let call_me_from_c () =
  Printf.printf "Will I survive?\n%!" ;
  Gc.compact () ;
  Printf.printf "I survived!\n%!"

let init =
  ignore (Thread.self ()) ;
  ignore (Callback.register "call_me_from_c" call_me_from_c)

[-- Attachment #4: b.c --]
[-- Type: application/octet-stream, Size: 704 bytes --]

#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
#include <caml/mlvalues.h>
#include <caml/startup.h>
#include <caml/callback.h>
#include <caml/threads.h>

static void *another_thread(void *wtv)
{
  (void)wtv;
  caml_c_thread_register();
  printf("in another_thread\n");
  caml_acquire_runtime_system();
  value *cb = caml_named_value("call_me_from_c");
  caml_callback(*cb, Val_unit);
  caml_release_runtime_system();
  caml_c_thread_unregister();
  return NULL;
}

int main(int argc, char *argv[])
{
  (void)argc;
  caml_startup(argv);
  caml_release_runtime_system();

  pthread_t th;
  (void)pthread_create(&th, NULL, another_thread, NULL);

  (void)pthread_join(th, NULL);

  return 0;
}

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Caml-list] C, threads, callbacks, and corrupted local_roots
  2019-06-20 18:03 [Caml-list] C, threads, callbacks, and corrupted local_roots rixed
@ 2019-06-21  7:30 ` Sébastien Hinderer
  2019-06-21 10:51   ` rixed
  0 siblings, 1 reply; 6+ messages in thread
From: Sébastien Hinderer @ 2019-06-21  7:30 UTC (permalink / raw)
  To: caml-list

Hi Cédric,

I'm sorry I can't help with your main question, butt re:the debug
runtime, at least upstream it is now enabled by default so I would
expect opam to provide it with the compiler for recent versions, rather
than in a separate package.

hth,

Sébastien.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Caml-list] C, threads, callbacks, and corrupted local_roots
  2019-06-21  7:30 ` Sébastien Hinderer
@ 2019-06-21 10:51   ` rixed
  2019-06-21 11:38     ` Sébastien Hinderer
  0 siblings, 1 reply; 6+ messages in thread
From: rixed @ 2019-06-21 10:51 UTC (permalink / raw)
  To: caml-list

> re:the debug
> runtime, at least upstream it is now enabled by default so I would
> expect opam to provide it with the compiler for recent versions, rather
> than in a separate package.

This makes sense.
Does it mean that the INSTALL.doc, which instruct to manually change the compilation flag and rebuild (see https://github.com/ocaml/ocaml/blob/trunk/INSTALL.adoc#if-something-goes-wrong) is not up to date?

I guess this is the difference between libasmrun and libasmrund (I indeed have both installed), although objdump shows debug sections in both.
Indeed, when I compile without -fPIC and link with libasmrund on Linux it seems to run in some debug mode; ie. it does print additional debug strings, but there are no more debug symbols and it seems to be compiled with optimisations on all the same. Also, it crash at the very same line with the very same 0xfff in local root address.

Under macos I can;t link with libasmrun nor libasmrund as the linker then complains about missing symbols such as _caml_extern_sp. That's why I've been linking with libasmrun_shared in the first place. This so file does not seem to come with a debug version.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Caml-list] C, threads, callbacks, and corrupted local_roots
  2019-06-21 10:51   ` rixed
@ 2019-06-21 11:38     ` Sébastien Hinderer
  2019-06-21 12:21       ` rixed
  0 siblings, 1 reply; 6+ messages in thread
From: Sébastien Hinderer @ 2019-06-21 11:38 UTC (permalink / raw)
  To: caml-list

Hi again, Cédric,

> This makes sense.
> Does it mean that the INSTALL.doc, which instruct to manually change
> the compilation flag and rebuild (see
> https://github.com/ocaml/ocaml/blob/trunk/INSTALL.adoc#if-something-goes-wrong)
> is not up to date?

Yes, you are perfectly right, very sorry about that and many thanks for
having pointed this out.

I have just submitted https://github.com/ocaml/ocaml/pull/8759 as an
attempt to improve the situation.

> I guess this is the difference between libasmrun and libasmrund

Yes and also between ocamlrun and ocamlrund

> (I indeed have both installed), although objdump shows debug sections
> in both.

The difference is not so much in the debug symbols, rather in the fact
that, as the INSTALL files rightly explains, the debug version contains
assertions and sanity checks that the valinna versioon does not contain.

> Indeed, when I compile without -fPIC and link with libasmrund on Linux
> it seems to run in some debug mode; ie. it does print additional debug
> strings, but there are no more debug symbols and it seems to be
> compiled with optimisations on all the same. Also, it crash at the
> very same line with the very same 0xfff in local root address.

Why did you try to link rather than using the runtime's executable,
actually?

Best wishes,

Sébastien.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Caml-list] C, threads, callbacks, and corrupted local_roots
  2019-06-21 11:38     ` Sébastien Hinderer
@ 2019-06-21 12:21       ` rixed
  2019-06-21 14:00         ` Sébastien Hinderer
  0 siblings, 1 reply; 6+ messages in thread
From: rixed @ 2019-06-21 12:21 UTC (permalink / raw)
  To: caml-list



On Fri, Jun 21, 2019, at 13:38, Sébastien Hinderer wrote:
> Why did you try to link rather than using the runtime's executable,
> actually?

Not sure I understand this question. By runtime's executable do you mean ocamlrun(d) ?
I'm using the native compiler, and my program entry point is in C (this is mandated by a library I'm using).

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Caml-list] C, threads, callbacks, and corrupted local_roots
  2019-06-21 12:21       ` rixed
@ 2019-06-21 14:00         ` Sébastien Hinderer
  0 siblings, 0 replies; 6+ messages in thread
From: Sébastien Hinderer @ 2019-06-21 14:00 UTC (permalink / raw)
  To: caml-list

rixed@happyleptic.org (2019/06/21 14:21 +0200):
> 
> 
> On Fri, Jun 21, 2019, at 13:38, Sébastien Hinderer wrote:
> > Why did you try to link rather than using the runtime's executable,
> > actually?
> 
> Not sure I understand this question. By runtime's executable do you mean ocamlrun(d) ?
> I'm using the native compiler, and my program entry point is in C
> (this is mandated by a library I'm using).

Oops sorry, the question was indeed not relevant.

Sébastien.

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2019-06-21 14:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-20 18:03 [Caml-list] C, threads, callbacks, and corrupted local_roots rixed
2019-06-21  7:30 ` Sébastien Hinderer
2019-06-21 10:51   ` rixed
2019-06-21 11:38     ` Sébastien Hinderer
2019-06-21 12:21       ` rixed
2019-06-21 14:00         ` Sébastien Hinderer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).