[Caml-list] OCaml 4.03.0 on Minix3 fails threading tests

[Caml-list] OCaml 4.03.0 on Minix3 fails threading tests

[Pierre Métras]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

Hello,

I'm trying to port an *almost* complete OCaml port on Minix3 while learning
this system at the same time, and I'm encountering a small blocker with
threads. Minix3 does not support native pre-emptive threads. So threads
code should be cooperative yielding the control flow regularly. I've linked
the OCaml binaries to the pthread library available in /usr/pkg/lib.

When I run the testsuite, it blocks in deadlocks in a simple test program
like lib-thread/close.ml, where a reading thread is consuming what a
writing thread puts in a pipe. In fact, the reading thread blocks on the
Unix.read system call and the scheduler never gives control back to the
writing thread that should close the pipe. This happens with systhreads
only as the lightweight threads play nicely.

$ ocamlc  -g -vmthread -w a unix.cma threads.cma tests/lib-threads/close.ml
-o ./program
$ ./program
reading...
closing fd...
read returned

$ ocamlopt  -g -thread -w a unix.cmxa threads.cmxa tests/lib-threads/
close.ml -o ./program
10.0.2.15$ ./program
reading...
^C


How to deal with that? What would be the best option?

1. I can try to have a minimal OCaml environment on Minix3 for the moment
and adapt the testsuite Makefiles not to run if threading is not
pre-emptive or to to skip the threading tests. One way to do that is to
have a new define HAVE_THREAD_COOPERATIVE added to OCaml configuration.
This would be useful only in rare situations like Minix3...

2. Because this OCaml version is linked to pthread and the system thinks
that systhreads are working, I could patch the Unix module to make it more
cooperative, adding sched_yield() calls only in the cases of
HAVE_THREAD_COOPERATIVE or __minix defines after blocking system calls.
This seems to me a bad workaround and a source of vicious bugs...

3. Or decide building OCaml environment without pthread on Minix
(./configure -no-pthread) but some modules that I tried to install with
opam, dependencies of utop or core I can't remember, required systhread...

Any advice on how to have the best OCaml environment and the maximum tests
passed?

Thanks
--
Pierre Métras

[-- Attachment #2: Type: text/html, Size: 3073 bytes --]

Re: [Caml-list] OCaml 4.03.0 on Minix3 fails threading tests

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 3310 bytes --]

When I remember correctly, the systhread implementation never worked
for user-space pthread libraries like the one that comes with Minix3.
That's simply because these libraries tend to be very buggy and full of
unresolved race conditions. In particular, OCaml requires that pthreads
and signals play well together, and this is one of those areas that are
probably not correctly implementable at all in user space.

Trying to fix that and find workarounds is very likely to be wasted
time. It could be easier to help package authors making systhreads
support an optional feature.

Gerd

Am Dienstag, den 16.08.2016, 23:09 -0400 schrieb Pierre Métras:
> Hello,
> 
> I'm trying to port an *almost* complete OCaml port on Minix3 while
> learning this system at the same time, and I'm encountering a small
> blocker with threads. Minix3 does not support native pre-emptive
> threads. So threads code should be cooperative yielding the control
> flow regularly. I've linked the OCaml binaries to the pthread library
> available in /usr/pkg/lib.
> 
> When I run the testsuite, it blocks in deadlocks in a simple test
> program like lib-thread/close.ml, where a reading thread is consuming
> what a writing thread puts in a pipe. In fact, the reading thread
> blocks on the Unix.read system call and the scheduler never gives
> control back to the writing thread that should close the pipe. This
> happens with systhreads only as the lightweight threads play nicely.
> 
> $ ocamlc  -g -vmthread -w a unix.cma threads.cma tests/lib-
> threads/close.ml -o ./program
> $ ./program
> reading...
> closing fd...
> read returned
> 
> $ ocamlopt  -g -thread -w a unix.cmxa threads.cmxa tests/lib-
> threads/close.ml -o ./program
> 10.0.2.15$ ./program
> reading...
> ^C
> 
> How to deal with that? What would be the best option?
> 
> 1. I can try to have a minimal OCaml environment on Minix3 for the
> moment and adapt the testsuite Makefiles not to run if threading is
> not pre-emptive or to to skip the threading tests. One way to do that
> is to have a new define HAVE_THREAD_COOPERATIVE added to OCaml
> configuration. This would be useful only in rare situations like
> Minix3...
> 
> 2. Because this OCaml version is linked to pthread and the system
> thinks that systhreads are working, I could patch the Unix module to
> make it more cooperative, adding sched_yield() calls only in the
> cases of HAVE_THREAD_COOPERATIVE or __minix defines after blocking
> system calls. This seems to me a bad workaround and a source of
> vicious bugs...
> 
> 3. Or decide building OCaml environment without pthread on Minix
> (./configure -no-pthread) but some modules that I tried to install
> with opam, dependencies of utop or core I can't remember, required
> systhread...
> 
> Any advice on how to have the best OCaml environment and the maximum
> tests passed?
> 
> Thanks
> --
> Pierre Métras
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]