Threads & Fork

Threads & Fork

[Jonathan Bryant]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

I'm confused as to why the attached code hangs.  My understanding of
Unix.fork () is that it completely clones the current process, which in
my understanding, clones the processes's threads as well.  Apparently,
though, that is not the case, because I can't join the thread in both
the parent and the child.  Are threads created external to the process? 
I've played around with the source code and I've found that I can only
join the thread in the parent process (i.e., if I remove the Thread.join
from the parent process, it still hangs).

Can someone explain this behavior?  I'm needing to create several
multi-threaded worker processes, and it would be much easier if I could
simply create one, have it wait for a signal, and, on that signal, fork
a clone of itself.  Also, I'm needing to fork a process from in the
middle of a multi-threaded application, so I assumed that my new process
needed to join all of the threads and close all of the file
descriptors.  Is this not the case?

-- 
--Jonathan Bryant
  jtbryant@valdosta.edu
  Student Intern
  Unix System Operations
  VSU Information Technology

"Das Leben ohne Music ist einfach ein Irrtum, eine Strapaze, ein" Exil."
("Life without music is simply an error, a pain, an exile.")
--Frederich Nietzsche

"The three cardinal values of a programmer are laziness, impatience, and
hubris."
--Perl Man Page



[-- Attachment #2: threadTest.ml --]
[-- Type: text/plain, Size: 944 bytes --]

(**
 * A simple thread + fork test.  Compile as:
 * ocamlopt.opt -threads -o threadTest unix.cmxa threads.cmxa threadTest.ml
 *)

let () =
    (* Create a thread to join *)
    let t = Thread.create (fun x -> ()) () in
    Printf.printf "Thread created.\n"; flush stdout;

    (* Fork a new process *)
    match Unix.fork () with
    | 0 ->
        (* Join the thread in the child process and exit *)
        Printf.printf "Child process created.\n"; flush stdout;
        Thread.join t;
        Printf.printf "Child joined thread.\n"; flush stdout;
        exit 0
    | _ as pid ->
        (* Join the thread and the child in the parent process and exit *)
        Printf.printf "Parent process continuing.\n"; flush stdout;
        Thread.join t;
        Printf.printf "Parent joined thread.\nWaiting on child.\n";
        flush stdout;
        let _ = Unix.waitpid [] pid in
        Printf.printf "Child joined.  Exiting.\n";
        exit 0

Re: [Caml-list] Threads & Fork

[Oliver Bandel]

Well,

I have not the answer to your question directly,
but:
I looked into the documentation of Unix.fork.
There is nothing mentioned about an exception,
when fork() mail fails.
But a fork() can fail.

How is this handled?

Ciao,
   Oliver

Re: [Caml-list] Threads & Fork

[Florian Weimer]

* Jonathan Bryant:

> I'm confused as to why the attached code hangs.  My understanding of
> Unix.fork () is that it completely clones the current process, which in
> my understanding, clones the processes's threads as well.  Apparently,
> though, that is not the case, because I can't join the thread in both
> the parent and the child.

I can't speak for the OCaml run-time, but POSIX fork only duplicates
the current thread, so the new process is essentially single-threaded.

Re: [Caml-list] Threads & Fork

[Oliver Bandel]

On Tue, Nov 22, 2005 at 04:39:07PM +0100, Florian Weimer wrote:
> * Jonathan Bryant:
> 
> > I'm confused as to why the attached code hangs.  My understanding of
> > Unix.fork () is that it completely clones the current process, which in
> > my understanding, clones the processes's threads as well.  Apparently,
> > though, that is not the case, because I can't join the thread in both
> > the parent and the child.
> 
> I can't speak for the OCaml run-time, but POSIX fork only duplicates
> the current thread, so the new process is essentially single-threaded.

I doubt that this is true.
Unix-fork() copies a complete process.
If the original has threads, the copy also have.
I don't think that POSIX handles this different to old Unix API.

Unix-module should do the same as the C-call of fork(2).

Threads are running inside a process, like any other
function. So all should be copied completely.

But using POSIX-threads and Unix-API can yield to many problems,
for example POSIX threading signals and Unix-signals are clashing
together and the behaviour can be undefined...

Ciao,
   Oliver

Re: [Caml-list] Threads & Fork

[David Teller]

On Tue, 2005-11-22 at 16:50 +0100, Oliver Bandel wrote:
> I doubt that this is true.
> Unix-fork() copies a complete process.
> If the original has threads, the copy also have.
> I don't think that POSIX handles this different to old Unix API.

Er... that would depend on the implementation of threads. Some thread
libraries rely on external processes and shared memory. I don't know how
ThreadUnix works to provide non-blocking system calls, but that might be
one source of the problem.

Cheers,
 David

Re: [Caml-list] Threads & Fork

[Alexander S. Usov]

On Tuesday 22 November 2005 16:50, Oliver Bandel wrote:
> On Tue, Nov 22, 2005 at 04:39:07PM +0100, Florian Weimer wrote:
> > * Jonathan Bryant:
> > > I'm confused as to why the attached code hangs.  My understanding of
> > > Unix.fork () is that it completely clones the current process, which in
> > > my understanding, clones the processes's threads as well.  Apparently,
> > > though, that is not the case, because I can't join the thread in both
> > > the parent and the child.
> >
> > I can't speak for the OCaml run-time, but POSIX fork only duplicates
> > the current thread, so the new process is essentially single-threaded.
>
> I doubt that this is true.
> Unix-fork() copies a complete process.
> If the original has threads, the copy also have.
> I don't think that POSIX handles this different to old Unix API.
>
> Unix-module should do the same as the C-call of fork(2).
>
> Threads are running inside a process, like any other
> function. So all should be copied completely.
>
> But using POSIX-threads and Unix-API can yield to many problems,
> for example POSIX threading signals and Unix-signals are clashing
> together and the behaviour can be undefined...

[Option End]A process shall be created with a single thread. If a 
multi-threaded process calls fork(), the new process shall contain a replica 
of the calling thread and its entire address space, possibly including the 
states of mutexes and other resources. Consequently, to avoid errors, the 
child process may only execute async-signal-safe operations until such time 
as one of the exec functions is called. [THR] [Option Start]  Fork handlers 
may be established by means of the pthread_atfork() function in order to 
maintain application invariants across fork() calls. [Option End]

(c) SUS v3

-- 
Best regards,
  Alexander.

Re: [Caml-list] Threads & Fork

[Oliver Bandel]

On Tue, Nov 22, 2005 at 06:32:55PM +0100, Alexander S. Usov wrote:
> On Tuesday 22 November 2005 16:50, Oliver Bandel wrote:
> > On Tue, Nov 22, 2005 at 04:39:07PM +0100, Florian Weimer wrote:
> > > * Jonathan Bryant:
> > > > I'm confused as to why the attached code hangs.  My understanding of
> > > > Unix.fork () is that it completely clones the current process, which in
> > > > my understanding, clones the processes's threads as well.  Apparently,
> > > > though, that is not the case, because I can't join the thread in both
> > > > the parent and the child.
> > >
> > > I can't speak for the OCaml run-time, but POSIX fork only duplicates
> > > the current thread, so the new process is essentially single-threaded.
> >
> > I doubt that this is true.
> > Unix-fork() copies a complete process.
> > If the original has threads, the copy also have.
> > I don't think that POSIX handles this different to old Unix API.
> >
> > Unix-module should do the same as the C-call of fork(2).
> >
> > Threads are running inside a process, like any other
> > function. So all should be copied completely.
> >
> > But using POSIX-threads and Unix-API can yield to many problems,
> > for example POSIX threading signals and Unix-signals are clashing
> > together and the behaviour can be undefined...
> 
> [Option End]A process shall be created with a single thread. If a 
> multi-threaded process calls fork(), the new process shall contain a replica 
> of the calling thread and its entire address space, possibly including the 
> states of mutexes and other resources. Consequently, to avoid errors, the 
> child process may only execute async-signal-safe operations until such time 
> as one of the exec functions is called. [THR] [Option Start]  Fork handlers 
> may be established by means of the pthread_atfork() function in order to 
> maintain application invariants across fork() calls. [Option End]
> 
> (c) SUS v3

OK, I looked into my literature...

    "Pthreads specifies that only the thread calling fork exists in the child."
                       (David R. Butenhof: Programming with POSIX-Threads, p. 197)

OK, so I was wrong with my doubting... ;-)

But... the other threads' states in the child are the same as in the
parent.... and other things are also very.... strange.
Mutexes and other things are living inb the child, but
threads that possibly would work with them don't exist...
...there is no cleanup done...
...well.. so better don't use it this way.
So Butenhof said the same as you found in SUS v3 .... 

Ciao,
   Oliver

Re: [Caml-list] Threads & Fork

[Matthew Hannigan]

On Tue, Nov 22, 2005 at 04:50:02PM +0100, Oliver Bandel wrote:
> On Tue, Nov 22, 2005 at 04:39:07PM +0100, Florian Weimer wrote:
> > * Jonathan Bryant:
> > 
> > > I'm confused as to why the attached code hangs.  My understanding of
> > > Unix.fork () is that it completely clones the current process, which in
> > > my understanding, clones the processes's threads as well.  Apparently,
> > > though, that is not the case, because I can't join the thread in both
> > > the parent and the child.
> > 
> > I can't speak for the OCaml run-time, but POSIX fork only duplicates
> > the current thread, so the new process is essentially single-threaded.
> 
> I doubt that this is true.
> Unix-fork() copies a complete process.
> If the original has threads, the copy also have.
> I don't think that POSIX handles this different to old Unix API.

from the solaris10 man page for fork; note the last sentence.


     A call to fork1()  replicates  only  the
     calling thread in the child process.

     In Solaris 10, a call to fork() is identical to  a  call  to
     fork1();  only the calling thread is replicated in the child
     process. This is the POSIX-specified behavior for fork().


--
Matt

Re: [Caml-list] Threads & Fork

[Christophe Raffalli]

I do not recall ... but this may be related to

PTHREAD_SCOPE_SYSTEM
and
PTHREAD_SCOPE_PROCESS

attribute or

PTHREAD_PROCESS_PRIVATE
and
PTHREAD_PROCESS_SHARED

look at the posix specif of pthread and try to ajust this parameters
from a C function call.



-- 
Christophe Raffalli
Université de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tél: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christophe.Raffalli@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution
can check this signature. The public key is
stored on www.keyserver.net
---------------------------------------------

Re: [Caml-list] Threads & Fork

[Oliver Bandel]

  "Avoid using fork in a threaded program (if you can)
   unless you intend to exec a new program immediately."

          (David R. Butenhof: Programming with POSIX Threads, p. 197)


So better first do the fork, then the threading-stuff...


Ciao,
   Oliver