[Caml-list] threads

[Caml-list] threads

[Francois Rouaix]

Is anyone using native threads in Ocaml 3.04 intensively on Linux these
days ?
I'm playing with a Gnutella servent, and I'm experimenting crashes
(bytecode and native).
I'm using the unix library heavily. So far, I'm aware that the DNS calls
are not reentrant,
but the rest of the library calls should be okay.

My "crashes" are weird... There is no core (ulimit -c unlimited, before
you ask).
The program "hangs", ps looks like this. Looks like a thread is stuck.

18964 pts/3    S      0:00 ./camella.opt
18965 pts/3    Z      0:00 [camella.opt <defunct>]
18967 pts/3    S      0:00 ./camella.opt

$strace -p 18964
Hangs
$strace -p 18967
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)

My log output on stderr ends abruptly, with no obvious trace of error
(and I log a lot, and I catch stray exceptions everywhere..)

Any hint or suggestion on how to track the problem would be
appreciated...

--f
François Rouaix



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] threads

[Remi VANICAT]

"Francois Rouaix" <frouaix@yahoo.com> writes:

> Is anyone using native threads in Ocaml 3.04 intensively on Linux these
> days ?
> I'm playing with a Gnutella servent, and I'm experimenting crashes
> (bytecode and native).
> I'm using the unix library heavily. So far, I'm aware that the DNS calls
> are not reentrant,
> but the rest of the library calls should be okay.
> 
> My "crashes" are weird... There is no core (ulimit -c unlimited, before
> you ask).
> The program "hangs", ps looks like this. Looks like a thread is stuck.
> 

Are you using lock ? (you should). This look like a deadlock, you
should log all you locking to see if the problem is there.

> 18964 pts/3    S      0:00 ./camella.opt
> 18965 pts/3    Z      0:00 [camella.opt <defunct>]

mmm, there is a dead thread. May be it has lock something before dying
with out releasing it...
-- 
Rémi Vanicat
vanicat@labri.u-bordeaux.fr
http://dept-info.labri.u-bordeaux.fr/~vanicat
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] threads

[Sylvain LE GALL]

On Tue, Mar 19, 2002 at 06:59:38PM -0800, Francois Rouaix wrote:
> Is anyone using native threads in Ocaml 3.04 intensively on Linux these
> days ?
> I'm playing with a Gnutella servent, and I'm experimenting crashes
> (bytecode and native).
> I'm using the unix library heavily. So far, I'm aware that the DNS calls
> are not reentrant,
> but the rest of the library calls should be okay.
> 

Hello,

I have tried this days to play with thread and mutex and condition.
Very funny but very messy. I do not use native thread but the problem is
that each thread is a process and when you exit because of a problem the
main thread if the child thread is detached, you can't make it ending. A
lot of problem also with signal handling ( a simple CtrL-C ). Each
thread catch it on is own... Not very simple. Of course i'm pretty sure
thread is very useful in certain situation but i think you can think
about using a non threaded process ( convert my program to this using
state for each object and a select on the different socket ... it has speed up the program ).

Thread have cause me also a lot of crash and non ending thread. I have
also a very non caml error : Segmentation Fault ( i do believe this can
not exist in Caml but now ... )

Sylvain LE GALL.

(of course if you intersted in getting your code in a non threaded
version i give you my solution... i know that all people in the list
will laugh at it because i am nearby a beginner :

type tcp_client_state = 
	TCP_Waiting 
	| TCP_Connected
	| TCP_Stopped


in my object 

	method run =
		match self#acq_state with
		TCP_Waiting -> self#waiting
		TCP_Connected -> self#connected

in the parent 

	method run =
	let (ready,_,_) = Unix.select self#list_socket [] [] delay_select
	in
	(* si les données dispo on réveille l'objet correspondant *)
	(* pour le serveur *)
		try
		while true do
			self#add_sortant
		done
		with Unix.Unix_error(Unix.EAGAIN,_,_) -> ()
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] threads

[Xavier Leroy]

> Is anyone using native threads in Ocaml 3.04 intensively on Linux these
> days ?

We occasionally get bug reports similar to yours, under Linux.  It's
hard to say whether it's a bug in the OCaml system thread library, or
in LinuxThreads itself.

> The program "hangs", ps looks like this. Looks like a thread is stuck.
> 
> 18964 pts/3    S      0:00 ./camella.opt
> 18965 pts/3    Z      0:00 [camella.opt <defunct>]
> 18967 pts/3    S      0:00 ./camella.opt

Actually, it looks like the thread manager thread (PID N+1, where N is
the initial PID of the program) died.  It should not, and it is not
surprising that the program hangs, since once the thread manager is
gone the program can no longer create new threads.

> I'm playing with a Gnutella servent, and I'm experimenting crashes
> (bytecode and native).
> I'm using the unix library heavily. So far, I'm aware that the DNS calls
> are not reentrant, but the rest of the library calls should be okay.

Yes, that's another issue: OCaml allows several threads to call
Unix.gethostbyname concurrently (without holding the master mutex),
yet most if not all implementations of gethostbyname() are not
reentrant.

As Nikolaj Bjorner mentioned, some systems provide a reentrant
gethostbyname_r() function, not necessarily with the same interface,
though, making it very hard to use in the Unix library.

(An easy way to see if your problems are related to gethostbyname is
to protect calls to Unix.gethostbyname by a mutex, to ensure that only
one at a time is performed.)

- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] threads

[ygrek]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

On Wed, 9 Sep 2009 01:20:06 +0200
Philippe Wang <philippe.wang.lists@gmail.com> wrote:

> >  let x = Array.make 100 []
> >  let update i n = x.(i) <- n :: x.(i)
> >  let read i = x.(i)
> 
> I don't think you can obtain funny results when you don't put a mutex
> on these two specific "update" and "read".
> What is sure is that "update" function is not atomic because you have
> a value allocation at the right of "<-" (with :: operator), and this
> may trigger garbage collection and/or make the scheduler change the
> running thread.

Yes, that's not a problem.

Thinking that over again it looks like the only issue remaining is the
possibility that compiler can be too smart and cache/reorder memory
operations, but ocamlopt is not of that kind, right?

-- 
 ygrek
 http://ygrek.org.ua

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [Caml-list] threads

[Philippe Wang]

Hi,

>  let x = Array.make 100 []
>  let update i n = x.(i) <- n :: x.(i)
>  let read i = x.(i)

I don't think you can obtain funny results when you don't put a mutex
on these two specific "update" and "read".
What is sure is that "update" function is not atomic because you have
a value allocation at the right of "<-" (with :: operator), and this
may trigger garbage collection and/or make the scheduler change the
running thread.

What you can be sure with the current official OCaml distribution is
that there won't be at the exact same time both an (<-)operation and a
(.())operation.
But it is actually possible, for instance, for a thread to compute
while another one is simultaneously writing on a socket. So it is
generally not a good idea to count on some operation atomicity to put
or not a mutex lock (well it's good to write some hard-to-debug
code)...

Cheers,

Philippe Wang


On Tue, Sep 8, 2009 at 7:33 PM, ygrek <ygrekheretix@gmail.com> wrote:
> Hello,
>
>  let x = Array.make 100 []
>  let update i n = x.(i) <- n :: x.(i)
>  let read i = x.(i)
>
>  Consider the following scenario: one thread is `update`ing x, another
> thread(s) uses only `read`. Is it safe to use these functions without
> locking on mutex?
>
>  I.e. is Array.set atomic? What about updating references (:=) ?
>
>  If I understand correctly these operations require only one cpu
> instruction to update one machine word and so should be atomic. Taking
> into account "single-cpu affinity" of ocaml program it should be safe
> to write such multithreaded code. Is it true?
>
>  Is it safe to assume that ocamlopt won't skip reads/writes to globally
> visible memory address using cached value in a register?
>
> --
>  ygrek
>  http://ygrek.org.ua
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>



-- 
Philippe Wang
   mail@philippewang.info

Re: [Caml-list] Threads

[Christophe Raffalli]

james woodyatt a écrit :

> On 11 Jun 2005, at 00:46, Christophe Raffalli wrote:
>
>>
>> No there is a global mutex for caml thread. The only thing you can do 
>> is have one Caml thread and many C threads running in the same time.
>>
>> This starts really to be an annoying limitation of the Caml runtime 
>> now that you can have easily two dual core processors in one machine 
>> ... (means four CPU)
>
>
> Not for me it isn't-- I gave up cooperative scheduling with Mac OS 9.
>
> Now, when I want concurrent, preëmptive multiprocessing, I call 
> fork(2) [and often exec(2)], like Nature intended.  I don't know why 
> you folks on Win32 platforms continue to put up with such abominable 
> support for POSIX interfaces.
>
>
Does not depend what you use (Posix, win32 threads etc ...), OCaml does 
not support two threads accessing the memory at the same time.

Re: [Caml-list] Threads

[james woodyatt]

On 11 Jun 2005, at 00:46, Christophe Raffalli wrote:
>
> No there is a global mutex for caml thread. The only thing you can do 
> is have one Caml thread and many C threads running in the same time.
>
> This starts really to be an annoying limitation of the Caml runtime 
> now that you can have easily two dual core processors in one machine 
> ... (means four CPU)

Not for me it isn't-- I gave up cooperative scheduling with Mac OS 9.

Now, when I want concurrent, preëmptive multiprocessing, I call fork(2) 
[and often exec(2)], like Nature intended.  I don't know why you folks 
on Win32 platforms continue to put up with such abominable support for 
POSIX interfaces.


-- 
j h woodyatt <jhw@wetware.com>
that's my village calling... no doubt, they want their idiot back.

Re: [Caml-list] Threads

[Christophe Raffalli]

Jonathan Bryant a écrit :

>I'm confused.  In the documentation on the threads library it says that
>the threads implementation is shared time on only a single processor. 
>Is that for VM and system threads, or are the POSIX threads able to take
>advantage of multiple processors?  I would think they would be able to
>because (unless I'm mistaken) they're just wrappers around the C system
>calls...
>
>  
>
No there is a global mutex for caml thread. The only thing you can do is 
have one Caml thread and many C threads running in the same time.

This starts really to be an annoying limitation of the Caml runtime now 
that you can have easily two dual core processors in one machine ... 
(means four CPU)





>--Jonathan
>
>_______________________________________________
>Caml-list mailing list. Subscription management:
>http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
>Archives: http://caml.inria.fr
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>Bug reports: http://caml.inria.fr/bin/caml-bugs
>  
>