From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 991B87EE4B for ; Thu, 26 Sep 2013 10:34:11 +0200 (CEST) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of edwin+ml-ocaml@etorok.net) identity=pra; client-ip=176.9.138.55; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="edwin+ml-ocaml@etorok.net"; x-sender="edwin+ml-ocaml@etorok.net"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of edwin+ml-ocaml@etorok.net designates 176.9.138.55 as permitted sender) identity=mailfrom; client-ip=176.9.138.55; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="edwin+ml-ocaml@etorok.net"; x-sender="edwin+ml-ocaml@etorok.net"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of postmaster@mail.etorok.net designates 176.9.138.55 as permitted sender) identity=helo; client-ip=176.9.138.55; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="edwin+ml-ocaml@etorok.net"; x-sender="postmaster@mail.etorok.net"; x-conformance=sidf_compatible; x-record-type="v=spf1" X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgMFAJLwQ1KwCYo3/2dsb2JhbABbgwc4iVi3Q4EiFnSCJQEBBScZAQE2AQEPCxgJFg8JAwIBAgFFEwEHAogGCKh7hE0BBY5gBo19gVQHCYQViTWGdIEvhieBL4UDi0WDJw X-IPAS-Result: AgMFAJLwQ1KwCYo3/2dsb2JhbABbgwc4iVi3Q4EiFnSCJQEBBScZAQE2AQEPCxgJFg8JAwIBAgFFEwEHAogGCKh7hE0BBY5gBo19gVQHCYQViTWGdIEvhieBL4UDi0WDJw X-IronPort-AV: E=Sophos;i="4.90,983,1371074400"; d="ml'?scan'208";a="28153798" Received: from mail.etorok.net ([176.9.138.55]) by mail3-smtp-sop.national.inria.fr with ESMTP; 26 Sep 2013 10:34:10 +0200 Received: from [172.30.42.25] (unknown [79.114.35.129]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.etorok.net (Postfix) with ESMTPSA id 0035446D8; Thu, 26 Sep 2013 10:34:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=etorok.net; s=mailout; t=1380184449; bh=bSQZ6gLZaAE5oq6dD6oK0ncaQb1Fx/3coJI0igULB60=; l=3896; h=Date:From:To:CC:References:In-Reply-To:From; b=gd0AkmQIfqrJTrrPU1PoeCXdXVkcdd23mK8fWHEPoh8v0goOkBpa287soP4Ibw1Pz RPb8stoECTgNO9J0yxbiMoMxacVWkxkkJPaJ8ks5qs8JApx6yUHuj6lVYcf+0IuNKu 3S3a4rSD4CJFfhCpan9rph96J1z6faOhNZNMF2Ao= Message-ID: <5243F180.3010102@etorok.net> Date: Thu, 26 Sep 2013 11:34:08 +0300 From: =?ISO-8859-1?Q?T=F6r=F6k_Edwin?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130922 Icedove/17.0.9 MIME-Version: 1.0 To: ocamlnet-devel@lists.sourceforge.net CC: caml-list@inria.fr References: <1354021383.3169.2@samsung> In-Reply-To: Content-Type: multipart/mixed; boundary="------------030107090305060204020302" X-Virus-Scanned: clamav-milter 0.97.8 at mail X-Virus-Status: Clean Subject: Re: [Caml-list] Multithreaded https requests in ocamlnet netclient This is a multi-part message in MIME format. --------------030107090305060204020302 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 11/28/2012 03:43 AM, Mike Lin wrote: > Thanks, Gerd! > > FWIW I could not reproduce the crash by using ocaml-ssl's blocking operations directly. > > https://gist.github.com/4152047#file_ssl_threads.ml > > This works fine- so, perhaps something nasty arises from using nonblocking I/O on ssl sockets from multiple threads. I'm sure if there is any other critical difference with how netclient/equeue-ssl and my example use ocaml-ssl. > > I also don't have time to pursue this much further, so I will try to put all of my http operations on one thread as your example suggests. I was debugging a similar bug, and found this old thread with a testcase. I figured a way to fix it, see the patch below (it didn't acquire the OCaml master lock before raising an exception, causing Ocaml code to be executed in parallel with other OCaml code ... leading to all sorts of nasty situations). Debugging thread-related bugs is hard, especially that none of the usual tools help here. I modified st_posix.h a bit by adding an m->owner field and checking it against pthread_self() to make sure a thread attempts to release only a lock it acquired itself, but there is more that could be done (check in raise_with_arg/etc. that we do hold the master lock, check after returning from each C call that we do hold the lock, same for C callbacks, etc.). Would it be possible to do add checks like this with '-runtime-variant d', i.e. can the thread implementation be changed to a "checking" one in that case? The patch: Index: src/equeue-ssl/ssl_exts_stubs.c =================================================================== --- src/equeue-ssl/ssl_exts_stubs.c (revision 1913) +++ src/equeue-ssl/ssl_exts_stubs.c (working copy) @@ -27,6 +27,7 @@ caml_enter_blocking_section(); ret = SSL_shutdown(ssl); if (ret == -1) { + caml_leave_blocking_section(); raise_with_arg(*caml_named_value("ssl_exn_shutdown_error"), Val_int(SSL_get_error(ssl, ret))); }; --------------030107090305060204020302 Content-Type: text/x-ocaml; name="netclient_https_threads.ml" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="netclient_https_threads.ml" (* ocamlfind ocamlopt -o netclient_https_threads -thread -linkpkg -package threads,netclient,ssl,equeue-ssl netclient_https_threads.ml http://docs.camlcity.org/docs/godipkg/3.12/godi-ocamlnet/doc/godi-ocamlnet/html/Https_client.html *) open Printf module HTTP = Http_client module HTTPS = Https_client ;; let mutex = Mutex.create () let compact () = Mutex.lock mutex; Gc.compact (); Mutex.unlock mutex;; let () = Ssl.init ~thread_safe:true ();; let fresh_pipeline () = let pipeline = new HTTP.pipeline in let ssl_ctx = Ssl.create_context Ssl.TLSv1 Ssl.Client_context in let tct = HTTPS.https_transport_channel_type ssl_ctx in pipeline#configure_transport HTTP.https_cb_id tct; pipeline ;; let f () = begin try Mutex.lock mutex; let pipeline = fresh_pipeline () in let call = new HTTP.get "https://www.google.com/" in pipeline#add call; pipeline#run (); Mutex.unlock mutex; printf "ok\n%!"; flush stdout; with e -> Printf.eprintf "Error: %s\n%!" (Printexc.to_string e) end; ;; let threads = Queue.create ();; for i = 1 to 10 do Queue.add (Thread.create f ()) threads; (*Thread.join (Queue.take threads)*) done;; while not (Queue.is_empty threads) do Thread.join (Queue.take threads) done;; --------------030107090305060204020302--