From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) by hurricane.the-brannons.com (Postfix) with ESMTPS id 3BCE4791EB for ; Fri, 8 Jan 2016 11:42:19 -0800 (PST) Received: by mail-wm0-x22d.google.com with SMTP id f206so147281821wmf.0 for ; Fri, 08 Jan 2016 11:43:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Hhv1p/yDxhs5szT+lJ4THD4R5OHAzz3pTfjvZUXxYBo=; b=cwy1WfmtaOkJBV3cwGi25KQAWMechQokGzj4tTF4iKU0bMTyXmyDq9vYoLrqnRqnqR WdpbC7XqA6QslnE7L2xxnWs4t0PivYfHjZqEx7esF3Er3BNUbc67AD0sVgZ/DyAPOe8g j+5iGfWzDKphqEg8Ibjzt/Xugh2A9GddL8CrExIUi3z3LKcTJnqz+hZdOII7rpXckSW/ r8VWFxA+RWstuqJVy/TJRts3c9Qx5mRFvIyM4EqX1JZ0jTnW6g1+IBfWm7vb2x3w5lve HU2W8hL/E6HmyrmjsINOLrJOjik1m0eDrNL5RwU2Mz5MEY/0zRnjYj2tim17lwosxUFz j3Hw== X-Received: by 10.28.126.84 with SMTP id z81mr606966wmc.29.1452282184627; Fri, 08 Jan 2016 11:43:04 -0800 (PST) Received: from 122oven.adamthompson.me.uk (c.2.2.f.4.7.e.f.f.f.d.1.4.2.2.0.2.4.0.9.2.4.1.1.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:1142:9042:224:1dff:fe74:f22c]) by smtp.gmail.com with ESMTPSA id g187sm573218wmf.8.2016.01.08.11.43.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Jan 2016 11:43:03 -0800 (PST) Date: Fri, 8 Jan 2016 19:43:01 +0000 From: Adam Thompson To: Chris Brannon Cc: Edbrowse-dev@lists.the-brannons.com Message-ID: <20160108194301.GJ12402@122oven.adamthompson.me.uk> References: <20151229191702.GB1766@122oven.adamthompson.me.uk> <8760z8c12v.fsf@mushroom.localdomain> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/qIPZgKzMPM+y5U5" Content-Disposition: inline In-Reply-To: <8760z8c12v.fsf@mushroom.localdomain> User-Agent: Mutt/1.5.24 (2015-08-30) Subject: Re: [Edbrowse-dev] curl handles and general comms design X-BeenThere: edbrowse-dev@lists.the-brannons.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Edbrowse Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 19:42:19 -0000 --/qIPZgKzMPM+y5U5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 04, 2016 at 04:38:32PM -0800, Chris Brannon wrote: > Adam Thompson writes: >=20 > > To solve the parallel curl handles accessing cookie databases issue, > > there's also the curl-shared interface. > > I believe this can be used with the curl-multi interface since the curl= -multi > > interface is single-threaded so no need for mutexes etc. >=20 > We're using that now, but with curl easy handles, rather than curl > multi. I don't know what would be involved in moving over to curl multi. I think curl multi is another object like curl shared, so I *think* an easy handle can be in a shared and multi object at the same time, though I'm really not sure. > > What this all means I think is that, by combining both interfaces, > > we should be able to create a single-threaded, essentially async, comms= layer. >=20 > The one problem is that you cannot share persistent connections across > curl easy handles. >=20 > Basically, you create a curl multi handle, and then add curl easy > handles to it. So when would we create the individual curl easy > handles? Ok, so I'm thinking of something like the following high level approach: Keep the global curl shared object, but not the global curl easy handle. The share object will be set for each new curl easy handle. We create a global curl multi object and have logic to keep checking this f= or active transfers and if any are found run the next chunk of the transfer. For each transfer we add a curl easy handle to the multi object, and have l= ogic to remove it when the transfer is finished. If I understand things correctly the shared object should keep track of the cookies whilst the multi object handles the connections and provides a way = of managing the connections so that the handles can be added when we need them. We may need some way to get the cookies into and out of curl, but I'm not sure of the cleanist way to do this. I'm surprised there isn't a way to get the shared object to write the cookie file on clean up but if not then we can always keep the global curl easy ha= ndle. Regards, Adam. --/qIPZgKzMPM+y5U5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJWkBFFAAoJELZ22lNQBzHO6TMIALWXpy+xdl7wyrNaRwDg/tyz BjeiVwLky/wXQL/MfgDKT6Uj8sJEdvaRtq9F37tSfioPdDUxYboAq8oi6y+X3A8W M/TJBS6NLEHCET7HdjwYKC1v6tjw67WLSg3Zn22hKGblLxuQmnBcZghEIqkRf0Z7 FRJ3yI0+t7rm9WFHZZ2PFKRMaFYYWTKclCgTgxCWSrsMKHvql8Id2QDtjykpsOyM LyeeoaAFM2PzRouTkF1RawvsIgOE/hYvSuCVP7FvC+A6CXVcwnB/SkiTknlr0oEa XYvu2cq0sjvbtaaKlzR7+AA5Gjq3ldiNvXeC5mPdYvsWkXjEfPXUb+6iZymvl54= =XFrB -----END PGP SIGNATURE----- --/qIPZgKzMPM+y5U5--