Re: [Edbrowse-dev] curl handles and general comms design

[Adam Thompson <arthompson1990@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2134 bytes --]

On Mon, Jan 04, 2016 at 04:38:32PM -0800, Chris Brannon wrote:
> Adam Thompson <arthompson1990@gmail.com> writes:
> 
> > To solve the parallel curl handles accessing cookie databases issue,
> > there's also the curl-shared interface.
> > I believe this can be used with the curl-multi interface since the curl-multi
> > interface is single-threaded so no need for mutexes etc.
> 
> We're using that now, but with curl easy handles, rather than curl
> multi.  I don't know what would be involved in moving over to curl multi.

I think curl multi is another object like curl shared,
so I *think* an easy handle can be in a shared and multi object at the same
time, though I'm really not sure.

> > What this all means I think is that, by combining both interfaces,
> > we should be able to create a single-threaded, essentially async, comms layer.
> 
> The one problem is that you cannot share persistent connections across
> curl easy handles.
> 
> Basically, you create a curl multi handle, and then add curl easy
> handles to it.  So when would we create the individual curl easy
> handles?

Ok, so I'm thinking of something like the following high level approach:
Keep the global curl shared object, but not the global curl easy handle.
The share object will be set for each new curl easy handle.
We create a global curl multi object and have logic to keep checking this for
active transfers and if any are found run the next chunk of the transfer.
For each transfer we add a curl easy handle to the multi object, and have logic to remove it when the transfer is finished.

If I understand things correctly the shared object should keep track of the
cookies whilst the multi object handles the connections and provides a way of
managing the connections so that the handles can be added when we need them.
We may need some way to get the cookies into and out of curl,
but I'm not sure of the cleanist way to do this.
I'm surprised there isn't a way to get the shared object to write the cookie
file on clean up but if not then we can always keep the global curl easy handle.

Regards,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]