From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <eklhad@comcast.net>
Received-SPF: Pass (mailfrom) identity=mailfrom;
 client-ip=2001:558:fe21:29:69:252:207:36; helo=resqmta-ch2-04v.sys.comcast.net;
 envelope-from=eklhad@comcast.net; receiver=<UNKNOWN> 
Received: from resqmta-ch2-04v.sys.comcast.net
 (resqmta-ch2-04v.sys.comcast.net [IPv6:2001:558:fe21:29:69:252:207:36])
 by hurricane.the-brannons.com (Postfix) with ESMTPS id D4A6677893
 for <edbrowse-dev@lists.the-brannons.com>;
 Fri,  2 Mar 2018 14:34:07 -0800 (PST)
Received: from resomta-ch2-06v.sys.comcast.net ([69.252.207.102])
 by resqmta-ch2-04v.sys.comcast.net with ESMTP
 id rtFee5R2Xpno4rtGweBvfi; Fri, 02 Mar 2018 22:35:26 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net;
 s=q20161114; t=1520030126;
 bh=1acwQY6ZZzfNW3eU6k4L+/BOGYdNtxOPNWOnCEacqUQ=;
 h=Received:Received:To:From:Reply-to:Subject:Date:Message-ID:
 Mime-Version:Content-Type;
 b=e3PEinu1E86v+1xT77H8Z4dH+oxKHAj59Pmg+lH2uEM95C60uTi8QJ2G9Lqir7yyp
 MyjgwXXIrNYW1RLqYgcT3BAgq0FI+1p3Ypx1GCxuDNEmOgVQjsxLTK960/e7N6Jx/m
 mBRcwzZ+1dpQ7eTRZjgjYam/CXro16CQitZTZAzezJRSCt8YotBYKBOwpfuirVIdcp
 KoHurtNahUD1Hrr70FRPJAWp4k/Y2M/h1n50qON/AP2aHkxDB1UeRECRCf1IAhB00c
 gMTQojEiLCpvnaGf0DImvTZs7RKWKocycTVVJuflxzhYS7dAaiG1YxNudy+uIM1KEO
 CERwgcb8ZH0GA==
Received: from unknown ([IPv6:2601:408:c300:8f09:21e:4fff:fec2:a0f1])
 by resomta-ch2-06v.sys.comcast.net with SMTP
 id rtGveA2CqnjgYrtGveHSHN; Fri, 02 Mar 2018 22:35:25 +0000
To: Edbrowse-dev@lists.the-brannons.com
From: Karl Dahlke <eklhad@comcast.net>
Reply-to: Karl Dahlke <eklhad@comcast.net>
User-Agent: edbrowse/3.7.2
Date: Fri, 02 Mar 2018 17:35:24 -0500
Message-ID: <20180202173524.eklhad@comcast.net>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary=nextpart-eb-854988
Content-Transfer-Encoding: 7bit
X-CMAE-Envelope: MS4wfHBxe0Ln7j0kuEgnhxQojpvjWtZLrtSAHgMjbd+DBp+2+lPXlZZQPJLKU6Izb034oiiNldQd5BTNSffhmohK4dvqjbrvNWupCDuhmhER/NV3Ufsjp2NZ
 hwQ81/yp0oNp8b7hBd/SMPamZ5oaEhvrLqLBVmd0shgsN/SrCXYZp3hy
Subject: [Edbrowse-dev] the i_get structure
X-BeenThere: edbrowse-dev@lists.the-brannons.com
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: Edbrowse Development List <edbrowse-dev.lists.the-brannons.com>
List-Unsubscribe: <http://lists.the-brannons.com/mailman/options/edbrowse-dev>, 
 <mailto:edbrowse-dev-request@lists.the-brannons.com?subject=unsubscribe>
List-Archive: <http://lists.the-brannons.com/mailman/private/edbrowse-dev/>
List-Post: <mailto:edbrowse-dev@lists.the-brannons.com>
List-Help: <mailto:edbrowse-dev-request@lists.the-brannons.com?subject=help>
List-Subscribe: <http://lists.the-brannons.com/mailman/listinfo/edbrowse-dev>, 
 <mailto:edbrowse-dev-request@lists.the-brannons.com?subject=subscribe>
X-List-Received-Date: Fri, 02 Mar 2018 22:34:08 -0000

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--nextpart-eb-854988
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Ok, this is kinda long. I rewrote about 800 lines of code, in 11 =
different files, and pushed, and hope it didn't break anything. Here is =
the roadmap.

1. It would be nice if we could fetch all the javascript files from the =
internet in parallel, and the css files too. I'm sure other browsers do =
that. It would speed things up.
Sometimes there are 10 or more of these files to fetch.
We already spin off processes to download files in the background, so =
you'd think the machinery is mostly there.
It's unix only, but I don't care, I don't think I have a single windows =
user at this point.
I could fork off and download to a temp file and when done read that =
temp file into the js <script> object and off we go.
But it seems less than ideal. There's just a gut feeling that threads =
would be better.

2. When a script is marked async, it can run asynchronously.
I don't think we're ever going to do that. I need a team of engineers =
for that, not just my spare time.
but ... async can mean postpone.
It means the browse can finish and you can start looking at the file =
while that script runs.
I can just put it on a timer.
Like in 10 seconds go ahead and run the async script.
Ok but in ten seconds you may find yourself locked out for a few =
seconds while the script runs, which is weird.
And if that script does an xhr, and the internet is slow, then you're =
blocked for 20 seconds.
Well, what if that postponed script, or perhaps timers in general, ran =
in another thread while you look around?
Now here's the thing, js + my dom + edbrowse will never be threadsafe.
So if a js timer or script is running over there, you can't run js in =
the foreground over here.
It runs here or there, not both.
If you try, like clicking on a button or doing anything that involves =
js, then I have to stop and block wait for the other thread to finish.
But this is not likely, and still better I think that doing all that =
stuff during browse and you have to wait for it even to see the page.
A lot of these async scripts are google analytics or google ads etc, =
that we just don't care about,
so let them run when they run, and they might update some lines on the =
page, filling in the ads, which you can look at if you care, or not.

3. The key to parallel downloads is a threadsafe curl system, not just =
curl itself but all the machinery we built around it.
I'm not expecting to run js in parallel in two separate threads, but we =
need to run curl in two separate threads, or in 10 separate threads.
So where does that leave us?

4. There is the basics of running curl in a threadsafe fashion, and =
maybe Chris can help me with this.
I know you have a 9 to 5, but maybe you remember reading about it and =
can determine if I have to do something differen, or special,
or if it's just well behaved already.
And what about the standard calls like stdio and such, are they =
threadsafe?
Remember that each thread could be dumping data to a common file in =
debug mode.

5. What about the framework around curl, primarily httpConnect?
Good lord there are about 20 static variables at the top of http.c that =
record values and states and the like as we step through the http fetch.
I mean it's as far away from threadsafe as the moon!

"Did you work at your regular job, then at 10 oclock at night just =
hammer this thing together as fast as you could, just to make it work?"

"Yeah, I kinda did."

So this push is about cleaning up my mess, at least some of it.
You're a programmer, so you know the drill.
All those static variables become members of a structure,
struct i_get    (internet get)
If a routine calls httpConnect, and this happens more often than you =
might think, including the xhr request, it has an auto variable
	struct i_get g;
We set it up with url and some parameters, and call httpConnect(&g),
and now everything is on the stack where it belongs.
I've been doing some surfing and it seems to work, but boy there was a =
lot of rewrite!

This isn't the end. There's the caching system, the web authorization =
system, finding proxy, the novs domains, I mean it calls things all =
over that might not be threadsafe.
But some of them are only used in the foreground thread, the =
interactive thread that is you typing at the keyboard.
And the whole cache system is probably threadsafe, because I set it up =
with a locking mechanism,
assuming many edbrowse processes would be accessing the same cache.
So these things can be managed one by one I think.
And you know, even if we never do any of the things in this post, it's =
still a better, cleaner design.

Karl Dahlke

--nextpart-eb-854988--