[Edbrowse-dev] a bundle of changes for drescher and generally

[Edbrowse-dev] a bundle of changes for drescher and generally

[Kevin Carhart]

The diedrescher.com website raises a lot of issues that extrapolate to all 
pages, so it has been a great place to start.  Amazon, dkb.de, drescher, 
fastmail, google groups, they all tie back to the DOM.  So now what I have 
ready is some edits that I am going to submit as a bundle.  It's possible 
that they do not all belong in the live program but will be more of a 
living illustration of what's missing and a neat way of showing how little 
distance there is between us and some tangible outcomes.  This is an 
acceptable way of working, right?  It's like forking the code for a 
specific purpose.  You can compile a bleeding-edge sandbox without 
worrying about anything, and it could lead to some if not all of the 
differences making it into edbrowse.

The idea to work on diedrescher.com came from Sebastian reporting that 
there is a series of links which don't respond or lead anywhere.  So the 
overall question is why not?  What does the edbrowse side need which it 
isn't finding in order to tie the click to the handler?  What is the site 
code and libraries expecting to exist that we haven't implemented?  It 
turns out to be a few things in combination.

Here is the illustration of "before".  If you click one of the links, 
you'll get a message like:


beginning of an edbrowse fragment -------------
* {KONTAKT}
g
label kontakt is not found
the edbrowse fragment ends here -------------



However, with the edits applied, it successfully goes through the entire 
circuit!  Here is a real run from "after":


beginning of an edbrowse fragment -------------
b http://diedrescher.com
11907
276
1
{DE} | {EN}



* {DRESCHER}

* {SHOP}

* {BIBLIOTHEK}

* {KONTAKT}
g
label kontakt is not found
rr
lines 19 through 74 have been updated
19
KONTAKT



Band



DRESCHER RECORDS
the edbrowse run fragment ends here -------------



It's a real proof of concept!  It goes the whole way and back!

I noticed that it didn't notify me that the lines changed, so I am missing 
something relating to how edbrowse knows to automatically report that 
lines changed.   Another flaw is that I can only retrieve the 
contact/kontact pages.  The other pages come back successfully from XHR, 
but they have their own wrinkles, some of which involves iframes.  I 
decided that for this bunch of edits, I would comment out 
"iframe" from availableTags because we have more work on this pending, so 
for the moment I am ignoring them so they can't crash edbrowse altogether.

It is kind of breathtaking how much stuff happens to get this result.  It 
turns out to be a little bit of everything working together: the xhr code, 
the timeout code, the DOM implementation, the events code, and giving 
libraries what they expect.

o First the page code is digested into memory.  A lot of functions will 
become resident, and some code will actually run prior to the first moment 
where it is interactive.
o jquery doles out event handlers to elements.  In this scenario, some of 
this distribution is done based on the value of the 'class' attribute, 
entirely managed by the library.  So this is very different than the 
easier style where the handler is out in the open in an onclick on a piece 
of html.
o Edbrowse picks up that these handlers exist
o So when I press 'g', edbrowse successfully found the handler on that 
anchor and ran it
o It happens that the page author's line of code uses $.get, which is 
jquery's wrapper around XMLHttpRequest.  The page author's call to $.get 
also includes a callback function which will run on success
o Now jquery's xhr wrapper uses our XHR code under the hood, to do the 
HTTP activity and to report a '200 OK' or other status
o startwindow calls fetchHTTP using the javascript conduit and returns the 
retrieved HTTP page back to jquery as a response.  It reports '200 
OK', which jquery's xhr code is expecting before it will run the 
callback function to run on success.  (Maybe something else happens on 
failure).
o jquery's xhr runs the callback, using setTimeout.  So our setTimeout 
code comes into it and I have made it more lenient so that one argument is 
allowed
o The callback code takes the retrieved html page and writes it to the 
innerHTML of a certain div
o This triggers our i{ side effect and we have now made the complete 
circuit back to the edbrowse side!  The new div contents go through 
rendering and there is a message that certain lines changed, which might 
or might not need a 'rr' to propagate.

hooray!
Patch will be in the next email.
Kevin

[Edbrowse-dev] a bundle of changes for drescher and generally

[Karl Dahlke]

> I noticed that it didn't notify me that the lines changed,

I'll have to read it again but it sounds like they were actually changed
by some timer code, not directly from the g command.
This is a feature that requires a lot of thought.
If lines change because of something you did then I tell you;
if they change from a timer then I don't.
The fear is some realtime site, maybe changing stock prices all the time,
and you don't want it spitting out those changes to you all the time.
Imagine if this spit out a message once a second.
http://www.eklhad.net/async
But maybe we don't have any of those websites since we don't have asyncronous
and all that stuff running,
and here in this example you would want to be told of the change,
or more accurately, you would want edbrowse to run the rr command,
for how would you know to do that,
and it would seem like it didn't work.
So we really have to give this one some thought.

Karl Dahlke

Re: [Edbrowse-dev] a bundle of changes for drescher and generally

[Kevin Carhart]

> I'll have to read it again but it sounds like they were actually changed
> by some timer code, not directly from the g command.

Oh yes- you're right!  I think.  The pivotal line inside of jquery is:
setTimeout (callback);
The callback function writes the retrieved html to the innerhtml of a div.
But it's ensconced by the Timer, so I think you're right.
That's why there needed to be an rr before we were notified.
So this distinction in its current form may be undermined
by the fact that it's out of our hands whether a page author or a
library author wrapped something in setTimeout on a whim or whyever.
A bit worrisome - I knew to press rr because I knew from all kinds
of other echoing that there was finally something there.
Idea: could it happen inline with all other commands?  Not
asynchronously immediate, but you find out as soon as you do
anything else.


Kevin

[Edbrowse-dev] substr

[Kevin Carhart]

Sorry - I had to make at least one mistake that I don't catch until after 
pressing the button.
When I add substr to the URL prototype, the function that is being done 
needs to actually be substr.  It's not just an alias but its own 
slightly different function.
Please swap it if you don't mind, or let me know and I will resubmit.
thanks
Kevin