[Edbrowse-dev] amazon and its JS

[Edbrowse-dev] amazon and its JS

[Kevin Carhart]

I haven't weighed in on amazon lately, because I can't get it to budge.  I 
consistently get kicked back to the login screen.

For lack of any progress on the HTTP sessions, I have gone back to 
javascript and am trying to tackle those two JS runtimes.
Also (and related) the events code is in an unfinished state - I'm sorry 
for starting this and not following through yet, and thank you Karl for 
what you implemented on events recently.

In a while, if not right away, the events code is going to be relevant to 
amazon.com.  Especially this certain phrasing where one central 
listener/dispatcher sits on document on document.body, and events bubble 
up from various elements and kick it off.  I think that is a common 
phrasing.  Amazon uses a lot of everything.  I noticed that they use 
document.cookie.

Here is a specific question.  What would happen if page code wants to 
overload window.alert?  This happens in amazon document.scripts[4].data:


amazon4.js:(function(a){var 
b=a.alert;window.alert=function(){a.ueLogError&&a.ueLogError({message:"[CSM] 
Alert invocation detected with argument: 
"+arguments[0],logLevel:"WARN"});Function.prototype.apply.apply(b,[a,arguments||[]])}})(window);


thanks
Kevin

[Edbrowse-dev] amazon and its JS

[Karl Dahlke]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

I too have been remiss in my work on edbrowse; there are so many life events clammering for my attention, some of them serious.

> I haven't weighed in on amazon lately, because I can't get it to budge.

This is very frustrating because it seems to work for me,
and I can't imagine what the difference is.
Worse still, Chuck gets a seg fault.
Maybe I could log into his machine and try to track that one down.
I don't deminimize; how does amazon behave if you don't deminimize. Just trying to cut out another variable.

You're right, amazon does so much that if we get it working that bodes well, and probably a lot of other sites will work too.

document.cookie should work. It's been around a while and reasonably well tested.

There should be no trouble replacing window.alert.

Karl Dahlke

Re: [Edbrowse-dev] amazon and its JS

[Chuck Hallenbeck]

Hi Kevin, Karl, and all,

Let me try to recap my situation a bit.  When I run edbrowse on my
ArchLinux OS, it does not segfault.  Not even when it is run with
db5 redirected to /tmp/log.  Like Kevin's experience though, it just
returns me to the login screen with minimum hint of what's wrong.

On Debian, running edbrowse without db5, it also never segfaults,
and like under Arch, does not recognize my credentials.  '

Edbrowse segfaults only under Debian, and only when db5 is invoked
and redirected.  I cannot run edbrowse with db5 set, unless it's also
redirected, because it's just too chatty.

I would be willing to give Karl access to my system if that would
help, except I have no access to my router to do any port-forwarding
or otherwise open it.  And frankly I'm feeling my age recently and
am having difficulty extending my skills to include such things as
mastering gdb, core dumps, and the like.  I'm an old dog and those
are new tricks.  But I'm good at following instructions.

And I'm patient, and appreciative, and am not about to abandon edbrowse
and my text console just to access Amazon.

Chuck


-- 
Here too, In Northeast Ohio, The Moon is Waning Crescent (10% of Full)
When your only tool is a hammer, everything looks like a nail.
Sent from Brittany's iPhone.

Re: [Edbrowse-dev] amazon and its JS

[Dominique Martinet]

Hi all,

On my end I cannot log in either, using duktape's master, but I've been
playing a bit with duktape's debugging.

One thing I find odd is that the loading time for www.amazon.com (not
the sign in page) seems to vary greatly from one run to another.
Sometimes it loads in tens of seconds (slow but bearable), but sometimes
it takes as much as five minutes to load, and I'd be hard pressed to say
what's different... I was blaming the duktape's debugger for speed but
it could be something else entierly.


On the other hand, I have implemented some of the duktape's debugging
protocol into edbrowse, you will need to build with a checkout of
duktape's and you need to edit duk_config.h to make at least
DUK_USE_DEBUGGER_SUPPORT into #define, then pull in my master's branch
(https://github.com/martinetd/edbrowse.git)


So far I don't do anything with debugger except listening to events,
duktape is kind enough to tell us about all 'throw' exceptions in the js
code even if they are caught -- without debugging we can only see the
uncaught ones, maybe some caught exceptions can be a hint.

I've got to say though that amazon's code really is a pain to debug with
the obfuscated variables though...

So far I've had these errors (without demin here so don't trust line
numbers too much)


uncaught throw: ReferenceError: identifier 'querySelectorAll' undefined
(line 1417)

uncaught throw: TypeError: undefined not callable (property
'appendChild' of [object Object]) (line 887)

caught throw: ReferenceError: identifier 'Canvas' undefined (line 1227)

caught throw: TypeError: cannot read property 'indexOf' of undefined
(line 13)

uncaught throw: TypeError: undefined not callable (property
'setAttribute' of [object Object]) (line 18)

uncaught throw: TypeError: undefined not callable (property
'setAttribute' of [object Object]) (line 17)


and in a timer (I removed prints when in timer in last commit):
caught throw: TypeError: undefined not callable (property 'getAttribute'
of [object Object]) (line 27)


I have only walked through the very first of these :
querySelectorAll seems to be invoked from document
 (e.g. document.querySelectorAll()) modulo tons of obfuscation.
I thought this would be easy to fix by just adding a line in
startwindows.js like this :
document.querySelectorAll = querySelectorAll
But that doesn't seem to work because querySelectorAll is defined in
third.js, not sure how to proceed here.


It took me a long time just to figure that querySelectorAll was looked
at from document though, mostly because of amazon's code, but I think
stopping in the context of the duktape debugger might help a bit. We
could have the js call stack at the time of the throw for example (might
not be of much help for amazon), or we could dump local variables.
See https://github.com/svaarala/duktape/blob/master/doc/debugger.rst for
available functions, anything under "Commands sent by debug client"
would be available with a bit more glue.

Initially I wanted to just let us run the jdb prompt from buffer.c
whenever a throw came in, but we can't because this assumes it can
process js normally (timers etc) and I think this will hang while
debugger is paused, we need to use the debugger eval at this point so I
might make a second jdb on that side if it works well.

-- 
Dominique

Re: [Edbrowse-dev] amazon and its JS

[Dominique Martinet]

Dominique Martinet wrote on Sat, Jan 13, 2018:
> I have only walked through the very first of these :
> querySelectorAll seems to be invoked from document
>  (e.g. document.querySelectorAll()) modulo tons of obfuscation.
> I thought this would be easy to fix by just adding a line in
> startwindows.js like this :
> document.querySelectorAll = querySelectorAll
> But that doesn't seem to work because querySelectorAll is defined in
> third.js, not sure how to proceed here.

Karl told me where to put that line in third.js, I now have two new
exceptions instead:

TypeError: undefined not callable (property 'getBoundingClientRect' of
[object Object]) (line 1)
(oddly enough not from duk debugger, only edbrowse sees this one?)

and

caught throw: TypeError: undefined not callable (property 'getExtension'
of [object Object]) (line 18)
(from a timer)

Looking this up getExtension seems to be a webGL thing so probably not
for us... getBoundingClientRect is also something graphical to get the
square coordinates of something, not sure how that'd block us either,
but anything goes.


With querySelectorAll hitting submit after entering my password takes
much longer than it previously did before sending the post request, but
I cannot see any difference with it (the openid.return_to token is different
but it might be everytime):

before:
appActionToken=duOgjKNIs7R4LCPldTzPEC8NUbwj3D&appAction=SIGNIN&openid.pape.max_auth_age=ape%3AMA%3D%3D&openid.identity=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjAvaWRlbnRpZmllcl9zZWxlY3Q%3D&pageId=ape%3AdXNmbGV4&ignoreAuthState=ape%3AMQ%3D%3D&openid.return_to=ape%3AaHR0cHM6Ly93d3cuYW1hem9uLmNvbT9fZW5jb2Rpbmc9VVRGOCZyZWZfPW5hdl95YV9zaWduaW4%3D&prevRID=ape%3AUzVFMDRRV0NSVzdCVjg5MFRLUFE%3D&openid.assoc_handle=ape%3AdXNmbGV4&openid.mode=ape%3AY2hlY2tpZF9zZXR1cA%3D%3D&openid.ns.pape=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvZXh0ZW5zaW9ucy9wYXBlLzEuMA%3D%3D&prepopulatedLoginId=&failedSignInCount=ape%3AMA%3D%3D&openid.claimed_id=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjAvaWRlbnRpZmllcl9zZWxlY3Q%3D&openid.ns=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjA%3D&email=mymail&create=0&password=mypass&signInSubmit=Submit

after:
appActionToken=duOgjKNIs7R4LCPldTzPEC8NUbwj3D&appAction=SIGNIN&openid.pape.max_auth_age=ape%3AMA%3D%3D&openid.identity=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjAvaWRlbnRpZmllcl9zZWxlY3Q%3D&pageId=ape%3AdXNmbGV4&ignoreAuthState=ape%3AMQ%3D%3D&openid.return_to=ape%3AaHR0cHM6Ly93d3cuYW1hem9uLmNvbS8%2FX2VuY29kaW5nPVVURjgmcmVmXz1uYXZfeWFfc2lnbmlu&prevRID=ape%3AM1BGOUJNOUFKTjhaUEJWM1JRVDk%3D&openid.assoc_handle=ape%3AdXNmbGV4&openid.mode=ape%3AY2hlY2tpZF9zZXR1cA%3D%3D&openid.ns.pape=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvZXh0ZW5zaW9ucy9wYXBlLzEuMA%3D%3D&prepopulatedLoginId=&failedSignInCount=ape%3AMA%3D%3D&openid.claimed_id=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjAvaWRlbnRpZmllcl9zZWxlY3Q%3D&openid.ns=ape%3AaHR0cDovL3NwZWNzLm9wZW5pZC5uZXQvYXV0aC8yLjA%3D&email=mymail&create=0&password=mypass&signInSubmit=Submit

so I guess this is a dead end...

I have repushed master with querySelectorAll added where Karl suggested,
can't hurt to have for everyone, and moved the debugger code to a new
branch called debugger (still in my tree) to separate the two.

-- 
Dominique

[Edbrowse-dev] amazon and its JS

[Karl Dahlke]

[-- Attachment #1: Type: text/plain, Size: 294 bytes --]

Most of these missing functions, setAttribute getAttribute appendChild etc, are standard methods for dom objectss, so we need to learn more about the object, either it isn't an instance of one of our understood classes, or we didn't link those methods into the class prototype.

Karl Dahlke