[Edbrowse-dev] a technique for the find-and-fix

[Edbrowse-dev] a technique for the find-and-fix

[Kevin Carhart]

I'm excited to share a discovery about one way to go about tackling a big 
and arcane site.

When I'm sitting in jdb, the JS routines that have been digested into 
memory can be echoed:

* ok(window)

a,b,c,d,e,f,blah

* blah

* function blah (a) { b = arcaneMethod(a) }

So then simple code injection is possible, so that the arguments to blah 
can become less opaque.  Just overwrite blah from the jdb command line:

* function blah (a) { exportable_a = a; b = arcaneMethod(a) }

Now I can find out about the arguments to blah:

* exportable_a

* [Object object]

And it will also be legal to pipe this argument back in:

* blah(exportable_a)

And it will also be legal to try things from that function step by step 
like:

* b=arcaneMethod(exportable_a)

In case I know blah produces a runtime error but I don't know which of 
its six subsidiary calls raises the error, now I can try each one and get 
down to the root cause more quickly!

Kevin

[Edbrowse-dev] deobfuscate(js_text)

[Kevin Carhart]

Hi group

Recently while debugging, I went back to fastmail.com, which we improved 
around December.  At one point I got a completely opaque message about a 
jquery error:
execute core-1.js at 1
TypeError: $(...) is null

I decided to bite the bullet and do something about this.  When JS code is 
minified and compacted, the line numbers tell you nothing.  It could 
report an error on line 1, but line 1 is several pages long.

So, I went back to a deobfuscator script that Chris and Karl sent me a 
link for last year.  It will space things out, indent and add newlines. 
The symbols still suck, however now the line number of an 
error can be way more helpful.   The deobfuscator is in Python so I 
pieced together how to embed the python interpreter and call it from C. 
And it works!!  Chunks of JS code from a page get newlines!

Currently I have this in prepareScript.  May not be the best place.  I 
want to do it prior to when the line count is found.

Near the end of prepareScript, instead of:
         set_property_string(t->jv, "data", js_text);
I changed it to
         set_property_string(t->jv, "data", deobfuscate(js_text));


I can give you this proof of concept code if you like- at the minimum, it 
is going to help me isolate more DOM errors.  I don't think it would be 
something that we would consider for the live edbrowse.  UNLESS - is there 
any reason to believe that putting JS code through a deobfuscator is not 
isomorphic?  Could it actually improve (or, change for the worse, 
unfortunately) the interpretation of the JS itself?  I am surprised to 
find that when I rerun fastmail, I don't get the TypeError at all.  I 
could be misinterpreting what I'm observing. Deobfuscation should be only 
a formatting change, but then, the python deobfuscator is a bunch of 
idiosyncratic decoders that people have handwritten for the output of the 
top five or six encoders, so I don't really know the algorithms and 
regular expressions they have used.  It might modify the code as it 
reformats it.

This is the link to the beautifier:
https://github.com/beautify-web/js-beautify

Kevin

[Edbrowse-dev] OT: Fastmail and the good old days

[Chuck Hallenbeck]

Hi friends,

I was a fastmail user for about ten years until a month or two ago.
When I began, they had a single interface which was totally accessible
from the command line, but later they named that interface "classic" and
introduced their "standard" interface.  Soon that became the default
and one had to ask for classic for a fully accessible experience.
In recent months they began disabling several specific functions in
the classic interface, so it became impossible to create a new account,
maintain websites on fastmail, configre aliases or spam/virus tools, etc.
One was told to use the standard interface to perform those functions.
It was clear that by "classic," they meant "old-fashioned."

I got nervous about becoming the last passenger with a ticket on the
Titanic, so I abandoned fastmail before they abandoned me.

Hopefully resolving the current issues on fastmail will lead to more
general solutions on other sites as well.

So far I remain a text console command line junky, but that world
is rapidly shrinking, as everyone knows.  No longer a groundbreaker,
I've been morphed into a bitter old caretaker.  I wish I had my first
computer back.  It was a Digital Group desktop box with a Zylog Z-80
8-bit CPU clocked at a blazing4.0 MHz, fully expanded to 64 MB of memory,
with a Votrax VS-1 synthesizer.  It originally used audio tape cassettes
for data storage and retrieval.

I added two eight inch floppy drives, and got it to run CP/M with speech
access in the late 1970's.  I wrote a grade 2 braille translator in
C for it when the Perkie was announced (The Cranmer Modified Perkins
Brailler), and it also provided me with a talking terminal to access
the KU campus mainframe.  Those were the days.

Chuck
-- 
Willoughby Ohio
   Temperature 66?F
   Conditions Scattered Clouds
The Moon is Waning Gibbous (96% of Full)

[Edbrowse-dev] OT: Fastmail and the good old days (correction)

[Chuck Hallenbeck]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1921 bytes --]

Woops! The memory size mentioned below is corrected to KB, not MB. Sorry.

Hi friends,

I was a fastmail user for about ten years until a month or two ago.
When I began, they had a single interface which was totally accessible
from the command line, but later they named that interface "classic" and
introduced their "standard" interface.  Soon that became the default
and one had to ask for classic for a fully accessible experience.
In recent months they began disabling several specific functions in
the classic interface, so it became impossible to create a new account,
maintain websites on fastmail, configre aliases or spam/virus tools, etc.
One was told to use the standard interface to perform those functions.
It was clear that by "classic," they meant "old-fashioned."

I got nervous about becoming the last passenger with a ticket on the
Titanic, so I abandoned fastmail before they abandoned me.

Hopefully resolving the current issues on fastmail will lead to more
general solutions on other sites as well.

So far I remain a text console command line junky, but that world
is rapidly shrinking, as everyone knows.  No longer a groundbreaker,
I've been morphed into a bitter old caretaker.  I wish I had my first
computer back.  It was a Digital Group desktop box with a Zylog Z-80
8-bit CPU clocked at a blazing4.0 MHz, fully expanded to 64 KB of memory,
with a Votrax VS-1 synthesizer.  It originally used audio tape cassettes
for data storage and retrieval.

I added two eight inch floppy drives, and got it to run CP/M with speech
access in the late 1970's.  I wrote a grade 2 braille translator in
C for it when the Perkie was announced (The Cranmer Modified Perkins
Brailler), and it also provided me with a talking terminal to access
the KU campus mainframe.  Those were the days.

Chuck
-- 
Willoughby Ohio
   Temperature 66°F
   Conditions Scattered Clouds
The Moon is Waning Gibbous (96% of Full)

[Edbrowse-dev] deobfuscate(js_text)

[Karl Dahlke]

> I am surprised to find that when I rerun fastmail,
> I don't get the TypeError at all.
> Deobfuscation should be only a formatting change,

Perhaps the minimized js is correct according to the js specs,
but is not parsed properly by smjs.
When you clean up the js code, smjs can handle it.
If so this is a bug in smjs and should be reported,
(more research needed here),
and would also be a case for embedding the deobfuscator in the meantime,
as a temp fix, though I would have concerns about that.
Course I would expect a syntax error, not a type error.

Karl Dahlke

Re: [Edbrowse-dev] deobfuscate(js_text)

[Kevin Carhart]

Hi Karl and Chuck

Well, it turns out that part two of my message was a false alarm. 
Reformatting does not have a mysterious effect on parsing after all, I 
just had a confounding variable at the same time.  Anyway, 
the incorporation of the python beautifier, in the first place, is the 
real news!  There is indeed a coherent change in the line number for the 
bug I was describing, from
$(..) in line 1
to something like
$(..) in line 5017

That's what I was hoping for - I'm still someplace in the middle with 
fastmail, but making extra progress as a result of the line numbers!

> Course I would expect a syntax error, not a type error.

Yes, it may be one of those things where the reported cause for runtimes 
is skewed relative to what is actually going on.  As a matter of fact, I 
was thinking recently that when I sit down with pages' worth of edbrowse 
output from the higher debug levels, I am generally starting at the 
bottom, which may be a mistake.  It suddenly hit me that runtimes in these 
web pages may be a little like the compilation errors from gcc or 
something!  Maybe there is a cascading effect, and the errors lower down 
are side effects of the first few.  So I should be going to the root cause 
and starting with the first thing first.  I discovered that it makes a 
nice debug mode if I add some exit(0) statements to readMessage in ebjs.c. 
So this way, the whole program will quit on the very first error, which 
forces me to address things in the order they occur.

K

Re: [Edbrowse-dev] OT: Fastmail and the good old days (correction)

[Kevin Carhart]

Thanks for writing that, Chuck.  That was fun.  Though I am sorry to hear 
that fastmail's accessibility is going downhill.  I hope eventually we 
will have working equivalents for whatever sorts of bells and whistles 
they have on their standard mode.  It may be too slow for some sites, and 
you'll find other means of doing something but I hope to get some things 
incorporated fairly soon.
>
> Hopefully resolving the current issues on fastmail will lead to more
> general solutions on other sites as well.

Yes, it does for sure.  This is what I have discovered.  All web pages are 
a window on a world, and it's the same world.  So I am really working on 
amazon, dkb.de, Drescher the metal band, google groups, fastmail, and 
others all at the same time, and I do it like a relay.  There is beginning 
to be a lot of overlap in the kinds of bugs.  I feel as though we 
are closing in on improvements and there are only so many unaddressed 
things lurking out there.

> access in the late 1970's.  I wrote a grade 2 braille translator in
> C for it when the Perkie was announced (The Cranmer Modified Perkins

That's awesome.. you shouldn't admit this so loud, or you will be enlisted 
to develop edbrowse :)

Kevin