From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out.smtp-auth.no-ip.com (smtp-auth.no-ip.com [8.23.224.61]) by hurricane.the-brannons.com (Postfix) with ESMTPS id 74F7C77FF5 for ; Tue, 21 Jun 2016 18:16:57 -0700 (PDT) X-No-IP: carhart.net@noip-smtp X-Report-Spam-To: abuse@no-ip.com Received: from carhart.net (unknown [99.52.200.227]) (Authenticated sender: carhart.net@noip-smtp) by smtp-auth.no-ip.com (Postfix) with ESMTPA id 4CD37400FFF for ; Tue, 21 Jun 2016 18:18:24 -0700 (PDT) Received: from carhart.net (localhost [127.0.0.1]) by carhart.net (8.13.8/8.13.8) with ESMTP id u5M1INZB007947 for ; Tue, 21 Jun 2016 18:18:23 -0700 Received: from localhost (kevin@localhost) by carhart.net (8.13.8/8.13.8/Submit) with ESMTP id u5M1INvu007944 for ; Tue, 21 Jun 2016 18:18:23 -0700 Date: Tue, 21 Jun 2016 18:18:23 -0700 (PDT) From: Kevin Carhart To: edbrowse-dev@lists.the-brannons.com In-Reply-To: Message-ID: References: User-Agent: Alpine 2.03 (LRH 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: [Edbrowse-dev] deobfuscate(js_text) X-BeenThere: edbrowse-dev@lists.the-brannons.com X-Mailman-Version: 2.1.21 Precedence: list List-Id: Edbrowse Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 01:16:57 -0000 Hi group Recently while debugging, I went back to fastmail.com, which we improved around December. At one point I got a completely opaque message about a jquery error: execute core-1.js at 1 TypeError: $(...) is null I decided to bite the bullet and do something about this. When JS code is minified and compacted, the line numbers tell you nothing. It could report an error on line 1, but line 1 is several pages long. So, I went back to a deobfuscator script that Chris and Karl sent me a link for last year. It will space things out, indent and add newlines. The symbols still suck, however now the line number of an error can be way more helpful. The deobfuscator is in Python so I pieced together how to embed the python interpreter and call it from C. And it works!! Chunks of JS code from a page get newlines! Currently I have this in prepareScript. May not be the best place. I want to do it prior to when the line count is found. Near the end of prepareScript, instead of: set_property_string(t->jv, "data", js_text); I changed it to set_property_string(t->jv, "data", deobfuscate(js_text)); I can give you this proof of concept code if you like- at the minimum, it is going to help me isolate more DOM errors. I don't think it would be something that we would consider for the live edbrowse. UNLESS - is there any reason to believe that putting JS code through a deobfuscator is not isomorphic? Could it actually improve (or, change for the worse, unfortunately) the interpretation of the JS itself? I am surprised to find that when I rerun fastmail, I don't get the TypeError at all. I could be misinterpreting what I'm observing. Deobfuscation should be only a formatting change, but then, the python deobfuscator is a bunch of idiosyncratic decoders that people have handwritten for the output of the top five or six encoders, so I don't really know the algorithms and regular expressions they have used. It might modify the code as it reformats it. This is the link to the beautifier: https://github.com/beautify-web/js-beautify Kevin