[edbrowse-dev] Rooted

[edbrowse-dev] Rooted

[Karl Dahlke]

[-- Attachment #1: Type: text/plain, Size: 3676 bytes --]

Spider monkey has a rooting guide, which I bookmarked, and I need to read, but sometimes I learn more just playing withthe the software.

First the existential crisis.
I stored a pointer to the document object, swallowed startwindow.js and third.js, which is a lot of stuff,
Fetched the document object from the global object, and verified that the pointer changed out from under me.
Pointer to object is not permanent and guaranteed, as it is in duktape.
Just one of many reasons duktape is sooooo much easier to use.
We have to have reliable pointers.
When I go to a hyperlink, <a> tag, there's an object with that,
edbrowse keeps a pointer to that object, I need to be able to bounce through that pointer and look at the object,
and see if the href has changed, or if there is onclick code to run, and so on and so on.
If the garbage collector has moved that object somewhere else, or if it moved by realloc because it got bigger,
then I'm fucking screwed.
A seg fault that I'll never be able to debug.
That is the existential crisis.
And the crisis is confirmed by my hello program, wherein the document object moved to a new location because of executing 16,000 lines of js.
This of course put me into a deep depression, along with everything else that is happening in my life.

But I wanted to learn something.
Still haven't read the rooting guide, but I knew it had something to do with rooting.

There is a general Rooted class.

template <t>
class Rooted<t> { ... }

(This is where you have to learn c++.)

Lots of things can be rooted but I'm mostly interested in objects.
So specialize the template as

Rooted<Object>

They have some convenient typedefs for the common ones.

typedef Rooted<Object> RootedObject;
typedef Rooted<Value> RootedValue;

A rooted thing is 12 bytes.
The first 8 bytes do the rooting, somehow, and the last 4 bytes point to the thing.
The * operator is overloaded to push the pointer out.
(Other important operators are overloaded as well.)
So if d is an object from class Rooted<Foo>, then *d returns those last 4 bytes, which is a pointer to something of type foo.
But only use that pointer in a transient way.
I thought rooting would prevent the pointer from changing, prevent the thing from moving.
WRONG!
But it does update the pointer in each root if it does move.
So, my document object moved, and it updated the pointer in my rooted document object.
In fact they really want you to deal with the rooted things, not the pointers at all, if you can help it.
Their functions take and return rooted things.
If objects move around in memory, all the rooted things are updated.

I have a structure htmlTag, with a member jv, javascript variable.
It's just a void *
I could point to the object and feel good about it, in duktape, because the object never moved.
In mozjs, I can't just point to the object, it might move, I have to use something rooted.
I could have RootedObject in the html tag, but then all of edbrowse has to read in jsapi.h,
and all of edbrowse has to know about RootedObject, and all of edbrowse has to be processed by g++.
There's no more encapsulation, keeping it all within jseng-moz.cpp.
That would piss me off!
(Yeah, that's how we handled it many years ago.)
Or I could just say ok, jv is an opaque 12 bytes that I don't know anything about, but that's hardly portable.
What if it's 16 bytes on some other machine, or even 24?
I could point to or index an array of rooted things that is only known inside jseng-moz.cpp.
I sort of like that idea but not sure how to implement it in practice.

Holy shit this stuff is complicated.

Karl Dahlke

Re: [edbrowse-dev] Rooted

[Adam Thompson]

Appologies if I'm missing a bunch of the discussion, haven't been keeping up
with IRC for a long time now.

On Sat, Oct 31, 2020 at 11:23:49PM -0400, Karl Dahlke wrote:
> Spider monkey has a rooting guide, which I bookmarked, and I need to read, but sometimes I learn more just playing withthe the software.
> 
> First the existential crisis.
> I stored a pointer to the document object, swallowed startwindow.js and third.js, which is a lot of stuff,
> Fetched the document object from the global object, and verified that the pointer changed out from under me.
> Pointer to object is not permanent and guaranteed, as it is in duktape.
> Just one of many reasons duktape is sooooo much easier to use.
> We have to have reliable pointers.
> When I go to a hyperlink, <a> tag, there's an object with that,
> edbrowse keeps a pointer to that object, I need to be able to bounce through that pointer and look at the object,
> and see if the href has changed, or if there is onclick code to run, and so on and so on.
> If the garbage collector has moved that object somewhere else, or if it moved by realloc because it got bigger,
> then I'm fucking screwed.
> A seg fault that I'll never be able to debug.
> That is the existential crisis.
> And the crisis is confirmed by my hello program, wherein the document object moved to a new location because of executing 16,000 lines of js.
> This of course put me into a deep depression, along with everything else that is happening in my life.
> 
> But I wanted to learn something.
> Still haven't read the rooting guide, but I knew it had something to do with rooting.
> 
> There is a general Rooted class.
> 
> template <t>
> class Rooted<t> { ... }
> 
> (This is where you have to learn c++.)
> 
> Lots of things can be rooted but I'm mostly interested in objects.
> So specialize the template as
> 
> Rooted<Object>
> 
> They have some convenient typedefs for the common ones.
> 
> typedef Rooted<Object> RootedObject;
> typedef Rooted<Value> RootedValue;
> 
> A rooted thing is 12 bytes.
> The first 8 bytes do the rooting, somehow, and the last 4 bytes point to the thing.
> The * operator is overloaded to push the pointer out.
> (Other important operators are overloaded as well.)
> So if d is an object from class Rooted<Foo>, then *d returns those last 4 bytes, which is a pointer to something of type foo.
> But only use that pointer in a transient way.
> I thought rooting would prevent the pointer from changing, prevent the thing from moving.
> WRONG!
> But it does update the pointer in each root if it does move.
> So, my document object moved, and it updated the pointer in my rooted document object.
> In fact they really want you to deal with the rooted things, not the pointers at all, if you can help it.
> Their functions take and return rooted things.
> If objects move around in memory, all the rooted things are updated.

That's not going to be nice for us to work with.

> I have a structure htmlTag, with a member jv, javascript variable.
> It's just a void *
> I could point to the object and feel good about it, in duktape, because the object never moved.
> In mozjs, I can't just point to the object, it might move, I have to use something rooted.
> I could have RootedObject in the html tag, but then all of edbrowse has to read in jsapi.h,
> and all of edbrowse has to know about RootedObject, and all of edbrowse has to be processed by g++.
> There's no more encapsulation, keeping it all within jseng-moz.cpp.
> That would piss me off!
> (Yeah, that's how we handled it many years ago.)

Yeah, that wouldn't be nice.

> Or I could just say ok, jv is an opaque 12 bytes that I don't know anything about, but that's hardly portable.
> What if it's 16 bytes on some other machine, or even 24?

Or the API changes which changes its size.

> I could point to or index an array of rooted things that is only known inside jseng-moz.cpp.
> I sort of like that idea but not sure how to implement it in practice.

Yeah.  The easy (and problematic) option is just to have an array and store
indices in the structure.  It'll really be horrible and require some method
of management.  Either that or we have an opaque type which is only defined
in the mozjs stuff which hides the c++ part but I suspect this may only work
properly with some compilers (and no I'm not sure on that).
 
> Holy shit this stuff is complicated.

Yes.  I have to ask at this stage, why would we want to go back to mozjs?
I've no doubt things've changed but last time I looked at it it was becoming
increasingly not suited to our design as well as having an approach where
they could alter it in any way they needed to to make Firefox work.  I
remember having all sorts of issues trying to work out what'd changed which
was part of the reason for changing engines I think.

As I said at the start of this email, I admit I've not really been keeping
up with this stuff.  However, I'm interested if there's something here I've
missed.

Cheers,
Adam.

[edbrowse-dev] Rooted

[Karl Dahlke]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

No problem Adam.   Thanks for your work long ago, which I'm revisiting, and your thoughts.

We almost certainly have to change to something, because duktape is now 2 years out of date, and shows no sign of catching up.
No proper function.toString(), no es6, no let, no Promise, etc etc.
They don't respond to our emails.
They aren't part of anything commercial and don't have a pressing need to stay current.
A lot of projects just use it as an engine for their own, in-house js, which they can write any way they want.
As a result, we are able to browse fewer and fewer sites each month.
nasa.gov just switch to some es6 features, for example.

We're not entirely sold on mozjs, also taking a hard look at v8.
But both are c++, and both have a lot of the same challenges for edbrowse integration.
The up side is, both will always stay current, as they are at the heart of commercial browsers.

Karl Dahlke

Re: [edbrowse-dev] Rooted

[Adam Thompson]

On Sun, Nov 01, 2020 at 06:27:41PM -0500, Karl Dahlke wrote:
> No problem Adam.   Thanks for your work long ago, which I'm revisiting, and your thoughts.

That's ok, I've been meaning to have another look at edbrowse code for a
*long* while now but never quite manage.
 
> We almost certainly have to change to something, because duktape is now 2 years out of date, and shows no sign of catching up.
> No proper function.toString(), no es6, no let, no Promise, etc etc.
> They don't respond to our emails.
> They aren't part of anything commercial and don't have a pressing need to stay current.
> A lot of projects just use it as an engine for their own, in-house js, which they can write any way they want.
> As a result, we are able to browse fewer and fewer sites each month.
> nasa.gov just switch to some es6 features, for example.

I thought they were looking at es6 last I heard but that was a while back.
It's a shame if that project's died like this.  On the other hand the github
repo shows commits from October this year including 2.6 release prep.  In
addition it looks like they got promise support (at least from the commit
log) a while ago so it may not be as bad as all that.  The deb package is
from at least late 2019 based on the version numbering so I'm not sure how
out of date they actually are.  It may be that the commit log makes things
look better than they are, I'm not sure.

How do you mean about the lack of email response? that's concerning if
that's the case.

> We're not entirely sold on mozjs, also taking a hard look at v8.
> But both are c++, and both have a lot of the same challenges for edbrowse integration.
> The up side is, both will always stay current, as they are at the heart of commercial browsers.

The problem is that, at least in my experience, neither is really treated as
a library for external consumption.  Sure they're available but they seem to
be somewhat more of a framework sometimes than just a js lib.  May be this
says something about how the web (and thus browsers) are going in terms of
needing more and more features and being increasingly js driven...  I don't
know.  Anyway, as you say, we really need to try and keep up as otherwise
we'll just end up with worse-than-no js support soon which wouldn't be a
good thing.

From what I can see (brief look for debian packages) they appear to have
multiple spidermonkey versions packaged (52 and 78) but nothing which (at
least superficially) looks like v8.  Even the version of nodejs (which I
believe uses v8 under the hood) doesn't have a v8-looking dependency.  this
concerns me from the v8 side as I remember it being...  difficult...  to
compile all those years ago whereas at least spidermonkey was doable.  On
the other hand, the following concerns me from a spidermonkey perspective:
"
Note: Standalone SpiderMonkey is not an official product.
Our continuous integration system does produce a tarball that is built into a
binary that runs our smoke tests, but we do not maintain it nor actively test
its suitability for general embedding.
We have periodically created "releases",
but they are best-effort and incomplete.
We do happily accept patches, and make some effort to keep the tip of the
Gecko tree minimally working as an embeddable source package.
We are very limited in our ability to support older versions,
including those labeled as "releases" on this page.
"

I'm not sure where this leaves us in terms of our ability to keep up.


Cheers,
Adam.