[Edbrowse-dev] interprocess messages

[Edbrowse-dev] interprocess messages

[Karl Dahlke]

As promised, here are some thoughts on messages between eb and the back-end
js process.
Please remember that this is intended as increment,
and so there are messsages to do the things we do today,
while other messages and functionality will come later.

Edbrowse currently maintains a session number,
which is printed by the e command.
In addition to this, edbrowse will maintain a stack number for
the buffers that stack up per session.
Session number and stack number
determines uniquely an edbrowse buffer, a window/document,
and a javascript context.
This combined number will be the context number,
passed between the two processes to identify the js context.

e www.ibm.com
session 1, stack 1, context 100001 (or some such).

Upon browse, if js is not disabled,
eb sends message to js to create the context.
If js is not yet launched, fork/exec the js process.
This process creates the js runtime, allocates the heap, etc.
If this initialization cannot happen it returns an error,
which edbrowse prints, and then disables js,
so you don't get this error again and again on every browse command.
So let's assume js launches and succeeds with a runtime environment.
It creates the js context for the corresponding edbrowse window and returns.

If at any time the js process dies, or is killed,
edbrowse cannot read or write to said process.
At this point it loops through all windows in all sessions
and sets js_dead = true in each of them.
The next browse will reinvoke the js process, as if it were the first browse.
If this succeeds, then in any other session you can unbrowse
and browse, and that action will create a fresh js context for that session.
You could perhaps restart a preexisting web page, even after a js disaster.
Of course we hope these disasters are rare.

There is something that is never done today, but must happen in the future.
One js context, through frames or parent pointers,
can access or modify variables in another js context.
This is another reason to keep them all in the same process.
I'm going to hold off on this for a future incremental step,
but it needs to happen some day.

So far we have the first message, and a status return:

1. create java context for this context number

The next message is of course, and predictably,
to destroy the context, because the window is closed.

2. distroy javascript context for this window

3. destroy javascript runtime process, issued when edbrowse quits completely.
Even if you didn't issue this message, edbrowse exits, and then js exits
on broken pipe.

Subsequent messages fetch or establish objects in the js world,
or properties of those objects.
The return from each request begins with a high level status code.
0 = ok
1 = syntax error or other problem with this particular command.
2 = this js context has failed, set js_dead = true
3 = the entire js heap is compromised, set js_dead = true
   across all sessions and all windows.
4 = js cannot run, no fork, no exec, no js runtime, etc.
   Mark each buffer with js_dead = true and also disable javascript
   so this error does not come again and again with each browse.
   Of course you can re enable javascript and try to browse again.
   Error 4 can only arise from the create java context request.

After the high status number is a low status number,
that edbrowse can use to print a particular error message.
If the status is 0 = ok then the rest of the message
returns whatever edbrowse was asking for.
With this in mind, here are some more ipc messages.

Just as contexts have numbers, so objects within contexts have numbers.
These are assigned by the js process.
They must be, since running javascript can create more objects,
quite apart from those initially created by html.
If edbrowse asks js to create an object corresponding to a dom entity,
js will return the number of said object for future reference.
More on this below.

4. compile and execute a chunk of js code.

5. get property from an object, passes object number and property name,
returns type indicator and a string representation of the property.
Property could be string, integer, float, boolean, function, or another object
whence the presentation is the object number.

6. set property within an object, passes object number,
type of property, and string representation of that property.
Additional parameters are create, when the property is created first time,
and readonly, if the property is created readonly.
Nothing is returned, other than the usual status bytes.

7. Run function, usually an event handler.
pass the dom object number that is the "this" for the running function,
and the name of the function.

Perhaps 2 or 3 more ipc messages that I haven't thought of,
but this should give us the functionality we have today.

Around this I would write wrappers that are essentially the
same functions we have today.
So get_property_string would do what it does now but do so by passing
a message and getting a response.
As much of this as I can do, would make it that much easier to switch over
to the interprocess system.
It would be a manageable increment.

Karl Dahlke

Re: [Edbrowse-dev] interprocess messages

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]

On Mon, Dec 08, 2014 at 02:27:20PM -0500, Karl Dahlke wrote:
> As promised, here are some thoughts on messages between eb and the back-end
> js process.
> Please remember that this is intended as increment,
> and so there are messsages to do the things we do today,
> while other messages and functionality will come later.

Ok, how are you planning on implementing this in terms of ipc?
Pipes, sockets or message queues would be the three I'd think of here.
Also, with the aim of incrementing in the right direction,
how would this play with blocking calls,
and the ability to run js asynchronously?
If we're going to do this kind of redesign we need to look at this stuff or in
a few years edbrowse will be basically unusable on the internet due to the
(often ridiculous) amount of ajax and friends being used on modern websites.
The more I think about how this is going to work,
the more I think we need to look at dynamically managing the js heap and having
a js process per session (at least) in order to make things work on the modern internet.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] interprocess messages

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> As promised, here are some thoughts on messages between eb and the back-end

This looks like a very solid start to me.

I assume that for convenience, you'll be calling these two processes eb
and js, so I use those names below.

I noticed that there's not a corresponding "destroy object" message, but
the main eb process never really needs to be able to do that, does it?
On the other hand, the js process might very well destroy objects that
are known to the eb process, so we need some kind of message that can be
sent from js to eb:
object_was_destroyed, with parameters of a context number and object
number.

-- Chris

Re: [Edbrowse-dev] interprocess messages

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]

On Tue, Dec 09, 2014 at 07:57:30AM -0800, Chris Brannon wrote:
> Karl Dahlke <eklhad@comcast.net> writes:
> 
> > and compiling js code into native cpu code for speed.
> > Really? A true js compiler?
> 
> I'm not sure how familiar you are with Google Docs, but they've
> implemented a word processor, spread sheet, and presentation
> manager as web apps.  So they definitely have an interest in making all
> this stuff fast.  Also, on devices like the ChromeBook, the OS is
> basically the browser.  Yeah, it's Linux under the hood, but unless you
> replace the software on it, you'll be doing everything through the
> browser.

Yeah, not to mention all the other websites which are following google's lead here.
However, I personally have reservations about it,
especially when people start trying to tell me how fast it is for doing proper
number crunching.

> And then there's just plain sillyness, like the x86 emulator implemented
> in JS.  Yes, you can actually run Linux in an x86 emulator inside your
> browser.  Why you'd want to do that is another question entirely.
Wow, I never new about the x86 thing.
I've heard of a lua vm written in js (again why?)
but an entire x86 setup's pushing things to the next level of wtf (however,
I kind of want to try it just once... primarily because it's so crazy).
> 
> Regarding my earlier post about V8, here are some links
> describing the issues that Gentoo had with it.
> 
> http://comments.gmane.org/gmane.linux.gentoo.devel/88222
> http://phajdan-jr.blogspot.com/2013/11/when-libraries-you-use-are-moving-too.html

Yeah, these comments don't exactly encourage confidence that we'll be getting
anything much more maintainable.

> I'm not at all against the switch, if it will be nicer to deal with.
> But if we do switch, we probably want to bundle our own version of V8
> with edbrowse.

That sounds a bit painful as far as handling security issues goes tbh.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] interprocess messages

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 2654 bytes --]

On Tue, Dec 09, 2014 at 09:13:02AM -0500, Karl Dahlke wrote:
> > Pipes, sockets or message queues would be the three I'd think of here.
> 
> Right. Probably pipes cause it's easy, but it mostly doesn't matter
> and could be easily changed one to another.

Right, when I did this last time I landed on message queues eventually since the interface was a bit nicer to use (you put a message in, you get a message out, no need to worry about reading the correct amount etc), but I don't really mind any of the three.
> 
> > how would this play with blocking calls,
> 
> All three of these, and any others you might think of,
> block by default, and can also be set nonblocking,
> but I've learned over the years nonblocking is a bad idea.
> Never make a nonblocking call.
> My jupiter speech adapter for example - reads index markers
> coming back from the synthesizer asynchronously, and your keyboard commands,
> perhaps telling speech to shut up, and maybe speech
> commands coming from other processes through the fifo /etc/jupiter.fifo.
> Three asynchronous streams to monitor simultaneously,
> and the right way to do this is by calling select.
> In other words, if a primary process is monitoring many asynchronous
> events from other processes / streams, select() is your friend.
> It does exactly what you want.
> This is deep in the kernel, and thank god they put it there,
> because you just can't simulate it through any kind of user space programming.
> Anyways, I believe we will be able to migrate to various asynchronous events,
> ajax, download to disk in background, etc etc, when the time comes.

Agreed (though on a scalability note, as we're on linux anyway,
I'd go for epoll over select, but select is more portable,
and with 3 descriptors there's no noticeable, or probably measurable,
performance difference).

> I will see if I can bang this first step out,
> partly because I haven't programmed in about a year,
> and I find that I really miss it.
> When you're not programming for your boss / professor, it really is fun.
> I estimate a couple of weeks.

Sounds good.

> Meantime you (plural) might look at v8 and just evaluate.
> I still haven't gotten a hello world program to compile,
> and honestly that concerns me.

Yeah, that does sound rather concerning.
When I've got a bit of time (this weekend may be) I'll have a look at the Debian packaged version and see how far I get.
I'm also investigating if there are any other js engines which are in a
suitable state for use in a web browser.
I've got no solid candidates yet but I've got a couple of ideas.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] interprocess messages

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> and compiling js code into native cpu code for speed.
> Really? A true js compiler?

I'm not sure how familiar you are with Google Docs, but they've
implemented a word processor, spread sheet, and presentation
manager as web apps.  So they definitely have an interest in making all
this stuff fast.  Also, on devices like the ChromeBook, the OS is
basically the browser.  Yeah, it's Linux under the hood, but unless you
replace the software on it, you'll be doing everything through the
browser.

And then there's just plain sillyness, like the x86 emulator implemented
in JS.  Yes, you can actually run Linux in an x86 emulator inside your
browser.  Why you'd want to do that is another question entirely.

Regarding my earlier post about V8, here are some links
describing the issues that Gentoo had with it.

http://comments.gmane.org/gmane.linux.gentoo.devel/88222
http://phajdan-jr.blogspot.com/2013/11/when-libraries-you-use-are-moving-too.html

I'm not at all against the switch, if it will be nicer to deal with.
But if we do switch, we probably want to bundle our own version of V8
with edbrowse.

Getting a V8 hello world is definitely on my to-do list.

-- Chris

[Edbrowse-dev] interprocess messages

[Karl Dahlke]

> Pipes, sockets or message queues would be the three I'd think of here.

Right. Probably pipes cause it's easy, but it mostly doesn't matter
and could be easily changed one to another.

> how would this play with blocking calls,

All three of these, and any others you might think of,
block by default, and can also be set nonblocking,
but I've learned over the years nonblocking is a bad idea.
Never make a nonblocking call.
My jupiter speech adapter for example - reads index markers
coming back from the synthesizer asynchronously, and your keyboard commands,
perhaps telling speech to shut up, and maybe speech
commands coming from other processes through the fifo /etc/jupiter.fifo.
Three asynchronous streams to monitor simultaneously,
and the right way to do this is by calling select.
In other words, if a primary process is monitoring many asynchronous
events from other processes / streams, select() is your friend.
It does exactly what you want.
This is deep in the kernel, and thank god they put it there,
because you just can't simulate it through any kind of user space programming.
Anyways, I believe we will be able to migrate to various asynchronous events,
ajax, download to disk in background, etc etc, when the time comes.

I will see if I can bang this first step out,
partly because I haven't programmed in about a year,
and I find that I really miss it.
When you're not programming for your boss / professor, it really is fun.
I estimate a couple of weeks.
Meantime you (plural) might look at v8 and just evaluate.
I still haven't gotten a hello world program to compile,
and honestly that concerns me.
Much of their documentation is all excited about setting speed records,
and compiling js code into native cpu code for speed.
Really? A true js compiler?
What the fuck for?
To see how fast you can jerk off?
As we've said so many times, performance is not really an issue here,
how about reliability and portability and flexibility?
The last thing I'd want to do is turn js into machine code.
Well google sometimes does things just to do them, just because they can.
Still it looks like a cleaner interface, without all the compartment confusion,
and I'm sure those compartments are really going to haunt us when one js
context travels through document.frames[2].whatever to access another context.
We don't do this stuff now and we need to and I'm not sure
how to cross into the other compartments
and honestly I'm not sure how firefox does it eithere, but it must.
I'm thinking intercontext stuff is easier in v8.
But I really don't have a clue.
Anyways, I'll get back to my ipc work, which is moving forward nicely.
Chris, as maintainer you might want to ask if we should mark
another version before taking the ipc step.
I think there were just little tweaks here and there since 3.5.1,
but some of them fixed some small bugs.
Look at logs and see what you think.

Karl Dahlke