[Edbrowse-dev] Non-technical rant

[Edbrowse-dev] Non-technical rant

[Chuck Hallenbeck]

Hi all,

I would also add my congratulations, and express my gratitude, to
the edbrowse development team.  I have been an edbrowse, archlinux,
and command line addict for many years, and have welcomed each advance
in functionality.  But I have become discouraged recently  because the
moving target we are pursuing seems to be pulling away from us.

Last summer I decided to abandon the linux world and emigrate to an
imac system, where there seemed to be a growing community of happy blind
users.  It was not a decision made lightly, and it was one I now regret.
But having resurrected my former favorites I now find, in addition to
the improvements of recent months, an increase in the accessibility
problem areas as well.  For example, Amazon now works extremely well,
but I can no longer access the third party email service I have used
for the past fifteen years, due to a change in their user interface.
I can continue to use it as is, but if it needs a configuration tweak,
or a subscription renewal, I will need sighted assistance.  My experiment
with the proprietary world was an expensive one, and I think I would
rather continue to rely on edbrowse, linux, and the command line, rather
than deal with the PITA presented by the iMac and its GUI prison.

Sorry for the rant.  I'll do what I can to help out here.  Let's not
worry about this year's Christmas shopping.  There's always next year.

Chuck

Re: [Edbrowse-dev] Non-technical rant

[Chris Brannon]

Chuck Hallenbeck <chuckh@ftml.net> writes:

> But I have become discouraged recently  because the
> moving target we are pursuing seems to be pulling away from us.

I would argue that it has been doing that since about 1995 or so.

> Last summer I decided to abandon the linux world and emigrate to an
> imac system, where there seemed to be a growing community of happy blind
> users.

Deedra has one too.  I made my peace with it, and I use it for browsing
when I have to access fancy sites.
But part of me would sure be glad if I could check my bank balance with
edbrowse.  Perhaps that day is closer than I think?

> For example, Amazon now works extremely well,
> but I can no longer access the third party email service I have used
> for the past fifteen years, due to a change in their user interface.

You've just pointed out one of my major problems with web
applications.  The user interface is subject to change at any moment, at
the sole whim of the developer, with no possibility to downgrade to a
version that you find more acceptable.  In effect, the developer is
dictating to you, and you have no control.
The user interface is tightly coupled to the communication.
Compare this to a non-web-based network service that uses an open
protocol.

Take Usenet for example.   If I want to use Usenet, I can choose among any
number of news readers.  If none of the options suits me for some
reason, I can choose to write my own.
Compare this to a hypothetical web forum, www.hamradioforum.com.  If
I wish to participate in it, I *must* use their user interface.  I
cannot swap it for one that appeals to me.
If their site is edbrowse-friendly, I can at least make the experience a
bit less painless.  If I'm really lucky, hamradioforum.com offers a web
API using JSON and HTTP.  Odds are good that even if they do, it is only
accessible to people who have registered for a developer key.  However,
even if that web API is open for my use, if I write a client against it,
it will most likely only work with hamradioforum.com.  On the other
hand, my Usenet client works with limitless newsgroups on countless
news servers.

I'd argue that one of the fundamental problems of the so-called "modern"
web is the tight coupling between user interface and data transfer.
There's also a severe impedence mismatch, as we are forcing a document
delivery system designed in the early 1990s to serve as some sort of
highly interactive application platform.  It is piles of hacks upon
piles of hacks, rather than sound engineering.  On the other hand, it is
extremely popular, and boat loads of people are making mega bucks doing
it.  So onward we go.

-- Chris

[Edbrowse-dev] Non-technical rant

[Karl Dahlke]

> Take Usenet for example.
> If I want to use Usenet, I can choose among any number of news readers.

Wow, I didn't know usenet still existed.
It was the primary distraction in the workplace in the 80's, Bell Labs
for instance, long before social media etc.
How many hours did I waste away reading and posting to said groups?
My boss none the wiser, since my system had no screen.
I got past that phase though, and figured I better work when I'm working,
but with the novelty of it all I can tell you I wasn't the only one.   :)

I hope that the scripting capabilities of edbrowse allow us
to slightly mash the recalcitrant web interface
into something that we individually want and like,
but of course at the heart of it edbrowse has to work with the website,
js quirks and all.
We may find at some point that the scripting needs more power,
to realize that dream, but first things first.

Karl Dahlke

Re: [Edbrowse-dev] Non-technical rant

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> Wow, I didn't know usenet still existed.

Yes it still does.  Hopefully it always will, but its glory days are
over, thanks to a combination of factors, including spammers and former
attorney general Andrew Quomo of New York.

In fact, you can read this mailing list with a newsreader.  It's on a
server called gmane.org, which aggregates lots of open-source
development lists like this one.  Just point your client at
news.gmane.org and visit the group gmane.comp.web.edbrowse.devel.
As I say, that server isn't linked into Usenet proper.  Think of it as a
disconnected node.

-- Chris

[Edbrowse-dev] alt.ensign.crusher.die.die.die

[Kevin Carhart]

On Thu, 17 Dec 2015, Chris Brannon wrote:

> Karl Dahlke <eklhad@comcast.net> writes:
>
>> Wow, I didn't know usenet still existed.
>
> Yes it still does.  Hopefully it always will, but its glory days are

I'm glad to hear this.  I was an undergrad from 90-95, so I read a lot of 
news also.  There was a group devoted to disliking the Wesley Crusher 
character on Star Trek.

> Sorry for the rant

This is half the fun. :)

Seriously, I have a fastmail account also, so I will pull this up now and 
find out what happens.  I think all roads lead to xhr.  I would be willing 
to bet that fastmail uses it.  BUT-- it's true, there may be something 
more easy that will help.

Kevin

Re: [Edbrowse-dev] alt.ensign.crusher.die.die.die

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]

On Thu, Dec 17, 2015 at 12:56:54PM -0800, Kevin Carhart wrote:
> On Thu, 17 Dec 2015, Chris Brannon wrote:
> >Sorry for the rant
> 
> This is half the fun. :)

Indeed, and I'm glad to find I'm not the only one who thinks this.
Having to work with some fairly inaccessible web stuff at work (read anything
made by Atlassian from my experience)
and consequently spending more than too much time grapling with json APIs to
try and do simple things like move tasks to "in progress"
or check what development tasks I've got to do is a constant frustration.
Fortunately some of these APIs have been scripted by other people,
but it'd be really nice if there was some kind of standard which wasn't "use
the crazy, ever evolving, UI which we force on you".

> Seriously, I have a fastmail account also, so I will pull this up now and
> find out what happens.  I think all roads lead to xhr.  I would be willing
> to bet that fastmail uses it.  BUT-- it's true, there may be something more
> easy that will help.

I don't personally have an account but,
since I've ended up with more time than I expected,
I'm beginning to think that getting a somewhat functional XHR object is more
pressing than a js engine switch.
As such, are there any incremental steps we can take which will get us at least
a basic XHR object which don't require rewriting (or duplicating using an async
design) Edbrowse's comms layer?

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[Edbrowse-dev] XHR

[Kevin Carhart]

>
> I don't personally have an account but,
> since I've ended up with more time than I expected,
> I'm beginning to think that getting a somewhat functional XHR object is more
> pressing than a js engine switch.
> As such, are there any incremental steps we can take which will get us at least
> a basic XHR object which don't require rewriting (or duplicating using an async
> design) Edbrowse's comms layer?

I have some code, but I'm not sure if you will be able
to use it or not.  I managed to get a little further with
some sites by giving it a crude S-JAX.

Currently in startwindow.js we have a stub of an XMLHttpRequest:

XMLHttpRequest = function(){
     this.headers = {};
     this.responseHeaders = {};
     this.aborted = false;//non-standard
};

But off in my not-yet-submitted code, I have an
implementation of open:

     open: function(method, url, async, user, password){

And also of 'send'.  To do the HTTP, I hooked it up with a
simple curl call back in the C++ file.  It accepts
parameters, it runs a request-response, and returns
the output to a javascript variable.  Probably insecure!
It is not intended for the robust edbrowse in its
current form.

So it's a preliminary stab, which doesn't yet have
two massive things, (a) to be asynchronous and
(b) bringing your response back to the edbrowse
tree (if needed - I thought maybe the process of
how to splice the new XHR response text back into
the tree on the JS side would cover it, because
then our existing side effects would take it back
to the edbrowse tree)

All it is right now is a side HTTP getter which
does not work as a continuous DOM splicer, but
executes a second GET/POST, puts the resulting
"raw material" in a variable and leaves it up
to you to do something with it.

Maybe the incremental steps of which you speak
would be to do some of this, only well.  :)

Kevin











This duplicates http.c,
but I didn't know how to get back to the C code.

And of the send method.  And for the HTTP action, I
just wrote something that works like other native implementations
of a javascript keyword.  It just accepts the appropriate
parameters and calls curl.

And of the send method, where all I did was redundantly
bring over a routine that can call curl, into jseng-moz.





>
> Cheers,
> Adam.
>

--------
Kevin Carhart * 415 225 5306 * The Ten Ninety Nihilists

Re: [Edbrowse-dev] XHR

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 3153 bytes --]

On Sat, Dec 19, 2015 at 03:40:32PM -0800, Kevin Carhart wrote:
> >
> >I don't personally have an account but,
> >since I've ended up with more time than I expected,
> >I'm beginning to think that getting a somewhat functional XHR object is more
> >pressing than a js engine switch.
> >As such, are there any incremental steps we can take which will get us at least
> >a basic XHR object which don't require rewriting (or duplicating using an async
> >design) Edbrowse's comms layer?
> 
> I have some code, but I'm not sure if you will be able
> to use it or not.  I managed to get a little further with
> some sites by giving it a crude S-JAX.

Ok, that sounds like something at least.

> Currently in startwindow.js we have a stub of an XMLHttpRequest:
> 
> XMLHttpRequest = function(){
>     this.headers = {};
>     this.responseHeaders = {};
>     this.aborted = false;//non-standard
> };
> 
> But off in my not-yet-submitted code, I have an
> implementation of open:
> 
>     open: function(method, url, async, user, password){
> 
> And also of 'send'.  To do the HTTP, I hooked it up with a
> simple curl call back in the C++ file.  It accepts
> parameters, it runs a request-response, and returns
> the output to a javascript variable.  Probably insecure!

Rather than a js variable I think it should produce some sort of response
object, but other than that I think that's how it's supposed to work,
not just splicing things directly into DOM.

> It is not intended for the robust edbrowse in its
> current form.

I don't think we're intending to do another release for a little while,
so if it isn't robust at the moment we can battle-test and fix it prior to that point.

> So it's a preliminary stab, which doesn't yet have
> two massive things, (a) to be asynchronous and
> (b) bringing your response back to the edbrowse
> tree (if needed - I thought maybe the process of
> how to splice the new XHR response text back into
> the tree on the JS side would cover it, because
> then our existing side effects would take it back
> to the edbrowse tree)

You're correct I think in that we should suck this back or discard it depending on the js. This should be existing functionality.

> All it is right now is a side HTTP getter which
> does not work as a continuous DOM splicer, but
> executes a second GET/POST, puts the resulting
> "raw material" in a variable and leaves it up
> to you to do something with it.

Sounds good.

> Maybe the incremental steps of which you speak
> would be to do some of this, only well.  :)

Sounds fine, perhaps we can alter it to use http.c rather than curl directly,
using some of the background downloading (but into memory via temp files or
some such magic) to do the async part almost "for free".
I'm not sure though.

It's definitely something we need to look at.
Then we just need to add OPTION requests and CORS support (and probably some
more async plumbing) and we'll have something like an AJAX feature.

Anyway, I think it'd be useful to clean up and submit what you've got so we can see where we are.

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] XHR

[Kevin Carhart]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 323 bytes --]



> Anyway, I think it'd be useful to clean up and submit what you've got so 
> we can see where we are.

Great, here is a zip of startwindow.js and jseng-moz.cpp.
The changes in startwindow begin after line 1074.
The change in jseng is a routine called fetchAsynchronous, which
is also added to document_methods[].

Kevin

[-- Attachment #2: Type: APPLICATION/zip, Size: 26507 bytes --]

Re: [Edbrowse-dev] XHR

[Kevin Carhart]

I also meant to accompany that code with a sample transcript.  I'm 
instantiating the XMLHttpRequest prototype.

The sample has no particular connection to the fastmail work, I
just used fastmail.com as an example URL that demonstrates that a real
HTTP action was undertaken.  You can tell because real response
contents are returned.


> jdb

> var client = new XMLHttpRequest

> var df = document.forms[0]

> var payload = form_serialize(df)

> ok(client)

< headers,responseHeaders,aborted

> client.method = "POST"

< POST

> client.url = df.action

< https://classic.fastmail.com/html/?u=4c8b41a9

> client.send(payload)

< * Adding handle: conn: 0x10b47a0... [a variety of detail is echoed]

> ok(client)

< headers,responseHeaders,aborted,method,url,responseText,readyState

> ok(client.responseHeaders)

< HTTP/1.1 403 Forbidden,Server,Date,Content-Type,Content-Length,Connection,X-Request-Id...

> client.responseText

< <!DOCTYPE html>.....

[Edbrowse-dev] XHR

[Karl Dahlke]

> Great, here is a zip of startwindow.js and jseng-moz.cpp.

Kevin this is definitely blazing a trail.
If you don't mind I'm going to postpone looking at it,
for a time, while I rebundle things,
because it seems we all agree that much of http.c should be shared
and reused for this purpose, otherwise we're going to be reinventing too much.
After that is done I'll dive in to your zip file
and maybe we, together, can find the right http routines
to call instead of curl directly,
to take advantage of all that machinery we have developed.

Meantime there are still plenty of small bugs to fix in the existing js
and cookies and so on.
I really thought I had read somewhere that id= stands in for name=,
but Chris says maybe it doesn't, so IDK.

Cheers.

Karl Dahlke

Re: [Edbrowse-dev] XHR

[Kevin Carhart]

>> Great, here is a zip of startwindow.js and jseng-moz.cpp.
>
> Kevin this is definitely blazing a trail.
> If you don't mind I'm going to postpone looking at it,
> for a time, while I rebundle things,
> because it seems we all agree that much of http.c should be shared

Thanks Karl!  Yes, any usage, whatever works.
It's going to be superceded by the ability to call http.c
and have a better foundation.

> Meantime there are still plenty of small bugs to fix in the existing js
> and cookies and so on.

Yes, one thing I think I noticed in the output from fastmail
is a JS runtime where it tried to find nextSibling or previousSibling.
I think that is another way of moving among nodes,
comparable to child and parent moves.

> I really thought I had read somewhere that id= stands in for name=,
> but Chris says maybe it doesn't, so IDK.

Hmmmm.  So here are some thoughts on fastmail and the
issues raised:
- According to Chuck, basic actions in fastmail work now!
- In retrospect, I'm not sure if the use_classic cookie and
the way it gets into document.cookie was the culprit, because
I think the two hiddens without a name were the cause of the
session key mismatch error.
- I reported a doubling at one point (use_classic; use_classic;)
but I never hit this again so it was probably a sidetrack.
- Possibly the two hiddens without a name have their name
attribute grafted in by javascript.  That would be
a plausible reason
for why this popular, robust website has something illegal
just laying around.  That it actually isn't illegal, but
there is JS that is supposed to deal with it and isn't
running.
We could keep this formulation in mind - it seems
like something I have stumbled on before: in the rendered
html, the user might hit something incongruous like
<Log in><Log out>
Combined with the fact that it doesn't work right.
One reason it could be this way is that site
authors have coded a
superset in their server-side code, and are intending
on using JS to always take away one of those on the
client side before it reaches the user.  But then maybe
if the JS file breaks on something else (like Sibling
or other things), the JS file bails out and the code
responsible for the take-away is never reached.  I
think maybe the candy-store website has this too.

happy holidays,
Kevin

Re: [Edbrowse-dev] XHR

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On Tue, Dec 22, 2015 at 12:04:29PM -0800, Kevin Carhart wrote:
> We could keep this formulation in mind - it seems
> like something I have stumbled on before: in the rendered
> html, the user might hit something incongruous like
> <Log in><Log out>
> Combined with the fact that it doesn't work right.
> One reason it could be this way is that site
> authors have coded a
> superset in their server-side code, and are intending
> on using JS to always take away one of those on the
> client side before it reaches the user.  But then maybe
> if the JS file breaks on something else (like Sibling
> or other things), the JS file bails out and the code
> responsible for the take-away is never reached.  I
> think maybe the candy-store website has this too.

I don't know about that specific website but yeah, I've seen this a lot.
there are also sites and frameworks (names escape me right now)
which use this kind of thing as a sort of app level cache mechanism by sending
everything then deleting bits and dynamically filling in others with...
you guessed it... AJAX.

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] Non-technical rant

[Kevin Carhart]

Also, to highlight the good news too:

> problem areas as well.  For example, Amazon now works extremely well,

Is that right!  Well that's great to hear!

[Edbrowse-dev] masking of passwords

[Kevin Carhart]

This is established edbrowse behavior and not new,
so it may already have been discussed or
concluded.
But does it seem a little worrisome that passwords
are echoed as they are typed?
I just noticed this on fastmail.

I know that at the time I type i2=password,
or something, edbrowse
has no way of knowing what I want to do next or
the fact that I'm about to address i2, which is
an html form element of type "password".
So it's impossible to mask out the password with
asterisks or dots the way you can if you're
typing into something that is pre-established to
be a password.
But what if there was an invisible mode?

i1=username
dne (or some keys meaning 'enter the do-not-echo mode')
i2=password
dne
i3*

Kevin




On Thu, 17 Dec 2015, Kevin Carhart wrote:

>
> Also, to highlight the good news too:
>
>> problem areas as well.  For example, Amazon now works extremely well,
>
> Is that right!  Well that's great to hear!
>
>
>
> _______________________________________________
> Edbrowse-dev mailing list
> Edbrowse-dev@lists.the-brannons.com
> http://lists.the-brannons.com/mailman/listinfo/edbrowse-dev
>

--------
Kevin Carhart * 415 225 5306 * The Ten Ninety Nihilists

Re: [Edbrowse-dev] masking of passwords

[Chris Brannon]

Kevin Carhart <kevin@carhart.net> writes:

> I know that at the time I type i2=password,
> or something, edbrowse
> has no way of knowing what I want to do next

Yeah, generally that is true.  However, I've seen programs be pretty
smart about this.  For example, the IRC client weechat will
start printing masking characters as soon as you type the string
/msg nickserv identify
For those not familiar with IRC, this is often how you authenticate your account,
by sending a private message to a bot named nickserv.
We could do that kind of cleverness in edbrowse,
but I like your "invisible mode" idea.

-- Chris

Re: [Edbrowse-dev] masking of passwords

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1965 bytes --]

On Thu, Dec 17, 2015 at 01:55:16PM -0800, Chris Brannon wrote:
> Kevin Carhart <kevin@carhart.net> writes:
> 
> > I know that at the time I type i2=password,
> > or something, edbrowse
> > has no way of knowing what I want to do next
> 
> Yeah, generally that is true.  However, I've seen programs be pretty
> smart about this.  For example, the IRC client weechat will
> start printing masking characters as soon as you type the string
> /msg nickserv identify
> For those not familiar with IRC, this is often how you authenticate your account,
> by sending a private message to a bot named nickserv.
> We could do that kind of cleverness in edbrowse,
> but I like your "invisible mode" idea.

Doing that would require a change in how edbrowse handles terminal input.
Specifically we currently wait for a line to be entered before we process it,
but in order for such fancy things as i2= to cause imediate password masking
we'd need to process each keystroke.

What I generally do (on linux) is use:
!stty -echo

To enter the password then (quickly):
!clear

To remove the printed line. I suspect displayLine could be altered in
some way to avoid the screen printing,
but I'm not so sure about the non-echoing.
I think that, rather than an invisible mode,
I'd prefer something like a pw command which'd take a field number and then
display a non-echoing prompt, i.e.:
pw2
Password for field 2: <password goes here>

And then have the field print as ... (i.e. for my gmail account this would read):
<arthompson1990@gmail.com> <...>

The reason for the fixed ... is that it means you can't guess password length
(probably rather paranoid). If the field was blank then it would print as a blank field.
If someone used i2 rather than the pw command then the field would still print as ...
but obviously the i2 line would remain both visible and in the readline history
(in readline mode).

Any thoughts?

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[Edbrowse-dev] masking of passwords

[Karl Dahlke]

> I'd prefer something like a pw command which'd take a field number and then

I like this idea all the way around,
perhaps I will make incremental steps in that direction.
I think displayLine can substitute <...>
as I wrote earlier, but I need to go back and look at the code.

Karl Dahlke

Re: [Edbrowse-dev] masking of passwords

[Kevin Carhart]

Note on the passwords thread, if someone runs
with it and knows how to do it, thank you.  It is basically
a feature request from me as a user, since I don't think
I know how to follow through and D.I.Y. and it
isn't trivial, as Karl was saying.

Kevin

[Edbrowse-dev] masking of passwords

[Karl Dahlke]

> But what if there was an invisible mode?

Well selfishly, I don't have a screen, so I don't care.
But seriously...

This is really a nontrivial effort.
Sure I can suppress echo at the tty,
but as soon as you hit return edbrowse prints the line with the password in position.
There it is on the screen.
Or if you ever reprint that line, there it is again.
You can't just replace the text with stars because that very text
is passed to our html tree and then to the server upon submit.
The only way around this is to keep the text in place but muck with displayLine()
so that if the tag is type password then everything between <> is replaced with stars.
That could be done but now it's a piece of work
at the input level, invisible mode, and the output level, displayLine.
The invisible mode would have to supress echo under both cooked tty and readline,
and then the same for the windows port,
the latter I definitely don't know how to do.

Karl Dahlke

Re: [Edbrowse-dev] masking of passwords

[Kevin Carhart]

On Thu, 17 Dec 2015, Karl Dahlke wrote:

>> But what if there was an invisible mode?
>
> Well selfishly, I don't have a screen, so I don't care.
> But seriously...

I know, but ... we could get a burst of popularity
from the Windows port, and those users could tend
to have a monitor, perhaps.

> This is really a nontrivial effort. Sure I can suppress echo at the tty,

Yeah.  Hmmmmmmmmmmm.

Kevin