edbrowse-dev - development list for edbrowse
 help / color / mirror / Atom feed
* [Edbrowse-dev] protocol handlers and outputting to buffer
@ 2018-01-26 18:01 Adam Thompson
  2018-01-26 18:13 ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-01-26 18:01 UTC (permalink / raw)
  To: Edbrowse-dev

Hi,

Whilst messing around with protocol handlers in edbrowse (specifically to make a gopher plugin), I found that the protocol based plugin support has no support for outputting to the buffer as in the converter plugin support.
Specifically, whereas running a converter plugin allows a pipe, the protocol handlers in http.c always call the command using eb_system().
Is there any reason for this or would it be possible to use the command running code from runPluginConverter (may be by pulling out into a generic function so the return codes work) in the protocol handler running code in http.c?
This'd allow other protocols to be added as plugins whilst still keeping the edbrowse interface for cases where one wants to download (and possibly preprocess) using an unsupported protocol
but display and interact using the edbrowse buffers.

If there's no objections to the change I can make the change as I've some time this weekend.

Cheers,
Adam.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-01-26 18:01 [Edbrowse-dev] protocol handlers and outputting to buffer Adam Thompson
@ 2018-01-26 18:13 ` Karl Dahlke
  2018-01-27  9:58   ` Adam Thompson
  0 siblings, 1 reply; 12+ messages in thread
From: Karl Dahlke @ 2018-01-26 18:13 UTC (permalink / raw)
  To: Edbrowse-dev

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

Yes that sounds like a good idea.
Go for it if you have the time, but be careful, httpConnect() is a bit of a house of cards, and I apologize for that.
Also none of that stuff is in jsrt so you might want to retest some of the existing plugins and connectors to make sure they all still work;
and I'll do the same once you have pushed.

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-01-26 18:13 ` Karl Dahlke
@ 2018-01-27  9:58   ` Adam Thompson
  2018-01-27 14:40     ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-01-27  9:58 UTC (permalink / raw)
  To: Karl Dahlke; +Cc: Edbrowse-dev

On Fri, Jan 26, 2018 at 01:13:46PM -0500, Karl Dahlke wrote:
> Yes that sounds like a good idea.
> Go for it if you have the time, but be careful, httpConnect() is a bit of a house of cards, and I apologize for that.
> Also none of that stuff is in jsrt so you might want to retest some of the existing plugins and connectors to make sure they all still work;
> and I'll do the same once you have pushed.

Thanks, I'll have a go at it.  It'll be good to get back into the edbrowse code rather than just using it.

Any idea how I can hook plugins up to use the rest of the edbrowse browsing mechanism (not massively required but could be useful once we get protocol handlers that can download into the buffers)?

Cheers,
Adam.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-01-27  9:58   ` Adam Thompson
@ 2018-01-27 14:40     ` Karl Dahlke
  2018-02-01 19:58       ` Adam Thompson
  0 siblings, 1 reply; 12+ messages in thread
From: Karl Dahlke @ 2018-01-27 14:40 UTC (permalink / raw)
  To: Edbrowse-dev

> Any idea how I can hook plugins up to use the rest of the edbrowse browsing

I'm not sure what you're asking, but it might already exist.
In the plugin descriptor in the config file, it is assumed to play, like music or video, unless outtype is set.

outtype = T

Text, like programs that convert word .doc into ascii, or rich text .rtf into ascii, etc.

outtype = H

Output is assumed html, and then browsed.

plugin {
type = pdf
desc = pdf file
suffix = pdf
content = application/pdf
program = pdftohtml -i -noframes %i %o >/dev/null
outtype = H
}

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-01-27 14:40     ` Karl Dahlke
@ 2018-02-01 19:58       ` Adam Thompson
  2018-02-01 20:28         ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-02-01 19:58 UTC (permalink / raw)
  To: Karl Dahlke; +Cc: Edbrowse-dev

On Sat, Jan 27, 2018 at 09:40:56AM -0500, Karl Dahlke wrote:
> > Any idea how I can hook plugins up to use the rest of the edbrowse browsing
> 
> I'm not sure what you're asking, but it might already exist.
> In the plugin descriptor in the config file, it is assumed to play, like music or video, unless outtype is set.

Yes, what I'd like to do is provide type information such that I can, for example, download the content then run a converter as if I'd done the download with a natively supported protocol.
Not a big thing but may be a useful ability and if I'm working on this stuff anyway I may as well do it if possible.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-01 19:58       ` Adam Thompson
@ 2018-02-01 20:28         ` Karl Dahlke
  2018-02-09 18:44           ` Adam Thompson
  0 siblings, 1 reply; 12+ messages in thread
From: Karl Dahlke @ 2018-02-01 20:28 UTC (permalink / raw)
  To: Edbrowse-dev

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

I've been working on a lot of stuff in the meantime, but I think it's all orthogonal, touching different sourcefiles, so we shouldn't collide.

I'm still not sure where you're headed.
If there is a change to the user interface, could you float that by us first, before you implement?
Maybe this situation is analogous.
If you have a file in buffer and you want to play it but it has no suffix and nothing to key on to tell you how to play it,
you can type pb.mp3  play buffer like mp3, or pb.wav etc.
Are you thinking something like   b.pdf   url
download, and render like it's pdf?
g.rtf go to this link and render like it's rich text?
Or is it all in the config file:
plugin {
if protocol is gopher than render this way.
}
Or something else????

Pretend like you're writing the usersguide documentation first (which nobody does),
and post that and then maybe I can say something intelligent.

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-01 20:28         ` Karl Dahlke
@ 2018-02-09 18:44           ` Adam Thompson
  2018-02-09 20:10             ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-02-09 18:44 UTC (permalink / raw)
  To: Karl Dahlke; +Cc: Edbrowse-dev

On Thu, Feb 01, 2018 at 03:28:18PM -0500, Karl Dahlke wrote:
> I've been working on a lot of stuff in the meantime, but I think it's all orthogonal, touching different sourcefiles, so we shouldn't collide.
> 
> I'm still not sure where you're headed.
> If there is a change to the user interface, could you float that by us first, before you implement?

It *shouldn't* need a ui change.
Basically gopher has a notion of content types and what I'd like to do is have a new output type for example:
outtype=v
Would mean that the output type of the handler is variable and then have some mechanism for the protocol handler to tell the rest of the browsing
mechanism what it downloaded.  For example one may download an html file via gopher or a sound file.
It'd be really nice if the rest of the plugins could run as normal, as if the browser had gone to the URL via http.
I'm not quite sure how one would accomplish this in an efficient way however.

Cheers,
Adam.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-09 18:44           ` Adam Thompson
@ 2018-02-09 20:10             ` Karl Dahlke
  2018-02-11 16:41               ` Adam Thompson
  0 siblings, 1 reply; 12+ messages in thread
From: Karl Dahlke @ 2018-02-09 20:10 UTC (permalink / raw)
  To: Edbrowse-dev

curl has gopher support.
Tell edbrowse to use curl to download anything gopher, just as it downloads http or https.
Look at http.c line 937, perhaps we can just add gopher to the disjunction.
Perhaps headers will come across, like http headers, with content type, and of course the file could have suffixes.
I'd play around with it but don't have a gopher server with various files to test.
Unless there is a public one.
Just change that one line and run with db4 and see what kind of headers come across.

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-09 20:10             ` Karl Dahlke
@ 2018-02-11 16:41               ` Adam Thompson
  2018-02-11 16:54                 ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-02-11 16:41 UTC (permalink / raw)
  To: Karl Dahlke; +Cc: Edbrowse-dev

On Fri, Feb 09, 2018 at 03:10:24PM -0500, Karl Dahlke wrote:
> curl has gopher support.
> Tell edbrowse to use curl to download anything gopher, just as it downloads http or https.
> Look at http.c line 937, perhaps we can just add gopher to the disjunction.
> Perhaps headers will come across, like http headers, with content type, and of course the file could have suffixes.
> I'd play around with it but don't have a gopher server with various files to test.
> Unless there is a public one.
> Just change that one line and run with db4 and see what kind of headers come across.

Yeah I know, I was contemplating patching the support directly into edbrowse but wasn't sure which would be
easier plus it'd be good if plugins could more closely integrate with the rest of edbrowse in any case.

On the subject of gopher specifically it has no headers, the content type is done in the selector.  It also has a menu which needs to be parsed (basically a list of selectors and text).  Quite simple but would need the
rendering code to be aware of it somehow.
I'm not sure if I could do something like the ftp code?

Anyway, sdf.org (and sdf-eu.org) run a gopher server, and there're a bunch more (including a project
gutenberg one but I'm not sure how standard that one is).

Cheers,
Adam.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-11 16:41               ` Adam Thompson
@ 2018-02-11 16:54                 ` Karl Dahlke
  2018-02-11 17:14                   ` Adam Thompson
  0 siblings, 1 reply; 12+ messages in thread
From: Karl Dahlke @ 2018-02-11 16:54 UTC (permalink / raw)
  To: Edbrowse-dev

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Thank you for raising this issue.
Edbrowse has native support for ftp, but ftp has no headers and no help, so edbrowse is a bit limited in its understanding, as is every other ftp client.
I know what you mean about a gopher menu, ftp presents each directory as a menu, showing subdirectories and files.
Clients understand this, as does edbrowse.
I turn a menu listing into html, with links, ftp links, that take you to subdirectories and/or files.
It works pretty well.
I imagine something like that could be done for gopher, and I'd say it has to be native, like ftp, I don't think this is a plugin situation.
I'll play around with the public gopher sites as time permits,
this is something edbrowse should do.

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-11 16:54                 ` Karl Dahlke
@ 2018-02-11 17:14                   ` Adam Thompson
  2018-02-11 17:40                     ` Karl Dahlke
  0 siblings, 1 reply; 12+ messages in thread
From: Adam Thompson @ 2018-02-11 17:14 UTC (permalink / raw)
  To: Karl Dahlke; +Cc: Edbrowse-dev

On Sun, Feb 11, 2018 at 11:54:53AM -0500, Karl Dahlke wrote:
> Thank you for raising this issue.
> Edbrowse has native support for ftp, but ftp has no headers and no help, so edbrowse is a bit limited in its understanding, as is every other ftp client.
> I know what you mean about a gopher menu, ftp presents each directory as a menu, showing subdirectories and files.
> Clients understand this, as does edbrowse.
> I turn a menu listing into html, with links, ftp links, that take you to subdirectories and/or files.
> It works pretty well.
> I imagine something like that could be done for gopher, and I'd say it has to be native, like ftp, I don't think this is a plugin situation.
> I'll play around with the public gopher sites as time permits,
> this is something edbrowse should do.

Yeah, if it's doing the ftp list to html thing natively, where's that code?  I could probably put together similar for gopher, the tricky part is
getting the type info from the selectors into edbrowse.  Also, gopher has simple forms which we should probably implement.
My thought for using a plugin was that I could hammer together a proof of concept then look at the edbrowse
patch but it seems we simply don't have a way to integrate protocol handlers with the rest of the mechanism
closely enough to make this happen.
Helpfully there's a (fairly readable) RFC for all this and it's a very simple protocol. Also we don't need to bother with
anything more than plain gopher (there was a Gopher plus but I don't know of anyone who cares about it really) which simplifies all this.

On the subject of protocol handlers, if we can't get a variable output type (which is where this all started) I'd still
like to support handlers which can output to the buffer rather than just be ran as external processes.  That seems like a sensible thing to allow to me.
Any thoughts?

Cheers,
Adam.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Edbrowse-dev] protocol handlers and outputting to buffer
  2018-02-11 17:14                   ` Adam Thompson
@ 2018-02-11 17:40                     ` Karl Dahlke
  0 siblings, 0 replies; 12+ messages in thread
From: Karl Dahlke @ 2018-02-11 17:40 UTC (permalink / raw)
  To: Edbrowse-dev

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

http.c  ftpConnect() and parse_directory_listing().
You might need gopher analogies for these two functions.
Obviously you'd rename the second to parse_ftp_directory() or some such, then make a gopher version.
Seems doable.
Some of the complexity of ftpConnect is that we can download, or even download in background, probably this carries over into gopher, don't see why it wouldn't or shouldn't.

I will look into the speed issue, and/or infinite loop, obviously edbrowse is still somewhat unusable.
Remember there are sites like nasa that you can't even browse without js.
My push for getComputedStyle will make a lot of sites run faster.
Yes, stackoverflow does get lost, but not in css, it's some other js that is brought in.

Karl Dahlke

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2018-02-11 17:40 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-26 18:01 [Edbrowse-dev] protocol handlers and outputting to buffer Adam Thompson
2018-01-26 18:13 ` Karl Dahlke
2018-01-27  9:58   ` Adam Thompson
2018-01-27 14:40     ` Karl Dahlke
2018-02-01 19:58       ` Adam Thompson
2018-02-01 20:28         ` Karl Dahlke
2018-02-09 18:44           ` Adam Thompson
2018-02-09 20:10             ` Karl Dahlke
2018-02-11 16:41               ` Adam Thompson
2018-02-11 16:54                 ` Karl Dahlke
2018-02-11 17:14                   ` Adam Thompson
2018-02-11 17:40                     ` Karl Dahlke

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).