[Edbrowse-dev] Ports

[Edbrowse-dev] Ports

[Karl Dahlke]

Just thinking about ports, in regards to sockets.
I don't know if it's the best way, but the easiest way is to decide
somewhat arbitrarily that edbrowse will use the ports between 47380 and 47390, or some such range.
Microsoft did that when they first came out with their sql server,
if you want to connect to it through odbc you just know it's 3280, or some such
I don't remember the numbers but I use to know.
And being microsoft it became the standard, of course,
so now nobody else touches those ports.
And if they do you can change them in the MS SQL configuration.
So I was thinking edbrowse might use a preset range of ports for its IPC,
and that range could be changed in .ebrc.

Karl Dahlke

Re: [Edbrowse-dev] Ports

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> Just thinking about ports, in regards to sockets.
> I don't know if it's the best way, but the easiest way is to decide
> somewhat arbitrarily that edbrowse will use the ports between 47380 and 47390, or some such range.

My primary concern would be multi-user systems.  How do you keep Alice's
edbrowse from conflicting with Bob's?
Then there's security.  You can have your thing listen on loopback, so
it isn't accessible from the big bad Internet, but if you're on a
multi-user system, it's still open to whoever is sharing the system with you.
Granted, most of us use this program on what are basically single-user
systems.  There aren't exactly a lot of edbrowse users out there,
either.  So maybe the answer is "don't care".

For me, the optimal answer would be Unix domain sockets.  If you don't
know about them, they're like sockets, but represented by named files in
the filesystem.  So to connect to a daemon, you might connect to
/run/my-daemon/socket or whatever, rather than some port on localhost.
The advantages are that they can only be used for local interprocess
communication between things running on the same machine, and access
control is based on the Unix filesystem permission checks that we all
know and love.
But I'm guessing that these aren't usable on Windows, are they?  Please
tell me it has an equivalent...

[Edbrowse-dev] Ports

[Karl Dahlke]

I know about domain sockets, but heard they were not available on windows.
If they were then we could create various /tmp/.edbrowse/sockets
and bounce through those.
If not then we use ports as usual.

Perhaps your security concerns are an argument in favor
of passing the internet data back from edbrowse-curl to edbrowse via temp files,
created 0600 for the user.
These temp files could not be "sniffed".

Karl Dahlke

[Edbrowse-dev] Ports

[Karl Dahlke]

On further thought, maybe my reaction is indeed, don't care.
Only root can sniff packets on the internal loopback,
and root can already do everything anyways, including reading all your files.
I don't think we need fear packet trafic within the machine.

Karl Dahlke

Re: [Edbrowse-dev] Ports

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

On Wed, Dec 30, 2015 at 10:05:28PM -0500, Karl Dahlke wrote:
> I know about domain sockets, but heard they were not available on windows.
> If they were then we could create various /tmp/.edbrowse/sockets
> and bounce through those.

That's my thoughts also. I seem to remember Windows had something like them or
at least some other method of IPC which doesn't involve network ports.
I'm not particularly comfortable with ports tbh.
I suspect whatever we do will have to have some differences between Unix and
Windows, hence why I'd like a more generic IPC interface.

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[Edbrowse-dev] Ports

[Karl Dahlke]

> I seem to remember Windows had something like domain sockets or
> at least some other method of IPC which doesn't involve network ports.

I really don't know.
I thought Geoff said traditional sockets were the best, perhaps only
practical form of IPC besides pipes,
but don't want to put words in his mouth so will cc to him.

> I'm not particularly comfortable with ports tbh.
> I suspect whatever we do will have to have some differences between Unix and
> windows.

Connecting using traditional sockets on network ports,
all through loopback, really doesn't bother me at all, if ports can be
reconfigured in .ebrc -
well, doesn't bother me as much as having different systems on the two OSs.
If this is the best / only way for windows,
or even a practical way on windows,
then let's just do it across the board.
We've already confirmed my socket layer works portably across both -
would probably be easy to add in udp capability if needed.
but I'm sure Geoff knows more than I do here. Other practical/flexible IPC
on Windows? Or any issues with loopback sockets in network ports?

Karl Dahlke

Re: [Edbrowse-dev] Ports

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

On Fri, Jan 01, 2016 at 10:01:35AM -0500, Karl Dahlke wrote:
> > I seem to remember Windows had something like domain sockets or
> > at least some other method of IPC which doesn't involve network ports.
> 
> I really don't know.
> I thought Geoff said traditional sockets were the best, perhaps only
> practical form of IPC besides pipes,
> but don't want to put words in his mouth so will cc to him.

Ok, I can't imagine chrome etc do things this way but I may well be wrong.
Additionally, I've seen programs like chrome and firefox spawning multiple
processes with the same name, presumably doing different things on Windows.
I wonder how that's managed and if we can do something similar?

> > I'm not particularly comfortable with ports tbh.
> > I suspect whatever we do will have to have some differences between Unix and
> > windows.
> 
> Connecting using traditional sockets on network ports,
> all through loopback, really doesn't bother me at all, if ports can be
> reconfigured in .ebrc -
> well, doesn't bother me as much as having different systems on the two OSs.
> If this is the best / only way for windows,
> or even a practical way on windows,
> then let's just do it across the board.
> We've already confirmed my socket layer works portably across both -
> would probably be easy to add in udp capability if needed.
> but I'm sure Geoff knows more than I do here. Other practical/flexible IPC
> on Windows? Or any issues with loopback sockets in network ports?

Ok, I'd feel slightly better about this if there was a way to make the port
selection automatic rather than configured,
it feels like multi-user would break in exciting ways otherwise.

Cheers,
Adam.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] Ports

[Geoff McLane]

[-- Attachment #1: Type: text/plain, Size: 2176 bytes --]

Hi Karl, Adam,

Well I think pipes and sockets seem certainly the most
practical cross-platform IPC mechanisms...

Pipes seems to have less difference in that only perhaps
a few #ifdef WIN32 are required, if any, in a cross-compile...

Sockets definitely does require a number of #ifdef WIN32,
but not really excessive... many can be handled as MACROS...
and the cross-porting has been done MANY times... in lots
and lots of libraries, apps, utilities, so is sort of very
mature... getting easy even...

But reading up a little on MSDN, and remembering, the
following IPC mechanisms are available in Windows, but
for sure some are **WINDOWS ONLY**!

1. Clipboard/DDE - can agree a format then do copy/paste
2. COM - OLE manage compound document interface
3. Data Copy - Using Windows messaging - WM_COPYDATA
4. RPC - have only ever used it over sockets...
5. File Mapping or shared memory mapping - just put data
6. Pipes and Sockets - are cross-platform...

Not sure which of these would fit "domain sockets", but maybe
I missed something else available... having coded and used
most of them, in various apps, at various time, I am not sure
which I would choose as the most 'generic' to Windows...

I am sure unix has some form of shared memory mapping (5)... just
copy a data block using a simple memory pointer would probably be
the fastest... but requires that the partner be monitoring that
space, polling... and what about thread safety? and maybe needs
some/many #ifdef to account for the differences...

But as Karl mentions he has already shown 6. Pipes and Sockets
both work... with no porting issues that I know of...

Concerning sockets, over the years I have collected some tcp,
udp samples, and this is where I added and tested Karl's
socket.c - and pushed them all to my 'new' tcp-tests repo -

https://github.com/geoffmcl/tcp-tests

See src/ebsocket.c... compiles without even a warning both
in WIN32 and UNIX... still to do a WIN64 compile... and
maybe a MinGW compile... sockets are fun ;=)) and really
now quite an old technology that has not been replaced...

Regards,
Geoff.

PS: I have now 'subscibed' to the dev list... so should
be no need to cc me...

[-- Attachment #2: Type: text/html, Size: 2977 bytes --]

[Edbrowse-dev] Ports

[Karl Dahlke]

> Ok, I'd feel slightly better about this if there was a way to make the port
> selection automatic rather than configured,
> it feels like multi-user would break in exciting ways otherwise.

Oops, you're right.
I had the model of one edbrowse-curl daemon serving all, and then sure it could listen
on the edbrowse-curl port, whatever that is, and all good,
but if we spin up one curl server per user,
it has to listen on a per-user edbrowse port,
so that's a bit more complicated.
We'll give this more thought / discussion.

Karl Dahlke

Re: [Edbrowse-dev] Ports

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 3869 bytes --]

On Fri, Jan 01, 2016 at 08:09:10PM +0100, Geoff McLane wrote:
> Hi Karl, Adam,
> 
> Well I think pipes and sockets seem certainly the most
> practical cross-platform IPC mechanisms...
I agree with that and if we were doing a traditional client server I'd go down
that route for sure.
> Pipes seems to have less difference in that only perhaps
> a few #ifdef WIN32 are required, if any, in a cross-compile...

But multiple processes can't monitor a pipe as far as I know.

> Sockets definitely does require a number of #ifdef WIN32,
> but not really excessive... many can be handled as MACROS...
> and the cross-porting has been done MANY times... in lots
> and lots of libraries, apps, utilities, so is sort of very
> mature... getting easy even...

I wonder if what we actually should be doing is looking for a cross-platform IPC lib?

> But reading up a little on MSDN, and remembering, the
> following IPC mechanisms are available in Windows, but
> for sure some are **WINDOWS ONLY**!
> 
> 1. Clipboard/DDE - can agree a format then do copy/paste

Yuck, is that really done within apps?

> 2. COM - OLE manage compound document interface

... ouch! That's gonna be ifdef hell.

> 3. Data Copy - Using Windows messaging - WM_COPYDATA

Don't know about this, sounds potentially interesting though.

> 4. RPC - have only ever used it over sockets...

Yeah, usually a socket technology as far as I know.

> 5. File Mapping or shared memory mapping - just put data

Unix has this via mmap, never used it for IPC but I've used it for other things.
There'll be sync issues this way, lots of 'em.
I know apps do IPC this way, and successfully though.

> 6. Pipes and Sockets - are cross-platform...

Yep.

> Not sure which of these would fit "domain sockets", but maybe
> I missed something else available... having coded and used
> most of them, in various apps, at various time, I am not sure
> which I would choose as the most 'generic' to Windows...

How about named pipes? I just looked them up on msdn and it looks like we could
may be do that. I'm not sure if it's directly equivalent to unix domain sockets
or may be the unix equivalent is more like a fifo, but either way that'd work.
Actually fifos would work quite nicely.
Edbrowse-curl, per user, monitors an input fifo,
then creates output fifos for each connected edbrowse instance.
That way we can have $tmpdir/edbrowse/$user/edbrowse-curl/input and then
numbered output fifos.

> I am sure unix has some form of shared memory mapping (5)... just
> copy a data block using a simple memory pointer would probably be
> the fastest... but requires that the partner be monitoring that
> space, polling... and what about thread safety? and maybe needs
> some/many #ifdef to account for the differences...

Yeah as I said above, certainly possible but the synchronisation issues will be
awkward, and there're probably differences as well as you point out.

> But as Karl mentions he has already shown 6. Pipes and Sockets
> both work... with no porting issues that I know of...

Agreed.

> Concerning sockets, over the years I have collected some tcp,
> udp samples, and this is where I added and tested Karl's
> socket.c - and pushed them all to my 'new' tcp-tests repo -
> 
> https://github.com/geoffmcl/tcp-tests
> 
> See src/ebsocket.c... compiles without even a warning both
> in WIN32 and UNIX... still to do a WIN64 compile... and
> maybe a MinGW compile... sockets are fun ;=)) and really
> now quite an old technology that has not been replaced...

Ok, not to forget cygwin...
and the 32 and 64 bit versions of those,
though I should imagine that we should be fine as long as we don't try running
64 bit libs on 32 bit Windows.
... fun as you say.

Cheers,
Adam.
PS: has any of this altered with Windows 10?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] Ports

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> but if we spin up one curl server per user,
> it has to listen on a per-user edbrowse port,
> so that's a bit more complicated.

Yes, it's terribly complicated, and this is just the sort of scenario
for which Unix-domain sockets were designed.  I'd say use Unix domain
sockets on Unix and Mac, and IP sockets with loopback on Windows, unless
there's an implementation of Unix domain sockets for Windows.
In any case, we only care about the socket type when it is created and
bound or connected.  I.E., the socket(), bind(), and connect() calls.
So most of the time nobody needs to care whether it is an IP socket or a
Unix domain socket.

-- Chris

[Edbrowse-dev] Ports

[Karl Dahlke]

I use fifos, unix named pipe, even in my speech adapter,
so I know them and like them.
There's a user level command to create these on the fly; you don't have to be root.
mkfifo()
If windows has similar this could work well.
I mean edbrowse and edbrowse-js are already piped,
there's almost no difference between that and named fifo.

Karl Dahlke