[Edbrowse-dev] imap

[Edbrowse-dev] imap

[Karl Dahlke]

This is perhaps more thinking ahead...

The most common request from users is:
why doesn't edbrowse work on this site,
and is usually an object or method that I haven't implemented yet.
The next request is:
what about imap?
I started looking at wikipedia imap and rfc3501.
There's a lot going on here.
I think there are three options for us.

1. Access an imap server much as we access a pop3 server today.
Use port 143 instead of 110.
Many of the commands are the same, at least functionally,
and of course there are many new commands.
We wouldn't have to use all the new commands.

2. Try to use a linux imap library to help us,
the way we use curl for http and ftp.
I don't even know if such exists and is easy to use.
It has now become a requirement that any libraries we use are
in linux and free bsd, since bsd is on board with us.

3. Ask the user to use a separate stand-alone program to pull
mail off an imap server and into a local mailbox file.
Edbrowse would then read the local mailbox file and pretend it was a mail server,
and present those mail messages to the user via the same interface as today.
Delete them (from the file rather than the mail server),
or same them to other files, or save attachments, etc, as we do today.

I would lean towards 3, but I might not understand what is going on here at all,
and I'm curious what others think.

Karl Dahlke

Re: [Edbrowse-dev] imap

[Chris Brannon]

Karl Dahlke <eklhad@comcast.net> writes:

> 1. Access an imap server much as we access a pop3 server today.

I'd rather not do this if we can avoid it.
Our SMTP implementation had undiscovered bugs for years.  Remember
dot-stuffing?  IMAP is an even more complex protocol.

> 2. Try to use a linux imap library to help us,

At a glance, vmime might be promising.
It's a C++ library, though.

> 3. Ask the user to use a separate stand-alone program to pull
> mail off an imap server and into a local mailbox file.

If we do this, we'll miss the great selling point of IMAP: keeping
mail in separate folders with state kept on the server.
We could handle separate folders with method 3 if we added code to
edbrowse to handle maildirs or multiple mailboxes.
The state of messages will be lost from the server, though.  I can't see
a way to do option three without losing some of the advantages of IMAP.
Here goes some IMAP propaganda.

The nice feature of IMAP is multiple folders.  I get lots and lots
of email.  Most of it is from mailing lists, but I also have quite a few
online collaborators and friends.  On my IMAP server, mail is filtered
into multiple folders by criteria such as sender, mailing list ID, and
so forth.  The filtering is done by an external program.  I have a
special folder just for personal mail from Karl.  I've had it for years.
When I connect via IMAP, I see all of these folders, and I can decide which
ones have messages that require my immediate attention.  This is how I
deal with the torrent of mail I receive every day.  If I had a flat
mailbox, I'd spend all my time reading mail!

Furthermore, I can keep the state of all my folders on the server where
it belongs, and all of the clients that I might care to use will have a
consistent view of the world.  This is important for people who want to
read their email with mobile devices or multiple computers.

-- Chris

Re: [Edbrowse-dev] imap

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

As I don't use edbrowse as an email client I'm probably not the most qualified
to talk about this side of the code.
However, I would strongly suggest either 2 or 3 as approaches since I think
writing a fully functional imap module is probably going to be a lot of work.

Given how many linux clients support imap (including a very ed-like email
client) I'd be very surprised and disapointed if a decent library isn't
available for this.

Cheers,
Adam.
PS: it looks like edbrowse currently breaks threading,
at least in mutt (probably something to do with message headers)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] imap

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

Wow, replying to my own message, sorry for the noise.
On Fri, Mar 21, 2014 at 05:11:36PM +0000, Adam Thompson wrote:
> Given how many linux clients support imap (including a very ed-like email
> client) I'd be very surprised and disapointed if a decent library isn't
> available for this.

From the output of apt-cache show libcurl3:
libcurl is an easy-to-use client-side URL transfer library, supporting DICT,
 FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S,
 RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP.

Hmmm, looks like we have our imap (and smtp and pop3) library,
and it does ssl as well, unless the description is incorrect.

I'm not sure how it's imap support works though.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Edbrowse-dev] imap

[Chris Brannon]

Adam Thompson <arthompson1990@gmail.com> writes:

> Given how many linux clients support imap (including a very ed-like email
> client) I'd be very surprised and disapointed if a decent library isn't
> available for this.

Do you mean heirloom-mailx (AKA nail)?
It's quite ed-like, being based on Berkeley mail from the teletype era.
The IMAP support is fine.  It handles multiple folders and multiple
accounts.

-- Chris

Re: [Edbrowse-dev] imap

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Fri, Mar 21, 2014 at 11:26:15AM -0700, Chris Brannon wrote:
> Adam Thompson <arthompson1990@gmail.com> writes:
> 
> > Given how many linux clients support imap (including a very ed-like email
> > client) I'd be very surprised and disapointed if a decent library isn't
> > available for this.
> 
> Do you mean heirloom-mailx (AKA nail)?
> It's quite ed-like, being based on Berkeley mail from the teletype era.
> The IMAP support is fine.  It handles multiple folders and multiple
> accounts.

Yeah that was the one. I admit, being a long-time mutt user,
I've not played with it that much but it looks kind of interesting.

Getting back to edbrowse though, we need to decide how much mail client support we want.
For example do we want to do maildir, mbox etc for local mailboxes as well as imap and pop3 (with their secure equivalents I hope) for remote ones?
What about pgp and the like for signing messages?
Or threads?

I only mention these because, just like with the browser element,
if we start adding things to the mail client, users will, quite rightly,
request more features.
Given the number of things edbrowse already does,
I guess I'm just concerned that it's going to become too big for its own good
by trying to do too many things.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[Edbrowse-dev] imap

[Karl Dahlke]

Adam raises a legitimate concern about "feature creep",
edbrowse trying to do too much.
I can see this point, but I also think edbrowse power is partly in its breadth.
Before I implemented odbc, some people would have thought that was
beyond the scope of what edbrowse should do.
And yet I have found it so helpful in my work
to be able to edit a database table the way I edit a file.
Similarly for managing a directory,
rename a file with a substitute command.
It is the same interface for many tasks.
When bringing up an email I can go to a hyperlink
within the email, the way explorer and outlook merge together
in a seamless whole.
I can also cut&paste things to and from emails as I formulate a reply.

The fact that people ask for imap suggests to me that they want
this unified experience to extend to imap, and quite honestly I probably
would too if I ever used or needed imap, if I ever had lots of email
to deal with, which I don't.

In fact you can see I moved in that direction from the outset, with my filters,
which automatically save certain emails to certain files based on subject
or sender etc.
What chris referred to in his imap settings,
a folder with all emails from me, can almost be done in edbrowse today with

fromfilter {
Karl Dahlke > kd-mail
}

My option to extend and continue this strategy is not ideal,
as chris points out, because the folders and arrangements of emails
are not on the server, and not seen in a consistent way from
all machines and devices.
That doesn't impact me because I only look at my mail from one computer,
but I know my friends at work had this concern as they viewed mail
from cell phones and many such devices, so I get it.

I guess I need to look at the imap support that is offered by curl,
since we already have curl in our product.
I would hope to present a similar email interface to the user,
close to what we have today,
but with more commands like switching folders (imap folders on the server),
or a list of all folders, etc,
and perhaps a way to reply to an email without necessarily having a copy
of it on your computer.

Karl Dahlke

Re: [Edbrowse-dev] imap

[Adam Thompson]

[-- Attachment #1: Type: text/plain, Size: 3114 bytes --]

On Sat, Mar 22, 2014 at 06:04:47AM -0400, Karl Dahlke wrote:
> Adam raises a legitimate concern about "feature creep",
> edbrowse trying to do too much.
> I can see this point, but I also think edbrowse power is partly in its breadth.
> Before I implemented odbc, some people would have thought that was
> beyond the scope of what edbrowse should do.
> And yet I have found it so helpful in my work
> to be able to edit a database table the way I edit a file.
> Similarly for managing a directory,
> rename a file with a substitute command.
> It is the same interface for many tasks.

I can see this and indeed quite like the common interface for some things.
However I'm worried about the increased complication,
particularly as it looks like the browser code's going to explode into much
more code than it is currently.

> When bringing up an email I can go to a hyperlink
> within the email, the way explorer and outlook merge together
> in a seamless whole.
> I can also cut&paste things to and from emails as I formulate a reply.

Yeah I can imagine that's quite nice.

> The fact that people ask for imap suggests to me that they want
> this unified experience to extend to imap, and quite honestly I probably
> would too if I ever used or needed imap, if I ever had lots of email
> to deal with, which I don't.

This is true, but I wonder if it would be possible to make a set of tools which
work together, sharing a common interface,
rather than a single combined program.

> In fact you can see I moved in that direction from the outset, with my filters,
> which automatically save certain emails to certain files based on subject
> or sender etc.
> What chris referred to in his imap settings,
> a folder with all emails from me, can almost be done in edbrowse today with
> 
> fromfilter {
> Karl Dahlke > kd-mail
> }
> 
> My option to extend and continue this strategy is not ideal,
> as chris points out, because the folders and arrangements of emails
> are not on the server, and not seen in a consistent way from
> all machines and devices.

In addition, if you have as much email as I do,
that'd probably come fairly close to filling my hard drive.

> That doesn't impact me because I only look at my mail from one computer,
> but I know my friends at work had this concern as they viewed mail
> from cell phones and many such devices, so I get it.

Yeah, I'd struggle with that as well.

> I guess I need to look at the imap support that is offered by curl,
> since we already have curl in our product.

Yeah, and we can probably use it for smtp and pop3 if we don't already.

> I would hope to present a similar email interface to the user,
> close to what we have today,
> but with more commands like switching folders (imap folders on the server),
> or a list of all folders, etc,
> and perhaps a way to reply to an email without necessarily having a copy
> of it on your computer.

You'll probably want to download the email to quote in most cases,
so replying without downloading is probably not needed.

Cheers,
Adam.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]