IMAP revisited

IMAP revisited

[Steinar Bang]

I've come to the conclusion that an IMAP backend for Gnus is something
I want.  It's been quiet on this front for a long while now, so I
wonder if anybody's actually working on it?

If nobody's working on it, I see no option except to start
investigating how to write one (todays snow means that the motorcycle
season is definitely *over*...).

So I'm starting today, by printing out the source for the NNTP and POP
backends, to see what they look like and what requirements a Gnus
backend has to satisfy.

But, I'm hoping I won't have to...:-/


- Steinar

Re: IMAP revisited

[Scott Blachowicz]

Steinar Bang <sb@metis.no> wrote:

> So I'm starting today, by printing out the source for the NNTP and POP
> backends, to see what they look like and what requirements a Gnus
> backend has to satisfy.

Are there any existing backends that...

1) deal with separate machine client/server type things.
2) do both read & writes.
3) don't have active files, etc on the server.
4) can maintain sequences/marks/etc on the server.
5) whatever I'm forgetting.

BTW, I'd be interested in such a backend, too...

Scott Blachowicz  Ph: 206/283-8802x240   Mathsoft (Data Analysis Products Div)
                                         1700 Westlake Ave N #500
scott@statsci.com                        Seattle, WA USA   98109
Scott.Blachowicz@seaslug.org

Re: IMAP revisited

[Sudish Joseph]

Scott Blachowicz <scott@statsci.com> writes:
> 3) don't have active files, etc on the server.
> 4) can maintain sequences/marks/etc on the server.

This is the key missing functionality.  Most of the work, I think,
would be in adding stuff to the backend api to let Gnus query the
backend for all state.  

Note that this is useful for lots more stuff than IMAP.  For instance,
such an api would allow the writing of a clean nnml-with-aynch-delivery 
backend.  

Essentially, Gnus has to stop telling backends what the article
number of a message should be.  For starters.

-Sudish

Re: IMAP revisited

[David Moore]

Sudish Joseph <sudish@mindspring.com> writes:

> Essentially, Gnus has to stop telling backends what the article
> number of a message should be.  For starters.

	I must be missing something, since only the fetch commands to
the backends have an article number provided by gnus.  Routines like
BACKEND-request-accept-article are given a group and the body of the
article and come up with their own number for it.

	Or are you referring to the backend renumbering old articles
when Gnus isn't looking?  That's a definite problem, certainly.


-- 
David Moore <dmoore@ucsd.edu>       | Computer Systems Lab      __o
UCSD Dept. Computer Science - 0114  | Work: (619) 534-8604    _ \<,_
La Jolla, CA 92093-0114             | Fax:  (619) 534-1445   (_)/ (_)
<URL:http://oj.egbt.org/dmoore/>    | Solo Furnace Creek 508 -- 1996!

Re: IMAP revisited

[visigoth]

David Moore <dmoore@UCSD.EDU> writes:

> 	I must be missing something, since only the fetch commands to
> the backends have an article number provided by gnus.  Routines like
> BACKEND-request-accept-article are given a group and the body of the
> article and come up with their own number for it.
> 
> 	Or are you referring to the backend renumbering old articles
> when Gnus isn't looking?  That's a definite problem, certainly.

The IMAP model is that the server does all the message management
internally, including purges, reordering, whatever.  It keeps track of
flags on each message.  There -is- a UID (unique identifier)
associated with each message, but it's a little harder to work with
than just the sequence number.  <shrug>

There are a number of ways of dealing with this sort of thing.

As a note--I know of a couple of people (including myself) have looked
at doing IMAP for Gnus.  However, I just haven't had time to make any
significant progress, and I suspect the same is true of the other guy
I know who was looking at it.  (I'm sure he'll speak up if he's still
working on it.)

John.

Re: IMAP revisited

[visigoth]

visigoth@naiad.fac.cs.cmu.edu writes:

> As a note--I know of a couple of people (including myself) have looked
> at doing IMAP for Gnus.  However, I just haven't had time to make any
> significant progress, and I suspect the same is true of the other guy
> I know who was looking at it.  (I'm sure he'll speak up if he's still
> working on it.)

Eeek!  Wouldn't you know it--I went and said this I'd go and spend
several hours hacking on it.  I now have a very grody parser that
works in some cases but not others.  I'm planning on spending another
hour or two making a -real- parser right now.  I'll be at MIT for a
conference on Kerberos over the next week, so progress will be slow
then, but look for results soon.

For hacking potential, see:

<file://localhost/afs/cs.cmu.edu/user/visigoth/src/emacs/imap.el> with AFS or
<ftp://olivier.pc.cs.cmu.edu/pub/imap.el.gz> otherwise.

The version in AFS will be updated as I work on it.  I'll try to
remember to put the "real" parser in place when I get it done.

Note--this isn't an attempted nnimap.el yet--this is lower-level
stuff.  Once the general imap code is done, nnimap should be pretty
easy.

John.

Re: IMAP revisited

[Kai Grossjohann]

>>>>> Steinar Bang writes:

  SB> If nobody's working on it, I see no option except to start
  SB> investigating how to write one (todays snow means that the
  SB> motorcycle season is definitely *over*...).

I've been working on something very simple lately.  A friend has
written a Perl module NNML which looks like an NNTP server and
reads/writes an nnml like directory structure (hence the name).  This
Perl module has been extended with a number of commands XACCEPT,
XDATE, XDELETE and XDELETEGROUP.  I have extended the nntp backend to
include mail-like backend functions using these additional commands.
For example, if you type `B c', Gnus uses two backend functions, one
to extract the article from the source group, the second to insert it
into the target group (the latter is called *-request-accept-article).
I wrote the insertion function which just sends XACCEPT to the server.
As another example, expiring articles uses XDATE to extract the date
from an article then uses the normal Gnus mechanisms to decide whether
or not the article should be deleted.

The backend now finally works (I'm using it right now) but is still
rather slow for some actions.

Please note that this is no fancy stuff: the normal Gnus mechanisms
are used to maintain marks and so on.

I think it will be quite useful for sharing mail between home and
office: set up an NNML server at both ends, tell the home server to
suck everything from the office server, then read the mail at home.
But this means I see some mails twice (because of the marks).

kai
-- 
I wonder why nobody don't like me,
or is it de fact dat I'm ugly? -- Harry Belafonte

Re: IMAP revisited

[Kai Grossjohann]

>>>>> Kai Grossjohann writes:

  Kai> Please note that this is no fancy stuff: the normal Gnus mechanisms
  Kai> are used to maintain marks and so on.

I forgot to mention that getting mail doesn't really work: it uses the
nnml functions to directly write to the disk :-(

kai
-- 
I wonder why nobody don't like me,
or is it de fact dat I'm ugly? -- Harry Belafonte

Re: IMAP revisited

[Paul Franklin]

>>>>> Sudish Joseph writes:

 > Scott Blachowicz <scott@statsci.com> writes:
 >> 3) don't have active files, etc on the server.

IMAP doesn't have to change to deal with this; LIST and EXAMINE can be
used.  However, it will be slow if the user has many mailboxes, as a
EXAMINE will be needed for each mailbox.

 >> 4) can maintain sequences/marks/etc on the server.

That's what nn*-request-update-info was created for.  I think.  It at
least provides a way for gnus to find out about changes on the server,
but I don't know if it allows gnus changes to propagate to the server.
This depends on when it is called.

 > Essentially, Gnus has to stop telling backends what the article
 > number of a message should be.  For starters.

I think this is too fundamental to Gnus to change.  However, the
mapping between Gnus message numbers and IMAP4 UIDs is the only thing
which needs to be stored locally to make this work.  (UIDs would work
if they didn't have gaps.  But I think some IMAP4 servers use
time-based values, and this leaves lots of gaps if you don't come
close to receiving one message per second.)

A lot of this could probably be patched over depending on when
nn*-request-update-info is called.  (Lars, is this documented yet?  It
isn't in red 0.47.)  The real killer is cached articles.

--Paul

Re: IMAP revisited

[Lars Magne Ingebrigtsen]

Paul Franklin <paul@cs.washington.edu> writes:

> A lot of this could probably be patched over depending on when
> nn*-request-update-info is called.  (Lars, is this documented yet?  It
> isn't in red 0.47.)

It's in the "Optional Backend Functions" section.

> The real killer is cached articles.

Yup.  It would be better if the nnimap backend would map the IMAP ids
into article numbers.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen