IMAP and IDLE...

IMAP and IDLE...

[Lars Magne Ingebrigtsen]

I was looking at Mail.app just now, and it is really appealing the way it
just updates all the stuff asynchronously.

I've been sceptical to supporting IDLE before, since you can only IDLE
on one mail box at a time.  But it just occurred to me that if you're
doing client-side splitting, just idling on the INBOX would suffice.

So, doing stuff asynchronously in Emacs is a major pain in the ass, and
it tends to hang Emacs spontaneously, especially when doing network
connects and stuff.

But here's my idea: We could open an extra connection to every IMAP
server (that we're doing client splits on).  We issue an IDLE on the
INBOX. If the connection goes away, we don't do anything asynchronously.
But otherwise, every time we get a new message in the INBOX, we run it
through the normal splitting methods and update the group buffer.

Since this will mostly be splitting one single message at a time, and we
only download the headers to do the splitting, this can probably be done
without hanging Emacs too badly.

Of course, updating the group buffer (and any summary buffer you have
open that gets a new message) from an async process will give you all
the same problems with buffer management (copying/yanking text etc) that
we see with the updated Date line, so we have to figure out how to fix
that problem generally.

(For viewers who have recently joined us -- updating text in the buffer
asynchronously (from timers and sentinels) is tricky, because we want to
avoid moving point and mark, even though they might be in the region
we're updating.  And there's some weird interaction between this and
transient-mark-mode.  We should aim to create a `with-no-buffer-change'
macro that we can run arbitrary buffer-altering functions inside of
safely.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]


Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've been sceptical to supporting IDLE before, since you can only IDLE
> on one mail box at a time.  But it just occurred to me that if you're
> doing client-side splitting, just idling on the INBOX would suffice.

True, but there's another situation where it is very useful.

I split on the server into about 100 IMAP folders, at delivery time.
INBOX is special, as its mail that (in theory) I want to see promptly,
and the rest I only look at when I'm in a mail-processing mode.

So in this case too, IDLE on just INBOX is not only useful but what I'd
want to configure if I could have IDLE on any subset.

Mail.app (not that I want to hold it up as a paragon of virtue, because
top-posting and sending long lines are both antisocial) also downloads
messages from non-INBOX in the background.  It seems to be a bug that
one can't turn that off, but I wonder if it's just doing it every few
minutes, and IDLEing on INBOX the rest of the time.

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: IMAP and IDLE...

[Frank Schmitt]

Greg Troxel <gdt@work.lexort.com> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> I've been sceptical to supporting IDLE before, since you can only IDLE
>> on one mail box at a time.  But it just occurred to me that if you're
>> doing client-side splitting, just idling on the INBOX would suffice.
>
> True, but there's another situation where it is very useful.
>
> I split on the server into about 100 IMAP folders, at delivery time.
> INBOX is special, as its mail that (in theory) I want to see promptly,
> and the rest I only look at when I'm in a mail-processing mode.

Yes, that's my setup too. Mailing lists are sorted into folders and mail
directly addressed to me goes to INBOX.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: IMAP and IDLE...

[Ted Zlatanov]

On Wed, 16 Mar 2011 12:31:00 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> But here's my idea: We could open an extra connection to every IMAP
LMI> server (that we're doing client splits on).  We issue an IDLE on the
LMI> INBOX. If the connection goes away, we don't do anything asynchronously.
LMI> But otherwise, every time we get a new message in the INBOX, we run it
LMI> through the normal splitting methods and update the group buffer.

LMI> Since this will mostly be splitting one single message at a time, and we
LMI> only download the headers to do the splitting, this can probably be done
LMI> without hanging Emacs too badly.

I'd rather just see a modeline indicator that I have new mail in INBOX.
I really don't want Emacs to freeze randomly and "instant" splitting
doesn't have much value for me (if it did, I'd be using Procmail or
Sieve on the server side).

Ted

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I'd rather just see a modeline indicator that I have new mail in INBOX.
> I really don't want Emacs to freeze randomly and "instant" splitting
> doesn't have much value for me (if it did, I'd be using Procmail or
> Sieve on the server side).

I assume that for people who use client-side splitting, the question "is
there new mail in INBOX?" can be emulated with 99.7% accuracy by this
function:

(defun nnimap-new-mail-p ()
  t)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Eric S Fraga]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:


[...]

> I assume that for people who use client-side splitting, the question "is
> there new mail in INBOX?" can be emulated with 99.7% accuracy by this
> function:
>
> (defun nnimap-new-mail-p ()
>   t)

how true! ;-)

I long ago got rid of any biff type functionality in my system as there
was no point.  on that note, I don't need any automatic loading of
email; I'm happy to git 'g' when *I* want to read email...  but that's
me.


-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 + No Gnus v0.14

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Wed, 16 Mar 2011 14:37:33 +0000, Eric S Fraga <e.fraga@ucl.ac.uk> said:

ESF> I long ago got rid of any biff type functionality in my system as
ESF> there was no point.  on that note, I don't need any automatic
ESF> loading of email; I'm happy to git 'g' when *I* want to read
ESF> email...  but that's me.

The trick to this is server-side-splitting with a biff system that can
watch and prioritize multiple boxes.  I ended up writing yet-another new
biff system (qmailcheck) that color codes "new" lines in it, displays
things by a prioritized list of folders so that the important ones are
at the top (and I concentrate on them).  Only some mailboxes cause the
window to pop forward and others (really important) issue task-bar
notifications.  The ding list doesn't even get listed in it (sorry
Lars).  If I had a single INBOX, you're right, biff would be useless.

So, the trick to mail (and I get tons) is still server-side-filtering.
I cringe at the thought of everyone doing client side filtering and
watching it all get sucked in and then filtered as they wait.
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Ted Zlatanov]

On Thu, 17 Mar 2011 11:08:20 -0700 Wes Hardaker <wes@hardakers.net> wrote: 

WH> So, the trick to mail (and I get tons) is still server-side-filtering.
WH> I cringe at the thought of everyone doing client side filtering and
WH> watching it all get sucked in and then filtered as they wait.

Oh, once you've used `gnus-registry-split-fancy-with-parent' you can
never go back ;)

This is actually on my TODO list for Cloudy Gnus: move the registry to
a shared storage mode so non-Gnus clients can figure out the right
mailbox for a reply.

On Thu, 17 Mar 2011 11:11:05 -0700 Wes Hardaker <wes@hardakers.net> wrote: 

WH> Maybe the right thing to do is do IDLE by default on INBOX (which,
WH> ironically, I don't use at all) and then have a group parameter (or
WH> something) set to open a new connection specifically for that box.  That
WH> way you could set the 'this box is important flag' on it to do IDLE
WH> updates for, say, the top 10 boxes out of the 100 that you might have?
WH> And the 11th for all the rest (or the 11th is the INBOX IDLE one that is
WH> also the main pull connection).

How about making IDLE-handlers an alist of (folder . handler) pairs?

When it's a single symbol or lambda you know it's for "INBOX".

Then if the user asks for 800 IDLE connections, well, they asked for it...

Ted

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Thu, 17 Mar 2011 13:31:12 -0500, Ted Zlatanov <tzz@lifelogs.com> said:

TZ> How about making IDLE-handlers an alist of (folder . handler) pairs?

TZ> When it's a single symbol or lambda you know it's for "INBOX".

TZ> Then if the user asks for 800 IDLE connections, well, they asked for it...

I'm actually ambivalent about how to actually specify you want 800.  I
just want to be able to ask for 800.  Or, more likely, at least 8.

-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Ted Zlatanov]

On Thu, 17 Mar 2011 13:52:06 -0700 Wes Hardaker <wes@hardakers.net> wrote: 

>>>>>> On Thu, 17 Mar 2011 13:31:12 -0500, Ted Zlatanov <tzz@lifelogs.com> said:
TZ> How about making IDLE-handlers an alist of (folder . handler) pairs?

TZ> When it's a single symbol or lambda you know it's for "INBOX".

TZ> Then if the user asks for 800 IDLE connections, well, they asked for it...

WH> I'm actually ambivalent about how to actually specify you want 800.  I
WH> just want to be able to ask for 800.  Or, more likely, at least 8.

I'm trying to make it hard to go crazy with connections by requiring
that the folder name be exact and not a regex or a list.  So if you
really want it you'll figure it out, but it shouldn't be trivial to ask
for "IDLE on 800 boxes".

Ted

Re: IMAP and IDLE...

[Eric S Fraga]

Wes Hardaker <wes@hardakers.net> writes:

[...]

> So, the trick to mail (and I get tons) is still server-side-filtering.
> I cringe at the thought of everyone doing client side filtering and
> watching it all get sucked in and then filtered as they wait.

Yeah but this assumes that you can do server-side filtering.  I cannot
unfortunately so I'm stuck with the client doing it.  Not ideal but, at
least with gnus, the process is relatively painless.  If only emacs had
proper multi-threading support, mind you...

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 + No Gnus v0.16

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Fri, 18 Mar 2011 08:51:26 +0000, Eric S Fraga <e.fraga@ucl.ac.uk> said:

ESF> Yeah but this assumes that you can do server-side filtering.  I cannot
ESF> unfortunately so I'm stuck with the client doing it.  Not ideal but, at
ESF> least with gnus, the process is relatively painless.  

Yep.  I agree.  I'm lucky enough to still be using procmail (which I
still also think needs a rewrite).

Fortunately, the advent of sieve seems to have made more servers support
server-side filtering at least.  But it's not in really really
wide-spead deployment.  yet.

ESF> If only emacs had proper multi-threading support, mind you...

Dare to dream, my friend, dare to dream.
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Eric S Fraga]

Wes Hardaker <wes@hardakers.net> writes:

>>>>>> On Fri, 18 Mar 2011 08:51:26 +0000, Eric S Fraga <e.fraga@ucl.ac.uk> said:
>
> ESF> Yeah but this assumes that you can do server-side filtering.  I cannot
> ESF> unfortunately so I'm stuck with the client doing it.  Not ideal but, at
> ESF> least with gnus, the process is relatively painless.  
>
> Yep.  I agree.  I'm lucky enough to still be using procmail (which I
> still also think needs a rewrite).

Have you tried maildrop which, IMO, works significantly better (safer
and easier to understand filters) than procmail?

I used to use both (procmail for a very long time and then mailfilter
for a year or two) until I moved to keeping my emails on an imap server,
for better or for worse...

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 + No Gnus v0.16

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Fri, 18 Mar 2011 16:00:51 +0000, Eric S Fraga <e.fraga@ucl.ac.uk> said:

ESF> Have you tried maildrop which, IMO, works significantly better (safer
ESF> and easier to understand filters) than procmail?

I think I looked at it a number of years ago, but hadn't gotten around
to converting my gazillions of procmail recipes to it.  I'll go look
again though :-)
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 348 bytes --]

On Fri, Mar 18 2011, Eric S Fraga wrote:

> Have you tried maildrop which, IMO, works significantly better (safer
> and easier to understand filters) than procmail?

For the record, the only good alternative which should be considered is
Sieve (RFC3028). If you can use it, of course. :)

-- 
Julien Danjou
❱ http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: IMAP and IDLE...

[Eric S Fraga]

Julien Danjou <julien@danjou.info> writes:

> On Fri, Mar 18 2011, Eric S Fraga wrote:
>
>> Have you tried maildrop which, IMO, works significantly better (safer
>> and easier to understand filters) than procmail?
>
> For the record, the only good alternative which should be considered is
> Sieve (RFC3028). If you can use it, of course. :)

And therein lies the rub...  I wish I could use Sieve (server side)
versus gnus (or maildrop or whatever) client side.  It would make it so
much easier to have my phone co-exist with my emacs usage.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 + No Gnus v0.16

Re: IMAP and IDLE...

[Olivier Sirven]

On 2011-03-16 14:53:46, Ted Zlatanov <tzz@lifelogs.com> wrote:
> I'd rather just see a modeline indicator that I have new mail in INBOX.
> I really don't want Emacs to freeze randomly and "instant" splitting
> doesn't have much value for me (if it did, I'd be using Procmail or
> Sieve on the server side).

Sometimes you just don't have the possibility to do server side
splitting and all you can do is to do it on the client side

--
Olivier

Re: IMAP and IDLE...

[Ted Zlatanov]

On Wed, 16 Mar 2011 15:20:22 +0100 Olivier Sirven <the.slaa@gmail.com> wrote: 

OS> On 2011-03-16 14:53:46, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> I'd rather just see a modeline indicator that I have new mail in INBOX.
>> I really don't want Emacs to freeze randomly and "instant" splitting
>> doesn't have much value for me (if it did, I'd be using Procmail or
>> Sieve on the server side).

OS> Sometimes you just don't have the possibility to do server side
OS> splitting and all you can do is to do it on the client side

My point was that instant splitting is not so useful to me with my
current work pattern.  That may change.

On Wed, 16 Mar 2011 14:58:04 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> I assume that for people who use client-side splitting, the question "is
LMI> there new mail in INBOX?" can be emulated with 99.7% accuracy by this
LMI> function:

LMI> (defun nnimap-new-mail-p ()
LMI>   t)

I'd need a count and to be able to peek at the sender and subject of the
INBOX, plus to be able to dump the INBOX messages' body text (without
attachments) in a temp buffer.  So I can say "oh there's mail...  let me
peek...  no, it's not worth splitting it yet."  This is how I work now.

I'm not suggesting anyone else needs this, but IMHO it's generally useful.

Ted

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I'd need a count and to be able to peek at the sender and subject of the
> INBOX, plus to be able to dump the INBOX messages' body text (without
> attachments) in a temp buffer.  So I can say "oh there's mail...  let me
> peek...  no, it's not worth splitting it yet."  This is how I work now.

Right, a count might be nice...  I'll keep it in mind when (or if) I
tackle the IDLE issue.

So far, there's three use cases for idling on INBOX:

1) Pull down and split everything there

2) For server-side splits, just update the INBOX group -- the
interesting mail is in the INBOX

3) Just report the number of unsplit messages in INBOX, so that it can
be acted on manually

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Ted Zlatanov]

On Wed, 16 Mar 2011 18:35:41 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I'd need a count and to be able to peek at the sender and subject of the
>> INBOX, plus to be able to dump the INBOX messages' body text (without
>> attachments) in a temp buffer.  So I can say "oh there's mail...  let me
>> peek...  no, it's not worth splitting it yet."  This is how I work now.

LMI> Right, a count might be nice...  I'll keep it in mind when (or if) I
LMI> tackle the IDLE issue.

LMI> So far, there's three use cases for idling on INBOX:

LMI> 1) Pull down and split everything there

LMI> 2) For server-side splits, just update the INBOX group -- the
LMI> interesting mail is in the INBOX

LMI> 3) Just report the number of unsplit messages in INBOX, so that it can
LMI> be acted on manually

Maybe a nnimap backend variable of "IDLE-handler" would be nice.  It
would receive a list of the new raw message bodies and then the user can
decide what to do by customizing that handler.  So we can provide
pre-defined defaults:

for (1), 'run-splits
for (2), 'update-INBOX
for (3), a custom lambda or symbol

That way you can implement the IDLE functionality without worrying about
writing all the rest of the code immediately.

Ted

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Maybe a nnimap backend variable of "IDLE-handler" would be nice.  It
> would receive a list of the new raw message bodies and then the user can
> decide what to do by customizing that handler.  So we can provide
> pre-defined defaults:
>
> for (1), 'run-splits
> for (2), 'update-INBOX
> for (3), a custom lambda or symbol
>
> That way you can implement the IDLE functionality without worrying about
> writing all the rest of the code immediately.

Yup; I think that's the way to go.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Ted Zlatanov]

On Thu, 17 Mar 2011 18:18:25 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Maybe a nnimap backend variable of "IDLE-handler" would be nice.  It
>> would receive a list of the new raw message bodies and then the user can
>> decide what to do by customizing that handler.  So we can provide
>> pre-defined defaults:
>> 
>> for (1), 'run-splits
>> for (2), 'update-INBOX
>> for (3), a custom lambda or symbol
>> 
>> That way you can implement the IDLE functionality without worrying about
>> writing all the rest of the code immediately.

LMI> Yup; I think that's the way to go.

I am actually not sure if the whole body needs to be transferred (I'd
like it but others may not).  Does IDLE offer granularity or is it just
one mode where you get everything?  Anyhow, I hope you have time to do
this at some point.

Ted

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I am actually not sure if the whole body needs to be transferred (I'd
> like it but others may not).  Does IDLE offer granularity or is it just
> one mode where you get everything?  Anyhow, I hope you have time to do
> this at some point.

IDLE just says "article FOO arrived now" (and stuff), so it's very low
bandwidth by itself.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Ted Zlatanov]

On Thu, 17 Mar 2011 18:44:31 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I am actually not sure if the whole body needs to be transferred (I'd
>> like it but others may not).  Does IDLE offer granularity or is it just
>> one mode where you get everything?  Anyhow, I hope you have time to do
>> this at some point.

LMI> IDLE just says "article FOO arrived now" (and stuff), so it's very low
LMI> bandwidth by itself.

OK, that will work for me.  Just pass the text to the handler I guess.

Ted

Re: IMAP and IDLE...

[Olivier Sirven]

On 2011-03-16 12:31:00, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> But here's my idea: We could open an extra connection to every IMAP
> server (that we're doing client splits on).  We issue an IDLE on the
> INBOX. If the connection goes away, we don't do anything asynchronously.
> But otherwise, every time we get a new message in the INBOX, we run it
> through the normal splitting methods and update the group buffer.
>
> Since this will mostly be splitting one single message at a time, and we
> only download the headers to do the splitting, this can probably be done
> without hanging Emacs too badly.

It would be a great feature to have (at least for me). 

I receive about 5K/6K mails a day in my IMAP INBOX and it's a real pain
each time I refresh my group buffer. During work days I do it manually
(maybe once every 10mn) and I have to wait for 1mn before being able to
use my emacs again. I coded an autochecker to do it automatically every
5mn that I can easily launch when I need to because after 2 days without
any refresh I would have to wait for 5mn before being able to use emacs
again (that time drastically went down with the new nnimap
implementation though).

So if I can just drop that macro and have an async gnus checking my
emails and doing the splitting word everytime I receive a new email and
leaving my emacs usable it would be just the best thing in the world ;)

That being said it would be quite useful to be able to specify the list
of IMAP folders that needs an IDLE command to be issued from time to
time.

--
Olivier

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Olivier Sirven <the.slaa@gmail.com> writes:

> So if I can just drop that macro and have an async gnus checking my
> emails and doing the splitting word everytime I receive a new email and
> leaving my emacs usable it would be just the best thing in the world ;)

Yes, it would be nice.  But it remains to be seen how usable Emacs would
actually be in this scenario.  :-)

> That being said it would be quite useful to be able to specify the list
> of IMAP folders that needs an IDLE command to be issued from time to
> time.

You can only IDLE on one mail box per connection.  It's a pretty stupid
limitation in the IMAP protocol, but...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Olivier Sirven]

On 2011-03-16 18:37:54, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>> That being said it would be quite useful to be able to specify the list
>> of IMAP folders that needs an IDLE command to be issued from time to
>> time.
>
> You can only IDLE on one mail box per connection.  It's a pretty stupid
> limitation in the IMAP protocol, but...

too bad

--
Olivier

Re: IMAP and IDLE...

[David Engster]

Olivier Sirven writes:
> On 2011-03-16 18:37:54, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>>> That being said it would be quite useful to be able to specify the list
>>> of IMAP folders that needs an IDLE command to be issued from time to
>>> time.
>>
>> You can only IDLE on one mail box per connection.  It's a pretty stupid
>> limitation in the IMAP protocol, but...
>
> too bad

Well, you can use multiple connections; that's what other clients are
doing (to much dismay of some server administrators...).

-David

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Wed, 16 Mar 2011 18:37:54 +0100, Lars Magne Ingebrigtsen <larsi@gnus.org> said:

LMI> You can only IDLE on one mail box per connection.  It's a pretty stupid
LMI> limitation in the IMAP protocol, but...

There are proposals to add a multiple-idle command to fix this.  But I
don't think it's implemented anywhere yet (but haven't checked).

The way other mail systems get around this is to open multiple
connections.  Sucks, I know, but is still the only solution.

Maybe the right thing to do is do IDLE by default on INBOX (which,
ironically, I don't use at all) and then have a group parameter (or
something) set to open a new connection specifically for that box.  That
way you could set the 'this box is important flag' on it to do IDLE
updates for, say, the top 10 boxes out of the 100 that you might have?
And the 11th for all the rest (or the 11th is the INBOX IDLE one that is
also the main pull connection).
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Wes Hardaker <wes@hardakers.net> writes:

> There are proposals to add a multiple-idle command to fix this.  But I
> don't think it's implemented anywhere yet (but haven't checked).

Last time I checked, nobody implemented the proposed extension.

> The way other mail systems get around this is to open multiple
> connections.  Sucks, I know, but is still the only solution.
>
> Maybe the right thing to do is do IDLE by default on INBOX (which,
> ironically, I don't use at all) and then have a group parameter (or
> something) set to open a new connection specifically for that box.  That
> way you could set the 'this box is important flag' on it to do IDLE
> updates for, say, the top 10 boxes out of the 100 that you might have?
> And the 11th for all the rest (or the 11th is the INBOX IDLE one that is
> also the main pull connection).

It seems kinda antisocial to open multiple connections in this way.  And
slow, too, since the connection stuff is difficult to do asynchronously
in Emacs without the user getting any noticeable pauses, unfortunately.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[Wes Hardaker]

>>>>> On Thu, 17 Mar 2011 19:14:59 +0100, Lars Magne Ingebrigtsen <larsi@gnus.org> said:

LMI> It seems kinda antisocial to open multiple connections in this way.  And
LMI> slow, too, since the connection stuff is difficult to do asynchronously
LMI> in Emacs without the user getting any noticeable pauses,
LMI> unfortunately.

It depends on the setup.  Certainly I probably wouldn't do it to my
work's servers, but I'd definitely open a bunch to my personal one.
Whether or not it's anti-social is dependent on both ends, not just the
client.
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

Wes Hardaker <wes@hardakers.net> writes:

> It depends on the setup.  Certainly I probably wouldn't do it to my
> work's servers, but I'd definitely open a bunch to my personal one.
> Whether or not it's anti-social is dependent on both ends, not just the
> client.

True.  It should be left up to the user to decide.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've been sceptical to supporting IDLE before, since you can only IDLE
> on one mail box at a time.  But it just occurred to me that if you're
> doing client-side splitting, just idling on the INBOX would suffice.

FWIW, this would be somewhat useful to me, although I'd like to be able
to pick the one mailbox that it IDLEs on. The usefulness of the
automatic updating would depend on how smooth it ran.

I'd also be very interested to see if this ends up fixing my bug with
incoming new mail being marked read without me seeing it :).

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> I'd also be very interested to see if this ends up fixing my bug with
> incoming new mail being marked read without me seeing it :).

Probably not.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[James Cloos]

>>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

LMI> John Sullivan <john@wjsullivan.net> writes:
>> I'd also be very interested to see if this ends up fixing my bug with
>> incoming new mail being marked read without me seeing it :).

LMI> Probably not.  :-)

The incoming mail marked read bug is almost certainly one of the many
bugs caused by trying to store the metadata on the imap server rather
than in .newsrc.eld.

The code which is run when exiting a group probably changes the marks
on all of the mail in the group rather than just on the mail of which
gnus is already aware.

If you run g, wait for some new mail to arrive, and only then enter the
group, the new mail does not show.  But, I bet, that unseen mail still
gets marked read (or otherwise modified) when you exit the group.  And
thus appears to gnus to be old the next time you enter that group.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: IMAP and IDLE...

[Lars Magne Ingebrigtsen]

James Cloos <cloos@jhcloos.com> writes:

> If you run g, wait for some new mail to arrive, and only then enter the
> group, the new mail does not show.  But, I bet, that unseen mail still
> gets marked read (or otherwise modified) when you exit the group.  And
> thus appears to gnus to be old the next time you enter that group.

It's possible...  but Gnus only sends the marks delta to the server, so
if Gnus doesn't know about message X, it won't send any data to the
server about message X.

I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: IMAP and IDLE...

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> James Cloos <cloos@jhcloos.com> writes:
>
>> If you run g, wait for some new mail to arrive, and only then enter the
>> group, the new mail does not show.  But, I bet, that unseen mail still
>> gets marked read (or otherwise modified) when you exit the group.  And
>> thus appears to gnus to be old the next time you enter that group.
>
> It's possible...  but Gnus only sends the marks delta to the server, so
> if Gnus doesn't know about message X, it won't send any data to the
> server about message X.
>
> I think.

That sounds like a credible explanation, but on the other hand, when
I've tried to reproduce it that way, I can't. I can definitely say that
not *all* of the unseen mail gets marked read that way.