* Returning to ticks on read-only imap servers @ 2010-10-05 19:16 Michael Welsh Duggan 2010-10-05 20:25 ` Steinar Bang 2010-10-07 18:59 ` Lars Magne Ingebrigtsen 0 siblings, 2 replies; 17+ messages in thread From: Michael Welsh Duggan @ 2010-10-05 19:16 UTC (permalink / raw) To: ding I'd like to revisit the issue of ticks on read-only nnimap servers. On a particular cyrus server, the select-result is: ((((t ("OK" ("READ-WRITE") "Completed") ("FLAGS" ("\\Answered" "\\Flagged" "\\Draft" "\\Deleted" "\\Seen" "$Forwarded")) ("OK" ("PERMANENTFLAGS" "(\\Seen)")) ("2659" "EXISTS") ("0" "RECENT") ("OK" ("UIDVALIDITY" "1022333920")) ("OK" ("UIDNEXT" "15114")) ("OK" ("NOMODSEQ") "Sorry," "modsequences" "have" "not" "been" "enabled" "on" "this" "mailbox")) Setting anything by \Seen will cause the response: NO Permission denied I need non-seen ticks to be handled by the client on this server. -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-05 19:16 Returning to ticks on read-only imap servers Michael Welsh Duggan @ 2010-10-05 20:25 ` Steinar Bang 2010-10-05 21:01 ` Michael Welsh Duggan 2010-10-07 18:59 ` Lars Magne Ingebrigtsen 1 sibling, 1 reply; 17+ messages in thread From: Steinar Bang @ 2010-10-05 20:25 UTC (permalink / raw) To: ding >>>>> Michael Welsh Duggan <md5i@md5i.com>: > I'd like to revisit the issue of ticks on read-only nnimap servers. My personal opinion is that read-only nnimap servers isn't such a hot idea. NNTP access to whatever would have been better. However... > I need non-seen ticks to be handled by the client on this server. ...a fallback to storing ticks in the .newsrc.eld is probably the expected behaviour (ie. if things are to work as transparently across backends as possible)...? ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-05 20:25 ` Steinar Bang @ 2010-10-05 21:01 ` Michael Welsh Duggan 0 siblings, 0 replies; 17+ messages in thread From: Michael Welsh Duggan @ 2010-10-05 21:01 UTC (permalink / raw) To: ding Steinar Bang <sb@dod.no> writes: >>>>>> Michael Welsh Duggan <md5i@md5i.com>: > >> I'd like to revisit the issue of ticks on read-only nnimap servers. > > My personal opinion is that read-only nnimap servers isn't such a hot > idea. NNTP access to whatever would have been better. > > However... Yeah. This used to be a server with read/write access to personal groups, and read-only access to public groups. Then for some reason *cough* government clients *cough* our personal mail stuff got switched to the-never-to-be-damned-enough Exchange. >> I need non-seen ticks to be handled by the client on this server. > > ...a fallback to storing ticks in the .newsrc.eld is probably the > expected behaviour (ie. if things are to work as transparently across > backends as possible)...? I agree that this is probably the correct fallback. -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-05 19:16 Returning to ticks on read-only imap servers Michael Welsh Duggan 2010-10-05 20:25 ` Steinar Bang @ 2010-10-07 18:59 ` Lars Magne Ingebrigtsen 2010-10-07 21:37 ` Michael Welsh Duggan 1 sibling, 1 reply; 17+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-10-07 18:59 UTC (permalink / raw) To: ding Michael Welsh Duggan <md5i@md5i.com> writes: > I'd like to revisit the issue of ticks on read-only nnimap servers. I think the last time I looked at this, I stopped when I found out that EXAMINE doesn't return enough data to say whether a group is read-only or not. So nnimap has to issue a SELECT (which is slow) and then possibly store the information somewhere for later. Perhaps the first time it sees the group? -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-07 18:59 ` Lars Magne Ingebrigtsen @ 2010-10-07 21:37 ` Michael Welsh Duggan 2010-10-07 22:40 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 17+ messages in thread From: Michael Welsh Duggan @ 2010-10-07 21:37 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > Michael Welsh Duggan <md5i@md5i.com> writes: > >> I'd like to revisit the issue of ticks on read-only nnimap servers. > > I think the last time I looked at this, I stopped when I found out that > EXAMINE doesn't return enough data to say whether a group is read-only > or not. So nnimap has to issue a SELECT (which is slow) and then > possibly store the information somewhere for later. Perhaps the first > time it sees the group? I'm going to brainstorm here, so apologies if this seems a little rambling. Hmm... When does the information about marks need to be known? In order to set a mark, you need to first enter the group, which necessitates a select. So, when it comes to setting marks, I think that the information is on hand is available. (EXAMINE probably doesn't give you the right information because when EXAMINE is in play, the group is read-only by definition.) As for knowing about existing marks beforehand... If the group is not read-only, this already works. If it is read-only, the marks are stored on the client (in the .newsrc.eld). Based on the fact that ticks that were set before nnimap was rewritten still exist and work, it would seem that client-side marks are heeded despite the fact that they do not exist on the server. Unless this is only the case because these marked articles are not in the last 100... (BTW, that 100 should be in a variable, instead of being a constant value. I can see wanting to possibly change that number, possibly even on a per-group basis.) -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-07 21:37 ` Michael Welsh Duggan @ 2010-10-07 22:40 ` Lars Magne Ingebrigtsen 2010-10-08 15:07 ` James Cloos 0 siblings, 1 reply; 17+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-10-07 22:40 UTC (permalink / raw) To: ding Michael Welsh Duggan <md5i@md5i.com> writes: > Hmm... When does the information about marks need to be known? Immediately. :-) When you do a `g', you get lists of marks and stuff from the server. (nnimap issues a series of EXAMINE + UID FETCH FLAGS commands.) If it's a normal read/write mailbox, then the absence of a \Seen, for instance, means that Gnus should consider the message to be unread. If it's a read-only mailbox, then the absence of \Seen doesn't mean anything, and Gnus shouldn't alter any of the data it has on the article in question. So Gnus needs to now the status before doing anything, really, which seems to imply that nnimap has to do a SELECT on any mailboxes it hasn't seen before. Hm. That's actually not very difficult, since it already knows this stuff. So it just means that it has to stash this flag somewhere, and the group info is as good as place as any, I guess. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-07 22:40 ` Lars Magne Ingebrigtsen @ 2010-10-08 15:07 ` James Cloos 2010-10-08 17:00 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 17+ messages in thread From: James Cloos @ 2010-10-08 15:07 UTC (permalink / raw) To: ding >>>>> "LMI" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes: LMI> So Gnus needs to now the status before doing anything, really, LMI> which seems to imply that nnimap has to do a SELECT on any LMI> mailboxes it hasn't seen before. Hm. That's actually not LMI> very difficult, since it already knows this stuff. A SELECT on any group it hasn't seen before is a good idea anyway. -JimC -- James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 15:07 ` James Cloos @ 2010-10-08 17:00 ` Lars Magne Ingebrigtsen 2010-10-08 18:22 ` Julien Danjou 0 siblings, 1 reply; 17+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-10-08 17:00 UTC (permalink / raw) To: ding James Cloos <cloos@jhcloos.com> writes: > LMI> So Gnus needs to now the status before doing anything, really, > LMI> which seems to imply that nnimap has to do a SELECT on any > LMI> mailboxes it hasn't seen before. Hm. That's actually not > LMI> very difficult, since it already knows this stuff. > > A SELECT on any group it hasn't seen before is a good idea anyway. Yeah. I think I have a clear idea how to proceed here now. For groups it knows already, it'll output either EXAMINE+FETCH (or EXAMINE QRESYNC) for the servers that support that. For groups it doesn't know, it'll output a SELECT+FETCH 1:*. That SELECT will tell nnimap whether it supports flags or not, and nnimap can then stash that info. In the same sweep, it'll examine the EXAMINE results for mismatches in UIDVALIDITY, and do an extra SELECT+FETCH 1:* for those groups. So for normal `g' work, it'll be no slower than today (and much, much faster for servers with QRESYNC). Only the appearance of new groups, or UIDVALIDITY mismatches, will trigger more chatter between Gnus and the IMAP server. I'll work on implementing this this weekend. Once it's implemented, the very first time you use Gnus after the push, Gnus will issue a SELECT+FETCH 1:* for all your groups to get a complete data set again. And I'll change where nnimap stashes the data. I was confused, so I stashed the active data (and stuff) in the info marks, while they should be in the info parameters section. This will also fix itself the first time you run this. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 17:00 ` Lars Magne Ingebrigtsen @ 2010-10-08 18:22 ` Julien Danjou 2010-10-08 18:26 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 17+ messages in thread From: Julien Danjou @ 2010-10-08 18:22 UTC (permalink / raw) To: ding [-- Attachment #1: Type: text/plain, Size: 1400 bytes --] On Fri, Oct 08 2010, Lars Magne Ingebrigtsen wrote: > Yeah. I think I have a clear idea how to proceed here now. For groups > it knows already, it'll output either EXAMINE+FETCH (or EXAMINE QRESYNC) > for the servers that support that. For groups it doesn't know, it'll > output a SELECT+FETCH 1:*. That SELECT will tell nnimap whether it > supports flags or not, and nnimap can then stash that info. > > In the same sweep, it'll examine the EXAMINE results for mismatches in > UIDVALIDITY, and do an extra SELECT+FETCH 1:* for those groups. > > So for normal `g' work, it'll be no slower than today (and much, much > faster for servers with QRESYNC). Only the appearance of new groups, or > UIDVALIDITY mismatches, will trigger more chatter between Gnus and the > IMAP server. > > I'll work on implementing this this weekend. Once it's implemented, the > very first time you use Gnus after the push, Gnus will issue a > SELECT+FETCH 1:* for all your groups to get a complete data set again. That overlaps was I was saying in an earlier mail today, using UIDVALIDITY to resync everything rather than fetch the 100 last mails. The only question left is, are you forced to to a FETCH 1:*, isn't a STATUS enough? That would be very faster. You'd do the FETCH on group entering. My 2¢, -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 18:22 ` Julien Danjou @ 2010-10-08 18:26 ` Lars Magne Ingebrigtsen 2010-10-08 18:28 ` Julien Danjou 0 siblings, 1 reply; 17+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-10-08 18:26 UTC (permalink / raw) To: ding Julien Danjou <julien@danjou.info> writes: > The only question left is, are you forced to to a FETCH 1:*, isn't a > STATUS enough? That would be very faster. You'd do the FETCH on group > entering. No, a STATUS only gives you the high watermark in the group. If the high watermark increases by 10000, there may be only a single new article in the mailbox. Or there may be 10000 new messages, but 9999 of them have already been read. So you need to do a FETCH MARKS to find out what's really happened. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 18:26 ` Lars Magne Ingebrigtsen @ 2010-10-08 18:28 ` Julien Danjou 2010-10-08 18:49 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 17+ messages in thread From: Julien Danjou @ 2010-10-08 18:28 UTC (permalink / raw) To: ding [-- Attachment #1: Type: text/plain, Size: 524 bytes --] On Fri, Oct 08 2010, Lars Magne Ingebrigtsen wrote: > No, a STATUS only gives you the high watermark in the group. If the > high watermark increases by 10000, there may be only a single new > article in the mailbox. Or there may be 10000 new messages, but 9999 of > them have already been read. No, it gives you if the box changed, and how many messages/unread it has. That looks like enough info to print the group buffer. Or not? -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 18:28 ` Julien Danjou @ 2010-10-08 18:49 ` Lars Magne Ingebrigtsen 2010-10-08 19:34 ` Julien Danjou 0 siblings, 1 reply; 17+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-10-08 18:49 UTC (permalink / raw) To: ding Julien Danjou <julien@danjou.info> writes: > No, it gives you if the box changed, and how many messages/unread it > has. That looks like enough info to print the group buffer. Or not? Not. :-) You only know the number, not which ones, so you can't really say what happened. If you had high 50, unseen 25, and it increase to high 55, unseen 23, how do you know whether that's 5 new articles, and two old that were read, or 1 new article, and three that were read? Just work through the scenarios, and keep in mind that the UIDNEXT isn't necessarily consecutive. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 18:49 ` Lars Magne Ingebrigtsen @ 2010-10-08 19:34 ` Julien Danjou 2010-10-08 21:32 ` Dan Christensen 0 siblings, 1 reply; 17+ messages in thread From: Julien Danjou @ 2010-10-08 19:34 UTC (permalink / raw) To: ding [-- Attachment #1: Type: text/plain, Size: 1034 bytes --] On Fri, Oct 08 2010, Lars Magne Ingebrigtsen wrote: > Not. :-) You only know the number, not which ones, so you can't really > say what happened. But you do not care about which ones in the group buffer, I think. > If you had high 50, unseen 25, and it increase to high 55, unseen 23, > how do you know whether that's 5 new articles, and two old that were > read, or 1 new article, and three that were read? What's "high"? The last article number (UIDNEXT?) ? > Just work through the scenarios, and keep in mind that the UIDNEXT isn't > necessarily consecutive. I did not talk about UIDNEXT, but about UIDVALIDITY, which if it changes, indicate a change in the mailbox. My point is that in the *Group* buffer we (mostly) need how many messages and how many unread message there's in the group, and STATUS give that for free, with an indicator (UIDVALIDITY) indicating *if* we need to do a FETCH 1:* on group entering. :-) -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 19:34 ` Julien Danjou @ 2010-10-08 21:32 ` Dan Christensen 2010-10-09 6:57 ` Julien Danjou 0 siblings, 1 reply; 17+ messages in thread From: Dan Christensen @ 2010-10-08 21:32 UTC (permalink / raw) To: ding Julien Danjou <julien@danjou.info> writes: > My point is that in the *Group* buffer we (mostly) need how many > messages and how many unread message there's in the group, and STATUS > give that for free, with an indicator (UIDVALIDITY) indicating *if* we > need to do a FETCH 1:* on group entering. :-) I'm no expert, but my understanding is that in normal operation, UIDVALIDITY never changes, even when messages arrive, are deleted, or flags change. So it can't help solve the problem at hand. Dan ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-08 21:32 ` Dan Christensen @ 2010-10-09 6:57 ` Julien Danjou 2010-10-09 8:10 ` Michael Welsh Duggan 0 siblings, 1 reply; 17+ messages in thread From: Julien Danjou @ 2010-10-09 6:57 UTC (permalink / raw) To: Dan Christensen; +Cc: ding [-- Attachment #1: Type: text/plain, Size: 606 bytes --] On Fri, Oct 08 2010, Dan Christensen wrote: > I'm no expert, but my understanding is that in normal operation, > UIDVALIDITY never changes, even when messages arrive, are deleted, or > flags change. So it can't help solve the problem at hand. That's totally the opposite. A UIDVALIDITY number design a mailbox in a state. If any change occurs, the UIDVALIDITY number changes. That's how you know you need to do a FETCH 1:* to resynchronize your (cached) version of the mailbox. Read section 2.3.1.1 of RFC3501. -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-09 6:57 ` Julien Danjou @ 2010-10-09 8:10 ` Michael Welsh Duggan 2010-10-09 8:26 ` Julien Danjou 0 siblings, 1 reply; 17+ messages in thread From: Michael Welsh Duggan @ 2010-10-09 8:10 UTC (permalink / raw) To: Dan Christensen; +Cc: ding Julien Danjou <julien@danjou.info> writes: > On Fri, Oct 08 2010, Dan Christensen wrote: > >> I'm no expert, but my understanding is that in normal operation, >> UIDVALIDITY never changes, even when messages arrive, are deleted, or >> flags change. So it can't help solve the problem at hand. > > That's totally the opposite. A UIDVALIDITY number design a mailbox in a > state. If any change occurs, the UIDVALIDITY number changes. That's how > you know you need to do a FETCH 1:* to resynchronize your (cached) > version of the mailbox. > > Read section 2.3.1.1 of RFC3501. After re-reading that section, I think you are mistaken. UIDVALIDITY is guaranteed not to change as long as the messages in the mailbox have not changed their UIDs (or rather, as long as the mailbox maintains its identity). It does not change with the addition, or deletion, of messages to that mailbox. On the other hand, the UIDNEXT does change with the addition of new messages to the mailbox. It will not, however, change with the deletion of messages. The triple of mailbox name, UIDVALIDITY, and message UID will always refer to the same message no matter what, as long as that message still exists. "First, the next unique identifier value MUST NOT change unless new messages are added to the mailbox; and second, the next unique identifier value MUST change whenever new messages are added to the mailbox, even if those new messages are subsequently expunged." In the above extract, the "next unique identifier value" refers to UIDNEXT, not UIDVALIDITY. -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Returning to ticks on read-only imap servers 2010-10-09 8:10 ` Michael Welsh Duggan @ 2010-10-09 8:26 ` Julien Danjou 0 siblings, 0 replies; 17+ messages in thread From: Julien Danjou @ 2010-10-09 8:26 UTC (permalink / raw) To: Michael Welsh Duggan; +Cc: Dan Christensen, ding [-- Attachment #1: Type: text/plain, Size: 886 bytes --] On Sat, Oct 09 2010, Michael Welsh Duggan wrote: > After re-reading that section, I think you are mistaken. UIDVALIDITY is > guaranteed not to change as long as the messages in the mailbox have not > changed their UIDs (or rather, as long as the mailbox maintains its > identity). It does not change with the addition, or deletion, of > messages to that mailbox. On the other hand, the UIDNEXT does change > with the addition of new messages to the mailbox. It will not, however, > change with the deletion of messages. The triple of mailbox name, > UIDVALIDITY, and message UID will always refer to the same message no > matter what, as long as that message still exists. Yeah, re-re-reading I think you're right indeed.. UIDVALIDITY is not the thing I wanted it to be. That sucks. -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2010-10-09 8:26 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-10-05 19:16 Returning to ticks on read-only imap servers Michael Welsh Duggan 2010-10-05 20:25 ` Steinar Bang 2010-10-05 21:01 ` Michael Welsh Duggan 2010-10-07 18:59 ` Lars Magne Ingebrigtsen 2010-10-07 21:37 ` Michael Welsh Duggan 2010-10-07 22:40 ` Lars Magne Ingebrigtsen 2010-10-08 15:07 ` James Cloos 2010-10-08 17:00 ` Lars Magne Ingebrigtsen 2010-10-08 18:22 ` Julien Danjou 2010-10-08 18:26 ` Lars Magne Ingebrigtsen 2010-10-08 18:28 ` Julien Danjou 2010-10-08 18:49 ` Lars Magne Ingebrigtsen 2010-10-08 19:34 ` Julien Danjou 2010-10-08 21:32 ` Dan Christensen 2010-10-09 6:57 ` Julien Danjou 2010-10-09 8:10 ` Michael Welsh Duggan 2010-10-09 8:26 ` Julien Danjou
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).