Re: nnimap and crossposting (Re: Moving from nnml to nnimap...)

[Simon Josefsson <jas@extundo.com>]

Ken Raeburn <raeburn@raeburn.org> writes:

> Simon Josefsson <jas@extundo.com> writes:
>>> My experimentation so far (playing with imtest, don't have the guest
>>> account and nnimap config set up yet) seems to indicate that the marks
>>> won't be propagated.  If this is the case, I think it would be a good
>>> idea for the documentation to indicate this.
>>
>> IMAP doesn't understand the concept of crossposting, I think, so this
>> sounds normal.  How does crosspost mark propagation work for nntp
>> groups?
>
> Using the xref header, I believe, which as far as I can tell, is
> generated on the fly for nnimap, and only lists the current group.

Ah.

> I wonder, is there a way to go through all the groups and ask the
> server if a certain set of message-ids are present in each?

Yes, iterate over the Gnus groups and use `imap-search' for each
group.  E.g., (imap-search "HEADER Message-Id foo@bar"
nnimap-server-buffer).

> It would probably be painfully slow if there are a lot of groups
> (I've got over 200 groups in my nnml hierarchy), and would require
> either staying connected until all the groups are scanned or storing
> the data in some magic place so the scan can be resumed if
> interrupted.

Yes, it doesn't sound like a realistic plan for a default behaviour,
but might be implemented as an add-on workaround.

> Worst case, perhaps Gnus could create a variant on the Xref header,
> which says which folders the article was going to be copied to, and
> leave out the message numbers, but that presumably requires Gnus to
> download, edit, and upload each message, and it wouldn't work with
> something like Sieve doing the refiling as messages come in, which is
> where I'd prefer to get eventually.

Hm.  What about: when marking an article, use the current
nnimap-split-rule to determine which groups would receive the current
message, and then use the imap-search logic above on only those
groups, to find any crossposted copies?  Will probably be rather slow
anyway, and assumes nnimap-split-rule knows about all split rules, but
it is an improvement.

>>    I suspect it might be possible to implement something like it
>> for nnimap by looking in the nnmail message-id split log when applying
>> marks to a group (i.e., for each article with new marks, look up the
>> message id in the split log and find out what other groups it was
>> crossposted to and apply the same mark to it).
>
> That means this split log would have to be permanent, not per-session,
> so I could read (and mark expirable across groups) in this session a
> message I got during the last session.  That per-server data will get
> very big, and slow to scan.

I think the nnmail split-log already is permanent, it is used for,
e.g., `nnmail-split-fancy-with-parent' which have similar needs (e.g.,
find destination groups for earlier messages based on message-id).

> And it would need to be shared between different client systems one
> might run Gnus on.  On the one hand, that might not be much worse than
> the .newsrc.eld file, but I'd kinda like to be able to browse my mail
> on my home server from my Gnus session at work on occasion, and vice
> versa, and they really shouldn't have to share all the .newsrc.eld
> data.  And isn't it part of the point of IMAP that such data lives on
> the server?

Yes.  It would only work if you only use one instance of Gnus, and it
does all the splitting, or if you somehow synchronize the information
between all Gnus instances.

So here is a more IMAPish, but still Gnus-specific, idea:

When nnimap crosspost split an article into several groups, it add
client-specific message flags on each message, indicating the mailbox
and article number of the other crossposted copies.  For example, if
article 4711 in mailbox INBOX.foo have a mark
gnus-crosspost-INBOX.bar-42, then when marking INBOX.foo:4711 the same
mark be applied to INBOX.bar:42 as well.  There is one immediate
problem: article numbers may change over time.  Therefor, perhaps just
indicating the destination mailbox name might be enough, and after
knowing the mailbox name, the imap-search logic above could be used.
Alternatively, the mark can contain UIDVALIDITY too, and nnimap would
only try to set the mark on the other articles if the destination
group have the same UIDVALIDITY and the UID still exist.  The latter
alternative would fast, the only delay is opening all other groups to
set flags in.  A minor problem would be that these crosspost marks are
never garbage collected, so they will stick around forever.  Perhaps
that's OK.