Deleting messages that are in several groups

Deleting messages that are in several groups

[Kai Grossjohann]

I asked this question on gnu.emacs.help but nobody answered.  Maybe
one of you gurus can help?

I put messages into at most one group, so I don't have a problem and
no experience with messages in more than one group.  But I've got a
colleague who wants to have each message in a lot of groups.  And thus
the following question.

In the following, `delete' means anything that causes the message to
be removed from a group, including the expiry process and `B DEL'.
But just being marked as read is not sufficient for a message to be
considered deleted (unless total-expire).

Is it possible to configure and/or use Gnus so that one has two
options for deleting a message that is in several groups:

  (a) remove it from the current group but leave it in all other
      groups
  (b) remove it from all groups it is in.

Some messages might be in several groups because of the splitting
process and nnmail-crosspost being t.  Some other messages might be in
several groups because they're explicitly put there with B c (or
should B C be used instead?).

Also, what happens when a message is `B m'ed from one group to
another?  Are all its old copies deleted or just the copy in the
current group?

I think the user in question would be happy with total-expire or
auto-expire or neither, your choice.  Just make it so that he has both
choices for `deleting'.

kai
-- 
~/.signature: No such file or directory

Re: Deleting messages that are in several groups

[Joe Wells]

[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]

>>>>> "Kai" == Kai Grossjohann <grossjohann@charly.cs.uni-dortmund.de> writes:

  Kai> In the following, `delete' means anything that causes the message to
  Kai> be removed from a group, including the expiry process and `B DEL'.
  Kai> But just being marked as read is not sufficient for a message to be
  Kai> considered deleted (unless total-expire).

  Kai> Is it possible to configure and/or use Gnus so that one has two
  Kai> options for deleting a message that is in several groups:

  Kai>   (a) remove it from the current group but leave it in all other
  Kai>       groups
  Kai>   (b) remove it from all groups it is in.

  Kai> Some messages might be in several groups because of the splitting
  Kai> process and nnmail-crosspost being t.  Some other messages might be in
  Kai> several groups because they're explicitly put there with B c (or
  Kai> should B C be used instead?).

  Kai> Also, what happens when a message is `B m'ed from one group to
  Kai> another?  Are all its old copies deleted or just the copy in the
  Kai> current group?

I fixed all of these by redefining the interface between Gnus and the back
end to support dealing with crossposted articles.  Then I defined some new
user commands to take advantage of the new interface.

The problem is that I only implemented the new inferface for the nnml back
end.  My changes can't be integrated into Gnus until someone implements
the new interface for all back ends.  I posted a long and very detailed
message about these changes back in January.  I am including a copy of
this message below.

I would be really happy to send copies of my changes to anyone who wanted
to do the work necessary to integrate them into Gnus.  Lars has already
agreed with the changes in principle.  I sent the changes to him a long
time ago, but I don't think he has the necessary motivation to do the
additional work himself.

-- 
Joe Wells <jbw@dcs.gla.ac.uk>

[-- Attachment #2: Type: message/rfc822, Size: 12350 bytes --]

From: Joe Wells <jbw@csb.bu.edu>
To: ding@ifi.uio.no
Subject: new back-end interface to fix article move/delete problems
Date: 23 Jan 1997 23:03:50 -0500
Message-ID: <xnjvi8nk9cp.fsf@csb.bu.edu>

Dear fellow Gnusians (of any color, but Red preferred),

I have recently done a great deal of reimplementation to fix some problems
in using the mail-back-end commands to move, crosspost, and delete
articles that are crossposted inside the mail back end.  This work
involves a new back-end interface *-request-modify-article which subsumes
the functionality of the *-request-move-article, *-request-accept-article,
and *-request-replace-article interfaces.  I have shown my changes to Lars
and he seems happy with them.

Unfortunately, I have only implemented the back-end portion of my changes
for the nnml back end.  So I would like to recruit Gnus-loving people to
implement this interface for the other back ends.  I have written most of
the nnml implementation in a manner that does not actually depend on nnml.
Unfortunately, I don't understand nnoo well enough to make these functions
available in that way.  I also don't use the other back ends which makes
it difficult for me to test my changes.

I have enclosed a description of the new back-end interface and some of
the changes I have made below.

-- 
Joe Wells <jbw@cs.bu.edu>

----------------------------------------------------------------------
SPECIFICATION OF *-request-modify-article INTERFACE

`(nnchoke-request-modify-article
             SERVER OLD-GROUP-ARTICLE NEW-GROUP-ARTICLE BUFFER
   &optional DONT-SAVE DONT-ADD-EXTRAS-TO-NEW)'

Replaces the article specified by SERVER and OLD-GROUP-ARTICLE with one
specified by NEW-GROUP-ARTICLE and BUFFER (and SERVER).  This may involve
removing the article from some old groups and adding (crossposting) it to
some new groups.

SERVER must be a string identifying an opened virtual server in the nnchoke
back end.

OLD-GROUP-ARTICLE is an alist whose members are of the form (GROUP-NAME
. ARTICLE-NUMBER).  The group names must be distinct from each other and
must not include any server prefix.  Each group must already exist in the
server.  Collectively, the group names and article numbers must belong to
a single cross-posted article in the server.  OLD-GROUP-ARTICLE is
automatically extended to include all other existing group/article pairs
reachable by Xref header links from articles already mentioned in
OLD-GROUP-ARTICLE.  (Call these additional group/article pairs the
"extra" pairs.)  OLD-GROUP-ARTICLE may be empty (indicating that a new
article is being created).

NEW-GROUP-ARTICLE is an alist whose members are of the form (GROUP-NAME
. ARTICLE-NUMBER) or (GROUP-NAME . nil).  The group names must be distinct
from each other and must not include any server prefix.  If any of the
named groups do not exist, they will be created.  For each (GROUP-NAME
. XXX) pair in NEW-GROUP-ARTICLE, one of the following must be true:
  1. (GROUP-NAME . XXX) is a member of OLD-GROUP-ARTICLE.
  2. GROUP-NAME does not appear in OLD-GROUP-ARTICLE and either:
     a. XXX is an article number not already used in group GROUP-NAME.
     b. XXX is nil (indicating an unused article number in group
        GROUP-NAME should be allocated).
NEW-GROUP-ARTICLE may be empty (causing only deletions of the article to
occur).  NEW-GROUP-ARTICLE is automatically extended so as to include the
"extra" group/article pairs mentioned above.  NEW-GROUP-ARTICLE may be
just t or contain the element t in addition to its ordinary alist pairs.
In this case, a list of groups will be added in place of the t based on
nnmail-split-methods and the contents of BUFFER.  For each additional
group not already in NEW-GROUP-ARTICLE, a new pair (GROUP-NAME . ART-NUM)
or (GROUP-NAME . nil) is added, depending on whether such a pair is in
OLD-GROUP-ARTICLE.

Either BUFFER names a buffer containing a correctly formatted article or
BUFFER is nil.  The "Xref" and "Lines" headers do not need to contain
valid data since they will be recomputed.  The "Message-ID" header will
also be replaced if OLD-GROUP-ARTICLE is not nil.  If a "Newsgroups"
header exists, it will be adjusted to agree with the new "Xref" header.
If BUFFER is nil and NEW-GROUP-ARTICLE is non-nil, then OLD-GROUP-ARTICLE
must be non-nil, in which case one of the articles (all probably
identical) it specifies will be used.

If optional parameter DONT-SAVE is non-nil, avoids saving the active file
and NOV database files if possible.

If optional parameter DONT-ADD-EXTRAS-TO-NEW is non-nil, the "extra"
group article pairs are not added to NEW-GROUP-ALIST.

Returns no data (i.e., doesn't do anything to nntp-server-buffer).  The
return value is nil if the operation fails.  The reason for a failure may
be recovered via nnchoke-status-message.  If the operation succeeds, the
return value depends on whether NEW-GROUP-ARTICLE is non-nil.  If
NEW-GROUP-ARTICLE is nil, the return value is t, otherwise the return
value will be the same as NEW-GROUP-ARTICLE except the nil values will be
replaced by the article numbers actually used.  This function
destructively modifies NEW-GROUP-ARTICLE and the contents of BUFFER.

----------------------------------------------------------------------
EXAMPLES OF HOW TO USE THE *-request-modify-article INTERFACE

In these examples, assume that at the start there exists an article
crossposted to groups in the "serv" virtual server of the nnchoke back end
as given by this Xref header:

  Xref: foo.bar.com abcdef:412 ghijkl:398 mnopqrs:43

  (nnchoke-request-modify-article 
             SERVER OLD-GROUP-ARTICLE NEW-GROUP-ARTICLE BUFFER
   &optional DONT-SAVE DONT-ADD-EXTRAS-TO-NEW)'

To add an additional crossposting of the article in group "zyxwvu":

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412))
                                  '(("abcdef" . 412) ("zyxwvu")) nil)

To remove all copies of the article:

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412)) nil nil nil t)

To replace the crossposting in group "abcdef" by one in "zyxwvu" without
affecting the other crosspostings:

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412)) '(("zyxwvu")) nil)

To replace all of the crosspostings of the article by one in "zyxwvu":

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412))
                                  '(("zyxwvu")) nil nil t)

To replace all of the crosspostings of the article by crosspostings
determined by consulting nnmail-split-methods:

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412)) t nil nil t)

To add additional crosspostings determined by consulting
nnmail-split-methods without affecting the existing crosspostings:

  (nnchoke-request-modify-article "serv" '(("abcdef" . 412))
                                  '(t ("abcdef" . 412)) nil)

With any of these examples, one can supply a buffer for the 4th argument
to simultaneously change the body and/or headers of the article.

To add a new article to the system (or copy an old article) into group
"zyxwvu" using the headers and body in BUFFER:

  (nnchoke-request-modify-article "serv" nil '(("zyxwvu")) BUFFER)

To add a new article to the system (or copy an old article) using groups
determined by consulting nnmail-split-methods and using the headers and
body in BUFFER:

  (nnchoke-request-modify-article "serv" nil t BUFFER)

----------------------------------------------------------------------
DESCRIPTION OF CHANGES (in pseudo-ChangeLog format)

gnus-sum.el:

  New Summary mode keybindings:

    B D       gnus-summary-backend-delete-map
    B D d     gnus-summary-delete-article
    B D a     gnus-summary-delete-article-crosspostings-also
    B D o     gnus-summary-delete-article-other-crosspostings-only

  gnus-summary-move-article:

    Reimplemented using new *-request-modify-article interface.  No longer
    uses *-request-move or *-request-accept.

    Doc string is now complete, including correct description of crosspost
    and copy behavior specified by the ACTION parameter.

    Fixes these problems/bugs:

      New articles were not crossposted with the source article (or
      preexisting articles already crossposted with the source article)
      when possible, even when ACTION was crosspost.  (There was code to
      do this, it was just broken in a number of different ways.) 

      Xref headers in article files were sometimes left different from the
      value in the NOV file.  (This happened for pre-existing articles
      that were crossposted with the source article.)

      Corrupt Xref headers were generated.  (Spaces were missing.)

      Marks in the wrong group could be updated after a move.

      Marks were not propagated when ACTION was crosspost.

      Would propagate marks when ACTION was copy.

      Would leave dangling Xref links.  (The Xref header would point to
      nonexistent articles.  This was not a serious problem since Gnus is
      accustomed to dealing with this in the case of expired articles.)

      If the code for handling Xref links when crossposting had worked at
      all, it could have done crossposts across multiple back ends and
      created Xref lines which mentioned the same newsgroup twice!  These
      would be serious semantic errors.
      
      Did not update the various copies of an article's header vector in
      active summary buffers.
      
      Did not flush the old version of an article (with the old Xref
      header) from various caches (backlog, async prefetch,
      original-article-buffer).  *** Still not fixed for the "cache". ***
      
      Did not check that at most one of TO-GROUP and SELECT-METHOD were
      non-nil.

    I'm still working on fixing these problems:

      Marks are never propagated to new groups created as part of the
      move.

      Does not update the Xref header in the Article buffer if viewing the
      article in question.
      
  gnus-summary-delete-article-crosspostings-also:
    New function.

  gnus-summary-delete-article-other-crosspostings-only:
    New function.

  gnus-summary-delete-article:

    Added optional parameter CROSSPOSTINGS to support previous two
    functions.

    Reimplemented using *-request-modify-article for two reasons:
      1. Supporting previous two functions.
      2. Fix bug where Xref headers were not kept up-to-date.  (This is
         less important.)

    Doc string now explains return value.

nnml.el:

  nnml-request-move-article, nnml-request-accept-article,
  nnml-request-replace-article:

    Completely reimplemented using nnml-request-modify-article.
    
  nnml-request-modify-article:
    
    New function.

    Preserves Xref headers properly in both articles and NOV files.
    Can add new crosspostings to an existing article.

    Subsumes functionality of *-request-move-article,
    *-request-accept-article, and *-request-replace-article interfaces,
    but also allows much more with a very easy-to-use interface.  New
    interface is completely documented in the doc string in a form
    suitable for pasting into the manual section on back-end interfaces.
    (Interface specification included below at end of this file.)

    Fixes bug where LAST parameter of nnml-request-move-article or
    nnml-request-accept-article might be t but the NOV files and active
    list are not flushed to disk due to an error causing an abort before
    the flushing code is reached.

  nnml-validate-existing-gaa,
  nnml-validate-potential-gaa,
  nnml-verify-writable-gaa,
  nnml-create-gaa-groups,
  nnml-allocate-gaa-article-numbers,
  nnml-delete-gaa,
  nnml-gaa-difference,
  nnml-gaa-union,
  nnml-find-other-crossrefs-for-gaa,
  nnml-article-header,
  nnml-article-message-id:
    New functions used by nnml-request-modify-article.

  nnml-add-nov:
    Wrote complete doc string.
    
    Extended to also be able to set the NOV entry for an arbitrary article
    number, existing or not.  Still runs fast for append case.
 
gnus-int.el:

  gnus-request-modify-article:
    New function.  Written in the standard manner for a back-end interface
    wrapper.  Only non-trivial aspect: It changes its method argument into
    a server name (i.e., address) argument.

----------------------------------------------------------------------

[-- Attachment #3: Type: message/rfc822, Size: 3123 bytes --]

From: Joe Wells <jbw@csb.bu.edu>
To: ding@ifi.uio.no
Subject: Re: new back-end interface to fix article move/delete problems
Date: 10 Feb 1997 23:32:26 -0500
Message-ID: <xnjraioezdx.fsf@csb.bu.edu>

>>>>> "Paul" == Paul Franklin <paul@cs.washington.edu> writes:

  Paul> Joe proposed some backend interface modifications to fix problems with
  Paul> moving, etc. crossposted articles, which I'm assuming will make it
  Paul> into qgnus.  ...

  Paul> ...  However, I'm considering writing a
  Paul> backend based on nnml (if I can figure out the nnoo stuff) which might
  Paul> not be able to handle case 2a for an element of NEW-GROUP-ARTICLE.  So
  Paul> I'm wondering when this case would get used.

The relevant part of the proposed back-end-interface:

  >> NEW-GROUP-ARTICLE is an alist whose members are of the form (GROUP-NAME
  >> . ARTICLE-NUMBER) or (GROUP-NAME . nil).  The group names must be distinct
  >> from each other and must not include any server prefix.  If any of the
  >> named groups do not exist, they will be created.  For each (GROUP-NAME
  >> . XXX) pair in NEW-GROUP-ARTICLE, one of the following must be true:
  >>   1. (GROUP-NAME . XXX) is a member of OLD-GROUP-ARTICLE.
  >>   2. GROUP-NAME does not appear in OLD-GROUP-ARTICLE and either:
  >>      a. XXX is an article number not already used in group GROUP-NAME.
  >>      b. XXX is nil (indicating an unused article number in group
  >>         GROUP-NAME should be allocated).

There are two goals behind possibility 2a (neither of which have been
implemented):

  1. An article is originally crossposted in groups A, B, and C as
     articles 5, 6, and 7 (respectively).  The user removes it from group
     C.  Later, the user asks for the the article to be crossposted in
     group C.  It should be possible to reuse the old number 7 rather than
     the number that would be given to a new article.

  2. The user asks for an article to be crossposted to group A.  The
     article was originally sent in 1994 (three years ago).  If the
     article is given the number in group A that would be given to a new
     article, then it may be listed in the summary buffer for that group
     after many articles that are actually newer than it.  If there were
     gaps in the sequence of existing article numbers in group A, it
     should be possible to use one of the numbers in the gaps instead, so
     that the article gets listed in a more natural order.

Goal number 2 is more questionable than goal number 1.

A reasonable restriction of the proposed back-end-interface would be to
require that a non-existing article number must be one that the article
used earlier that has since been deleted.  This would support goal 1 but
not goal 2.  It would also probably avoid the problems that Paul
envisioned.

-- 
Joe Wells <jbw@cs.bu.edu>

Re: Deleting messages that are in several groups

[Kai Grossjohann]

>>>>> On 12 Nov 1997, Joe Wells said:

  Joe> I fixed all of these by redefining the interface between Gnus
  Joe> and the back end to support dealing with crossposted articles.
  Joe> Then I defined some new user commands to take advantage of the
  Joe> new interface.

I like your ideas a lot.  Would be a very good thing to have.  If only
I groked Gnus enough to be able to help...

But my immediate problem is what does the current version of Gnus (I'm
talking about Emacs 20/Gnus 5.5 here) do?  And is it possible for that
prospective new user of Gnus to be able to both delete one and all
copies of a message using some combination of auto-expire,
total-expire and mail backend?

kai
-- 
~/.signature: No such file or directory