Gnus Questions #1: Article Expiry

Gnus Questions #1: Article Expiry

[Dave Abrahams]

Hi Lars and all,

I am attempting a return to Gnus...

...but I've decided that this time around I can't afford to do that
without actually grokking the system I'm using.  I've been reading
carefully through the manual and doing some experiments so I can verify
my understanding, such as it is.

I'm also growing a list of questions. I'll try to separate them by topic
and contribute doc patches on the basis of the answers.  Please also
correct any false statements below:

* What's the difference between marking an article "expirable (E)" and
  `marking it expired' as described by [[info:gnus#Spam and Ham
  Processors]]?

* [[info:gnus#Expiring Mail]] and [[info:gnus#Group Parameters]] both
  mention articles being `put through the expiry process,' but that
  process is never spelled out.  What exactly is involved?

* I presume that "expiring" an article means the same as "putting the
  article through the expiry process" (?)

* What's the difference between `gnus-summary-expire-articles' and
  `gnus-summary-expire-articles-now'?  The documentation doesn't make
  that clear.

* What's the point of backend-specific expiry settings like
  nnfolder-inhibit-expiry (I'm referring to the `nnfolder-' part when I
  say `backend-specific')?  Don't we have enough other ways to say "this
  group/server isn't expirable?"

* [[info:gnus#Expiring Mail]] mentions `having auto expiry switched on'
  but doesn't say how to do that.  Are we talking about the auto-expire
  group parameter here, or something else?

* "Total Expire" and "Auto Expire"

  * The main point of using "Total Expire" instead of "Auto Expire"
    seems to be that with "total expire" you can keep a distinction
    between expirable (`E') and other marks that indicate an article was
    read... until expiry actually runs.  At that point, if you're using
    total expire they're all treated the same.  With "auto expire," on
    the other hand, you know that only articles marked `E' will be
    put through the expiry process.

  * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
    scoring takes effect at expiry time, and "auto-expire" changes all
    read marks to `E' too early for adaptive scoring to do its work.  Is
    that right?
   
  * [[info:gnus#Expiring Mail]] seems to contradict my understanding,
    though: it claims that "auto-expire" gives me "more marks to work
    with."

    ,----
    | Another advantage of auto-expire is that you get more marks to work
    | with: for the articles that are supposed to stick around, you can
    | still choose between tick and dormant and read marks.  But with
    | total-expire, you only have dormant and ticked to choose from
    `----

    Okay, now that I read it again I think it's saying that with
    "auto-expire," if I can somehow produce a mark other than `E' for an
    article that's been read,, that article can persist even if it's
    neither dormant or ticked.  That's fine as far as it goes but
    mentioning it seems almost pointless, since Gnus is going to
    automatically mark everything I read as `E'.  What am I missing?

  * This (from [[info:gnus#Expiring Mail]]) seems confusing for what I
    think are related reasons:
   
     ,----
     | Note that making a group auto-expirable doesn't mean that all read
     | articles are expired--only the articles marked as expirable will be
     | expired
     `----

    If auto-expire automatically marks articles expirable when you read
    them, doesn't that /necessarily/ mean all read articles are expired
    (except articles you read in the past?).  Is there a common usage
    model where people set auto-expire and then use some explicit
    commands to change the read mark on some of their articles after
    Gnus has already marked them `E'?

  * Aren't there a bajillion other ways to do the following, including
    by customizing the "auto-expire" group parameter?  Why would I do it
    as below (see [[info:gnus#Expiring Mail]]) instead?
   
    ,----
    | To avoid having articles marked as read marked as
    | expirable automatically, you can put something like the following in
    | your `~/.gnus.el' file:
    | 
    |      (remove-hook 'gnus-mark-article-hook
    |                   'gnus-summary-mark-read-and-unread-as-read)
    |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
    | 
    `----
    

* How does the expire-age group parameter come into play?
  - Does it prevent me from marking articles as expirable for a period?
  - Does it prevent auto-expire from marking articles expirable?
  - Does it simply exempt articles that are too young from expiry?

* Suggestion: Rename `gnus-auto-expirable-newsgroups'
  `gnus-auto-expirable-groups' since, generally, auto-expire only
  applies to mail and not to nntp.
      
* As far as I can tell, the user experience of Agent expiry is
  completely different from that of regular article expiry.  This isn't
  explained anywhere, but IIUC from experiments, agent expiry doesn't
  delete any mail except from the agent's cache.  Instead it merely
  flushes (some part of) the agent's cache.  Is that right?

  I and several of my friends have long been plagued by the symptom that
  if I delete an IMAP message in some other mail client, it still hangs
  around in Gnus.  I've beat my head against
  `gnus-agent-regenerate[-group]' and `gnus-agent-flush-*' and other
  trix for years trying to correct it, but never found a reliable
  formula.  Agent expiry seems to be the key.  I think.  Have I got that
  right?

Thanks for taking the time with these questions,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: Gnus Questions #1: Article Expiry

[lee]

Dave Abrahams <dave@boostpro.com> writes:

> * I presume that "expiring" an article means the same as "putting the
>   article through the expiry process" (?)

Well, when I mark an article `E', I'm "expiring" it.  It'll be purged
some time later --- apparently only *if* I enter and leave the group the
article is in later.  If I don't, the article will never be purged: is
that true?

> * What's the difference between `gnus-summary-expire-articles' and
>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>   that clear.

The difference is in "delete all expirable articles in the group /that
have been around for a while/" vs. "This means that *all* articles
eligible for expiry in the current group will disappear" /right now/.
The documentation could be more clear on this, like:


,----
| `B e'
|      Perform an expiry process on the articles in the current
|      group. Only articles that are expirable *and due to be purged*
|      will be purged.  The expiry process is performed by the function
|      `gnus-summary-expire-articles'.  (See Expiring Mail::).
| 
| `B C-M-e'
|      Purge all the expirable articles in the current group *right
|      now*. This means that all expirable articles will be purged
|      immediately.  The purging process is performed by the function
|      `gnus-summary-expire-articles-now'.  (See Expiring Mail::).
`----

How about renaming "gnus-summary-expire-articles-now" to
"gnus-summary-purge-articles" (or to "gnus-purge-articles")?

There also needs to be some clarification about what "expiring" means.
There seem to be actually quite a few stages of expirableness:

* not expirable, depends on gnus-inhibit-user-auto-expire,
                            nnfolder-inhibit-expiry??
* marked as `E', depends on age
* marked as `r', depends on age, total-expire
* marked as `R', depends on age, total-expire
* marked as `O', depends on age, total-expire
* marked as `K', depends on age, total-expire
* marked as `Y', depends on age, total-expire
* marked as "and so on"[1], apparently depends total-expire only
* marked as `G': depends on gnus-inhibit-user-auto-expire,
                            nnfolder-inhibit-expiry??


Then there's the purgability of articles.  It seems that articles marked
`G' are always purged on leaving a group.  Perhaps they cannot be purged
eventually, depending on some settings (and marks, maybe):

,---- [ info:gnus#Expiring Mail ]
|    If `gnus-inhibit-user-auto-expire' is non-`nil', user marking
| commands will not mark an article as expirable, even if the group has
| auto-expire turned on.
`----

This mentions marking only. Will articles in stages of expirableness
eventually be purged nonetheless?

What disables the purgability of an article?


[1] {
      ,---- [ info:gnus#Expiring Mail ]
      | So, in addition to the articles marked `E', also the articles
      | marked `r', `R', `O', `K', `Y' and so on are considered
      | expirable.
      `----

      What other marks are there?  Can I prevent articles from being
      marked as `O' automatically?  What means `r`?  Can I configure
      which marks put articles into stages of expirableness? There needs
      to be an overview of the possible marks in the chapter about
      expiring mail, or somewhere.
}

> * What's the point of backend-specific expiry settings like
>   nnfolder-inhibit-expiry (I'm referring to the `nnfolder-' part when I
>   say `backend-specific')?  Don't we have enough other ways to say "this
>   group/server isn't expirable?"

Hm, nnfolder-inhibit-expiry doesn't seem to be explained in the
documentation.  What does it inhibit, the purgability or the
expirableness?

Which other ways to say that a server/group isn't expirable (or
purgeable, which is a difference) have you found?

Is there a way to request information about the expirableness and
purgability of articles? You'd get an information like "This article is
[not expirable | expired | expirable and will expire when ...] because
.... This article is [not purgable | purgable and will be purged when
...] because ....".  Gnus must have some way of figuring this out to be
able to expire and purge articles correctly.

> * [[info:gnus#Expiring Mail]] mentions `having auto expiry switched on'
>   but doesn't say how to do that.  Are we talking about the auto-expire
>   group parameter here, or something else?

Yeah, it should be clarified.

> * "Total Expire" and "Auto Expire"
>
>   * The main point of using "Total Expire" instead of "Auto Expire"
>     seems to be that with "total expire" you can keep a distinction
>     between expirable (`E') and other marks that indicate an article was
>     read... until expiry actually runs.  At that point, if you're using
>     total expire they're all treated the same.  With "auto expire," on
>     the other hand, you know that only articles marked `E' will be
>     put through the expiry process.

Yes: Total-expire makes articles purgeable *without* marking
them. Auto-expire makes articles purgeable *by* marking them.

>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>     scoring takes effect at expiry time, and "auto-expire" changes all
>     read marks to `E' too early for adaptive scoring to do its work.  Is
>     that right?

Non-adaptive scoring seems to happen when you enter a group.  Each time
you enter the group, the scoring rules are applied, resulting in
incredible scores.  I wonder what the idea behind this way of
implementing it is.  Is that different with adaptive scoring?

>     Okay, now that I read it again I think it's saying that with
>     "auto-expire," if I can somehow produce a mark other than `E' for an
>     article that's been read,, that article can persist even if it's
>     neither dormant or ticked.  That's fine as far as it goes but
>     mentioning it seems almost pointless, since Gnus is going to
>     automatically mark everything I read as `E'.  What am I missing?

Auto-expire does *not* lead to eventually purging articles you don't
touch, while total-expire *does* lead to eventually purging articles you
don't touch (because gnus touches them with scoring and `O' marks, for
example).

>   * This (from [[info:gnus#Expiring Mail]]) seems confusing for what I
>     think are related reasons:
>    
>      ,----
>      | Note that making a group auto-expirable doesn't mean that all read
>      | articles are expired--only the articles marked as expirable will be
>      | expired
>      `----
>
>     If auto-expire automatically marks articles expirable when you read
>     them, doesn't that /necessarily/ mean all read articles are expired
>     (except articles you read in the past?).

The documentation somewhat struggles trying to explain that the
auto-expire option doesn't do anything else than marking articles with
`E' when they otherwise would be marked `R'.

To say just that and not much more would probably suffice.


Thinking of this, what's the point behind gnus changing marks
automatically?  When I mark something read, it'll later be marked `O'.
When marking something as killed, that doesn't seem to be any different
from marking it as read (unless you use adaptive scoring maybe).  Gnus
automatically marks it as `O' as well.

Why doesn't gnus keep the distinction between "read" and "killed"?  When
I enter a group after having killed some of the threads, I still want to
be able to see what threads I've killed, what I have actually read and
what I didn't read but "cought up" with.  I don't trust the scoring
because it might hide things from me I want to see, so seeing what I
read, killed and cought up with is an important information to me.

Can't gnus mark as `R' what I've read, as `K' what I've killed and
as `O' what I cought up with and keep these marks intact?


-- 
html messages are obsolete

Re: Gnus Questions #1: Article Expiry

[Dave Abrahams]

Lee, thanks a *lot* for jumping in.  I know this is a big one...

on Mon Jul 04 2011, lee <lee-AT-yun.yagibdah.de> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> * I presume that "expiring" an article means the same as "putting the
>>   article through the expiry process" (?)
>
> Well, when I mark an article `E', I'm "expiring" it.  

I don't believe that interpretation consistent with the terminology as
used in the majority of the document.  IIUC, marking an article `E'
makes it "expirable" but the actual expiration happens later, if at all.

> It'll be purged some time later --- apparently only *if* I enter and
> leave the group the article is in later.  

There are other ways to trigger the expiry process, e.g. `M-g'
(gnus-summary-rescan-group) and expiry doesn't necessarily mean the
article will be deleted (or "purged").  For example, the article could
be refiled.  Or nothing at all could happen. It all depends how you set
things up, as far as I can tell.

It would be nice to have a detailed description of exactly what options
and user hooks are checked/called by the system in what order.

> If I don't, the article will never be purged: is that true?

The article can be deleted in other ways, e.g. `B DEL'
(gnus-summary-delete-article).

>> * What's the difference between `gnus-summary-expire-articles' and
>>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>>   that clear.
>
> The difference is in "delete all expirable articles in the group /that
> have been around for a while/" vs. "This means that *all* articles
> eligible for expiry in the current group will disappear" /right now/.

Yeah, I read all those words, but they didn't add up to any obvious
distinction for me. The phrases "have been around for a while" and
"eligible for expiry" are vague at best.  By contrast the use of
"delete" and "disappear" seem implausibly concrete for a system whose
expiration behavior can range from "do nothing" (which is undocumented,
but that's what you get from nnimap if the expiry target is nil) to
"forward the article to my mother-in-law" (if I write the function to do
that).  I would rather say that the articles are "sent to the group's
expiry target" (a phrase I'd suitably define in Expiring Mail::).

> The documentation could be more clear on this, 

Thus my posting :-)

> like:
>
>
> ,----
> | `B e'
> |      Perform an expiry process on the articles in the current
> |      group. Only articles that are expirable *and due to be purged*
> |      will be purged.  The expiry process is performed by the function
> |      `gnus-summary-expire-articles'.  (See Expiring Mail::).
> | 
> | `B C-M-e'
> |      Purge all the expirable articles in the current group *right
> |      now*. This means that all expirable articles will be purged
> |      immediately.  The purging process is performed by the function
> |      `gnus-summary-expire-articles-now'.  (See Expiring Mail::).
> `----

Unfortunately, that change doesn't improve the clarity for me.
It doubles down on the overly-concrete aspects of the existing text.  It
also makes it sound like `B e' isn't directly bound to
`gnus-summary-expire-articles', but instead invokes it indirectly as
/part/ of its functionality.

If the following changes are accurate, I prefer them:

`B e'  - `gnus-summary-expire-articles'
     Run all articles in the current group that are marked expirable
     (`E') through the expiry process (*note Expiring Mail::).

`B C-M-e' - `gnus-summary-expire-articles-now'
     Send all articles in the current group marked expirable (`E')
     directly to their expiry target regardless of their age
     (*note Expiring Mail::).

> How about renaming "gnus-summary-expire-articles-now" to
> "gnus-summary-purge-articles" (or to "gnus-purge-articles")?
>
> There also needs to be some clarification about what "expiring" means.

+1

> There seem to be actually quite a few stages of expirableness:

Using an informal definition of `expirable' ("when expiry runs, this
article will be sent to the expiry target"), I agree.  I think the docs
are pretty strict about using `expirable' only to refer to articles
marked `E', though.  Now, we could discuss the merits of changing that,
but it's the status quo.

> * not expirable, depends on gnus-inhibit-user-auto-expire,

Based on my reading of the docs, I don't think it depends on that.
`gnus-inhibit-user-auto-expire' seems to prevent
`gnus-summary-mark-as-expirable' (normally bound to `E') from having any
effect:

,----
| *If non-nil, user marking commands will not mark an article as
| expirable, even if the group has auto-expire turned on.
`----

AFAICT an article that already has an `E' mark is still expirable in
both the formal and informal sense, though.

[I don't know what other "user marking commands" this text might be
referring to.  Are there any?]

>                             nnfolder-inhibit-expiry??

> * marked as `E', depends on age
> * marked as `r', depends on age, total-expire
> * marked as `R', depends on age, total-expire
> * marked as `O', depends on age, total-expire
> * marked as `K', depends on age, total-expire
> * marked as `Y', depends on age, total-expire

The above matches my understanding.

> * marked as "and so on"[1], apparently depends total-expire only

Yeah, that's disturbingly vague.

> * marked as `G': depends on gnus-inhibit-user-auto-expire,

Again, unless the docs are just wrong or extremely misleading, I don't
think so.

>                             nnfolder-inhibit-expiry??
>
>
> Then there's the purgability of articles.  It seems that articles marked
> `G' are always purged on leaving a group.  
> Perhaps they cannot be purged
> eventually, depending on some settings (and marks, maybe):

My (weak) understanding is that `G' (cancelled) only applies to nntp
groups and what happens to cancelled articles depends on whether the
nntp server responds to cancellation.

> ,---- [ info:gnus#Expiring Mail ]
> |    If `gnus-inhibit-user-auto-expire' is non-`nil', user marking
> | commands will not mark an article as expirable, even if the group has
> | auto-expire turned on.
> `----
>
> This mentions marking only. Will articles in stages of expirableness
> eventually be purged nonetheless?

To answer precisely, I believe they will be sent to their expiry targets
according to the usual other criteria (e.g. age) and regardless of this
setting.  At least, that's what the doc seems to say.  A quick
experiment should be enough to confirm this, though, if you have doubts.

> What disables the purgability of an article?

If "purge" means "delete," then as far as I can tell nothing disables it
other than, potentially, the server itself (e.g. if something like
dovecot is serving you a read-only IMAP group or an nntp server
disallows article cancellation)

> [1] {
>       ,---- [ info:gnus#Expiring Mail ]
>       | So, in addition to the articles marked `E', also the articles
>       | marked `r', `R', `O', `K', `Y' and so on are considered
>       | expirable.
>       `----
>
>       What other marks are there?  

Lots.  My most complete list of marks to date is below, but I wouldn't
be surprised to learn that I missed something.:

"R"
"K"
"-"
" " (gnus-unread-mark)
"r" (explicit delete)
"X" (explicit kill)"
"Y" (low score)
"=" (gnus-unsendable-mark)
"%" (gnus-downloadable-mark")
"!" (gnus-ticked-mark)
"$" (gnus-spam-mark)
"?" (gnus-dormant-mark)
"E" (gnus-expirable-mark)
"O" (gnus-ancient-mark) Articles that were marked as read in previous sessions and are now "old"
"A" (gnus-replied-mark) All articles that you have replied to or made a followup to
"*" Article is "[[info:gnus#Persistent%20Articles][persistent]]"
`C' Marked as read by a catchup (`gnus-catchup-mark').
`G' Canceled article (`gnus-canceled-mark')
`Q' Sparsely reffed article (`gnus-sparse-mark').  
`M' Article marked as read by duplicate suppression (`gnus-duplicate-mark'). 

>       Can I prevent articles from being marked as `O' automatically?

Good question

> What means `r`?  

The documentation (somewhere) says it means the article was "explicitly
deleted," but near as I can tell, it means you invoked
gnus-summary-mark-as-read-forward (`d') on the article.  Aside from that
and the effects of potentially having different membership in
`gnus-auto-expirable-marks', it's equivalent to `R'.  I think.

>       Can I configure
>       which marks put articles into stages of expirableness? 

See gnus-auto-expirable-marks.  Not sure if that's what you have in
mind, though.

>       There needs
>       to be an overview of the possible marks in the chapter about
>       expiring mail, or somewhere.
> }

Indeed: http://debbugs.gnu.org/db/89/8978.html

>> * What's the point of backend-specific expiry settings like
>>   nnfolder-inhibit-expiry (I'm referring to the `nnfolder-' part when I
>>   say `backend-specific')?  Don't we have enough other ways to say "this
>>   group/server isn't expirable?"
>
> Hm, nnfolder-inhibit-expiry doesn't seem to be explained in the
> documentation.  What does it inhibit, the purgability or the
> expirableness?

My guess from the name is that it prevents the expiry process from
running on that group.

> Which other ways to say that a server/group isn't expirable (or
> purgeable, which is a difference) have you found?

I think, among others:

  setting nnmail-expiry-wait to `never', and if undocumented behaviors
  count, setting nnmail-expiry-target to nil.

and similar settings in the individual group parameters.

> Is there a way to request information about the expirableness and
> purgability of articles? You'd get an information like "This article is
> [not expirable | expired | expirable and will expire when ...] because
> .... This article is [not purgable | purgable and will be purged when
> ...] because ....".  Gnus must have some way of figuring this out to be
> able to expire and purge articles correctly.

That is an *excellent* idea for a new feature.  I suggest `M-x gnus-bug
RET [feature-request]...'

>> * [[info:gnus#Expiring Mail]] mentions `having auto expiry switched on'
>>   but doesn't say how to do that.  Are we talking about the auto-expire
>>   group parameter here, or something else?
>
> Yeah, it should be clarified.
>
>> * "Total Expire" and "Auto Expire"
>>
>>   * The main point of using "Total Expire" instead of "Auto Expire"
>>     seems to be that with "total expire" you can keep a distinction
>>     between expirable (`E') and other marks that indicate an article was
>>     read... until expiry actually runs.  At that point, if you're using
>>     total expire they're all treated the same.  With "auto expire," on
>>     the other hand, you know that only articles marked `E' will be
>>     put through the expiry process.
>
> Yes: Total-expire makes articles purgeable *without* marking
> them. Auto-expire makes articles purgeable *by* marking them.

I obviously don't understand your definition of "purgeable."  I used to
think it meant "able to be deleted," but now I think it's something else.

I think I disagree with your use of "*without* marking," as IIUC
an article must acquire one of the "read marks" in order to be expired.

>>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>>     scoring takes effect at expiry time, and "auto-expire" changes all
>>     read marks to `E' too early for adaptive scoring to do its work.  Is
>>     that right?
>
> Non-adaptive scoring seems to happen when you enter a group.  Each time
> you enter the group, the scoring rules are applied, resulting in
> incredible scores.  I wonder what the idea behind this way of
> implementing it is.  Is that different with adaptive scoring?

I don't know. I don't think it's particularly relevant to my
question or the topic of expiry (which I'm trying to handle in this
thread) so if you don't mind I'd prefer not to discuss that here, just
to keep this from spiralling way beyond everyone's TL;DR limit.

>>     Okay, now that I read it again I think it's saying that with
>>     "auto-expire," if I can somehow produce a mark other than `E' for an
>>     article that's been read,, that article can persist even if it's
>>     neither dormant or ticked.  That's fine as far as it goes but
>>     mentioning it seems almost pointless, since Gnus is going to
>>     automatically mark everything I read as `E'.  What am I missing?
>
> Auto-expire does *not* lead to eventually purging articles you don't
> touch, while total-expire *does* lead to eventually purging articles you
> don't touch (because gnus touches them with scoring and `O' marks, for
> example).

Well, define "touch."  I think you mean "read."  But maybe you don't,
because IIUC `O' is only applied by Gnus to articles you've read (as you
seem to acknowledge below).  I suppose scoring can mark articles
automatically.

>>   * This (from [[info:gnus#Expiring Mail]]) seems confusing for what I
>>     think are related reasons:
>>    
>>      ,----
>>      | Note that making a group auto-expirable doesn't mean that all read
>>      | articles are expired--only the articles marked as expirable will be
>>      | expired
>>      `----
>>
>>     If auto-expire automatically marks articles expirable when you read
>>     them, doesn't that /necessarily/ mean all read articles are expired
>>     (except articles you read in the past?).
>
> The documentation somewhat struggles trying to explain that the
> auto-expire option doesn't do anything else than marking articles with
> `E' when they otherwise would be marked `R'.
>
> To say just that and not much more would probably suffice.

I think, if that's accurate, it would be *far* preferable.

> Thinking of this, what's the point behind gnus changing marks
> automatically?  When I mark something read, it'll later be marked `O'.

I happen to find that mark useful: I strive to maintain "Inbox Zero" and
`O' lets me know that an article has been sitting around in my Inbox a
bit too long.

> When marking something as killed, that doesn't seem to be any different
> from marking it as read (unless you use adaptive scoring maybe).  Gnus
> automatically marks it as `O' as well.

Yeah, that's odd.  `R', `r', and `K' all seem to mean roughly the same
thing.  And depending on how you have expiry set up for your group it's
quite possible that a number of other marks also mean roughly the same
thing.  I'm thinking of replacing the display of those marks in my
summary line with something more semantically useful.

> Why doesn't gnus keep the distinction between "read" and "killed"?  

Hmm... My guess is that under the covers, it does, and the mark just
gets hidden by `O' when an article is old enough.

> When I enter a group after having killed some of the threads, I still
> want to be able to see what threads I've killed, what I have actually
> read and what I didn't read but "cought up" with.  I don't trust the
> scoring because it might hide things from me I want to see, so seeing
> what I read, killed and cought up with is an important information to
> me.
>
> Can't gnus mark as `R' what I've read, as `K' what I've killed and
> as `O' what I cought up with and keep these marks intact?

Well, you can customize the summary line format almost infinitely and
use your own marks.  That's what I meant by "replacing the display of
those marks in my summary line with something more semantically useful."
However, I also think Gnus could/should have a more useful (and simpler)
default right out of the box.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: Gnus Questions #1: Article Expiry

[Lars Magne Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> * What's the difference between marking an article "expirable (E)" and
>   `marking it expired' as described by [[info:gnus#Spam and Ham
>   Processors]]?

I'm not that familiar with the spam processing thing, so I'm not sure.
I mean, there is no mark for "expired articles" (i.e. deleted articles)
as far as I know.  Just "expirable articles".

> * [[info:gnus#Expiring Mail]] and [[info:gnus#Group Parameters]] both
>   mention articles being `put through the expiry process,' but that
>   process is never spelled out.  What exactly is involved?

It's what's described on the "Expiring Mail" node?

> * I presume that "expiring" an article means the same as "putting the
>   article through the expiry process" (?)

Yes.

> * What's the difference between `gnus-summary-expire-articles' and
>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>   that clear.

The latter says:

"This means that *all* articles that are marked as expirable will be
deleted forever, right now."

> * What's the point of backend-specific expiry settings like
>   nnfolder-inhibit-expiry (I'm referring to the `nnfolder-' part when I
>   say `backend-specific')?  Don't we have enough other ways to say "this
>   group/server isn't expirable?"

It's sometimes more convenient to inhibit it on a server basis than on a
Gnus group basis.

> * [[info:gnus#Expiring Mail]] mentions `having auto expiry switched on'
>   but doesn't say how to do that.  Are we talking about the auto-expire
>   group parameter here, or something else?

That, and `gnus-auto-expirable-newsgroups'.  See the index entry for
`auto-expire'. 

> * "Total Expire" and "Auto Expire"
>
>   * The main point of using "Total Expire" instead of "Auto Expire"
>     seems to be that with "total expire" you can keep a distinction
>     between expirable (`E') and other marks that indicate an article was
>     read... until expiry actually runs.  At that point, if you're using
>     total expire they're all treated the same.  With "auto expire," on
>     the other hand, you know that only articles marked `E' will be
>     put through the expiry process.

Well, sort of.  With total expire, you expire all old articles.  If you
don't use total expire, the expiration process will only consider
"E"-marked articles.

You can "E"-mark them manually, or you can switch on auto-expire and
have Gnus set the "E" mark automatically.

>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>     scoring takes effect at expiry time, and "auto-expire" changes all
>     read marks to `E' too early for adaptive scoring to do its work.  Is
>     that right?

No.  If you set the "E" mark on all articles (whether automatically or
manually), Adaptive Scoring won't be able to tell whether you're read an
article or not, so it can't do its thing.

>   * [[info:gnus#Expiring Mail]] seems to contradict my understanding,
>     though: it claims that "auto-expire" gives me "more marks to work
>     with."
>
>     ,----
>     | Another advantage of auto-expire is that you get more marks to work
>     | with: for the articles that are supposed to stick around, you can
>     | still choose between tick and dormant and read marks.  But with
>     | total-expire, you only have dormant and ticked to choose from
>     `----
>
>     Okay, now that I read it again I think it's saying that with
>     "auto-expire," if I can somehow produce a mark other than `E' for an
>     article that's been read,, that article can persist even if it's
>     neither dormant or ticked.  That's fine as far as it goes but
>     mentioning it seems almost pointless, since Gnus is going to
>     automatically mark everything I read as `E'.  What am I missing?

Is it talking about adaptive scoring there?

>   * This (from [[info:gnus#Expiring Mail]]) seems confusing for what I
>     think are related reasons:
>
>      ,----
>      | Note that making a group auto-expirable doesn't mean that all read
>      | articles are expired--only the articles marked as expirable will be
>      | expired
>      `----
>
>     If auto-expire automatically marks articles expirable when you read
>     them, doesn't that /necessarily/ mean all read articles are expired
>     (except articles you read in the past?).

No.  You can remove the "E" mark manually.

>     Is there a common usage
>     model where people set auto-expire and then use some explicit
>     commands to change the read mark on some of their articles after
>     Gnus has already marked them `E'?

Yes.  I used to do that back before SpamAssassin.  I used auto-expire to
"E" mark everything (which was mostly spam), and then I manually "d"'d
the non-spam message.

>   * Aren't there a bajillion other ways to do the following, including
>     by customizing the "auto-expire" group parameter?  Why would I do it
>     as below (see [[info:gnus#Expiring Mail]]) instead?
>
>     ,----
>     | To avoid having articles marked as read marked as
>     | expirable automatically, you can put something like the following in
>     | your `~/.gnus.el' file:
>     | 
>     |      (remove-hook 'gnus-mark-article-hook
>     |                   'gnus-summary-mark-read-and-unread-as-read)
>     |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
>     | 
>     `----

Yes, that seems rather excessive.

> * How does the expire-age group parameter come into play?
>   - Does it prevent me from marking articles as expirable for a period?
>   - Does it prevent auto-expire from marking articles expirable?
>   - Does it simply exempt articles that are too young from expiry?

The latter.

> * Suggestion: Rename `gnus-auto-expirable-newsgroups'
>   `gnus-auto-expirable-groups' since, generally, auto-expire only
>   applies to mail and not to nntp.

Gnus has historically used "newsgroups" and "groups" as synonyms, and I
think that boat has sailed a long time ago.

> * As far as I can tell, the user experience of Agent expiry is
>   completely different from that of regular article expiry.

Yes.  :-)

>   This isn't
>   explained anywhere, but IIUC from experiments, agent expiry doesn't
>   delete any mail except from the agent's cache.  Instead it merely
>   flushes (some part of) the agent's cache.  Is that right?

Yes.

>   I and several of my friends have long been plagued by the symptom that
>   if I delete an IMAP message in some other mail client, it still hangs
>   around in Gnus.  I've beat my head against
>   `gnus-agent-regenerate[-group]' and `gnus-agent-flush-*' and other
>   trix for years trying to correct it, but never found a reliable
>   formula.  Agent expiry seems to be the key.  I think.  Have I got that
>   right?

There have been many reports about nnimap and the Agent not playing nice
with each other.  I have yet to delve into that morass.  But I will.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Gnus Questions #1: Article Expiry

[lee]

Dave Abrahams <dave@boostpro.com> writes:

> Lee, thanks a *lot* for jumping in.  I know this is a big one...

You're welcome :)  Perhaps we can contribute to improving the
documentation, I'd be happy to do that.

> on Mon Jul 04 2011, lee <lee-AT-yun.yagibdah.de> wrote:
>
>> Dave Abrahams <dave@boostpro.com> writes:
>>
>>> * I presume that "expiring" an article means the same as "putting the
>>>   article through the expiry process" (?)
>>
>> Well, when I mark an article `E', I'm "expiring" it.  
>
> I don't believe that interpretation consistent with the terminology as
> used in the majority of the document.  IIUC, marking an article `E'
> makes it "expirable" but the actual expiration happens later, if at all.

Yes, the difference between "marking articles as expirable" and
"expiring" them is an important difference the documentation makes.  The
chapter "Expiring Mail" goes like this:

+ how to get rid of unwanted mail, when are articles deleted
+ options to mark articles as expirable when reading
+ suggestions for when to use which option
+ group options about marking articles as expirable
+ adaptive scoring interfering with marking options
+ options specifying how long it takes before expirable articles are deleted
+ alternatives to deleting articles
+ some additional side notes

It comes down to that articles marked as expirable will eventually be
deleted at some time.  How the expiry process actually works isn't
explained.  It's all about marking articles.

What I learned first from this chapter was that with default settings,
my email won't ever be deleted unless I mark it with `E', which was all
I wanted to know at that time.  It turned out to be probably not true
because when the active file is messed up for some reason, gnus will
overwrite existing articles with new ones because they get the same
number.  If that's true, I wish it would be changed.

Anyway, from the point of view of the user who's reading the
documentation, there isn't a relevant difference between marking an
article `E' and expiring it.  So you could say that the documentation,
though it's good, is missing the point and gets into its own way:

It is missing the point because the user isn't interested in a
neglectable difference between "expiring" and "expiring".  It gets into
its own way in that it's supposedly explaining "expiring", but instead
it tries to explain "expiring" and fails.  ---  I hope you see what I'm
trying to say, it's difficult to express :/  The documentation is like
reading three different threads in a newsgroup while replying to a
fourth one all at the same time.  In the end, everyone is confused and
they can't even tell why.

The way out may be to split the chapter about "Expiring Mail" into
several sections, narrow each section down as much as possible to its
specific topic and to begin the whole chapter with telling the users
that their email won't be deleted unless they make it so.  Perhaps an
explanation of the terminology and checking the terminology might be
helpful, too.  I would also add an overview of the available marks and
the key combinations for setting them so that "Expiring Mail" can refer
to that.  Perhaps we can even come up with a drawing to give an
overview.

>> It'll be purged some time later --- apparently only *if* I enter and
>> leave the group the article is in later.  
>
> There are other ways to trigger the expiry process, e.g. `M-g'
> (gnus-summary-rescan-group)

Is that mentioned in "Expiring Mail"? I didn't know that ...

> and expiry doesn't necessarily mean the article will be deleted (or
> "purged").  For example, the article could be refiled.  Or nothing at
> all could happen. It all depends how you set things up, as far as I
> can tell.

Yeah --- we could use some sort of flowchart to show what happens under
which conditions.

> It would be nice to have a detailed description of exactly what options
> and user hooks are checked/called by the system in what order.

Ha!  That's what I mean when I'm saying that "Expiring Mail" doesn't
explain "expiring mail" :)

>> If I don't, the article will never be purged: is that true?
>
> The article can be deleted in other ways, e.g. `B DEL'
> (gnus-summary-delete-article).

Oh, what I was asking is what happens when I don't enter and leave a
group that has "expirable" articles in it: Are they ever going to be
purged?  The question is: "When does the "expiry process" happen?"  Is
gnus going through all groups in irregular intervals to purge
"expirable" articles automatically?  Is gnus purging articles when I
enter or leave groups?  Is gnus purging articles only when I somehow
tell it to?  My thinking that articles are purged when leaving a group
only comes from seeing the message "Expiring articles ..." in the
minibuffer when leaving a group.

>>> * What's the difference between `gnus-summary-expire-articles' and
>>>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>>>   that clear.
>>
>> The difference is in "delete all expirable articles in the group /that
>> have been around for a while/" vs. "This means that *all* articles
>> eligible for expiry in the current group will disappear" /right now/.
>
> Yeah, I read all those words, but they didn't add up to any obvious
> distinction for me. The phrases "have been around for a while" and
> "eligible for expiry" are vague at best.  By contrast the use of
> "delete" and "disappear" seem implausibly concrete for a system whose
> expiration behavior can range from "do nothing" (which is undocumented,
> but that's what you get from nnimap if the expiry target is nil) to
> "forward the article to my mother-in-law" (if I write the function to do
> that).  I would rather say that the articles are "sent to the group's
> expiry target" (a phrase I'd suitably define in Expiring Mail::).

That's all because the chapter "Expiring Mail" is missing the point.

>> ,----
>> | `B e'
>> |      Perform an expiry process on the articles in the current
>> |      group. Only articles that are expirable *and due to be purged*
>> |      will be purged.  The expiry process is performed by the function
>> |      `gnus-summary-expire-articles'.  (See Expiring Mail::).
>> | 
>> | `B C-M-e'
>> |      Purge all the expirable articles in the current group *right
>> |      now*. This means that all expirable articles will be purged
>> |      immediately.  The purging process is performed by the function
>> |      `gnus-summary-expire-articles-now'.  (See Expiring Mail::).
>> `----
>
> Unfortunately, that change doesn't improve the clarity for me.
> It doubles down on the overly-concrete aspects of the existing text.  It
> also makes it sound like `B e' isn't directly bound to
> `gnus-summary-expire-articles', but instead invokes it indirectly as
> /part/ of its functionality.
>
> If the following changes are accurate, I prefer them:
>
> `B e'  - `gnus-summary-expire-articles'
>      Run all articles in the current group that are marked expirable
>      (`E') through the expiry process (*note Expiring Mail::).
>
> `B C-M-e' - `gnus-summary-expire-articles-now'
>      Send all articles in the current group marked expirable (`E')
>      directly to their expiry target regardless of their age
>      (*note Expiring Mail::).

Hm.  I see the advantage of pointing out that the functions are bound to
the keys.  There are other chapters in the documentation doing it
likewise.

Do you know for sure that these functions run on all articles?  What if
you have defined a region?  I've found that you can sometimes do things
on a region and sometimes not, like, IIRC, mark read and set the process
mark, respectively.

More importantly, there are circumstances in which articles /not/ marked
`E' will be expired.  So:

`B e'
     Expire articles (`gnus-summary-expire-articles')

     The function (`gnus-summary-expire-articles') sends expirable
     articles to their expiry targets only after the articles have been
     expirable for some time.  The amount of time depends on settings.
     (See Expiring Mail::).

`B C-M-e'
     Expire articles at once (`gnus-summary-expire-articles-now')

     The function (`gnus-summary-expire-articles') sends expirable
     articles to their expiry targets regardless of how long the
     articles have been expirable.  (*note Expiring Mail::).

There's also:

"Group Maintenance"
`C-c C-x'
`C-c C-M-x'
"Topic Commands"
`C-c C-x'

>> How about renaming "gnus-summary-expire-articles-now" to
>> "gnus-summary-purge-articles" (or to "gnus-purge-articles")?
>>
>> There also needs to be some clarification about what "expiring" means.
>
> +1

Since one possible meaning of "expiring" is "sending articles to expiry
targets", renaming the function to "purge" won't be such a good idea ...

>> There seem to be actually quite a few stages of expirableness:
>
> Using an informal definition of `expirable' ("when expiry runs, this
> article will be sent to the expiry target"), I agree.  I think the docs
> are pretty strict about using `expirable' only to refer to articles
> marked `E', though.  Now, we could discuss the merits of changing that,
> but it's the status quo.

That the documentation is formally strict with using "expire" and
"expirable" without explaining what they mean and by assigning them
different meanings makes it not easy to understand.  It's understandable
why the documentation is like that: It follows the logic of the
software.  Following that logic, to "expire" an article does have
different meanings (or results), depending on settings.

Is it really strict in referring only to articles marked `E' as
expirable ones?  It also knows the possibility that articles with other
marks can be expirable.

>> * not expirable, depends on gnus-inhibit-user-auto-expire,
>
> Based on my reading of the docs, I don't think it depends on that.
> `gnus-inhibit-user-auto-expire' seems to prevent
> `gnus-summary-mark-as-expirable' (normally bound to `E') from having any
> effect:

It depends on what "expiring" means: When the user marks an article as
expirable, he's expiring it.  A setting that prevents the user from
marking articles as expirable has an effect on the expirableness of
articles.

> ,----
> | *If non-nil, user marking commands will not mark an article as
> | expirable, even if the group has auto-expire turned on.
> `----
>
> AFAICT an article that already has an `E' mark is still expirable in
> both the formal and informal sense, though.
>
> [I don't know what other "user marking commands" this text might be
> referring to.  Are there any?]

That's arguable: When auto-expire is turned on, marking an article `R'
makes it expirable.  Scoring can also make articles expirable then.

Now turn on gnus-inhibit-user-auto-expire and have the user change the
scoring rules in such a way that articles become expirable.  Then you
can say that articles have been marked as expirable because of a user
command.

Does gnus-inhibit-user-auto-expire prevent users from changing scoring
rules in such a way that articles become expirable?  When auto-expire is
turned on and you read articles, are they marked are `R' and become
expirable by reading them despite gnus-inhibit-user-auto-expire is
turned on?

>> * marked as "and so on"[1], apparently depends total-expire only
>
> Yeah, that's disturbingly vague.
>
>> * marked as `G': depends on gnus-inhibit-user-auto-expire,
>
> Again, unless the docs are just wrong or extremely misleading, I don't
> think so.

It would be something to try out :)

>> Then there's the purgability of articles.  It seems that articles marked
>> `G' are always purged on leaving a group.  
>> Perhaps they cannot be purged
>> eventually, depending on some settings (and marks, maybe):
>
> My (weak) understanding is that `G' (cancelled) only applies to nntp
> groups and what happens to cancelled articles depends on whether the
> nntp server responds to cancellation.

Hm, I thought `G' means "deleted".  Just enter a mail group and move an
article into another group, it'll be marked `G'.

>> What disables the purgability of an article?
>
> If "purge" means "delete," then as far as I can tell nothing disables it
> other than, potentially, the server itself (e.g. if something like
> dovecot is serving you a read-only IMAP group or an nntp server
> disallows article cancellation)

Let's say "purging" means "sending articles to their expiry targets".
Settings like nnfolder-inhibit-expiry seem to be supposed to prevent
purging.

>> [1] {
>>       ,---- [ info:gnus#Expiring Mail ]
>>       | So, in addition to the articles marked `E', also the articles
>>       | marked `r', `R', `O', `K', `Y' and so on are considered
>>       | expirable.
>>       `----
>>
>>       What other marks are there?  
>
> Lots.  My most complete list of marks to date is below, but I wouldn't
> be surprised to learn that I missed something.:

Wow, that's an interesting list :)  I'll keep it for reference.

> "R"
> "K"
> "-"
> " " (gnus-unread-mark)
> "r" (explicit delete)
> "X" (explicit kill)"

What's the difference between "explicit delete" and "explicit kill"?

> "Y" (low score)
> "=" (gnus-unsendable-mark)
> "%" (gnus-downloadable-mark")
> "!" (gnus-ticked-mark)
> "$" (gnus-spam-mark)
> "?" (gnus-dormant-mark)
> "E" (gnus-expirable-mark)
> "O" (gnus-ancient-mark) Articles that were marked as read in previous sessions and are now "old"

... and if you have total-expire turned on, the articles marked `O' (and
 some others) are expirable, too.  Does gnus display it that way?

How about a flag for articles that gnus considers as "expirable" under
current conditions?  With that, you might see articles flagged as "EP"
or "YP" or "OP" --- or if two letters are too many, there could be the
single-lettered `P' flag for articles that can be purged.  This would
make it more clear to the user that an article will be purged, which is
something they cannot figure out easily because there are so many global
and group specific settings involved.

Hm, where do I send feature requests?

> "A" (gnus-replied-mark) All articles that you have replied to or made a followup to
> "*" Article is "[[info:gnus#Persistent%20Articles][persistent]]"
> `C' Marked as read by a catchup (`gnus-catchup-mark').

Hm, when do you ever see that?

> `G' Canceled article (`gnus-canceled-mark')
> `Q' Sparsely reffed article (`gnus-sparse-mark').  

What does this one mean?

> `M' Article marked as read by duplicate suppression (`gnus-duplicate-mark'). 

>>       Can I prevent articles from being marked as `O' automatically?
>
> Good question
>
>> What means `r`?  
>
> The documentation (somewhere) says it means the article was "explicitly
> deleted," but near as I can tell, it means you invoked
> gnus-summary-mark-as-read-forward (`d') on the article.  Aside from that
> and the effects of potentially having different membership in
> `gnus-auto-expirable-marks', it's equivalent to `R'.  I think.

Then it wouldn't mean "explicit delete" as in your list of marks.

>>       Can I configure
>>       which marks put articles into stages of expirableness? 
>
> See gnus-auto-expirable-marks.  Not sure if that's what you have in
> mind, though.

Hm, `gnus-auto-expirable-marks' isn't in my documentation.  It would be
better if I was using the latest version, I guess.  So far, I've
downloaded emacs 24 from bzr and compiled it; the only thing that seems
to be missing is bbdb.  Isnt' bbdb part of emacs?  And how would I
install it for emacs 24 (on Debian testing, without interfering with the
version installed by the package management)?

>
>>       There needs
>>       to be an overview of the possible marks in the chapter about
>>       expiring mail, or somewhere.
>> }
>
> Indeed: http://debbugs.gnu.org/db/89/8978.html

Hm, I really need to switch to emacs 24 ...

>>> * What's the point of backend-specific expiry settings like
>>>   nnfolder-inhibit-expiry (I'm referring to the `nnfolder-' part when I
>>>   say `backend-specific')?  Don't we have enough other ways to say "this
>>>   group/server isn't expirable?"
>>
>> Hm, nnfolder-inhibit-expiry doesn't seem to be explained in the
>> documentation.  What does it inhibit, the purgability or the
>> expirableness?
>
> My guess from the name is that it prevents the expiry process from
> running on that group.

That would be something to add to the documentation.  How do we do that?

>> Which other ways to say that a server/group isn't expirable (or
>> purgeable, which is a difference) have you found?
>
> I think, among others:
>
>   setting nnmail-expiry-wait to `never', and if undocumented behaviors
>   count, setting nnmail-expiry-target to nil.
>
> and similar settings in the individual group parameters.

Hm, this too calls for the 'P` flag ...

>> Is there a way to request information about the expirableness and
>> purgability of articles? You'd get an information like "This article is
> [...]
>
> That is an *excellent* idea for a new feature.  I suggest `M-x gnus-bug
> RET [feature-request]...'

Cool, I've sent one.

>>> * "Total Expire" and "Auto Expire"
>>>
>>>   * The main point of using "Total Expire" instead of "Auto Expire"
>>>     seems to be that with "total expire" you can keep a distinction
>>>     between expirable (`E') and other marks that indicate an article was
>>>     read... until expiry actually runs.  At that point, if you're using
>>>     total expire they're all treated the same.  With "auto expire," on
>>>     the other hand, you know that only articles marked `E' will be
>>>     put through the expiry process.
>>
>> Yes: Total-expire makes articles purgeable *without* marking
>> them. Auto-expire makes articles purgeable *by* marking them.
>
> I obviously don't understand your definition of "purgeable."  I used to
> think it meant "able to be deleted," but now I think it's something else.

Well, at first it meant to delete an article, either immediately or
after some time.  Since articles are not necessarily deleted when they
are "expired" (gah!), I'll put it your way and say that "purging" an
article means "sending an article to its expiry target".  The "expiry
target" can be whatever, like deleting the article, moving it into
another group, mailing it to someone, saving it as a file somewhere ...
Please note that when an article is "expirable" and purging it would not
do anything, the article is not purgeable.  That's an important
distinction.

When you start contemplating that an "expiry target" could be called a
"purge target" as well, you could say that purging an article means to
expire it and vice versa.  You could also say that expiring an article
means to send it to its expiry target.  With that, we're back to
confusion.  Let's see ...:

"Purge" basically seems to refer to "removing something from
something"[1].  "Expire" has various meanings; I'd conclude that it
mainly has to do with "getting rid of or emitting something without the
possibility of undoing the deed"[2].

That isn't very helpful, either, other than that I would think that
/purge/ is a word better suited than /expire/ in this case because
"expiring" an article means to remove it from a group it's in, while it
doesn't necessarily mean to get rid of the article without the
possibility of getting it back.

Perhaps we're still confused because we haven't figured out yet what
"expiring" means.


[1]: http://www.websters-online-dictionary.org/definitions/purge?cx=partner-pub-0939450753529744%3Av0qd01-tdlq&cof=FORID%3A9&ie=UTF-8&q=purge&sa=Search#922
[2]:
http://www.websters-online-dictionary.org/definitions/expire?cx=partner-pub-0939450753529744%3Av0qd01-tdlq&cof=FORID%3A9&ie=UTF-8&q=expire&sa=Search#922


> I think I disagree with your use of "*without* marking," as IIUC an
> article must acquire one of the "read marks" in order to be expired.

Yes, I'm wrong --- I was a little confused about the `O' marks because I
thought they would be set automatically by gnus in the same way as mutt
does, and that isn't the case.

However, since total-expire considers articles with so many marks as
expirable and since gnus removes all distinctions like between read,
cought up with, killed and scored down, what's there to distinguish
between?  All the articles *not* marked unread are "expirable".

So isn't it the other way round: With auto-expire you have more marks
left to distinguish between articles than you have with total-expire
because total-expire basically reduces you to unread articles while,
with auto-expire, you can use all marks other than `E' without making
articles expirable.

What would be the point in distinguishing between articles that will all
be purged anyway?

> I don't know. I don't think it's particularly relevant to my
> question or the topic of expiry (which I'm trying to handle in this
> thread) so if you don't mind I'd prefer not to discuss that here, just
> to keep this from spiralling way beyond everyone's TL;DR limit.

That's fine with me :)  It doesn't seem to matter to the topic which of
the scoring methods in particular might make articles "expirable".

>> Auto-expire does *not* lead to eventually purging articles you don't
>> touch, while total-expire *does* lead to eventually purging articles you
>> don't touch (because gnus touches them with scoring and `O' marks, for
>> example).
>
> Well, define "touch."  I think you mean "read."

Yes, I mean "read" or "expire" (i. e. mark `E'), in a distinction to
articles marked automatically (by scoring rules).  Since gnus turns
"killed" and "cought up with" into "read" --- and "read" into "ancient"
(`O'), there isn't much distinction left for the user to work with.  (I
don't know what it does with articles marked `Y'; does it mark them `O'
without the user seeing that?)

I have a misconception about the `O' marks because I have a hard time
getting into my head that gnus sets those only for articles you
"touched" and not by itself.  It's important because it means that gnus
never sets any marks unless the user tells it to.  ---  Mutt, by
default, sets `O' marks on all articles you don't "touch" when leaving a
group, and I had always turned that off, not seeing any `O' marks at all
in mutt.  Now with gnus, the `O' marks are back and I get confused about
them and wish I could turn them off.

>> The documentation somewhat struggles trying to explain that the
>> auto-expire option doesn't do anything else than marking articles with
>> `E' when they otherwise would be marked `R'.
>>
>> To say just that and not much more would probably suffice.
>
> I think, if that's accurate, it would be *far* preferable.

It does say so:

,---- [ info (gnus) Expiring Mail ]
| In a nutshell, "auto-expire" means that Gnus hits `E' for
| you when you select an article.  And "total-expire" means that Gnus
| considers all articles as expirable that are read.
`----

>> Thinking of this, what's the point behind gnus changing marks
>> automatically?  When I mark something read, it'll later be marked `O'.
>
> I happen to find that mark useful: I strive to maintain "Inbox Zero" and
> `O' lets me know that an article has been sitting around in my Inbox a
> bit too long.

That's what the "recent" mark (`.') seems to be for.

>> When marking something as killed, that doesn't seem to be any different
>> from marking it as read (unless you use adaptive scoring maybe).  Gnus
>> automatically marks it as `O' as well.
>
> Yeah, that's odd.  `R', `r', and `K' all seem to mean roughly the same
> thing.  And depending on how you have expiry set up for your group it's
> quite possible that a number of other marks also mean roughly the same
> thing.  I'm thinking of replacing the display of those marks in my
> summary line with something more semantically useful.

Yeah we need a "purge" mark ...

Unless you can stop gnus from turning these marks to `O', `O' would seem
to be the only useful mark for this.  And if you can stop gnus from
turning to `O', you can as well use the actual marks.

>> Why doesn't gnus keep the distinction between "read" and "killed"?  
>
> Hmm... My guess is that under the covers, it does, and the mark just
> gets hidden by `O' when an article is old enough.

Hm.  That would mean we might be able to get these marks displayed ...
That's something to check out more closely :)

>> Can't gnus mark as `R' what I've read, as `K' what I've killed and
>> as `O' what I cought up with and keep these marks intact?
>
> Well, you can customize the summary line format almost infinitely and
> use your own marks.  That's what I meant by "replacing the display of
> those marks in my summary line with something more semantically useful."
> However, I also think Gnus could/should have a more useful (and simpler)
> default right out of the box.

Yeah, that would help ...  I need to find out how to write a function
that can display these marks, if they are preserved, and modify
gnus-summary-line-format accordingly ...

Re: Gnus Questions #1: Article Expiry

[lee]

lee <lee@yun.yagibdah.de> writes:

>> Well, you can customize the summary line format almost infinitely and
>> use your own marks.  That's what I meant by "replacing the display of
>> those marks in my summary line with something more semantically
>> useful."

Ah, it goes like this:


(defun gnus-user-format-function-M (header)
  (if (equal gnus-tmp-unread gnus-ancient-mark) ; then
      (make-string 1 ?a) ; else
    (make-string 1 ?-) )  )


... which uses the %u specifier of gnus-summary-line-format.

This is probably a very stupid way of making a function return a string;
I just don't know better yet ...  At least it shows that the marks are
available and how to make them visible :)

Re: Gnus Questions #1: Article Expiry

[Dave Abrahams]

Hi Lars,

Let me say in advance, thanks a lot for your answers.  Where I have
elided text you wrote below, please assume my response is, "got it,
thanks."

on Tue Jul 05 2011, Lars Magne Ingebrigtsen <larsi-AT-gnus.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> * What's the difference between marking an article "expirable (E)" and
>>   `marking it expired' as described by [[info:gnus#Spam and Ham
>>   Processors]]?
>
> I'm not that familiar with the spam processing thing, so I'm not sure.
> I mean, there is no mark for "expired articles" (i.e. deleted articles)
> as far as I know.  Just "expirable articles".

I agree and am inclined to think we need to s/expired/expirable/ in the
referenced info node.  I'll put that in my patch.

>> * [[info:gnus#Expiring Mail]] and [[info:gnus#Group Parameters]] both
>>   mention articles being `put through the expiry process,' but that
>>   process is never spelled out.  What exactly is involved?
>
> It's what's described on the "Expiring Mail" node?

Sorry to be a pain, but that doesn't look like a description of a
process to me.  It looks like a description of a bunch of settings and
the effects they have, but there's nothing like "first this happens,
then we do that, ..."


>> * What's the difference between `gnus-summary-expire-articles' and
>>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>>   that clear.
>
> The latter says:
>
> "This means that *all* articles that are marked as expirable will be
> deleted forever, right now."

Of course I've read that line over and over and it isn't getting
clearer.  As I wrote to Lee,

,----
| Yeah, I read all those words, but they didn't add up to any obvious
| distinction for me. The phrases "have been around for a while" and
| "eligible for expiry" are vague at best.  By contrast the use of
| "delete" and "disappear" seem implausibly concrete for a system whose
| expiration behavior can range from "do nothing" (which is undocumented,
| but that's what you get from nnimap if the expiry target is nil) to
| "forward the article to my mother-in-law" (if I write the function to do
| that).  I would rather say that the articles are "sent to the group's
| expiry target" (a phrase I'd suitably define in Expiring Mail::).
`----

>> * "Total Expire" and "Auto Expire"
>>
>>   * The main point of using "Total Expire" instead of "Auto Expire"
>>     seems to be that with "total expire" you can keep a distinction
>>     between expirable (`E') and other marks that indicate an article was
>>     read... until expiry actually runs.  At that point, if you're using
>>     total expire they're all treated the same.  With "auto expire," on
>>     the other hand, you know that only articles marked `E' will be
>>     put through the expiry process.
>
> Well, sort of.  With total expire, you expire all old articles.  

Do you really mean "old?" The doc seems to say you expire all articles
with a read mark.

> If you don't use total expire, the expiration process will only
> consider "E"-marked articles.
>
> You can "E"-mark them manually, or you can switch on auto-expire and
> have Gnus set the "E" mark automatically.

OK.

>>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>>     scoring takes effect at expiry time, and "auto-expire" changes all
>>     read marks to `E' too early for adaptive scoring to do its work.  Is
>>     that right?
>
> No.  If you set the "E" mark on all articles (whether automatically or
> manually), Adaptive Scoring won't be able to tell whether you're read an
> article or not, so it can't do its thing.

I think you mean "Yes" above.  Otherwise, it's not consistent with the
next sentence, which seems to confirm that my understanding was spot-on
:-).  Am I missing something?

>>   * [[info:gnus#Expiring Mail]] seems to contradict my understanding,
>>     though: it claims that "auto-expire" gives me "more marks to work
>>     with."
>>
>>     ,----
>>     | Another advantage of auto-expire is that you get more marks to work
>>     | with: for the articles that are supposed to stick around, you can
>>     | still choose between tick and dormant and read marks.  But with
>>     | total-expire, you only have dormant and ticked to choose from
>>     `----
>>
>>     Okay, now that I read it again I think it's saying that with
>>     "auto-expire," if I can somehow produce a mark other than `E' for an
>>     article that's been read,, that article can persist even if it's
>>     neither dormant or ticked.  That's fine as far as it goes but
>>     mentioning it seems almost pointless, since Gnus is going to
>>     automatically mark everything I read as `E'.  What am I missing?
>
> Is it talking about adaptive scoring there?

I don't think so; at that point in the text it hasn't raised adaptive
scoring yet.  It's in the paragraph that begins, "Which one is better,
auto-expire or total-expire?"

>>   * Aren't there a bajillion other ways to do the following, including
>>     by customizing the "auto-expire" group parameter?  Why would I do it
>>     as below (see [[info:gnus#Expiring Mail]]) instead?
>>
>>     ,----
>>     | To avoid having articles marked as read marked as
>>     | expirable automatically, you can put something like the following in
>>     | your `~/.gnus.el' file:
>>     | 
>>     |      (remove-hook 'gnus-mark-article-hook
>>     |                   'gnus-summary-mark-read-and-unread-as-read)
>>     |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
>>     | 
>>     `----
>
> Yes, that seems rather excessive.

Any objections if I remove that passage in my patch?

>> * Suggestion: Rename `gnus-auto-expirable-newsgroups'
>>   `gnus-auto-expirable-groups' since, generally, auto-expire only
>>   applies to mail and not to nntp.
>
> Gnus has historically used "newsgroups" and "groups" as synonyms, and I
> think that boat has sailed a long time ago.

Yeah, but Emacs supports aliases.  Is there any reason not to make
things clearer for future generations?

>>   I and several of my friends have long been plagued by the symptom that
>>   if I delete an IMAP message in some other mail client, it still hangs
>>   around in Gnus.  I've beat my head against
>>   `gnus-agent-regenerate[-group]' and `gnus-agent-flush-*' and other
>>   trix for years trying to correct it, but never found a reliable
>>   formula.  Agent expiry seems to be the key.  I think.  Have I got that
>>   right?
>
> There have been many reports about nnimap and the Agent not playing
> nice with each other.  I have yet to delve into that morass.  But I
> will.  :-)

We, the morassed, are very grateful. :-)

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: Gnus Questions #1: Article Expiry

[Dave Abrahams]

on Tue Jul 05 2011, lee <lee-AT-yun.yagibdah.de> wrote:

> lee <lee@yun.yagibdah.de> writes:
>
>>> Well, you can customize the summary line format almost infinitely and
>>> use your own marks.  That's what I meant by "replacing the display of
>>> those marks in my summary line with something more semantically
>>> useful."
>
> Ah, it goes like this:
>
>
> (defun gnus-user-format-function-M (header)
>   (if (equal gnus-tmp-unread gnus-ancient-mark) ; then
>       (make-string 1 ?a) ; else
>     (make-string 1 ?-) )  )
>
>
> ... which uses the %u specifier of gnus-summary-line-format.
>
> This is probably a very stupid way of making a function return a string;
> I just don't know better yet ...  

I think this is easier ;-)

 (defun gnus-user-format-function-M (header)
   (if (equal gnus-tmp-unread gnus-ancient-mark) "a" "-"))

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: Gnus Questions #1: Article Expiry

[lee]

Dave Abrahams <dave@boostpro.com> writes:

> on Tue Jul 05 2011, lee <lee-AT-yun.yagibdah.de> wrote:
>>
>> Ah, it goes like this:
>>
>>
>> (defun gnus-user-format-function-M (header)
>>   (if (equal gnus-tmp-unread gnus-ancient-mark) ; then
>>       (make-string 1 ?a) ; else
>>     (make-string 1 ?-) )  )
>>
>>
>> ... which uses the %u specifier of gnus-summary-line-format.
>>
>> This is probably a very stupid way of making a function return a string;
>> I just don't know better yet ...  
>
> I think this is easier ;-)
>
>  (defun gnus-user-format-function-M (header)
>    (if (equal gnus-tmp-unread gnus-ancient-mark) "a" "-"))

Hm, interesting, it's something I tried first and didn't seem to work.
Now the problem seems to be that I don't know how to get information
about the marks ...

Re: Gnus Questions #1: Article Expiry

[lee]

Dave Abrahams <dave@boostpro.com> writes:

>> I mean, there is no mark for "expired articles" (i.e. deleted
>> articles) as far as I know.  Just "expirable articles".
>
> I agree and am inclined to think we need to s/expired/expirable/ in
> the referenced info node.  I'll put that in my patch.

,---- [ info (gnus) Read Articles ]
| `r'
|      These are articles that the user has marked as read with the `d'
|      command manually, more or less (`gnus-del-mark').
`----

,---- [ info (gnus) Setting Marks ]
| `M d'
| `d'
|      Mark the current article as read
|      (`gnus-summary-mark-as-read-forward').
| 
| `D'
|      Mark the current article as read and move point to the previous
|      line (`gnus-summary-mark-as-read-backward').
`----

The `gnus-del-mark' still exists in gnus-score.el, gnus.el and
gnus-sum.el.

> [...]
>>>     ,----
>>>     | To avoid having articles marked as read marked as
>>>     | expirable automatically, you can put something like the following in
>>>     | your `~/.gnus.el' file:
>>>     | 
>>>     |      (remove-hook 'gnus-mark-article-hook
>>>     |                   'gnus-summary-mark-read-and-unread-as-read)
>>>     |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
>>>     | 
>>>     `----
>>
>> Yes, that seems rather excessive.
>
> Any objections if I remove that passage in my patch?

Yes :)  It's nice to have examples like this in the documentation.  How
else are users supposed to figure out what they could do?  See also the
chapter "3.3.2 Choosing Variables" (which also mentions `gnus-del-mark',
btw.).

Re: Gnus Questions #1: Article Expiry

[Dan Christensen]

Dave Abrahams <dave@boostpro.com> writes:

>>> * What's the difference between `gnus-summary-expire-articles' and
>>>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>>>   that clear.
>>
>> The latter says:
>>
>> "This means that *all* articles that are marked as expirable will be
>> deleted forever, right now."
>
> Of course I've read that line over and over and it isn't getting
> clearer.  

As far as I understand, the only difference between the two functions
is that the "now" version ignores the expiry-wait setting (i.e. it
assumes it is 0).

>> Well, sort of.  With total expire, you expire all old articles.  
>
> Do you really mean "old?" The doc seems to say you expire all articles
> with a read mark.

You are right, Lars was being brief.

>>>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>>>     scoring takes effect at expiry time, and "auto-expire" changes all
>>>     read marks to `E' too early for adaptive scoring to do its work.  Is
>>>     that right?
>>
>> No.  If you set the "E" mark on all articles (whether automatically or
>> manually), Adaptive Scoring won't be able to tell whether you're read an
>> article or not, so it can't do its thing.
>
> I think you mean "Yes" above.  Otherwise, it's not consistent with the
> next sentence, which seems to confirm that my understanding was spot-on
> :-).  Am I missing something?

I think he meant "No".  It doesn't happen at expiry time, but when it
happens, it depends on how articles are marked.  

>>>   * [[info:gnus#Expiring Mail]] seems to contradict my understanding,
>>>     though: it claims that "auto-expire" gives me "more marks to work
>>>     with."
>>>
>>>     ,----
>>>     | Another advantage of auto-expire is that you get more marks to work
>>>     | with: for the articles that are supposed to stick around, you can
>>>     | still choose between tick and dormant and read marks.  But with
>>>     | total-expire, you only have dormant and ticked to choose from
>>>     `----
>>>
>>>     Okay, now that I read it again I think it's saying that with
>>>     "auto-expire," if I can somehow produce a mark other than `E' for an
>>>     article that's been read,, that article can persist even if it's
>>>     neither dormant or ticked.  That's fine as far as it goes but
>>>     mentioning it seems almost pointless, since Gnus is going to
>>>     automatically mark everything I read as `E'.  What am I missing?
>>
>> Is it talking about adaptive scoring there?
>
> I don't think so; at that point in the text it hasn't raised adaptive
> scoring yet.  It's in the paragraph that begins, "Which one is better,
> auto-expire or total-expire?"

It's definitely talking about auto-expire/total-expire, and that
paragraph is the reason I use auto-expire.  It lets me use the
read mark on an article without it being considered for deletion.

>>>   * Aren't there a bajillion other ways to do the following, including
>>>     by customizing the "auto-expire" group parameter?  Why would I do it
>>>     as below (see [[info:gnus#Expiring Mail]]) instead?
>>>
>>>     ,----
>>>     | To avoid having articles marked as read marked as
>>>     | expirable automatically, you can put something like the following in
>>>     | your `~/.gnus.el' file:
>>>     | 
>>>     |      (remove-hook 'gnus-mark-article-hook
>>>     |                   'gnus-summary-mark-read-and-unread-as-read)
>>>     |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
>>>     | 
>>>     `----
>>
>> Yes, that seems rather excessive.
>
> Any objections if I remove that passage in my patch?

I use the above settings and don't know any other way to accomplish it.
If you change auto-expire to nil, then an article will never
automatically get the expire mark when read.  The above changes mean
that when you read an unread message, it gets the expirable mark, but
when you read an already read message, it doesn't get marked expirable.
This seems like it should be the default:  if I have changed the 
expirable mark to a plain read mark once, it's because I don't want
the article to expire, so why would I want it to get marked expirable
when I re-read the article?  Data loss ensues...

Maybe it will help to clear your confusion about the last two issues to
emphasize that an article can be read or expirable, and it's easy to
switch an article between the two marks.

Dan

Re: Gnus Questions #1: Article Expiry

[Lars Magne Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

>>> * [[info:gnus#Expiring Mail]] and [[info:gnus#Group Parameters]] both
>>>   mention articles being `put through the expiry process,' but that
>>>   process is never spelled out.  What exactly is involved?
>>
>> It's what's described on the "Expiring Mail" node?
>
> Sorry to be a pain, but that doesn't look like a description of a
> process to me.  It looks like a description of a bunch of settings and
> the effects they have, but there's nothing like "first this happens,
> then we do that, ..."

Well, it explains about expirable messages, and then it explains all the
variables that affect the process...

>>> * What's the difference between `gnus-summary-expire-articles' and
>>>   `gnus-summary-expire-articles-now'?  The documentation doesn't make
>>>   that clear.
>>
>> The latter says:
>>
>> "This means that *all* articles that are marked as expirable will be
>> deleted forever, right now."
>
> Of course I've read that line over and over and it isn't getting
> clearer.  As I wrote to Lee,
>
> ,----
> | Yeah, I read all those words, but they didn't add up to any obvious
> | distinction for me. The phrases "have been around for a while" and
> | "eligible for expiry" are vague at best.  By contrast the use of
> | "delete" and "disappear" seem implausibly concrete for a system whose
> | expiration behavior can range from "do nothing" (which is undocumented,
> | but that's what you get from nnimap if the expiry target is nil) to
> | "forward the article to my mother-in-law" (if I write the function to do
> | that).  I would rather say that the articles are "sent to the group's
> | expiry target" (a phrase I'd suitably define in Expiring Mail::).
> `----

"sent to the group's expiry target" is more correct, yes.  If you've
changed the expiry defaults.  The normal "target" is "delete".

>>> * "Total Expire" and "Auto Expire"
>>>
>>>   * The main point of using "Total Expire" instead of "Auto Expire"
>>>     seems to be that with "total expire" you can keep a distinction
>>>     between expirable (`E') and other marks that indicate an article was
>>>     read... until expiry actually runs.  At that point, if you're using
>>>     total expire they're all treated the same.  With "auto expire," on
>>>     the other hand, you know that only articles marked `E' will be
>>>     put through the expiry process.
>>
>> Well, sort of.  With total expire, you expire all old articles.  
>
> Do you really mean "old?" The doc seems to say you expire all articles
> with a read mark.

All old, read (sort of) articles.

>>>   * From [[info:gnus#Adaptive Scoring]] I think I conclude that adaptive
>>>     scoring takes effect at expiry time, and "auto-expire" changes all
>>>     read marks to `E' too early for adaptive scoring to do its work.  Is
>>>     that right?
>>
>> No.  If you set the "E" mark on all articles (whether automatically or
>> manually), Adaptive Scoring won't be able to tell whether you're read an
>> article or not, so it can't do its thing.
>
> I think you mean "Yes" above.  Otherwise, it's not consistent with the
> next sentence, which seems to confirm that my understanding was spot-on
> :-).  Am I missing something?

No means no, mister!!!

:-)

Adaptive scoring does not take effect at expiry time.  But the reason is
what you say, otherwise.

>>>   * Aren't there a bajillion other ways to do the following, including
>>>     by customizing the "auto-expire" group parameter?  Why would I do it
>>>     as below (see [[info:gnus#Expiring Mail]]) instead?
>>>
>>>     ,----
>>>     | To avoid having articles marked as read marked as
>>>     | expirable automatically, you can put something like the following in
>>>     | your `~/.gnus.el' file:
>>>     | 
>>>     |      (remove-hook 'gnus-mark-article-hook
>>>     |                   'gnus-summary-mark-read-and-unread-as-read)
>>>     |      (add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)
>>>     | 
>>>     `----
>>
>> Yes, that seems rather excessive.
>
> Any objections if I remove that passage in my patch?

No, go ahead.

>>> * Suggestion: Rename `gnus-auto-expirable-newsgroups'
>>>   `gnus-auto-expirable-groups' since, generally, auto-expire only
>>>   applies to mail and not to nntp.
>>
>> Gnus has historically used "newsgroups" and "groups" as synonyms, and I
>> think that boat has sailed a long time ago.
>
> Yeah, but Emacs supports aliases.  Is there any reason not to make
> things clearer for future generations?

It's too much churn for too little benefit.  At some point the aliases
will have to go away, and people will have broken setups.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/