deleting email, expiring, mail-source-delete-incoming

deleting email, expiring, mail-source-delete-incoming

[lee]

Hi,

what exactly does mail-source-delete-incoming do? The documentation
says:

,----
|      If non-`nil', delete incoming files after handling them.
`----

I´m wondering what is considered as "handling". For example, in my case
basically all splitting is done through the exim ~/.forward file, so
"handling" would mean that the user is doing all the handling of
incoming messages himself (using gnus).

When the user moves an incoming message to another group, is that
considered "handling", and will the message be treated according to what
mail-source-delete-incoming is set to? Or will it be delted immediately
because it was moved?


What´s behind this questions is that I´m trying to figure out what gnus
does with email I don´t want to keep stored on my harddisk. If I
understood the manual correctly, it´s like this (unless I modify the
default behaviour):


  1.) Gnus basically does not delete any email.
  2.) To have an email deleted, I should mark it as expired.
  3.) At some time, emails marked as expired and more than a week old
      will be deleted.
  4.) Apparently I don´t need to do anything for 3.) to happen, besides
      marking emails as expired.


The default behaviour is modified in that I have set
mail-source-delete-incoming to nil. Does this mean that expired emails
will never be deleted from incoming?

Re: deleting email, expiring, mail-source-delete-incoming

[Reiner Steib]

On Fri, Jun 10 2011, lee wrote:

> I´m wondering what is considered as "handling".

"handling" here starts with fetching the mail from `mail-sources' and
ends when the mail is stored in the group of your mail back end (nnml,
...) according to your split methods.

> What´s behind this questions is that I´m trying to figure out what gnus
> does with email I don´t want to keep stored on my harddisk. If I
> understood the manual correctly, it´s like this (unless I modify the
> default behaviour):
>
>   1.) Gnus basically does not delete any email.
>   2.) To have an email deleted, I should mark it as expired.
>   3.) At some time, emails marked as expired and more than a week old
>       will be deleted.
>   4.) Apparently I don´t need to do anything for 3.) to happen, besides
>       marking emails as expired.

Correct.

> The default behaviour is modified in that I have set
> mail-source-delete-incoming to nil. Does this mean that expired emails
> will never be deleted from incoming?

Expiry and mail-source-delete-incoming are completely unrelated.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: deleting email, expiring, mail-source-delete-incoming

[Philipp Haselwarter]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Fri, Jun 10 2011, lee wrote:
>
>> I´m wondering what is considered as "handling".
>
> "handling" here starts with fetching the mail from `mail-sources' and
> ends when the mail is stored in the group of your mail back end (nnml,
> ...) according to your split methods.
>
>> What´s behind this questions is that I´m trying to figure out what gnus
>> does with email I don´t want to keep stored on my harddisk. If I
>> understood the manual correctly, it´s like this (unless I modify the
>> default behaviour):
>>
>>   1.) Gnus basically does not delete any email.
>>   2.) To have an email deleted, I should mark it as expired.
>>   3.) At some time, emails marked as expired and more than a week old
>>       will be deleted.
>>   4.) Apparently I don´t need to do anything for 3.) to happen, besides
>>       marking emails as expired.
>
> Correct.
>
>> The default behaviour is modified in that I have set
>> mail-source-delete-incoming to nil. Does this mean that expired emails
>> will never be deleted from incoming?
>
> Expiry and mail-source-delete-incoming are completely unrelated.
>
> Bye, Reiner.

So IIUC gnus will with the default setting delete mail from my Inbox
after 10 days? Without confirmation?! (looking at
`mail-source-delete-old-incoming-confirm')
Sounds scary.

-- 
Philipp Haselwarter

Re: deleting email, expiring, mail-source-delete-incoming

[Adam Sjøgren]

On Sat, 11 Jun 2011 18:46:51 +0200, Philipp wrote:

> So IIUC gnus will with the default setting delete mail from my Inbox
> after 10 days? Without confirmation?! (looking at
> `mail-source-delete-old-incoming-confirm')

No.

By default development Gnus store a copy of all email imported from
mail-sources in ~/Mail/Incoming* files, as an extra precaution should
anything go wrong (with splitting or what have you).

Those backup copies will by default be deleted after 10 days.

I think in release versions usually the Incoming*-files are usually
deleted immediately.

> Sounds scary.

I think it sounds like a belt-and-suspenders kind of approach.


  Best regards,

     Adam

-- 
 "So this is what being a morning person is like, I           Adam Sjøgren
  thought. It's like being 80 years old."                asjo@koldfront.dk

Re: deleting email, expiring, mail-source-delete-incoming

[Reiner Steib]

On Sat, Jun 11 2011, Adam Sjøgren wrote:

> Those backup copies will by default be deleted after 10 days.
>
> I think in release versions usually the Incoming*-files are usually
> deleted immediately.

For released version it is supposed to be 2 days.

,----[ (info "(gnus)Mail Source Customization") ]
| `mail-source-delete-incoming'
|      If non-`nil', delete incoming files after handling them.  If `t',
|      delete the files immediately, if `nil', never delete any files.
|      If a positive number, delete files older than number of days (the
|      deletion will only happen when receiving new mail).  You may also
|      set `mail-source-delete-incoming' to `nil' and call
|      `mail-source-delete-old-incoming' from a hook or interactively.
|      `mail-source-delete-incoming' defaults to `10' in alpha Gnusae and
|      `2' in released Gnusae.  *Note Gnus Development::.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: deleting email, expiring, mail-source-delete-incoming

[Adam Sjøgren]

On Tue, 14 Jun 2011 22:08:19 +0200, Reiner wrote:

> On Sat, Jun 11 2011, Adam Sjøgren wrote:

> For released version it is supposed to be 2 days.

Ah, thanks for the correction.


  Best regards,

    Adam

-- 
 "So this is what being a morning person is like, I           Adam Sjøgren
  thought. It's like being 80 years old."                asjo@koldfront.dk

Re: deleting email, expiring, mail-source-delete-incoming

[Sivaram Neelakantan]

On Sat, Jun 11 2011,Philipp Haselwarter wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:

[snipped 23 lines]

>>> The default behaviour is modified in that I have set
>>> mail-source-delete-incoming to nil. Does this mean that expired emails
>>> will never be deleted from incoming?
>>
>> Expiry and mail-source-delete-incoming are completely unrelated.
>>
>> Bye, Reiner.
>
> So IIUC gnus will with the default setting delete mail from my Inbox
> after 10 days? Without confirmation?! (looking at
> `mail-source-delete-old-incoming-confirm')
> Sounds scary.

I don't think that's what's being said.  The incoming message file is
deleted after 10 days, the split mail in your INBOX stays.  The
downloaded temp files of all the messages are the ones being referred
to. 

 sivaram
 -- 

Re: deleting email, expiring, mail-source-delete-incoming

[lee]

Sivaram Neelakantan <nsivaram.net@gmail.com> writes:

> On Sat, Jun 11 2011,Philipp Haselwarter wrote:
>
>> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>>>
>>> Expiry and mail-source-delete-incoming are completely unrelated.
>>>
>>
>> So IIUC gnus will with the default setting delete mail from my Inbox
>> after 10 days? Without confirmation?!
>
> I don't think that's what's being said.  The incoming message file is
> deleted after 10 days, the split mail in your INBOX stays.  The
> downloaded temp files of all the messages are the ones being referred
> to. 

Well, I now have:


,----
| (setq
|  gnus-select-method '(nnml "")
|  mail-sources '((maildir :path "~/Maildir/")))
`----


The mail-source-delete-incoming-mail variable isn´t changed anymore in
~/.gnus.el. My MTA delivers all mail (with one exception for nndiary
mails) to ~/Maildir, in maildir format. From there, gnus fetches it.

So that would mean all the mail gnus has fetched from ~/Maildir stays
in ~/Maildir for another 10 days before it´s being deleted?

And the temp files? Is all the mail actually quadrupled until the files
in incoming (~/Maildir) and the temporary files are expired?

Re: deleting email, expiring, mail-source-delete-incoming

[Reiner Steib]

On Sun, Jun 12 2011, lee wrote:

> Well, I now have:
>
> ,----
> | (setq
> |  gnus-select-method '(nnml "")
> |  mail-sources '((maildir :path "~/Maildir/")))
> `----
>
> The mail-source-delete-incoming-mail variable isn´t changed anymore in
> ~/.gnus.el. My MTA delivers all mail (with one exception for nndiary
> mails) to ~/Maildir, in maildir format. From there, gnus fetches it.

Maybe it would make more sense to use the nnmaildir back end?  I.e. no
entry in `mail-source´ and an nnmaildir method in `gnus-select-method´
(or `gnus-secondary-select-methods').  See (info "(gnus)Maildir").

> So that would mean all the mail gnus has fetched from ~/Maildir stays
> in ~/Maildir for another 10 days before it´s being deleted?

The files corresponding to `mail-source-delete-incoming´ are stored in
`mail-source-directory´ and are called "Incoming*"

> And the temp files? Is all the mail actually quadrupled until the files
> in incoming (~/Maildir) and the temporary files are expired?

Not sure what you are counting.  The usual chain is:

$MAIL (/var/mail/USER) -> Incoming* -> ~/Mail/some.group/123

Incoming* is the only place where a redundant copy of a message lives.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: deleting email, expiring, mail-source-delete-incoming

[lee]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Sun, Jun 12 2011, lee wrote:
>
>> Well, I now have:
>>
>> ,----
>> | (setq
>> |  gnus-select-method '(nnml "")
>> |  mail-sources '((maildir :path "~/Maildir/")))
>> `----
>>
>> The mail-source-delete-incoming-mail variable isn´t changed anymore in
>> ~/.gnus.el. My MTA delivers all mail (with one exception for nndiary
>> mails) to ~/Maildir, in maildir format. From there, gnus fetches it.
>
> Maybe it would make more sense to use the nnmaildir back end?  I.e. no
> entry in `mail-source´ and an nnmaildir method in `gnus-select-method´
> (or `gnus-secondary-select-methods').  See (info "(gnus)Maildir").

Well, yes, it might have if I had known better when starting to switch
from mutt to gnus. Alas, I made the whole thing rather complicated
because first I found out that gnus cannot access my email stored in
maildir format under ~/Mail. Later, during moving my email, I found out
that there were two reasons for that: The tmp folder in ~/Mail/Maildir
(which was the incoming folder) was a maildir itself, and the directory
hierachy in which I stored my mail was a mixup of mailfolders and
folders, which is something that doesn´t fit into what gnus expects.

Then I thought I would use IMAP for all my mail and thus have the same
method of accessing my email with whatever client I might use, plus the
advantage of potentially being able to access it from anywhere. IMAP
turned out to be rather troublesome for two reasons: The version of
courier-imap currently in Debian testing has a bug that makes it
impossible to access some of the folders with gnus. I tried to compile
courier-imap from svn myself because this bug apparently has already
been fixed in more recent versions, and I found out that it doesn´t
compile (without a lot of effort). There doesn´t seem to be any other
IMAP server supporting maildir ...

Anyway, I had moved part of my emails to IMAP aready when it became too
annoying that some folders were inaccessible, so I finallly decided to
use nnml. Since nnml appears to be the native format of gnus, it´s
probably going to work best with that. Should I seriously need to access
all my email remotely, I can use ssh. For the cell phone, I can use IMAP
to see what mail has newly arrived, and that´s enough. The IMAP client
in the phone doesn´t even support folders, anyway.

>> So that would mean all the mail gnus has fetched from ~/Maildir stays
>> in ~/Maildir for another 10 days before it´s being deleted?
>
> The files corresponding to `mail-source-delete-incoming´ are stored in
> `mail-source-directory´ and are called "Incoming*"

Could I delete those manually? I currently have 89140 of those, while a
gnus-slave is moving mails from mailing lists into the appropriate
groups since three days or so. I made the mistake to dump those (with
mutt) into the incoming directory to get them into gnus. Unfortunately,
gnus is awfully slow with processing them.

In the meantime, I´ve switched the MTA to deliver to another directory
temporarily so that I can see what´s coming in with a gnus using the
temporary directory as its mail-source until the slave is done. Once the
slave is done, I could either switch back and hope that the Incoming*
files will be removed after 10 days, or I can delete them manually
before switching back, unless that would confuse gnus (which is somewhat
messed up now because I tried to configure it the way I wanted, see my
other post).

>> And the temp files? Is all the mail actually quadrupled until the files
>> in incoming (~/Maildir) and the temporary files are expired?
>
> Not sure what you are counting.  The usual chain is:
>
> $MAIL (/var/mail/USER) -> Incoming* -> ~/Mail/some.group/123
>
> Incoming* is the only place where a redundant copy of a message lives.

Well, I´ve been using maildir (and mutt) for about 15 years now, so
/var/spool/mail/user or the like isn´t exactly usual for me. It just
seems way too dangerous to keep all the mail in one huge file, or in a
few huge files.

So in this context, "temp files" makes me think of what might appear in
the "tmp" directory of a maildir folder.

Re: deleting email, expiring, mail-source-delete-incoming

[Reiner Steib]

On Tue, Jun 14 2011, lee wrote:

> Could I delete those manually? I currently have 89140 of those, while a
> gnus-slave is moving mails from mailing lists into the appropriate
> groups since three days or so. 

You can delete them using rm or `mail-source-delete-old-incoming':

,----[ <f1> f mail-source-delete-old-incoming RET ]
| mail-source-delete-old-incoming is an interactive compiled Lisp function in
| `mail-source.el'.
| 
| (mail-source-delete-old-incoming &optional AGE CONFIRM)
| 
| Remove incoming files older than AGE days.
| If CONFIRM is non-nil, ask for confirmation before removing a file.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: deleting email, expiring, mail-source-delete-incoming

[lee]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Tue, Jun 14 2011, lee wrote:
>
>> Could I delete those manually? I currently have 89140 of those, while a
>> gnus-slave is moving mails from mailing lists into the appropriate
>> groups since three days or so. 
>
> You can delete them using rm or `mail-source-delete-old-incoming':

Oh, that´s cool, thanks :)

Now I don´t dare anymore to run a gnus-slave or a master with the
incoming directory as the mail-source: It´ll still take quite a long
time (as in days) to process all the messages. I could let gnus run over
night and interrupt the processing with C-g to be able to use it, but
I´m afraid that might screw up the active file or other things again.

Is there a way to make the processing faster?