mail archival idea/feature request

mail archival idea/feature request

[Andrew Hobson]

The recent thread about mail archival got me thinking.  Maybe I'm
completely off the boat in the deep end, but here's how I'd like it to 
work.

At expiry time, instead of deleting the message, I'd like to move it
into another, non-expiring group.  That way I could archive all of my
incoming mail, but it would remain in the usual group until
expiration.  I could then determine at expiry time if I wanted to save
the message, and if so, which archival group to put it in.

I took a quick peek and I didn't see an easy way to do this, although
I'm a gnus neophyte.  It would seem to me that a customizable hook run
at expiry time that defaults to nnmail-delete-file-function (or
whatever is appropriate for the backend) would enable me to do what I
wanted.

I suppose I could deliver the mail to two groups at split time: one
for archival and one for usual purposes, but that seems less that
optimal.

Is there a way in pgnus to do this kind of archival?

Drew

Re: mail archival idea/feature request

[Lars Magne Ingebrigtsen]

Andrew Hobson <ahobson@eng.mindspring.net> writes:

> Is there a way in pgnus to do this kind of archival?

There is no overall functionality for moving articles instead of
deleting them upon expiry, but for nnml you can alter
nnmail-delete-file-function, and for nnfolder you can do stuff in
nnfolder-delete-mail-hook.  Archival expiry functionality could be
added, but what should it look like?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: mail archival idea/feature request

[Andrew Hobson]

On 09 Jul 1999 23:04:33 +0200, Lars Magne Ingebrigtsen <larsi@gnus.org> said:

> There is no overall functionality for moving articles instead of
> deleting them upon expiry, but for nnml you can alter
> nnmail-delete-file-function, and for nnfolder you can do stuff in
> nnfolder-delete-mail-hook.  Archival expiry functionality could be
> added, but what should it look like?

Because I might not want to archive mail that I explicitly deleted
(but then again I might), I was thinking of adding something similar
to gnus-decay-score-function for expiry time and delete time.  It
would be nice if all information known about the article was passed to
the function, such as which group it was in and the path to the file.
And like write-file-hooks, if the function returned non-nil, the file
is considered already deleted.

What do you think?

Drew
-- 
"Baby, will you eat that there snack cracker in your special outfit
 for me, please?"

Re: mail archival idea/feature request

[Lars Magne Ingebrigtsen]

Andrew Hobson <ahobson@eng.mindspring.net> writes:

> Because I might not want to archive mail that I explicitly deleted
> (but then again I might), I was thinking of adding something similar
> to gnus-decay-score-function for expiry time and delete time.  It
> would be nice if all information known about the article was passed to
> the function, such as which group it was in and the path to the file.
> And like write-file-hooks, if the function returned non-nil, the file
> is considered already deleted.

Well, I don't think the archive-expired-messages thing would be run on 
explicitly deleted files at all.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: mail archival idea/feature request

[Kai.Grossjohann]

Andrew Hobson <ahobson@eng.mindspring.net> writes:

> At expiry time, instead of deleting the message, I'd like to move it
> into another, non-expiring group.  That way I could archive all of my
> incoming mail, but it would remain in the usual group until
> expiration.  I could then determine at expiry time if I wanted to save
> the message, and if so, which archival group to put it in.

Why do you not want to just keep those messages which you want to keep
right in the group they are in?  Keep it simple...

kai (keep, keep)
-- 
Life is hard and then you die.

Re: mail archival idea/feature request

[Andrew Hobson]

On 10 Jul 1999 00:49:44 +0200, Kai.Grossjohann@CS.Uni-Dortmund.DE said:

> Andrew Hobson <ahobson@eng.mindspring.net> writes:
>> At expiry time, instead of deleting the message, I'd like to move it
>> into another, non-expiring group.  That way I could archive all of my
>> incoming mail, but it would remain in the usual group until
>> expiration.  I could then determine at expiry time if I wanted to save
>> the message, and if so, which archival group to put it in.

> Why do you not want to just keep those messages which you want to keep
> right in the group they are in?  Keep it simple...

It would be nice when searching for old mail (using nnir[1]) to know by
the group name the date it arrived.  I archive my outgoing mail by
date sent and I like that very much.

I guess I could change my format string to print out the date, but I
really don't want it there except when doing searches.

Not a very compelling reason, huh?  :-)

Drew

Footnotes: 
[1]  I can't begin to tell you how much I *love* nnir.

Re: mail archival idea/feature request

[Kai.Grossjohann]

Andrew Hobson <ahobson@eng.mindspring.net> writes:

> I guess I could change my format string to print out the date, but I
> really don't want it there except when doing searches.

You can even add stuff to gnus-summary-mode-hook which only puts in
the date when in nnir groups.  Let's see...

(defun my-gnus-summary-line-format ()
  (when (string-match "^nnir" gnus-newsgroup-name)
    (make-local-variable 'gnus-summary-line-format)
    (setq gnus-summary-line-format "SPECIAL FORMAT GOES HERE")))
(add-hook 'gnus-summary-mode-hook 'my-gnus-summary-line-format)

Does this work?

kai
-- 
Life is hard and then you die.

Re: mail archival idea/feature request

[Andrew Hobson]

On 10 Jul 1999 23:23:02 +0200, Kai.Grossjohann@CS.Uni-Dortmund.DE said:

> Andrew Hobson <ahobson@eng.mindspring.net> writes:
>> I guess I could change my format string to print out the date, but I
>> really don't want it there except when doing searches.

> You can even add stuff to gnus-summary-mode-hook which only puts in
> the date when in nnir groups.  Let's see...

[...]

> Does this work?

I tried it out and yes, I like it very much.  Although, I did get to
thinking and I thought that if I archived all my incoming mail like
Per for 10(!) years, my groups would become so huge, I might want to
split them up or archive some of them.  Having them already split up
by date would make that easier.

Thank you for your suggestions.  I may play around and see if I can do 
what I want with only a small patch.

Drew

Re: mail archival idea/feature request

[Karl Kleinpaste]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:
> You can even add stuff to gnus-summary-mode-hook which only puts in
> the date when in nnir groups.

Yup -- I do this for gnus-summary-line-format as well as
gnus-summary-mode-line-format.

(setq gnus-select-group-hook
      (function
       (lambda ()
	 ;; Ignore From: in summary for certain groups.
	 (cond ((string-match "clari." gnus-newsgroup-name)
		(setq gnus-summary-line-format "%U%R%z%I%(%[%4L: %-6,6n%]%)   %s\n"))
	       ((string-match "comp.sources" gnus-newsgroup-name)
		(setq gnus-summary-line-format "%U%R%z%I%(%[%4L%]%)   %s\n"))
	       ((string-match "nnir:" gnus-newsgroup-name)
		(setq gnus-summary-line-format "%U%R%z [%d] %I%(%[%4L: %-20,20f%]%) %s\n"))
	       (t
		(setq gnus-summary-line-format "%U%R%z%I%(%[%4L: %-20,20f%]%) %s\n")))
	 ;; Put full newsgroup name in summary mode line for nnir.
	 (setq gnus-summary-mode-line-format
	       (if (string-match "nnir:" gnus-newsgroup-name)
		   "Gnus: %G [%A] %Z" "Gnus: %g [%A] %Z"))
	 )))

See "[%d]", plus "%G" in the mode line, for nnir groups.

The latter is needed due to the fact that %g's provision of a
"shortish" group name makes the resulting nnir ephemeral group name
completely nonsensical, so I want the complete, long name there.

Re: mail archival idea/feature request

[Wes Hardaker]

>>>>> On 10 Jul 1999 00:49:44 +0200, Kai.Grossjohann@CS.Uni-Dortmund.DE said:

Kai> Why do you not want to just keep those messages which you want to
Kai> keep right in the group they are in?  Keep it simple...

Speeeeeed.  It's all well and dandy (mostly) for nnml groups (which
are fairly fast, but do get slower as they get larger), but its a real 
pain for network related backends like nnimap.  It also starts taking
too long to thread them.

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: mail archival idea/feature request

[Kai Großjohann]

Wes Hardaker <wjhardaker@ucdavis.edu> writes:

> Speeeeeed.  It's all well and dandy (mostly) for nnml groups (which
> are fairly fast, but do get slower as they get larger), but its a real 
> pain for network related backends like nnimap.  It also starts taking
> too long to thread them.

I noticed, too.  I set gnus-fetch-old-headers to 500.

kai
-- 
Life is hard and then you die.

Re: mail archival idea/feature request

[Wes Hardaker]

>>>>> On 09 Jul 1999 14:27:37 -0400, Andrew Hobson <ahobson@eng.mindspring.net> said:

Andrew> The recent thread about mail archival got me thinking.  Maybe
Andrew> I'm completely off the boat in the deep end, but here's how
Andrew> I'd like it to work.

Actually, having a group parameter called "(archivegroup "nnml:blah")"
or something would be really cool..  Then you could just hit, um, "A"
(yeah, I know, its in use) to send a message to the archivegroup (the
equivalent of an automatted "B m").  And if (expire-to-archive t) was
set, it would send expired messages there instead of deleting them...

God, if Gnus were a car, how much would it cost?

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: mail archival idea/feature request

[Lars Magne Ingebrigtsen]

Wes Hardaker <wjhardaker@ucdavis.edu> writes:

> Actually, having a group parameter called "(archivegroup "nnml:blah")"
> or something would be really cool..  Then you could just hit, um, "A"
> (yeah, I know, its in use) to send a message to the archivegroup (the
> equivalent of an automatted "B m").  And if (expire-to-archive t) was
> set, it would send expired messages there instead of deleting them...

Or -- the old, expirable articles are deleted, and the old,
non-expirable articles are moved to the archive group.  Hee hee.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: mail archival idea/feature request

[Wes Hardaker]

>>>>> On 27 Aug 1999 19:39:46 +0200, Lars Magne Ingebrigtsen <larsi@gnus.org> said:

Lars> Or -- the old, expirable articles are deleted, and the old,
Lars> non-expirable articles are moved to the archive group.  Hee hee.

That'd work too (though in some cases I want to pick when things move
to the archive group (and I want to archive everything) and when they
stick around as old (hence, E).

It comes from disk space being so cheap and most mail messages being
so small...  Why not just archive my entire life.

Quick calculations;
  Random sample of my old nnml inbox (now using nnimap):

  % ls -l | wc -l
      325
  % ls -l | awk '{total = total + $5} END { print (total/325); }'
  17278

Ok, so each message is on average 17k.  (most are much much smaller,
and a threshold on the max size archived would be even better).

I get about 300 messages a day at last count.  Ok, lets count
yesterday just to be sure:

% grep "Aug 30" MAIL_LOG | grep '^From' | wc -l
    287

Yep, still roughly accurate.

So, how much space for a years worth of messages?

  (* 17 300 365) = 1.8gig.

So, I'd gladly waste a 1.8 gig disk to archive my entire years worth
of mail providing I didn't have to store it in the current folder
where it would eat away my summary buffer creation speed.

However, I would not of course archive my entire set of mail lists.
Mostly important groups like inbox, work, etc, which would reduce the
size of the above as well.

So, uh, got it written yet?

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: mail archival idea/feature request

[Andrew Hobson]

On 01 Sep 1999 09:53:43 -0700, Wes Hardaker <wjhardaker@ucdavis.edu> said:

> However, I would not of course archive my entire set of mail lists.
> Mostly important groups like inbox, work, etc, which would reduce the
> size of the above as well.

> So, uh, got it written yet?

What I decided to do was crosspost all mail I wanted archived to a
group[1] that I don't expire.

It works well so far.

Drew


Footnotes: 
[1] (format-time-string "incoming.mail.%Y-%m")

-- 
There followed a period of embarrassment as the rumour went round
London that Joey Mellen had trepanned himself, whereas in fact he
had failed to do so.

Re: mail archival idea/feature request

[Wes Hardaker]

>>>>> On 01 Sep 1999 15:06:43 -0400, Andrew Hobson <ahobson@eng.mindspring.net> said:

Andrew> What I decided to do was crosspost all mail I wanted archived
Andrew> to a group[1] that I don't expire.

I'm talking about incoming mail as well as outgoing...

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: mail archival idea/feature request

[Andrew Hobson]

On 03 Sep 1999 14:32:54 -0700, Wes Hardaker <wjhardaker@ucdavis.edu> said:

> I'm talking about incoming mail as well as outgoing...

Right.  I use nnmail-split-fancy.  I do:

(defun ahobson-incoming-group ()
  (format-time-string "incoming.mail.%Y-%m"))

(setq nnmail-split-fancy
      '(&
	(: ahobson-incoming-group)
	(|
        ;; normal splitting
)))


All of my incoming mail is saved in a separate group by month.  I
don't expire those groups.  I then have a crossposted copy of the mail 
in my usual folders which I expire as usual.

Maybe I'm not understanding what you want to do.

Drew

Re: mail archival idea/feature request

[Wes Hardaker]

>>>>> On 03 Sep 1999 18:11:04 -0400, Andrew Hobson <ahobson@eng.mindspring.net> said:

Andrew> Right.  I use nnmail-split-fancy.  I do:

Well, thats true.  I could split it before hand, but I'd rather do it
at deletion time so I could keep the marks it contains.  That could
get lisp intensive (.eld) I suppose though...


-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: mail archival idea/feature request

[Lars Magne Ingebrigtsen]

Andrew Hobson <ahobson@eng.mindspring.net> writes:

> What I decided to do was crosspost all mail I wanted archived to a
> group[1] that I don't expire.

That's a very good idea.  Hm.  Perhaps what I want is a "shadow
hierarchy" on unexpirable groups for all my expirable groups.

The problem is that it probably won't scale very well.  Splitting mail 
to groups that contain 100M+ number of articles will be slow.

Darn.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen