* External delivery: Pros and cons
@ 1995-11-15 8:01 Sudish Joseph
1995-11-15 15:38 ` Brian Edmonds
0 siblings, 1 reply; 4+ messages in thread
From: Sudish Joseph @ 1995-11-15 8:01 UTC (permalink / raw)
Pros:
- Much faster startup.
Cons:
- It is possible to lose mail if you're not careful in setting things up.
- Not easy to set up, you'll have to juggle the startup files for
both your filtering agent and gnus.
Losing mail in this context comes in two flavors.
a) You actually lose the message.
b) The message itself isn't lost, but the NOV/active file entry gets
clobbered, so you might not know you received the message. (Not a
problem if you use nnmh instead of nnml.)
Avoiding lossage for the two cases:
a) Actual lossage of the entire message is possible if you use the
copy and move commands from the summary ("B m" "B c"). In this
case, gnus generates the new article number on the basis of
potentially old information, so it could overwrite a newly
delivered message.
Solution:
Don't move/copy into groups you deliver to. Sounds stupid, yes, but
it's not a big pain in practice. It's possible to make this safe
using locking, but looks like it'd be a pain to implement.
b) The active file getting clobbered is not a big deal, it'll get
corrected again by the next mail that arrives for that group.
The NOV file can get clobbered in two ways. It's possible for an
entry to get deleted due to gnus overwriting a newer version. It's
also possible to get your nov entries entered in non-ascending
order.
Solution:
For the active file problem, modify gnus-group-get-new-news-this-group
(M-g in the group buffer) to do an nnmh-style lookup for the nnml
group. This would make it simple to correct the active file entry.
This is also more consistent with how nntp groups are handled (uses
the active file for mass lookup/does a GROUP for a -this-group
lookup). A better solution is to use locking (see below).
For the NOV entry being deleted problem, use locking. Essentially,
provide hooks for the user to run whatever locking method they want
(procmail users might run `lockfile' off such hooks). Four hooks
would suffice: one pre- and post- pair for active file update, another
pair for NOV update.
The non-ascending NOV file should really be the filters
problem...except that doing this isn't efficient for some filters.
It's not a problem in mailagent, since only one mailagent is active
at any given time; doing a sort from procmail for _every_ message you
get is prohibitively expensive--sleeping for earlier articles to be
updated leads to the possibility of long queues of processes hanging
on one possibly-dead process)
It'd be simple enough to get GNUS to read such malformed NOV files
properly, but Lars fears that this might cause a slow down. I don't
think it'd be that bad. If this is unacceptable, we could have a
variable that would let the user select a version of
gnus-get-newsgroup-headers-xover that allows unsorted nov files, while
leaving the default the way it is.
If none of the above are unacceptable, use an external script to
sanity check.correct your overviews and active file. Or you could use
nnml-generate-nov-databases for this, but it clobbered something or
the other the last time I used it. :-)
None of the above are very difficult to implement, the only real issue
is whether any such code will be put into gnus itself. I'd rather not
have _another_ large chunk of code I have to patch into every release
I fetch. Besides, sooner or later, people're going to hack their
version of GNUS 5.x to do stuff like this...(I will, for one :)
-Sudish
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: External delivery: Pros and cons
1995-11-15 8:01 External delivery: Pros and cons Sudish Joseph
@ 1995-11-15 15:38 ` Brian Edmonds
1995-11-16 4:37 ` Scott Blachowicz
0 siblings, 1 reply; 4+ messages in thread
From: Brian Edmonds @ 1995-11-15 15:38 UTC (permalink / raw)
Cc: joseph
> Date: 15 Nov 1995 03:01:39 -0500
>
> Cons:
One con you forgot that used to bother me a lot: when you filter
incoming messages, you end up with one (potentially large: perl) process
per message. If you're on a system like mine where I retrieve mail via
UUCP once per hour, all those processes starting up at once can put a
serious strain on the system. I have blown out my virtual space once in
the past when I was only running with 20M.
I find splitting the mail inside emacs is pretty quick, unless you've
got hundreds of messages, and far kinder on my hard drive and system
performance. As it is, just punting everything through sendmail at once
produces a noticable lag.
Brian. http://www.cs.ubc.ca/spider/edmonds
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: External delivery: Pros and cons
1995-11-15 15:38 ` Brian Edmonds
@ 1995-11-16 4:37 ` Scott Blachowicz
0 siblings, 0 replies; 4+ messages in thread
From: Scott Blachowicz @ 1995-11-16 4:37 UTC (permalink / raw)
Cc: ding, joseph
Brian Edmonds <edmonds@cs.ubc.ca> wrote:
> One con you forgot that used to bother me a lot: when you filter
> incoming messages, you end up with one (potentially large: perl) process
> per message.
The "potentially large" part depends on what happens when your MTA does the
delivery. Mailagent, for instance, seems to have a 2 step method - the MTA
runs a "filter" program that just puts the message in a queue directory. The
potentially large perl process runs as a single process to clear that queue.
Of course, I haven't tried many alternatives, so I can't really comment on
relative merits...just on my current playthings. :-)
Scott Blachowicz Ph: 206/283-8802x240 StatSci, a div of MathSoft, Inc.
1700 Westlake Ave N #500
scott@statsci.com Seattle, WA USA 98109
Scott.Blachowicz@seaslug.org
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: External delivery: Pros and cons
@ 2002-10-20 20:13 Unknown
0 siblings, 0 replies; 4+ messages in thread
From: Unknown @ 2002-10-20 20:13 UTC (permalink / raw)
Cc: joseph
X-Geek-3: GE/CS d+(--) s:++>: a C++++$ ULUHO++++$ P+++$ L+++ E+++
W+++$ N+++ K-? !w--- O-? !M-- !V-- PS+ PE- Y+ PGP++ t@ 5++ !X R++
b+++ DI+++ D- G e+++ h+ r++ y+
X-Face: #q.#]5@vq!Jz+E0t_/;Y^gTjR\T^"B'fbeuVGiyKrvbfKJl!^e|e:iu(kJ6c|QYB57LP*|t&YlP~HF/=h:GA6o6W@I#deQL-%#.6]!z:6Cj0kd#4]>*D,|0djf'CVlXkI,>aV4\}?d_KEqsN{Nnt778"OsbQ["56/!nisvyB/uA5Q.{)gm6?q.j71ww.>b9b]-sG8zNt%KkIa>xWg&1VcjZk[hBQ>]j~`WqXl,y1a!(>6`UM{~'X[Y_,Bv+}=L\SS*mA8=s;!=O`ja|@PEzb&i0}Qp,`Z\:6:OmRi*
X-Organization: University of Massachusetts, Amherst, MA 01003
X-Time: Fri Nov 17 04:40:49 1995
X-Mail-System: Vm 5.95 (beta) for GNU Emacs 19.14 XEmacs Lucid (beta5)
References: <199511151538.HAA00082@edmonds.home.cs.ubc.ca>
From: Manoj Srivastava <srivasta@pilgrim.umass.edu>
Date: 17 Nov 1995 04:40:48 -0500
In-Reply-To: Brian Edmonds's message of Wed, 15 Nov 1995 07:38:25 -0800
Message-ID: <gvxbuqbh93z.fsf@plymouth.pilgrim.umass.edu>
Organization: Project Pilgrim, University of Massachusetts at Amherst
Lines: 42
X-Mailer: September Gnus v0.14
Hi,
>>"Brian" == Brian Edmonds <edmonds@cs.ubc.ca> writes:
Date> 15 Nov 1995 03:01:39 -0500
>>
Cons>
Brian> One con you forgot that used to bother me a lot: when you
Brian> filter incoming messages, you end up with one (potentially
Brian> large: perl) process per message. If you're on a system like
Brian> mine where I retrieve mail via UUCP once per hour, all those
Brian> processes starting up at once can put a serious strain on the
Brian> system. I have blown out my virtual space once in the past
Brian> when I was only running with 20M.
If you use mailagent as your external splitting agent, then
the filtering is two phased, each mail message invokes a small filter
written in C, that spools the files locally, and calls the
(potentially large) perl mailagent just once for a batch of files. I
think it would solve your problem with batch downloads (works great
for me on a linux box).
Brian> I find splitting the mail inside emacs is pretty quick, unless
Brian> you've got hundreds of messages, and far kinder on my hard
Brian> drive and system performance. As it is, just punting
Brian> everything through sendmail at once produces a noticable lag.
Well, I need mailagent for other things, and did not feel like
maintaining two sets of filtering rules.
Brian> Brian. http://www.cs.ubc.ca/spider/edmonds
manoj
-- Burnt Sienna. That's the best thing that ever happened to
Crayolas. Ken Weaver
Manoj Srivastava Project Pilgrim, Department of Computer Science
Phone: (413) 545-3918 A143B Lederle Graduate Research Center
Fax: (413) 545-1249 University of Massachusetts, Amherst, MA 01003
email:srivasta@pilgrim.umass.edu http://www.pilgrim.umass.edu/~srivasta/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2002-10-20 20:13 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1995-11-15 8:01 External delivery: Pros and cons Sudish Joseph
1995-11-15 15:38 ` Brian Edmonds
1995-11-16 4:37 ` Scott Blachowicz
2002-10-20 20:13 Unknown
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).