nnimap-split-download-body feature request

nnimap-split-download-body feature request

[Jake Colman]

I sometimes receieve work email with very large attachments.  I don't think
that large atachments are typical of spam.  It would be great if there could
be some way to indicate that message bodies should only be downloaded if they
are smaller than some (default) reasonable number.  Otherwise, the processing
time for spam.el becomes horrendous.

-- 
Jake Colman                     

Principia Partners LLC                    Phone: (201) 209-2467
Harborside Financial Center                 Fax: (201) 946-0320
902 Plaza Two                          E-mail: colman@ppllc.com
Jersey City, NJ 07311                 www.principiapartners.com

Re: nnimap-split-download-body feature request

[Ted Zlatanov]

On Fri, 14 Nov 2003, colman@ppllc.com wrote:

> I sometimes receieve work email with very large attachments.  I
> don't think that large atachments are typical of spam.  It would be
> great if there could be some way to indicate that message bodies
> should only be downloaded if they are smaller than some (default)
> reasonable number.  Otherwise, the processing time for spam.el
> becomes horrendous.

The key function call in nnimap.el is:

;; in nnimap-split-articles

(nnimap-request-article article) 

which is invoked if nnimap-split-download-body is on.  I don't know if
partial requests are possible, this would be a question for Simon
Josefsson probably.

Ted

Re: nnimap-split-download-body feature request

[Simon Josefsson]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Fri, 14 Nov 2003, colman@ppllc.com wrote:
>
>> I sometimes receieve work email with very large attachments.  I
>> don't think that large atachments are typical of spam.  It would be
>> great if there could be some way to indicate that message bodies
>> should only be downloaded if they are smaller than some (default)
>> reasonable number.  Otherwise, the processing time for spam.el
>> becomes horrendous.
>
> The key function call in nnimap.el is:
>
> ;; in nnimap-split-articles
>
> (nnimap-request-article article) 
>
> which is invoked if nnimap-split-download-body is on.  I don't know if
> partial requests are possible, this would be a question for Simon
> Josefsson probably.

It is not difficult to find out if a message is "too large", but the
problem is: what should nnimap do if the message is too large?  Throw
an error?  Return no data?  Query the user?  Return dummy data (that
will stick around in the agent and cached forever)?  I fear a backend
interface change would be request to properly support this...

Re: nnimap-split-download-body feature request

[Ted Zlatanov]

On Thu, 20 Nov 2003, jas@extundo.com wrote:

> It is not difficult to find out if a message is "too large", but the
> problem is: what should nnimap do if the message is too large?
> Throw an error?  Return no data?  Query the user?  Return dummy data
> (that will stick around in the agent and cached forever)?  I fear a
> backend interface change would be request to properly support
> this...

Nah, just fetch the headers.  I think that's all you can reasonably
expect as a Gnus user.  Maybe fake the body with "BODY TOO LARGE" or
something like that, or add a header, but I personally think that
retrieving just the headers in such a case is a perfectly good
solution.

This reminds me: is it possible to alter a message while splitting?
The reason I ask is that I'd like to insert a header that will tell
me, later, what the eventual destination of that message should be.
This is useful when the destination is "nnml:mail" (as returned by the
registry sometimes) but we're in the nnimap server.

If I can't insert headers, I can do it with the registry.

Ted

Re: nnimap-split-download-body feature request

[Simon Josefsson]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 20 Nov 2003, jas@extundo.com wrote:
>
>> It is not difficult to find out if a message is "too large", but the
>> problem is: what should nnimap do if the message is too large?
>> Throw an error?  Return no data?  Query the user?  Return dummy data
>> (that will stick around in the agent and cached forever)?  I fear a
>> backend interface change would be request to properly support
>> this...
>
> Nah, just fetch the headers.  I think that's all you can reasonably
> expect as a Gnus user.  Maybe fake the body with "BODY TOO LARGE" or
> something like that, or add a header, but I personally think that
> retrieving just the headers in such a case is a perfectly good
> solution.

The asynchronous prefetch, agent, cache (and possibly more things)
would cache this incomplete article.  How would they know the message
was incomplete?  After requesting a re-fetch of the entire article,
all those cached copies will need to be purged.  Sounds like work.

> This reminds me: is it possible to alter a message while splitting?

With nnimap, currently the answer is no.  It doesn't upload articles
to the server during splitting, it just use the COPY command.

Re: nnimap-split-download-body feature request

[Ted Zlatanov]

On Thu, 20 Nov 2003, jas@extundo.com wrote:

> Ted Zlatanov <tzz@lifelogs.com> writes:

>> Nah, just fetch the headers.  I think that's all you can reasonably
>> expect as a Gnus user.  Maybe fake the body with "BODY TOO LARGE"
>> or something like that, or add a header, but I personally think
>> that retrieving just the headers in such a case is a perfectly good
>> solution.
> 
> The asynchronous prefetch, agent, cache (and possibly more things)
> would cache this incomplete article.  How would they know the
> message was incomplete?  After requesting a re-fetch of the entire
> article, all those cached copies will need to be purged.  Sounds
> like work.

I'm not sure I understand.  Here's a patch to show you what I think
could be done, since nnimap-split-articles already decides between
the head and the whole body.  The nnimap-check-body-size function
needs to be provided, but I hope you see what I mean.

Ted.

--- nnimap.el   4 Sep 2003 22:22:18 -0000       6.71
+++ nnimap.el   20 Nov 2003 14:20:24 -0000
@@ -1271,9 +1271,10 @@
          (when (setq rule (nnimap-split-find-rule server inbox))
            ;; iterate over articles
            (dolist (article (imap-search nnimap-split-predicate))
-             (when (if (if (eq nnimap-split-download-body 'default)
-                           nnimap-split-download-body-default
-                         nnimap-split-download-body)
+             (when (if (and (nnimap-check-body-size article)
+                            (if (eq nnimap-split-download-body 'default)
+                                nnimap-split-download-body-default
+                              nnimap-split-download-body))
                        (and (nnimap-request-article article)
                             (with-current-buffer nntp-server-buffer (mail-narrow-to-head)))
                      (nnimap-request-head article))

Re: nnimap-split-download-body feature request

[Simon Josefsson]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 20 Nov 2003, jas@extundo.com wrote:
>
>> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>>> Nah, just fetch the headers.  I think that's all you can reasonably
>>> expect as a Gnus user.  Maybe fake the body with "BODY TOO LARGE"
>>> or something like that, or add a header, but I personally think
>>> that retrieving just the headers in such a case is a perfectly good
>>> solution.
>> 
>> The asynchronous prefetch, agent, cache (and possibly more things)
>> would cache this incomplete article.  How would they know the
>> message was incomplete?  After requesting a re-fetch of the entire
>> article, all those cached copies will need to be purged.  Sounds
>> like work.
>
> I'm not sure I understand.  Here's a patch to show you what I think
> could be done, since nnimap-split-articles already decides between
> the head and the whole body.  The nnimap-check-body-size function
> needs to be provided, but I hope you see what I mean.

Ah!  Sorry, I forgot we were still talking about nnimap split.  Yes,
it sounds like a good idea.  nnimap-check-body-size would probably
look like (< (imap-fetch ARTICLE "RFC822.SIZE" 'RFC822.SIZE)
nnimap-split-download-body-size) or something.