Automatically processing plain text attachments

Automatically processing plain text attachments

[Richard Lawrence]

Hi all,

[I'm new to the list, so apologies if this is a rather newb-ish question.
I looked in the manual, and tried to search the archives at Gmane, but
didn't come up with much, perhaps because I'm not exactly certain which
terms I should be using.  Pointers to documentation of all kinds would
be greatly appreciated...]

I'm hoping I can solicit a little advice about how to pull plain text
email attachments off of a subset of incoming messages and do some batch
processing on them, without seriously slowing down the rest of my mail
reading.

The background: I am about to begin teaching a writing-intensive course.
Students will email me their papers every week.  I have no desire to
download, print, and read a bunch of .doc files by hand every week.  So
I am considering asking my students to email their papers in plain text.
I would like to then apply some automated processing on my end that
would:

- download each student's paper 
- apply some (hopefully) simple transformations on the text
- save the resulting document in my "teaching" directory

The goal is to have these papers end up in Org mode format, so I can do
further batch processing that will export them using LaTeX, add them to
my to-do list, etc.  Since I will have about 100 papers to read over the
course of this semester, it seems like automating this is the right way
to go.

I currently use a very simple Gnus setup.  I am running Gnus 5.13 in
Emacs 23 on Debian.  I read my mail over nnimap.

Here's what I was thinking might work: 

1) Tell my students that they must indicate in the subject line of their
email that it contains a paper submission, so I can split those emails
to a special group using nnimap-split-rule.

2) Somehow, further process the emails that end up in that group.  For
example:
 - the group could represent a local Maildir, and I could have a cron job
   process new emails found there
 - the group could represent an IMAP folder, and when I read articles in that
   group, Gnus runs a hook that extracts attachments, processes them
   in a temporary buffer, and saves the result to a file (or perhaps
   refiles in an existing Org file)

Perhaps my question boils down to: what hooks are available in Gnus that
would allow me batch process the full content of these paper-submission
emails at the time they are split from my inbox (without having to
download the full content of *every* mail, e.g., by setting
nnimap-split-download-body)?

Thanks so much for any advice you can offer!

Best,
Richard

Re: Automatically processing plain text attachments

[Lars Ingebrigtsen]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Perhaps my question boils down to: what hooks are available in Gnus that
> would allow me batch process the full content of these paper-submission
> emails at the time they are split from my inbox (without having to
> download the full content of *every* mail, e.g., by setting
> nnimap-split-download-body)?

Hm...  I don't think there's any easy way to hook that up.  The
-download-body thing is done sort of "top level" before noticing what
groups the articles are supposed to go to.

The only this that occurs to me immediately is to look at the
`nnmail-split-trace' variable after the split has finished.  If new
messages has shown up in your special group, then do, er, stuff based on
that.  For instance, request the new messages one by one, and run some
filtering stuff...

It's just a small matter of programming.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Automatically processing plain text attachments

[Richard Lawrence]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> Perhaps my question boils down to: what hooks are available in Gnus that
>> would allow me batch process the full content of these paper-submission
>> emails at the time they are split from my inbox (without having to
>> download the full content of *every* mail, e.g., by setting
>> nnimap-split-download-body)?
>
> Hm...  I don't think there's any easy way to hook that up.  The
> -download-body thing is done sort of "top level" before noticing what
> groups the articles are supposed to go to.

I guess it wouldn't be *too* big a deal to download the body of all
messages...I don't get that much mail, thanks to the combination of Gnus
and Gmane...

In that case, would I be able to do what I want by installing a function
as the second element of my split rule in nnimap-split-rule?  Would this
function be called in a buffer containing the entire message body?

Another option would be to do this semi-interactively, which may be what
I want anyway.  Is it easier to call a hook on a message when it is
*read*, rather than when it is split?

> The only this that occurs to me immediately is to look at the
> `nnmail-split-trace' variable after the split has finished.  If new
> messages has shown up in your special group, then do, er, stuff based on
> that.  For instance, request the new messages one by one, and run some
> filtering stuff...

Where would I look to learn how to do these things?  Is there a guide to
hacking somewhere?  Or should I just dive into the Gnus code?

> It's just a small matter of programming.  :-)

Indeed.  

Thanks for your help, Lars!

Richard

Re: Automatically processing plain text attachments

[Lars Ingebrigtsen]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> In that case, would I be able to do what I want by installing a function
> as the second element of my split rule in nnimap-split-rule?  Would this
> function be called in a buffer containing the entire message body?

Yes, I think so.

> Another option would be to do this semi-interactively, which may be what
> I want anyway.  Is it easier to call a hook on a message when it is
> *read*, rather than when it is split?

Yes, that's a much more controlled situation.  There's a bazillion
article mode hooks/functions you can use, depending on what's most
convenient. 

> Where would I look to learn how to do these things?  Is there a guide to
> hacking somewhere?  Or should I just dive into the Gnus code?

I'd suggest looking at the manual for what hook to use, and then look at
the code.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen