Email folders and threading

Email folders and threading

[Jack Twilley]

I've noticed something frustrating about extended email conversations.

This is what the summary buffer looks like for
nnml:people.Charlotte_Evans:

OA [  40: +Charlotte Evans     ] conversation
Q      [   0:                      ] 
OA         [  27: +Charlotte Evans     ] 

Here's what the References: header looks like in the second message:

References: <01e001bf1a7a$09ab3250$2280d0d1@evolution.com> <m3wvsjowcq.fsf@duchess.twilley.org>

So the second message knows that there's a message between it and the
first message -- namely, my reply to the first message, which is
sitting in nnfolder+archive:mail.1999-10.

WIBNI the summary buffer for nnml:people.Charlotte_Evans looked like
this: 

OA [  40: +Charlotte Evans     ] conversation
Q      [  48: +Jack Twilley        ]
OA         [  27: +Charlotte Evans     ] 

And that the summary buffer for nnfolder+archive:mail.1999-10 looked
like this:

O  [  48: +Jack Twilley        ] Re: conversation
Q      [  27: +Charlotte Evans     ]
O +        [  57: +Jack Twilley        ] 

Or maybe even like this:

OA [  40: +Charlotte Evans     ] conversation
Q      [  48: +Jack Twilley        ]
OA         [  27: +Charlotte Evans     ] 
O +            [  57: +Jack Twilley        ] 

After all, the last message has all the references.

This might require the creation of some big hash of message-id's, such
that $hash{message-id} = "protocol+server:folder[articleno]" or
something to that effect.  

Thoughts?

Jack.
(I'm better at perl than elisp.)
-- 
Jack Twilley
jmt at nycap dot rr dot com
http colon slash slash jmt dot dhs dot org slash tilde jmt slash

Re: Email folders and threading

[Kai Großjohann]

Jack Twilley <jmt+usenet@nycap.rr.com> writes:

> [[ wants Gnus to add missing overview lines from other groups ]]

Why don't you save outgoing messages in the same group as incoming
ones?  This would solve your problems, I think.

kai
-- 
This gubblick contains many nonsklarkish English flutzpahs,
but the overall pluggandisp can be glorked from context. -- David Moser

Re: Email folders and threading

[Jack Twilley]

>>>>> "Kai" == Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

Kai> Why don't you save outgoing messages in the same group as
Kai> incoming ones?  This would solve your problems, I think.

I want all my outgoing mail to be saved in
nnfolder+archive:mail.year-mo and that works pretty well.

It just seems that since I have all the overview information for my
own emails, the summary buffer should fill in instead of leaving
gaping holes.

Jack.
-- 
Jack Twilley
jmt at nycap dot rr dot com
http colon slash slash jmt dot dhs dot org slash tilde jmt slash

Re: Email folders and threading

[Lars Magne Ingebrigtsen]

Jack Twilley <jmt+usenet@nycap.rr.com> writes:

> This might require the creation of some big hash of message-id's, such
> that $hash{message-id} = "protocol+server:folder[articleno]" or
> something to that effect.  
> 
> Thoughts?

You could stuff that hash-adding function down into
`message-sent-hook'.  And for display, you could just do a
user-defined format spec.  Shouldn't be too complicated to do. 

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Email folders and threading

[Kai Großjohann]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Jack Twilley <jmt+usenet@nycap.rr.com> writes:
> 
> > This might require the creation of some big hash of message-id's, such
> > that $hash{message-id} = "protocol+server:folder[articleno]" or
> > something to that effect.  
> 
> You could stuff that hash-adding function down into
> `message-sent-hook'.  And for display, you could just do a
> user-defined format spec.  Shouldn't be too complicated to do. 

Hm.  It turns out that my little nnmail.el patch already adds the
group name to ~/.nnmail-cache.  I guess it wouldn't be hard to make it
add the server name as well as the article number, too.

Patch below.  Information on what the heck it's supposed to do will
gladly be provided on request.

kai
-- 
This gubblick contains many nonsklarkish English flutzpahs,
but the overall pluggandisp can be glorked from context. -- David Moser


[-- Attachment #2: Patch for adding info to .nnmail-cache --]
[-- Type: application/emacs-lisp, Size: 3389 bytes --]

Re: Email folders and threading

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Hm.  It turns out that my little nnmail.el patch already adds the
> group name to ~/.nnmail-cache.  I guess it wouldn't be hard to make it
> add the server name as well as the article number, too.
> 
> Patch below.  Information on what the heck it's supposed to do will
> gladly be provided on request.

*request*  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Email folders and threading

[Kai Großjohann]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > Patch below.  Information on what the heck it's supposed to do will
> > gladly be provided on request.
> 
> *request*  :-)

Ok.  I have split methods, but I find that the split methods do not
catch all cases, so I manually move some messages.  I want followups
to those manually-moved messages in the same group where the
manually-moved message is.

So the patch has two parts: The first part frobs the ~/.nnmail-cache
handling such that, in addition to the message id, the group name is
entered into the file.  The second part is a function which can use
used in nnmail-split-fancy.  It looks at the References header and
searches a group name for the message ids mentioned there.  If it
finds one, the group is returned, else nil is returned.

Well, actually the group name is only put in ~/.nnmail-cache if the
group comes from the mail-receiving mail backend, since mail splitting
only applies to the mail-receiving mail backend.  Hence, the group
name is without a server name, ie foo.bar rather than nnml:foo.bar.

I forgot what exactly the thread was about, but I think for the
purposes of the thread, you may wish to enter both the server name (ie
nnml:foo.bar) and the article number into the message id cache.

What would you like to do now?  What would you like me to do now?

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: Email folders and threading

[Toby Speight]

Kai> Kai Großjohann <URL:mailto:Kai.Grossjohann@CS.Uni-Dortmund.DE>

0> In article <vafu2m2tq9n.fsf@lucy.cs.uni-dortmund.de>, Kai wrote:

Kai> Well, actually the group name is only put in ~/.nnmail-cache if
Kai> the group comes from the mail-receiving mail backend, since mail
Kai> splitting only applies to the mail-receiving mail backend.  Hence,
Kai> the group name is without a server name, ie foo.bar rather than
Kai> nnml:foo.bar.

I have both POP (nnml) and IMAP backends; will it still work?  (I
currently do the IMAP splitting on the server, but would like to do it
with Gnus; having this functionality makes Gnus splitting even more
attractive).

Re: Email folders and threading

[Simon Josefsson]

Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

> Kai> Well, actually the group name is only put in ~/.nnmail-cache if
> Kai> the group comes from the mail-receiving mail backend, since mail
> Kai> splitting only applies to the mail-receiving mail backend.  Hence,
> Kai> the group name is without a server name, ie foo.bar rather than
> Kai> nnml:foo.bar.
> 
> I have both POP (nnml) and IMAP backends; will it still work?

The patch doesn't modify the nnimap splitter, I think, so; no.

OTOH I don't think it would be very difficult to port it the nnimap
splitter (it's basicly the nnmail stuff ripped out) so if/when the
patch is applied to Gnus I could take a look at how hard it would be
to port it to the nnimap splitter.

Re: Email folders and threading

[Kai Großjohann]

Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

> I have both POP (nnml) and IMAP backends; will it still work?  (I
> currently do the IMAP splitting on the server, but would like to do it
> with Gnus; having this functionality makes Gnus splitting even more
> attractive).

There was some talk on the nnimap mailing list about this.  The idea
was to maintain the ~/.nnmail-cache information in an nnimap group, I
think.

I thought that it would be cool if the `split followups with parents'
feature was available for all IMAP clients, though, so I thought about
(and suggested) an extension of Cyrus and its Sieve implementation.
Nothing has become of this, though.

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: Email folders and threading

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> So the patch has two parts: The first part frobs the ~/.nnmail-cache
> handling such that, in addition to the message id, the group name is
> entered into the file.  The second part is a function which can use
> used in nnmail-split-fancy.  It looks at the References header and
> searches a group name for the message ids mentioned there.  If it
> finds one, the group is returned, else nil is returned.

I see.  Sounds useful.  Could you re-send me the patch against 5.8?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Email folders and threading

[Kai Großjohann]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I see.  Sounds useful.  Could you re-send me the patch against 5.8?

The patch that worked for pgnus 0.64 or thereabouts has continued to
work for me right up to 0.97 (or was it 0.98?  when CVS happened).
But I'll test if it still works against the current CVS.

Yes, it still works:

    patching file `lisp/nnmail.el'
    Hunk #1 succeeded at 1311 (offset -178 lines).

Patch attached.

kai
-- 
A preposition is not a good thing to end a sentence with.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch against pgnus 0.63 nnmail.el still works --]
[-- Type: text/x-patch, Size: 3389 bytes --]

--- lisp/nnmail.el.orig	Mon Nov 30 22:47:02 1998
+++ lisp/nnmail.el	Fri Dec  4 13:34:02 1998
@@ -1489,12 +1489,78 @@
 
 (defun nnmail-cache-insert (id)
   (when nnmail-treat-duplicates
-    (unless (gnus-buffer-live-p nnmail-cache-buffer)
-      (nnmail-cache-open))
+    ;; Store some information about the group this message is written
+    ;; to.  This function might have been called from various places.
+    ;; Sometimes, a function up in the calling sequence has an
+    ;; argument GROUP which is bound to a string, the group name.  At
+    ;; other times, there is a function up in the calling sequence
+    ;; which has an argument GROUP-ART which is a list of pairs, and
+    ;; the car of a pair is a group name.  Should we check that the
+    ;; length of the list is equal to 1? -- kai
+    (let ((g nil))
+      (cond ((and (boundp 'group) group)
+             (setq g group))
+            ((and (boundp 'group-art-list) group-art-list
+                  (listp group-art-list))
+             (setq g (caar group-art-list)))
+            ((and (boundp 'group-art) group-art (listp group-art))
+             (setq g (caar group-art)))
+            (t (setq g "")))
+      (unless (gnus-buffer-live-p nnmail-cache-buffer)
+        (nnmail-cache-open))
+      (save-excursion
+        (set-buffer nnmail-cache-buffer)
+        (goto-char (point-max))
+        (if (and g (not (string= "" g))
+                 (gnus-methods-equal-p gnus-command-method
+                                       (nnmail-cache-primary-mail-backend)))
+            (insert id "\t" g "\n")
+          (insert id "\n"))))))
+
+(defun nnmail-cache-primary-mail-backend ()
+  (let ((be-list (cons gnus-select-method gnus-secondary-select-methods))
+        (be nil)
+        (res nil))
+    (while (and (null res) be-list)
+      (setq be (car be-list))
+      (setq be-list (cdr be-list))
+      (when (and (gnus-method-option-p be 'respool)
+                 (eval (intern (format "%s-get-new-mail" (car be)))))
+        (setq res be)))
+    res))
+
+;; Fetch the group name corresponding to the message id stored in the
+;; cache.
+(defun nnmail-cache-fetch-group (id)
+  (when (and nnmail-treat-duplicates nnmail-cache-buffer)
     (save-excursion
       (set-buffer nnmail-cache-buffer)
       (goto-char (point-max))
-      (insert id "\n"))))
+      (when (search-backward id nil t)
+        (beginning-of-line)
+        (skip-chars-forward "^\n\r\t")
+        (unless (eolp)
+          (forward-char 1)
+          (buffer-substring (point)
+                            (progn (end-of-line) (point))))))))
+
+;; Function for nnmail-split-fancy: look up all references in the
+;; cache and if a match is found, return that group.
+(defun nnmail-split-fancy-with-parent ()
+  (let* ((refstr (or (message-fetch-field "references")
+                     (message-fetch-field "in-reply-to")))
+         (references nil)
+         (res nil))
+    (when refstr
+      (setq references (nreverse (gnus-split-references refstr)))
+      (unless (gnus-buffer-live-p nnmail-cache-buffer)
+        (nnmail-cache-open))
+      (mapcar (lambda (x)
+                (setq res (or (nnmail-cache-fetch-group x) res))
+                (when (string= "drafts" res)
+                  (setq res nil)))
+              references)
+      res)))
 
 (defun nnmail-cache-id-exists-p (id)
   (when nnmail-treat-duplicates