* mail group by author? @ 2009-03-30 9:22 Stephen Berman 2009-03-30 11:57 ` David Engster 0 siblings, 1 reply; 22+ messages in thread From: Stephen Berman @ 2009-03-30 9:22 UTC (permalink / raw) To: ding Is it possible to make a temporary group consisting of mails by a specific person whose mail articles may be scattered across many different groups? I would like to do this to this to be able to see all mail by given people, regardless of which groups the individual mails are stored in. Steve Berman ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 9:22 mail group by author? Stephen Berman @ 2009-03-30 11:57 ` David Engster 2009-03-30 12:49 ` Tassilo Horn 2009-04-14 16:39 ` Ted Zlatanov 0 siblings, 2 replies; 22+ messages in thread From: David Engster @ 2009-03-30 11:57 UTC (permalink / raw) To: ding Stephen Berman <stephen.berman@gmx.net> writes: > Is it possible to make a temporary group consisting of mails by a > specific person whose mail articles may be scattered across many > different groups? I would like to do this to this to be able to see all > mail by given people, regardless of which groups the individual mails > are stored in. nnkiboze should be able to do this, but I for one never got it to work with my mail groups. It is also pretty slow. If you store your mails locally, you can set up mairix to index them and use the nnmairix back end to create search groups like you describe. There is also the nnir search back end which works with several different search engines, including the IMAP search features, so if you're using nnimap this might be the easiest solution (nnmairix can be used via ssh, but that's a bit cumbersome to set up). Maybe the Gnus registry could be used for creating such temporary groups, but as far as I know there's nothing implemented yet that does what you want right away. -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 11:57 ` David Engster @ 2009-03-30 12:49 ` Tassilo Horn 2009-03-30 13:13 ` David Engster 2009-03-30 15:59 ` Andreas Schwab 2009-04-14 16:39 ` Ted Zlatanov 1 sibling, 2 replies; 22+ messages in thread From: Tassilo Horn @ 2009-03-30 12:49 UTC (permalink / raw) To: ding David Engster <deng@randomsample.de> writes: Hi David, > There is also the nnir search back end which works with several > different search engines, including the IMAP search features, so if > you're using nnimap this might be the easiest solution (nnmairix can > be used via ssh, but that's a bit cumbersome to set up). nnir seaches only in the group point is on. Bye, Tassilo -- Whenever Richard Stallman points at a Windows computer, it segfaults. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 12:49 ` Tassilo Horn @ 2009-03-30 13:13 ` David Engster 2009-03-30 15:25 ` Tassilo Horn 2009-03-30 15:59 ` Andreas Schwab 1 sibling, 1 reply; 22+ messages in thread From: David Engster @ 2009-03-30 13:13 UTC (permalink / raw) To: ding Tassilo Horn <tassilo@member.fsf.org> writes: > David Engster <deng@randomsample.de> writes: >> There is also the nnir search back end which works with several >> different search engines, including the IMAP search features, so if >> you're using nnimap this might be the easiest solution (nnmairix can >> be used via ssh, but that's a bit cumbersome to set up). > > nnir seaches only in the group point is on. Oh. Thanks for the correction. Do you happen to know if this is a principal restriction in nnir, or only when using the IMAP search back end? -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 13:13 ` David Engster @ 2009-03-30 15:25 ` Tassilo Horn 0 siblings, 0 replies; 22+ messages in thread From: Tassilo Horn @ 2009-03-30 15:25 UTC (permalink / raw) To: ding David Engster <deng@randomsample.de> writes: Hi David, >>> There is also the nnir search back end which works with several >>> different search engines, including the IMAP search features, so if >>> you're using nnimap this might be the easiest solution (nnmairix can >>> be used via ssh, but that's a bit cumbersome to set up). >> >> nnir seaches only in the group point is on. > > Oh. Thanks for the correction. Do you happen to know if this is a > principal restriction in nnir, or only when using the IMAP search back > end? I don't know, I use it only for searching nnimap groups. But I think about switching to nnmairix anyway: - It's much faster (the imap search on my local dovecot server seems to do simple grepping, no indexing involved) - I get strange results for `G G Tassilo Horn RET'. It seems to find all mails in the group, although most of them don't contain my name, neither in any header nor in the body. If I restrict the search to some field using `C-u G G <query> RET <field>' it seems to work, though... Bye, Tassilo -- My opinions may have changed, but not the fact that I am right. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 12:49 ` Tassilo Horn 2009-03-30 13:13 ` David Engster @ 2009-03-30 15:59 ` Andreas Schwab 2009-03-31 6:42 ` Tassilo Horn 1 sibling, 1 reply; 22+ messages in thread From: Andreas Schwab @ 2009-03-30 15:59 UTC (permalink / raw) To: ding Tassilo Horn <tassilo@member.fsf.org> writes: > David Engster <deng@randomsample.de> writes: > > Hi David, > >> There is also the nnir search back end which works with several >> different search engines, including the IMAP search features, so if >> you're using nnimap this might be the easiest solution (nnmairix can >> be used via ssh, but that's a bit cumbersome to set up). > > nnir seaches only in the group point is on. The last time I used it (with namazu) it searched whatever the search index covers. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 15:59 ` Andreas Schwab @ 2009-03-31 6:42 ` Tassilo Horn 2009-03-31 9:18 ` David Engster 2009-03-31 9:52 ` mail group by author? Vegard Vesterheim 0 siblings, 2 replies; 22+ messages in thread From: Tassilo Horn @ 2009-03-31 6:42 UTC (permalink / raw) To: ding Andreas Schwab <schwab@linux-m68k.org> writes: Hi Andreas, >>> There is also the nnir search back end which works with several >>> different search engines, including the IMAP search features, so if >>> you're using nnimap this might be the easiest solution (nnmairix can >>> be used via ssh, but that's a bit cumbersome to set up). >> >> nnir seaches only in the group point is on. > > The last time I used it (with namazu) it searched whatever the search > index covers. Yes, this may be different for indexing search engines. But for nnimap it seems to be local for a group. (But maybe this can be changed with some configuration bits.) Bye, Tassilo ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-31 6:42 ` Tassilo Horn @ 2009-03-31 9:18 ` David Engster 2009-03-31 10:36 ` Dovecot 1.2 virtual mailboxes (was: mail group by author?) Tassilo Horn 2009-03-31 9:52 ` mail group by author? Vegard Vesterheim 1 sibling, 1 reply; 22+ messages in thread From: David Engster @ 2009-03-31 9:18 UTC (permalink / raw) To: ding Tassilo Horn <tassilo@member.fsf.org> writes: > Andreas Schwab <schwab@linux-m68k.org> writes: > > Hi Andreas, > >>>> There is also the nnir search back end which works with several >>>> different search engines, including the IMAP search features, so if >>>> you're using nnimap this might be the easiest solution (nnmairix can >>>> be used via ssh, but that's a bit cumbersome to set up). >>> >>> nnir seaches only in the group point is on. >> >> The last time I used it (with namazu) it searched whatever the search >> index covers. > > Yes, this may be different for indexing search engines. But for nnimap > it seems to be local for a group. (But maybe this can be changed with > some configuration bits.) It seems according to IMAPrev4 the SEARCH command only works in the currently selected mailbox, so this is primarily a restriction in the IMAP protocol. Of course, one could extend nnir to select several mailboxes, do a SEARCH in each of them and then combine the results. BTW, for people using Dovecot: version 1.2 (still in beta) has a new "virtual mailbox" extension which looks pretty interesting for this kind of stuff: http://wiki.dovecot.org/Plugins/Virtual -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Dovecot 1.2 virtual mailboxes (was: mail group by author?) 2009-03-31 9:18 ` David Engster @ 2009-03-31 10:36 ` Tassilo Horn 2009-03-31 11:35 ` Dovecot 1.2 virtual mailboxes David Engster 0 siblings, 1 reply; 22+ messages in thread From: Tassilo Horn @ 2009-03-31 10:36 UTC (permalink / raw) To: ding David Engster <deng@randomsample.de> writes: Hi David, > BTW, for people using Dovecot: version 1.2 (still in beta) has a new > "virtual mailbox" extension which looks pretty interesting for this > kind of stuff: > > http://wiki.dovecot.org/Plugins/Virtual Looks quite similar to kiboze groups, right? I imagine that I could benefit from that feature by creating virtual groups for my most important contacts, but I guess it would conflict with things like `gcc-self', i.e. when I reply to a mail in a virtual group I want to gcc the original group, not the virtual one. But probably the registry could jump in here (at least if I've read the message in the original group before and omit virtual groups from registration). Bye, Tassilo ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Dovecot 1.2 virtual mailboxes 2009-03-31 10:36 ` Dovecot 1.2 virtual mailboxes (was: mail group by author?) Tassilo Horn @ 2009-03-31 11:35 ` David Engster 2009-03-31 12:23 ` Tassilo Horn 0 siblings, 1 reply; 22+ messages in thread From: David Engster @ 2009-03-31 11:35 UTC (permalink / raw) To: ding Tassilo Horn <tassilo@member.fsf.org> writes: > David Engster <deng@randomsample.de> writes: >> BTW, for people using Dovecot: version 1.2 (still in beta) has a new >> "virtual mailbox" extension which looks pretty interesting for this >> kind of stuff: >> >> http://wiki.dovecot.org/Plugins/Virtual > > Looks quite similar to kiboze groups, right? It seems so. I didn't try it yet, though. > I imagine that I could benefit from that feature by creating virtual > groups for my most important contacts, but I guess it would conflict > with things like `gcc-self', i.e. when I reply to a mail in a virtual > group I want to gcc the original group, not the virtual one. > > But probably the registry could jump in here (at least if I've read the > message in the original group before and omit virtual groups from > registration). Yes, I guess this could be scripted somehow, since the virtual groups are in their own namespace anyway. But indeed, the registry would have to see the original message in the original group beforehand. I already struggled with that problem with nnmairix, and don't yet know how to solve that efficiently, especially since I'm splitting with procmail on the server and not with Gnus. -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Dovecot 1.2 virtual mailboxes 2009-03-31 11:35 ` Dovecot 1.2 virtual mailboxes David Engster @ 2009-03-31 12:23 ` Tassilo Horn 0 siblings, 0 replies; 22+ messages in thread From: Tassilo Horn @ 2009-03-31 12:23 UTC (permalink / raw) To: ding David Engster <deng@randomsample.de> writes: Hi David, >> But probably the registry could jump in here (at least if I've read >> the message in the original group before and omit virtual groups from >> registration). > > Yes, I guess this could be scripted somehow, since the virtual groups > are in their own namespace anyway. But indeed, the registry would have > to see the original message in the original group beforehand. I > already struggled with that problem with nnmairix, and don't yet know > how to solve that efficiently, especially since I'm splitting with > procmail on the server and not with Gnus. Yes, I do so with SIEVE, too + some very restricted local fancy splitting with gnus-registry-split-fancy-with-parent. Therefore it would be nice to do the registration of messages not on reading them but somewhere after fetching and splitting. Bye, Tassilo ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-31 6:42 ` Tassilo Horn 2009-03-31 9:18 ` David Engster @ 2009-03-31 9:52 ` Vegard Vesterheim 2009-03-31 10:25 ` Tassilo Horn 1 sibling, 1 reply; 22+ messages in thread From: Vegard Vesterheim @ 2009-03-31 9:52 UTC (permalink / raw) To: ding On Tue, 31 Mar 2009 08:42:59 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: > Andreas Schwab <schwab@linux-m68k.org> writes: > > Hi Andreas, > >>>> There is also the nnir search back end which works with several >>>> different search engines, including the IMAP search features, so if >>>> you're using nnimap this might be the easiest solution (nnmairix can >>>> be used via ssh, but that's a bit cumbersome to set up). >>> >>> nnir seaches only in the group point is on. >> >> The last time I used it (with namazu) it searched whatever the search >> index covers. > > Yes, this may be different for indexing search engines. But for nnimap > it seems to be local for a group. (But maybe this can be changed with > some configuration bits.) It is possible to mark (gnus-topic-mark-topic) several groups, and issue a nnimap search (gnus-group-make-nnir-group) across all marked groups. - Vegard V - ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-31 9:52 ` mail group by author? Vegard Vesterheim @ 2009-03-31 10:25 ` Tassilo Horn 0 siblings, 0 replies; 22+ messages in thread From: Tassilo Horn @ 2009-03-31 10:25 UTC (permalink / raw) To: Vegard Vesterheim; +Cc: ding Vegard Vesterheim <vegard.vesterheim@uninett.no> writes: Hi Vegard, >>>> nnir seaches only in the group point is on. >>> >>> The last time I used it (with namazu) it searched whatever the >>> search index covers. >> >> Yes, this may be different for indexing search engines. But for >> nnimap it seems to be local for a group. (But maybe this can be >> changed with some configuration bits.) > > It is possible to mark (gnus-topic-mark-topic) several groups, and > issue a nnimap search (gnus-group-make-nnir-group) across all marked > groups. Hey, excellent! Thanks for the tip. Bye, Tassilo -- My software never has bugs. It just develops random features. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-03-30 11:57 ` David Engster 2009-03-30 12:49 ` Tassilo Horn @ 2009-04-14 16:39 ` Ted Zlatanov 2009-04-17 9:00 ` David Engster 1 sibling, 1 reply; 22+ messages in thread From: Ted Zlatanov @ 2009-04-14 16:39 UTC (permalink / raw) To: ding On Mon, 30 Mar 2009 13:57:02 +0200 David Engster <deng@randomsample.de> wrote: DE> Stephen Berman <stephen.berman@gmx.net> writes: >> Is it possible to make a temporary group consisting of mails by a >> specific person whose mail articles may be scattered across many >> different groups? I would like to do this to this to be able to see all >> mail by given people, regardless of which groups the individual mails >> are stored in. DE> Maybe the Gnus registry could be used for creating such temporary DE> groups, but as far as I know there's nothing implemented yet that does DE> what you want right away. This is really easy to implement as far as locating the message IDs and article groups+numbers. The hard part is to make all those articles into one group, I don't know how to do that. But I can write the first part if you need it. On Tue, 31 Mar 2009 13:35:10 +0200 David Engster <deng@randomsample.de> wrote: DE> Tassilo Horn <tassilo@member.fsf.org> writes: >> But probably the registry could jump in here (at least if I've read the >> message in the original group before and omit virtual groups from >> registration). DE> Yes, I guess this could be scripted somehow, since the virtual DE> groups are in their own namespace anyway. But indeed, the registry DE> would have to see the original message in the original group DE> beforehand. I already struggled with that problem with nnmairix, and DE> don't yet know how to solve that efficiently, especially since I'm DE> splitting with procmail on the server and not with Gnus. The registry could periodically visit all groups to see new messages. But that's messy. It could also read a "dribble" file with the message IDs and groups it needs to refresh. The gnus.registry.eld file could live on the server and get rsynced to the clients or opened remotely through Tramp. Since the registry is pure data storage, it is not hard to serialize it to and from a database. Registry queries currently assume the data structure is local, but that can be easily modified to query the remote. A remote DB may in fact be the best option, either something relaxed like CouchDB or SimpleDB, or something more constrained like a standard RDBMS schema. A benefit of CouchDB or SimpleDB is that users could make their registry publically readable, so I'm in favor of that. Tying mairix and other search backends into the same DB is also a good idea. On Tue, 31 Mar 2009 14:23:19 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: TH> Yes, I do so with SIEVE, too + some very restricted local fancy TH> splitting with gnus-registry-split-fancy-with-parent. Therefore it TH> would be nice to do the registration of messages not on reading them but TH> somewhere after fetching and splitting. You mean after SIEVE splitting? I don't know much about SIEVE on the server, can you hook scripts to a step? If yes it may be possible to tie it into the above database discussion. Ted ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-14 16:39 ` Ted Zlatanov @ 2009-04-17 9:00 ` David Engster 2009-04-17 10:07 ` Vegard Vesterheim 2009-04-17 16:53 ` mail group by author? Ted Zlatanov 0 siblings, 2 replies; 22+ messages in thread From: David Engster @ 2009-04-17 9:00 UTC (permalink / raw) To: ding Ted Zlatanov <tzz@lifelogs.com> writes: > On Mon, 30 Mar 2009 13:57:02 +0200 David Engster <deng@randomsample.de> wrote: > > DE> Stephen Berman <stephen.berman@gmx.net> writes: >>> Is it possible to make a temporary group consisting of mails by a >>> specific person whose mail articles may be scattered across many >>> different groups? I would like to do this to this to be able to see all >>> mail by given people, regardless of which groups the individual mails >>> are stored in. > > DE> Maybe the Gnus registry could be used for creating such temporary > DE> groups, but as far as I know there's nothing implemented yet that does > DE> what you want right away. > > This is really easy to implement as far as locating the message IDs and > article groups+numbers. Yes, I tested that some time ago using maphash, and it was pretty fast. My idea was to extend nnmairix to also build groups containing all messages which have a certain label. I still plan on doing that when the feature freeze is over. > The hard part is to make all those articles into one group, I don't > know how to do that. But I can write the first part if you need it. Making these kinds of "virtual" groups needs a back end which implements the Gnus API functions. This is already implemented in nnir, so I think extending nnir to also be able to query the registry shouldn't be difficult to do. > On Tue, 31 Mar 2009 13:35:10 +0200 David Engster <deng@randomsample.de> wrote: > DE> Tassilo Horn <tassilo@member.fsf.org> writes: >>> But probably the registry could jump in here (at least if I've read the >>> message in the original group before and omit virtual groups from >>> registration). > > DE> Yes, I guess this could be scripted somehow, since the virtual > DE> groups are in their own namespace anyway. But indeed, the registry > DE> would have to see the original message in the original group > DE> beforehand. I already struggled with that problem with nnmairix, and > DE> don't yet know how to solve that efficiently, especially since I'm > DE> splitting with procmail on the server and not with Gnus. > > The registry could periodically visit all groups to see new messages. > But that's messy. It could also read a "dribble" file with the message > IDs and groups it needs to refresh. The gnus.registry.eld file could > live on the server and get rsynced to the clients or opened remotely > through Tramp. Visiting all groups for new messages surely isn't nice, but I'd still prefer it because it's self-contained. One could call this in the gnus-after-getting-new-news-hook. Using external dribble files or scripts is probably faster and more elegant, but also more difficult to set up. > Since the registry is pure data storage, it is not hard to serialize it > to and from a database. Registry queries currently assume the data > structure is local, but that can be easily modified to query the remote. > > A remote DB may in fact be the best option, either something relaxed > like CouchDB or SimpleDB, or something more constrained like a standard > RDBMS schema. A benefit of CouchDB or SimpleDB is that users could make > their registry publically readable, so I'm in favor of that. Tying > mairix and other search backends into the same DB is also a good idea. Well, having one general database for all search related things would surely be a nice thing to have. But searching is just one feature; what I really wanted to have in Gnus is something resembling "Filters" in Opera Mail. I remember a message from Kai Großjohann years ago where he praised Opera for this feature. It's an impressive mail reader, and using filters instead of splitting immediately made sense to me. And most importantly, filtering is not restricted to the mail headers. You can filter with a full text search, including MIME parsing. For example, Opera directly shows you groups containing all attachments you have received, filtered according to type. To do something like that in Gnus, you need an external indexer. Mairix is just one possibility, but it's well suited for Gnus because it directly creates the resulting mailbox, so you don't have to parse large amounts of output in Emacs Lisp. Alternatively, you could surely also use search engines like Lucene, Xapian, Sphinx, etc., but those would need much more work to interface them with Gnus. Regards, David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 9:00 ` David Engster @ 2009-04-17 10:07 ` Vegard Vesterheim 2009-04-17 10:46 ` David Engster 2009-04-17 16:53 ` mail group by author? Ted Zlatanov 1 sibling, 1 reply; 22+ messages in thread From: Vegard Vesterheim @ 2009-04-17 10:07 UTC (permalink / raw) To: ding On Fri, 17 Apr 2009 11:00:16 +0200 David Engster <deng@randomsample.de> wrote: > To do something like that in Gnus, you need an external indexer. Mairix > is just one possibility, but it's well suited for Gnus because it > directly creates the resulting mailbox, so you don't have to parse large > amounts of output in Emacs Lisp. Alternatively, you could surely also > use search engines like Lucene, Xapian, Sphinx, etc., but those would > need much more work to interface them with Gnus. I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend for retrieving messages. Storing headers (and body) for mail messages into a database, opens possibilities for flexible and efficient queries and retrieval of email messages. One of the missing parts seems to be support for Virtual Folders: http://dbmail.org/dokuwiki/doku.php?id=v-folders http://tools.ietf.org/id/draft-ietf-lemonade-vfolder-01.txt Has anyone tried the combination dbmail/GNUS? - Vegard V - ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 10:07 ` Vegard Vesterheim @ 2009-04-17 10:46 ` David Engster 2009-04-17 11:50 ` Vegard Vesterheim 0 siblings, 1 reply; 22+ messages in thread From: David Engster @ 2009-04-17 10:46 UTC (permalink / raw) To: ding Vegard Vesterheim <vegard.vesterheim@uninett.no> writes: > On Fri, 17 Apr 2009 11:00:16 +0200 David Engster <deng@randomsample.de> wrote: > >> To do something like that in Gnus, you need an external indexer. Mairix >> is just one possibility, but it's well suited for Gnus because it >> directly creates the resulting mailbox, so you don't have to parse large >> amounts of output in Emacs Lisp. Alternatively, you could surely also >> use search engines like Lucene, Xapian, Sphinx, etc., but those would >> need much more work to interface them with Gnus. > > I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend > for retrieving messages. Storing headers (and body) for mail messages > into a database, opens possibilities for flexible and efficient > queries and retrieval of email messages. Thanks for that link, I didn't know of dbmail. This looks pretty similar to what archiveopteryx does (www.archiveopteryx.org), which however has support for virtual folders (they call it "views"). Both seem a bit oversized for a single user though, and I guess you would need your own server to easily access your mail from different machines? Also, I'm really happy with Dovecot, and there would have to be some serious advantages to make me switch to something else. And there are many people who happily use commercial IMAP services (Fastmail, Google, whatever), which should also be able to have fast full text search. What I'd really love to see is a client-only solution, e.g. a program which creates a *local* index from a *remote* IMAP server. -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 10:46 ` David Engster @ 2009-04-17 11:50 ` Vegard Vesterheim 2009-04-17 13:27 ` Syncing Gnus (was: mail group by author?) David Engster 0 siblings, 1 reply; 22+ messages in thread From: Vegard Vesterheim @ 2009-04-17 11:50 UTC (permalink / raw) To: ding On Fri, 17 Apr 2009 12:46:09 +0200 David Engster <deng@randomsample.de> wrote: > Vegard Vesterheim <vegard.vesterheim@uninett.no> writes: >> I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend >> for retrieving messages. Storing headers (and body) for mail messages >> into a database, opens possibilities for flexible and efficient >> queries and retrieval of email messages. > > Thanks for that link, I didn't know of dbmail. This looks pretty similar > to what archiveopteryx does (www.archiveopteryx.org), which however has > support for virtual folders (they call it "views"). Heh, thanks for that link, I didn't know of archiveopteryx ;-) It does indeed look similar to dbmail, in fact there is a special FAQ entry that compares the two: http://www.archiveopteryx.org/faq/mailstore#dbmail I'm intrigued about the possibility for virtual folders in archiveopteryx, I hope to get a chance to try this soon. > Both seem a bit oversized for a single user though, and I guess you > would need your own server to easily access your mail from different > machines? Also, I'm really happy with Dovecot, and there would have to > be some serious advantages to make me switch to something else. And > there are many people who happily use commercial IMAP services > (Fastmail, Google, whatever), which should also be able to have fast > full text search. > > What I'd really love to see is a client-only solution, e.g. a program > which creates a *local* index from a *remote* IMAP server. I typically use several different computers, so I prefer a server-side solution. One of my biggest gripes about Gnus is that some meta-information (.newsrc.eld etc.) is stored on the client instead of the server. This means that the count of unread messages typically is wrong when I move from one computer to another. Yes, I know about solutions like this: http://www.emacswiki.org/emacs/GnusSync - Vegard V - ^ permalink raw reply [flat|nested] 22+ messages in thread
* Syncing Gnus (was: mail group by author?) 2009-04-17 11:50 ` Vegard Vesterheim @ 2009-04-17 13:27 ` David Engster 0 siblings, 0 replies; 22+ messages in thread From: David Engster @ 2009-04-17 13:27 UTC (permalink / raw) To: ding Vegard Vesterheim <vegard.vesterheim@uninett.no> writes: > I'm intrigued about the possibility for virtual folders in > archiveopteryx, I hope to get a chance to try this soon. Please let me know how this works out for you. > I typically use several different computers, so I prefer a server-side > solution. One of my biggest gripes about Gnus is that some > meta-information (.newsrc.eld etc.) is stored on the client instead of > the server. This means that the count of unread messages typically is > wrong when I move from one computer to another. > > Yes, I know about solutions like this: > http://www.emacswiki.org/emacs/GnusSync Maybe the following is helpful: - Tramp: In your .emacs set: (require 'gnus-load) (require 'message) (setq gnus-startup-file "/scpx:login@server:.newsrc") (setq message-directory "/scpx:login@server:Mail/") (setq gnus-directory "/scpx:login@server:News/") Disclaimer: I've only tested this very shortly, so I'm not sure if this is working correctly. To speed this up, set up a ssh master connection beforehand (e.g. do "ssh -f -N -M login@server" in the shell and specify ControlMaster for the host in your .ssh/config). Otherwise, it will not only be dead slow, but might also raise some administrator's eyebrows. One day there might even be IMAP storage in Tramp. :-) - sshfs: http://fuse.sourceforge.net/sshfs.html Mount single directory using sshfs and again set gnus-startup-file, message-directory and gnus-directory accordingly. Never tried this, but I don't see why this shouldn't just work. Needless to say, both these methods are slower than just using rsync, which is why I prefer the latter. -David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 9:00 ` David Engster 2009-04-17 10:07 ` Vegard Vesterheim @ 2009-04-17 16:53 ` Ted Zlatanov 2009-04-17 20:15 ` Vegard Vesterheim 1 sibling, 1 reply; 22+ messages in thread From: Ted Zlatanov @ 2009-04-17 16:53 UTC (permalink / raw) To: ding; +Cc: Magnus Henoch On Fri, 17 Apr 2009 11:00:16 +0200 David Engster <deng@randomsample.de> wrote: DE> Ted Zlatanov <tzz@lifelogs.com> writes: >> The registry could periodically visit all groups to see new messages. >> But that's messy. It could also read a "dribble" file with the message >> IDs and groups it needs to refresh. The gnus.registry.eld file could >> live on the server and get rsynced to the clients or opened remotely >> through Tramp. At least for IMAP, it's possible to do this kind of update without Gnus-level calls. It would probably be more efficient. Hmm, maybe it makes sense to open a second IMAP session just for registry and search updates, and refresh it periodically. That would be much better than updating things synchronously in the main IMAP session. DE> Well, having one general database for all search related things would DE> surely be a nice thing to have. But searching is just one feature; what DE> I really wanted to have in Gnus is something resembling "Filters" in DE> Opera Mail. I remember a message from Kai Großjohann years ago where he DE> praised Opera for this feature. It's an impressive mail reader, and DE> using filters instead of splitting immediately made sense to me. And DE> most importantly, filtering is not restricted to the mail headers. You DE> can filter with a full text search, including MIME parsing. For example, DE> Opera directly shows you groups containing all attachments you have DE> received, filtered according to type. DE> To do something like that in Gnus, you need an external indexer. Mairix DE> is just one possibility, but it's well suited for Gnus because it DE> directly creates the resulting mailbox, so you don't have to parse large DE> amounts of output in Emacs Lisp. Alternatively, you could surely also DE> use search engines like Lucene, Xapian, Sphinx, etc., but those would DE> need much more work to interface them with Gnus. I prefer splitting, but perhaps I'm old-fashioned. Splitting works locally and remotely, whereas indexers need to be run and maintained. Also, splitting costs are paid once and splitting rules can be complicated. By contrast, filtering rules must be cheap, so they often drop down to simple tags or a domain-specific query language (as with nnir.el). I don't know the right solution now, but I'll think about it (ETA 2020 :) On Fri, 17 Apr 2009 12:07:49 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: VV> I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend VV> for retrieving messages. Storing headers (and body) for mail messages VV> into a database, opens possibilities for flexible and efficient VV> queries and retrieval of email messages. Considering the large amounts of people using IMAP on third-party servers nowadays, I think it's best to find an approach that works with any IMAP server. I know this is unfair to solutions like dbmail, but otherwise we end up with very limited solutions. I think David agrees with me from his followup. On Fri, 17 Apr 2009 13:50:54 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: VV> I typically use several different computers, so I prefer a VV> server-side solution. One of my biggest gripes about Gnus is that VV> some meta-information (.newsrc.eld etc.) is stored on the client VV> instead of the server. I would also like that, and in fact that's where we'll end up if we develop the facilities to have a global Gnus registry (since the newsrc file can be serialized and synchronized exactly like the registry). VV> This means that the count of unread messages typically is wrong when VV> I move from one computer to another. This actually could be avoided with IMAP, because it supports flags for exactly this purpose. I don't remember why the flags are not used in preference to the local read/unread lists in newsrc.eld. On Fri, 17 Apr 2009 15:27:36 +0200 David Engster <deng@randomsample.de> wrote: DE> One day there might even be IMAP storage in Tramp. :-) Unfortunately, plain file storage would not be a good solution for storing large data sets. This is a problem with any rsync/SSH/etc. file-level synchronization: speed, reliability, and accessibility all suffer. I think we need a way to serialize a hashtable like the Gnus registry to an IMAP mailbox and make it searchable. At least for the registry this is easy, since every piece is self-sufficient in a single hashtable. The Gnus newsrc.eld would be a bit harder to serialize. This approach essentially makes the IMAP server a weakly organized database server and each mailbox a table. I like it. Magnus, have you considered starting work on the IMAP-based file storage? Does any of the above seem interesting or are you looking for simple file persistence and nothing more? Ted ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 16:53 ` mail group by author? Ted Zlatanov @ 2009-04-17 20:15 ` Vegard Vesterheim 2009-04-17 20:42 ` Ted Zlatanov 0 siblings, 1 reply; 22+ messages in thread From: Vegard Vesterheim @ 2009-04-17 20:15 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, Magnus Henoch On Fri, 17 Apr 2009 11:53:01 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: > On Fri, 17 Apr 2009 12:07:49 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: > > VV> I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend > VV> for retrieving messages. Storing headers (and body) for mail messages > VV> into a database, opens possibilities for flexible and efficient > VV> queries and retrieval of email messages. > > Considering the large amounts of people using IMAP on third-party > servers nowadays, I think it's best to find an approach that works with > any IMAP server. I know this is unfair to solutions like dbmail, but > otherwise we end up with very limited solutions. I think David agrees > with me from his followup. But dbmail *is* a IMAP server, I do not necessarily suggest any special support for dbmail in Gnus. Having a proper database as storage with search indexes for headers, instead of relying on the filesystem, will hopefully result in a IMAP server with more flexible and efficienct query and retrieval functionality. Support for virtual folders (predefined searches), is more easily accomplished with an infrastructure like this. Presorting of mail into statically defined folders is IMHO the wrong approach. I have wasted much time searching for a specific email, only to discover it was filtered into another folder than I had thought. With virtual folders, the same message can be found in several folders, but will still be stored only once, since the folder is really a query on the IMAP server. Sitewide de-duplication of messages would be an added benefit of this approach. For instance, I would like to have a folder called 'People' with automatically generated subfolders for each person having sent, or recieved, an email to/from me. I would also like to have the possibilities to define folders with names (and content) like: 'Messages_recieved_this_month', 'Messages_I_sent_yesterday' etc. - Vegard V - ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: mail group by author? 2009-04-17 20:15 ` Vegard Vesterheim @ 2009-04-17 20:42 ` Ted Zlatanov 0 siblings, 0 replies; 22+ messages in thread From: Ted Zlatanov @ 2009-04-17 20:42 UTC (permalink / raw) To: ding On Fri, 17 Apr 2009 22:15:33 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: VV> On Fri, 17 Apr 2009 11:53:01 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: >> On Fri, 17 Apr 2009 12:07:49 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: >> VV> I have great hopes for dbmail (dbmail.org). dbmail has an IMAP-backend VV> for retrieving messages. Storing headers (and body) for mail messages VV> into a database, opens possibilities for flexible and efficient VV> queries and retrieval of email messages. >> >> Considering the large amounts of people using IMAP on third-party >> servers nowadays, I think it's best to find an approach that works with >> any IMAP server. I know this is unfair to solutions like dbmail, but >> otherwise we end up with very limited solutions. I think David agrees >> with me from his followup. VV> But dbmail *is* a IMAP server, I do not necessarily suggest any VV> special support for dbmail in Gnus. Having a proper database as VV> storage with search indexes for headers, instead of relying on the VV> filesystem, will hopefully result in a IMAP server with more flexible VV> and efficienct query and retrieval functionality. Support for virtual VV> folders (predefined searches), is more easily accomplished with an VV> infrastructure like this. Sorry, I thought you were implying we should have special support for dbmail's queries (instead of just using the regular IMAP commands). But the IMAP client needs at least some awareness of dbmail's abilities, no? VV> Presorting of mail into statically defined folders is IMHO the wrong VV> approach. I have wasted much time searching for a specific email, only VV> to discover it was filtered into another folder than I had VV> thought. With virtual folders, the same message can be found in VV> several folders, but will still be stored only once, since the folder VV> is really a query on the IMAP server. Managing large collections of data is expensive. No matter how good the database, it's slower to manage all the messages in one set as opposed to partitioning them in some way. In effect, "static" mailboxes are really partitions of the full message set, and the server's storage backend (Maildir+ or some other mechanism) handles the partitioning transparently. VV> Sitewide de-duplication of messages would be an added benefit of VV> this approach. I don't think the savings there are significant. VV> For instance, I would like to have a folder called 'People' with VV> automatically generated subfolders for each person having sent, or VV> recieved, an email to/from me. I would also like to have the VV> possibilities to define folders with names (and content) like: VV> 'Messages_recieved_this_month', 'Messages_I_sent_yesterday' etc. This can be done with the Gnus registry currently. What I'm slowly realizing is that the registry needs to be integrated deeper with Gnus as a sort of "mail crawler" instead of an opportunistic voyeur that can miss mail. In addition, the registry data needs to be storeable somewhere outside the local filesystem. Ted ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2009-04-17 20:42 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-03-30 9:22 mail group by author? Stephen Berman 2009-03-30 11:57 ` David Engster 2009-03-30 12:49 ` Tassilo Horn 2009-03-30 13:13 ` David Engster 2009-03-30 15:25 ` Tassilo Horn 2009-03-30 15:59 ` Andreas Schwab 2009-03-31 6:42 ` Tassilo Horn 2009-03-31 9:18 ` David Engster 2009-03-31 10:36 ` Dovecot 1.2 virtual mailboxes (was: mail group by author?) Tassilo Horn 2009-03-31 11:35 ` Dovecot 1.2 virtual mailboxes David Engster 2009-03-31 12:23 ` Tassilo Horn 2009-03-31 9:52 ` mail group by author? Vegard Vesterheim 2009-03-31 10:25 ` Tassilo Horn 2009-04-14 16:39 ` Ted Zlatanov 2009-04-17 9:00 ` David Engster 2009-04-17 10:07 ` Vegard Vesterheim 2009-04-17 10:46 ` David Engster 2009-04-17 11:50 ` Vegard Vesterheim 2009-04-17 13:27 ` Syncing Gnus (was: mail group by author?) David Engster 2009-04-17 16:53 ` mail group by author? Ted Zlatanov 2009-04-17 20:15 ` Vegard Vesterheim 2009-04-17 20:42 ` Ted Zlatanov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).