Re: emac gnus-centric gmail-phobic multi-host mail management

[regcl <regcl@channing.harvard.edu>]

W. Greenhouse <wgreenhouse-sGOZH3hwPm2sTnJN9+BGXg@public.gmane.org>
writes:

> Taking your last two questions:
>
> regcl <regcl-layyI3BE+OXOHkwriC13pfZ8FUJU4vz8@public.gmane.org> writes:
>
> [...]
>
>> C) Compatible mail repo/backup scheme
>>
>> I need to park the mail I am sweeping somewhere, once again with the
>> compartmentalization issues. While I use emacs as my primary email
>> client, it would probably be wise not to lock myself in to emacs, so
>> maybe I want the mail in a form compatible with other clients on my mac
>> os x desktops, most likely mac mail.app. What is the lightest weight
>> approach here? Is it maildir with dovecot/localhost?
>>
>>
>> D) Replication of the mail repos.
>>
>> I don't think I want to put up a pop or imap server on the net, at least
>> not now. So I am thinking the problem here is primarily one of
>> replication of my  mail directories across machines. Am I right?
>>
>> Based on my limited reading about mail, it seems like this could be done
>> with IMAP, but I know zilch about IMAP and I don't have a place to park
>> a personal IMAP server on the net.  
>>
>> On the other hand I use git for all of the other replication/syncing
>> that I ever do. So here is the question: Can I park my mail in maildir
>> with sub-repos for work and play, and use git for replication between
>> machines? What would be the pros and cons of this approach vs learning
>> using IMAP?
>
> Git would be sub-optimal for syncing Maildirs around, as would a
> standard file-moving tool like rsync, because a fundamental idea of
> Maildir is that mail states are expressed by renaming/moving the file.
> Whenever Gnus flags something or marks it read, therefore, git or rsync
> would treat this like a whole new file, leading to inefficiencies.
> Fortunately, there are tools to parse the Maildir format and figure out
> which messages are in fact the same on both ends, so we can quickly
> rename them to express a change in "flaggedness", instead of resending
> the whole thing and deleting the file on the other end (yuck).
>

Actually, if the contents of a file don't change, git does not create
another object, since it recognizes it as the same file because the
file's SHA-1 does not change when you move it or rename it. The only
overhead of moving or renaming a file is to store its new name and
directory, which has to happen anyway for each commit. Since we don't
edit incoming mail messages, each mail message will create at most one
git storage object. This is why I was thinking git might work well for
Maildir.

> Here are the two implementations I know of to efficiently sync maildirs
> over SSH, without exposing a new network service such as IMAP.  (They
> also bypass the system MTAs on both ends, as they operate on the end
> user's maildirs directly):
>
> 1. Maildirsync <http://search.cpan.org/dist/maildirsync/maildirsync.pod>:
>    Single perl script.  Requires you to write your own short shell script
>    or cron job to handle synchronization.
>
> 2. http://syncmaildir.sourceforge.net/: a complete rewrite of
>    maildirsync.  Evidently meant to "just work" without writing own shell
>    hax; I had trouble getting it to install on Slackware, however, so I've
>    been happily using maildirsync.  Syncmaildir seems to be packaged for
>    Debian-like distros, so that's a consideration.
>

Thank you. I will compare the features of these to git to try to
understand what the relative merits might be.

> My configuration looks something like this:
>
> 1. My almost-always-on homebox slurps various mail sites via fetchmail
>    using POP3.  On landing at the system MTA, the mail gets delivered
>    (along with any local/daemon-generated mail) to various Maildirs
>    according to rules in my ~/.procmailrc.
>
> 2. I read some mail on Gnus at the homebox.  When fetchmail->procmail
>    delivers some new mail to my priority inbox, `display-time-mode'
>    notifies me by putting a little picture of an envelope on my mode
>    line. :) Otherwise I don't get prodded about new mail until I poll
>    Gnus again myself.
>
> 3. I go on the road with my netbook.  On the netbook, I run my maildirsync
>    script when I have reliable net, and then M-x gnus to read the mail.
>
> 4. Gnus nnmaildir is pretty self-contained (flags/unread state is all
>    inside the maildir itself), so if you are only using maildir,
>    maildirsync will be enough to share state between the two machines.
>    Since I also read NNTP news with Gnus, however, and also use niceties
>    like adaptive scoring, I also sync my emacs configuration dir (which
>    contains the gnus score files and newsrc* files) using Bzr.
>

Thank you for laying out these details.

> TODO: be able to sync just one thing, instead of the sync
> mailidr-then-sync-bzr-repo dance.  Maybe
> http://lars.ingebrigtsen.no/2014/02/01/emacs-cloud/ will help with this
> someday.  Also, cronify the maildirsync end on the netbook.

If we can do both of these functions with git, this issues may be resolved.
>
> Hope this helps.  Please don't hesitate to ask for clarification; I'd be
> happy to share any of the configuration files (fetchmail, procmail,
> gnus/message/smtpmail/bbdb, maildirsync) that might be interesting.

Yes, this is all very helpful, thank you. Once I select an approach I
may well want more details.

Best,
regcl