Re: nnimap backend performances ?

[myglc2 <myglc2@gmail.com>]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> On 12/29/15 14:25 PM, myglc2 wrote:
>>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> My understanding is that the nnimap backend was re-written quite
>>> significantly two or three years ago (?). At any rate, the people who
>>> used the first iteration and were horrified now seem mostly happy with
>>> the new version, YMMV. I used to use isync and local dovecot, now use
>>> the direct connection, and it doesn't kill me. It isn't great (I'm in
>>> China), but it doesn't kill me.
>>>
>>> eric
>>
>> So Eric, it sounds like you abandoned ...
>>
>> https://ericabrahamsen.net/tech/2014/oct/gnus-dovecot-lucene.html
>
>
> No, I haven't abandoned that setup! Or rather, I've moved halfway away
> from it, but I'm going back.
>
> I recently switched my two main email accounts from Google Apps to
> self-hosted, and for the moment I'm having Gnus talk directly to the
> remote server for the new accounts.
>
> Basically I deleted the two Google Apps accounts, but left the
> corresponding imap accounts on my local Dovecot. I didn't want to move
> all the old mail into the new accounts, because there's a lot of it.
>
> That means I've got two "legacy" accounts for two addresses, which only
> exist locally, and two "new" accounts for the same two addresses, which
> only exist on my remote server. At some point I'll explain to my local
> Dovecot that these are actually four separate imap stores, and start
> syncing the two new accounts locally, as well.
>
> For the time being the two "legacy" accounts are still working just fine
> with the local Dovecot arrangement, and I should say are much easier to
> search than the new remote accounts. I want to move back mostly because
> the connections are still too slow/unstable for my taste. It's not Gnus'
> fault.

This is really good to hear. Your "legacy" situation closely parallels
the archive I need for a work account that I am loosing.

>> ... which I was following & hit the "plain" auth issue you discussed
>> ~1yr ago on the isync list.
>
> Ugh, I went through a dark period of confusion and despair for a bit
> there, when I was trying to learn too many new things all at the same
> time. In fact, I believe the problem was partially solved by upgrading
> isync (see the later patch posted to that isync thread). I'm now using
> 1.3.0, what are you using?

isync 1.3.0 built from git://git.code.sf.net/p/isync/isync master 46e792c

> I need to update the blog post: the ~/.mbsyncrc file syntax changed for
> the 1.3 version. There's another possibly important difference:
>
> 1) /etc/dovecot/dovecot.conf: In the userdb clause, on the "args" line,
> I've added at the end:
>
> username_format=%u

Thanks, I had that from your reply on a later thread (quoted below)
showing your more recent dovecot config ...

***
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Subject: Re: mail vs news
Newsgroups: gmane.emacs.gnus.general
To: ding@gnus.org
Date: Thu, 17 Dec 2015 19:13:40 +0800

Here's my output of "doveconf -n". This is all the configuration I've
got, apart from /etc/dovecot/passwd.

# 2.2.21 (5345f22): /etc/dovecot/dovecot.conf
# OS: Linux 4.2.5-1-ARCH i686  
auth_verbose = yes
disable_plaintext_auth = no
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
mail_plugins = "fts fts_lucene"
passdb {
  args = username_format=%u /etc/dovecot/passwd
  driver = passwd-file
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
}
protocols = imap
ssl = no
userdb {
  args = uid=eric gid=users home=/home/eric/.mail/%d/%n username_format=%u
  default_fields = mail=maildir:/home/eric/.mail/%d/%n/mail
  driver = static
}
***

That "doveconf -n" command is a beautiful thing! This also work for me
on dovecot 2.2.13 installed on debian 8 this way:

apt-get install dovecot-core dovecot-imapd dovecot-lucene

> 2) /etc/dovecot/passwd: The lines that used to look like
>
> eric@ericabrahamsen.net:{PLAIN}passwurd
>
> Now look like
>
> eric@ericabrahamsen.net:{PLAIN}passwurd:eric@ericabrahamsen.net

Nice. This is the puzzle piece I was missing. Now it works!

> I think the crux of the problem is that _authentication account_ and
> _user account_ are actually separate concerns. Either the previous
> version of isync was doing it wrong, or my dovecot config had to be
> fixed to separate the two things correctly, or maybe both.
>
> To be honest I've forgotten how it works, a bit, and that's the main
> reason I haven't synced my two new accounts to the local dovecot.
>
> I'll try to do that this weekend, and update that blog post! In the
> meantime, I hope something in here is helpful to you.

A huge help. Thank You.

FWIW, My goal is to sweep multiple gmail accounts into bullet-proof
archives with tools for searching and clustering several decades of
work. To get a handle on what is doable, I am experimenting with gnus
backends and search schemes, mu/mu4e, dovecot, and mbsync.

I would love to hear suggestions of other emacs-compatible solutions I
should try.

George