Announcements and discussions for Gnus, the GNU Emacs Usenet newsreader
 help / color / mirror / Atom feed
* nnir-notmuch-engine setup with gnus/offlineimap/dovecot
@ 2016-09-22 23:00 Arkady Grudzinsky
  2016-09-23  5:21 ` Dmitry Alexandrov
  2016-09-23  7:58 ` Rainer M Krug
  0 siblings, 2 replies; 26+ messages in thread
From: Arkady Grudzinsky @ 2016-09-22 23:00 UTC (permalink / raw)
  To: info-gnus-english

Hi,

I am trying to set up notmuch to search my mail in all my mail
groups while I want to use Gnus to read and manipulate messages.

I retrieve mail from Gmail into ~/Mail using offlineimap.
Offlineimap runs "notmuch new" as a postsynchook.
Notmuch is set up, and "notmuch search" works from the
command line.

Gnus reads the email from a local Dovecot imap server using
nnimap backend.

Here is a snippet from my .gnus:

(setq gnus-secondary-select-methods
      '(
        (nnimap "local"
                (nnimap-address "localhost")
                (nnimap-stream network)
                (nnimap-authenticator login)

                (nnir-search-engine notmuch)
                ...
                )))

When I run "G G" in Gnus group buffer, I get the following
messages:

Doing notmuch query from:agrudzinsky...
notmuch args: search --format=text --output=files from:agrudzinsky
Massaging notmuch output...done
Search produced empty results.
gnus-select-newsgroup: Couldn’t request group nnir:nnir-s5obmzfd0j4.fsf: Search produced empty results.

Shell command

notmuch search --format=text --output=files from:agrudzinsky

produces a long list of maildir files.

Most discussions on this topic point to
http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html

The name implies that this may be exactly what I want, but the
link appears to be dead now.  I get an empty page when I try to
view it.

If anyone uses gnus-dovecot-offlineimap set up with notmuch, can
you, please, post a snippet of your configuration?

Thanks.

-- 
Arkady


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-22 23:00 nnir-notmuch-engine setup with gnus/offlineimap/dovecot Arkady Grudzinsky
@ 2016-09-23  5:21 ` Dmitry Alexandrov
  2016-09-23  7:56   ` Rainer M Krug
  2016-09-23  7:58 ` Rainer M Krug
  1 sibling, 1 reply; 26+ messages in thread
From: Dmitry Alexandrov @ 2016-09-23  5:21 UTC (permalink / raw)
  To: Arkady Grudzinsky; +Cc: info-gnus-english

> Most discussions on this topic point to
> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>
> The name implies that this may be exactly what I want, but the
> link appears to be dead now.  I get an empty page when I try to
> view it.

The Internet Archive Wayback Machine comes to your aid :-)

http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html

(I have no idea whether this page is in fact helpful though.)


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-23  5:21 ` Dmitry Alexandrov
@ 2016-09-23  7:56   ` Rainer M Krug
  2016-09-29 21:06     ` Dave Abrahams
  0 siblings, 1 reply; 26+ messages in thread
From: Rainer M Krug @ 2016-09-23  7:56 UTC (permalink / raw)
  To: Dmitry Alexandrov; +Cc: info-gnus-english, Arkady Grudzinsky


[-- Attachment #1.1: Type: text/plain, Size: 1476 bytes --]

Dmitry Alexandrov <321942@gmail.com> writes:

>> Most discussions on this topic point to
>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>
>> The name implies that this may be exactly what I want, but the
>> link appears to be dead now.  I get an empty page when I try to
>> view it.

I am using exactly the same setup, except that I am using

,----
| (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
`----

to access my local mail and I don't have a dovecot daemon running
permanently.

>
> The Internet Archive Wayback Machine comes to your aid :-)
>
> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>
> (I have no idea whether this page is in fact helpful though.)

This link is helpful, but it did not actually help with getting the
search to work. There was another discy=ussion abl=out using lucene in
dovecot for the search, but as you, I would prefer notmuch search.

I have simply defined a shortcut for me to do the search - much
easier. You could even rebiung GG ?

Cheers,

Rainer


>
> _______________________________________________
> info-gnus-english mailing list
> info-gnus-english@gnu.org
> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-22 23:00 nnir-notmuch-engine setup with gnus/offlineimap/dovecot Arkady Grudzinsky
  2016-09-23  5:21 ` Dmitry Alexandrov
@ 2016-09-23  7:58 ` Rainer M Krug
  2016-09-23 21:27   ` Adam Sjøgren
  1 sibling, 1 reply; 26+ messages in thread
From: Rainer M Krug @ 2016-09-23  7:58 UTC (permalink / raw)
  To: Arkady Grudzinsky; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 1953 bytes --]

Arkady Grudzinsky <agrudzinsky@gmail.com> writes:

> Hi,
>
> I am trying to set up notmuch to search my mail in all my mail
> groups while I want to use Gnus to read and manipulate messages.
>
> I retrieve mail from Gmail into ~/Mail using offlineimap.
> Offlineimap runs "notmuch new" as a postsynchook.
> Notmuch is set up, and "notmuch search" works from the
> command line.
>
> Gnus reads the email from a local Dovecot imap server using
> nnimap backend.
>
> Here is a snippet from my .gnus:
>
> (setq gnus-secondary-select-methods
>       '(
>         (nnimap "local"
>                 (nnimap-address "localhost")
>                 (nnimap-stream network)
>                 (nnimap-authenticator login)
>
>                 (nnir-search-engine notmuch)

As far as I understand, the notmuch search only works on a maildir
(nnmaildir I think?) and not nnimap.

Rainer


>                 ...
>                 )))
>
> When I run "G G" in Gnus group buffer, I get the following
> messages:
>
> Doing notmuch query from:agrudzinsky...
> notmuch args: search --format=text --output=files from:agrudzinsky
> Massaging notmuch output...done
> Search produced empty results.
> gnus-select-newsgroup: Couldn’t request group nnir:nnir-s5obmzfd0j4.fsf: Search produced empty results.
>
> Shell command
>
> notmuch search --format=text --output=files from:agrudzinsky
>
> produces a long list of maildir files.
>
> Most discussions on this topic point to
> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>
> The name implies that this may be exactly what I want, but the
> link appears to be dead now.  I get an empty page when I try to
> view it.
>
> If anyone uses gnus-dovecot-offlineimap set up with notmuch, can
> you, please, post a snippet of your configuration?
>
> Thanks.

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-23  7:58 ` Rainer M Krug
@ 2016-09-23 21:27   ` Adam Sjøgren
  2016-09-26 23:51     ` Arkady Grudzinsky
  0 siblings, 1 reply; 26+ messages in thread
From: Adam Sjøgren @ 2016-09-23 21:27 UTC (permalink / raw)
  To: info-gnus-english

Rainer writes:

> As far as I understand, the notmuch search only works on a maildir
> (nnmaildir I think?) and not nnimap.

Side note: notmuch search also works with nnml.


  Best regards,

    Adam

-- 
 "So I was off a little bit, sue me."                         Adam Sjøgren
                                                         asjo@koldfront.dk


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-23 21:27   ` Adam Sjøgren
@ 2016-09-26 23:51     ` Arkady Grudzinsky
  2016-09-27  2:54       ` Adam Sjøgren
  2016-09-27  7:27       ` Rainer M Krug
  0 siblings, 2 replies; 26+ messages in thread
From: Arkady Grudzinsky @ 2016-09-26 23:51 UTC (permalink / raw)
  To: info-gnus-english

On Fri, Sep 23 2016, Adam Sjøgren wrote:

> Rainer writes:
>
>> As far as I understand, the notmuch search only works on a maildir
>> (nnmaildir I think?) and not nnimap.
>
> Side note: notmuch search also works with nnml.

Notmuch works on any local maildir directory.  So, if mail is
retrieved by offlineimap and stored locally in maildir, notmuch
will work even though Gnus is accessing the maildir through
Dovecot and nnimap backend.  Notmuch is separate from Gnus.

I got it working.  The code from https://goo.gl/VAXWxk works
perfectly.  The idea is to map GG to notmuch-search.  Notmuch
does its job and shows the messages in its own buffer.  Then in
notmuch view buffer, I can press C-c C-c and go to that message
in Gnus.  The trick is to replace the file name with Gnus news
group name.  This is what replace-regexp-in-string code does.

Notmuch searches all groups regardless of their status in Gnus
and shows the whole message thread.  The C-c C-c combination
opens the message in Gnus even if the message is located in a
killed group.

> (require 'notmuch)
> (add-hook 'gnus-group-mode-hook 'lld-notmuch-shortcut)
> (require 'org-gnus)

> (defun lld-notmuch-shortcut ()
>   (define-key gnus-group-mode-map "GG" 'notmuch-search)
>   )

> (defun lld-notmuch-file-to-group (file)
>   "Calculate the Gnus group name from the given file name.
> "
>   (let ((group (file-name-directory (directory-file-name (file-name-directory file)))))
>     (setq group (replace-regexp-in-string ".*/Maildir/" "nnimap+local:" group))
>     (setq group (replace-regexp-in-string "/$" "" group))
>     (if (string-match ":$" group)
>         (concat group "INBOX")
>       (replace-regexp-in-string ":\\." ":" group))))

> (defun lld-notmuch-goto-message-in-gnus ()
>   "Open a summary buffer containing the current notmuch
> article."
>   (interactive)
>   (let ((group (lld-notmuch-file-to-group (notmuch-show-get-filename)))
>         (message-id (replace-regexp-in-string
>                      "^id:" "" (notmuch-show-get-message-id))))
>     (setq message-id (replace-regexp-in-string "\"" "" message-id))
>     (if (and group message-id)
>         (progn 
>     (switch-to-buffer "*Group*")
>     (org-gnus-follow-link group message-id))
>       (message "Couldn't get relevant infos for switching to Gnus."))))

> (define-key notmuch-show-mode-map (kbd "C-c C-c") 'lld-notmuch-goto-message-in-gnus)


-- 
Arkady


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-26 23:51     ` Arkady Grudzinsky
@ 2016-09-27  2:54       ` Adam Sjøgren
  2016-09-27  7:27       ` Rainer M Krug
  1 sibling, 0 replies; 26+ messages in thread
From: Adam Sjøgren @ 2016-09-27  2:54 UTC (permalink / raw)
  To: info-gnus-english

Arkady writes:

> On Fri, Sep 23 2016, Adam Sjøgren wrote:

>> Rainer writes:

>>> As far as I understand, the notmuch search only works on a maildir
>>> (nnmaildir I think?) and not nnimap.

>> Side note: notmuch search also works with nnml.

> Notmuch works on any local maildir directory. [...]

Yes, that is what Rainer wrote. I just added that notmuch works fine
with nnml as well.


  :-),

   Adam

-- 
 "Stockholm vaknar långsamt                                   Adam Sjøgren
  På droger och på sorg"                                 asjo@koldfront.dk


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-26 23:51     ` Arkady Grudzinsky
  2016-09-27  2:54       ` Adam Sjøgren
@ 2016-09-27  7:27       ` Rainer M Krug
  2016-09-28 22:10         ` Arkady Grudzinsky
  1 sibling, 1 reply; 26+ messages in thread
From: Rainer M Krug @ 2016-09-27  7:27 UTC (permalink / raw)
  To: Arkady Grudzinsky; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 2777 bytes --]

Arkady Grudzinsky <agrudzinsky@gmail.com> writes:

> On Fri, Sep 23 2016, Adam Sjøgren wrote:
>
>> Rainer writes:
>>
>>> As far as I understand, the notmuch search only works on a maildir
>>> (nnmaildir I think?) and not nnimap.
>>
>> Side note: notmuch search also works with nnml.
>
> Notmuch works on any local maildir directory.  So, if mail is
> retrieved by offlineimap and stored locally in maildir, notmuch
> will work even though Gnus is accessing the maildir through
> Dovecot and nnimap backend.  Notmuch is separate from Gnus.

Correct. 

>
> I got it working.  The code from https://goo.gl/VAXWxk works
> perfectly.

Could you please post the un-shortened URL as it has been disabled by
google?

> The idea is to map GG to notmuch-search.  Notmuch
> does its job and shows the messages in its own buffer.  Then in
> notmuch view buffer, I can press C-c C-c and go to that message
> in Gnus.  The trick is to replace the file name with Gnus news
> group name.  This is what replace-regexp-in-string code does.
>

Yup - correct.

> Notmuch searches all groups regardless of their status in Gnus
> and shows the whole message thread.  The C-c C-c combination
> opens the message in Gnus even if the message is located in a
> killed group.
>
>> (require 'notmuch)
>> (add-hook 'gnus-group-mode-hook 'lld-notmuch-shortcut)
>> (require 'org-gnus)
>
>> (defun lld-notmuch-shortcut ()
>>   (define-key gnus-group-mode-map "GG" 'notmuch-search)
>>   )
>
>> (defun lld-notmuch-file-to-group (file)
>>   "Calculate the Gnus group name from the given file name.
>> "
>>   (let ((group (file-name-directory (directory-file-name (file-name-directory file)))))
>>     (setq group (replace-regexp-in-string ".*/Maildir/" "nnimap+local:" group))
>>     (setq group (replace-regexp-in-string "/$" "" group))
>>     (if (string-match ":$" group)
>>         (concat group "INBOX")
>>       (replace-regexp-in-string ":\\." ":" group))))
>
>> (defun lld-notmuch-goto-message-in-gnus ()
>>   "Open a summary buffer containing the current notmuch
>> article."
>>   (interactive)
>>   (let ((group (lld-notmuch-file-to-group (notmuch-show-get-filename)))
>>         (message-id (replace-regexp-in-string
>>                      "^id:" "" (notmuch-show-get-message-id))))
>>     (setq message-id (replace-regexp-in-string "\"" "" message-id))
>>     (if (and group message-id)
>>         (progn 
>>     (switch-to-buffer "*Group*")
>>     (org-gnus-follow-link group message-id))
>>       (message "Couldn't get relevant infos for switching to Gnus."))))
>
>> (define-key notmuch-show-mode-map (kbd "C-c C-c") 'lld-notmuch-goto-message-in-gnus)

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-27  7:27       ` Rainer M Krug
@ 2016-09-28 22:10         ` Arkady Grudzinsky
  2016-09-29  6:57           ` Rainer M Krug
  0 siblings, 1 reply; 26+ messages in thread
From: Arkady Grudzinsky @ 2016-09-28 22:10 UTC (permalink / raw)
  To: info-gnus-english

On Tue, Sep 27 2016, Rainer M Krug wrote:

>> I got it working.  The code from https://goo.gl/VAXWxk works
>> perfectly.
>
> Could you please post the un-shortened URL as it has been disabled by
> google?

That was the link to the missing page referred to above

https://web.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html

It got too long with the web archive extension

-- 
Arkady



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-28 22:10         ` Arkady Grudzinsky
@ 2016-09-29  6:57           ` Rainer M Krug
  0 siblings, 0 replies; 26+ messages in thread
From: Rainer M Krug @ 2016-09-29  6:57 UTC (permalink / raw)
  To: Arkady Grudzinsky; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 621 bytes --]

Arkady Grudzinsky <agrudzinsky@gmail.com> writes:

> On Tue, Sep 27 2016, Rainer M Krug wrote:
>
>>> I got it working.  The code from https://goo.gl/VAXWxk works
>>> perfectly.
>>
>> Could you please post the un-shortened URL as it has been disabled by
>> google?
>
> That was the link to the missing page referred to above
>
> https://web.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>

Thanks,

Rainer

> It got too long with the web archive extension

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-23  7:56   ` Rainer M Krug
@ 2016-09-29 21:06     ` Dave Abrahams
  2016-09-30  1:33       ` myglc2
  2016-09-30  6:59       ` Rainer M Krug
  0 siblings, 2 replies; 26+ messages in thread
From: Dave Abrahams @ 2016-09-29 21:06 UTC (permalink / raw)
  To: info-gnus-english


on Fri Sep 23 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:

> Dmitry Alexandrov <321942@gmail.com> writes:
>
>>> Most discussions on this topic point to
>>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>
>>> The name implies that this may be exactly what I want, but the
>>> link appears to be dead now.  I get an empty page when I try to
>>> view it.
>
> I am using exactly the same setup, except that I am using
>
> ,----
> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
> `----
>
> to access my local mail and I don't have a dovecot daemon running
> permanently.

Yes.  Other things that work really well in a setup like that are:

1. Using dovecot's mdbox storage format, which is much more efficient

2. Building dovecot with clucene and stemmer support for search

3. Using mbsync from the isync project (scales much better than
   offlineimap IMO, though I am about to try dropping it because doveadm
   sync is already part of dovecot)

>>
>> The Internet Archive Wayback Machine comes to your aid :-)
>>
>> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>
>> (I have no idea whether this page is in fact helpful though.)
>
> This link is helpful, but it did not actually help with getting the
> search to work. There was another discy=ussion abl=out using lucene in
> dovecot for the search, but as you, I would prefer notmuch search.
>
> I have simply defined a shortcut for me to do the search - much
> easier. You could even rebiung GG ?
>
> Cheers,
>
> Rainer
>
>>
>> _______________________________________________
>> info-gnus-english mailing list
>> info-gnus-english@gnu.org
>> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
-Dave



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-29 21:06     ` Dave Abrahams
@ 2016-09-30  1:33       ` myglc2
  2016-09-30  7:04         ` Rainer M Krug
  2016-09-30 12:02         ` Dave Abrahams
  2016-09-30  6:59       ` Rainer M Krug
  1 sibling, 2 replies; 26+ messages in thread
From: myglc2 @ 2016-09-30  1:33 UTC (permalink / raw)
  To: info-gnus-english

Dave Abrahams <dave@boostpro.com> writes:
>
> Yes.  Other things that work really well in a setup like that are:
>
> 1. Using dovecot's mdbox storage format, which is much more efficient
>
> 2. Building dovecot with clucene and stemmer support for search
>
> 3. Using mbsync from the isync project (scales much better than
>    offlineimap IMO,

I saw this too. mbsync (AKA isync) was ~30 times faster in my tests. It
was also important to build from the git repo because, at least a month
or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
out to be important in my gmail sync setup.

Built from the git repo it shows this version info:

g1@g1 ~$ mbsync --version
isync 1.3.0

> though I am about to try dropping it because doveadm sync is already
> part of dovecot)

A couple months ago I put a fair amount of effort into trying to use
dovecot sync. I ended up concluding that mbsync is faster and easier to
config.



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-29 21:06     ` Dave Abrahams
  2016-09-30  1:33       ` myglc2
@ 2016-09-30  6:59       ` Rainer M Krug
  2016-09-30  9:51         ` Alan Schmitt
  2016-09-30 11:44         ` Dave Abrahams
  1 sibling, 2 replies; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30  6:59 UTC (permalink / raw)
  To: Dave Abrahams; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 2718 bytes --]

Dave Abrahams <dave@boostpro.com> writes:

> on Fri Sep 23 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>
>> Dmitry Alexandrov <321942@gmail.com> writes:
>>
>>>> Most discussions on this topic point to
>>>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>
>>>> The name implies that this may be exactly what I want, but the
>>>> link appears to be dead now.  I get an empty page when I try to
>>>> view it.
>>
>> I am using exactly the same setup, except that I am using
>>
>> ,----
>> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
>> `----
>>
>> to access my local mail and I don't have a dovecot daemon running
>> permanently.
>
> Yes.  Other things that work really well in a setup like that are:
>
> 1. Using dovecot's mdbox storage format, which is much more efficient

There is, among others, one reason why I don't use mbox format:
backup. If I add an email to a folder, the whole mbox file will be
backed up - in the case of maildir, only the new mail file. This saves
backup space, especially as I am using a Mac and Time Machine which is
doing hourly backups.

>
> 2. Building dovecot with clucene and stemmer support for search

Does this really work with nnimap-shell-program? I thought one has to
use the dovecot daemon for this?

>
> 3. Using mbsync from the isync project (scales much better than
>    offlineimap IMO, though I am about to try dropping it because doveadm
>    sync is already part of dovecot)

offlineimap works for me - and never change a running system (well -
that's the theory).

Interesting - didn't know about doveadm sync - any experiences with it?

How does it compare to isync and offlineimap?

>
>>>
>>> The Internet Archive Wayback Machine comes to your aid :-)
>>>
>>> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>
>>> (I have no idea whether this page is in fact helpful though.)
>>
>> This link is helpful, but it did not actually help with getting the
>> search to work. There was another discy=ussion abl=out using lucene in
>> dovecot for the search, but as you, I would prefer notmuch search.
>>
>> I have simply defined a shortcut for me to do the search - much
>> easier. You could even rebiung GG ?
>>
>> Cheers,
>>
>> Rainer
>>
>>>
>>> _______________________________________________
>>> info-gnus-english mailing list
>>> info-gnus-english@gnu.org
>>> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  1:33       ` myglc2
@ 2016-09-30  7:04         ` Rainer M Krug
  2016-09-30 14:01           ` myglc2
  2016-09-30 12:02         ` Dave Abrahams
  1 sibling, 1 reply; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30  7:04 UTC (permalink / raw)
  To: myglc2; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 1731 bytes --]

myglc2 <myglc2@gmail.com> writes:

> Dave Abrahams <dave@boostpro.com> writes:
>>
>> Yes.  Other things that work really well in a setup like that are:
>>
>> 1. Using dovecot's mdbox storage format, which is much more efficient
>>
>> 2. Building dovecot with clucene and stemmer support for search
>>
>> 3. Using mbsync from the isync project (scales much better than
>>    offlineimap IMO,
>
> I saw this too. mbsync (AKA isync) was ~30 times faster in my tests. It

Very interesting - I might look into isync than.

> was also important to build from the git repo because, at least a month
> or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
> out to be important in my gmail sync setup.

Which feature was this, if I may ask? I would also use isync for
syncing a gmail account. Could you share your config so that I can have
an idea what I am looking at in a change ti isync?

How do these two compare in regards to stability (offlineimap has
e.g. an issue wit suspend on the Mac)?



>
> Built from the git repo it shows this version info:
>
> g1@g1 ~$ mbsync --version
> isync 1.3.0

Yup - that is the one I also have now from git.

>
>> though I am about to try dropping it because doveadm sync is already
>> part of dovecot)
>
> A couple months ago I put a fair amount of effort into trying to use
> dovecot sync. I ended up concluding that mbsync is faster and easier to
> config.

Thanks,

Rainer

>
>
> _______________________________________________
> info-gnus-english mailing list
> info-gnus-english@gnu.org
> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  6:59       ` Rainer M Krug
@ 2016-09-30  9:51         ` Alan Schmitt
  2016-09-30 10:33           ` Rainer M Krug
  2016-09-30 11:50           ` Dave Abrahams
  2016-09-30 11:44         ` Dave Abrahams
  1 sibling, 2 replies; 26+ messages in thread
From: Alan Schmitt @ 2016-09-30  9:51 UTC (permalink / raw)
  To: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 677 bytes --]

On 2016-09-30 08:59, Rainer M Krug <Rainer@krugs.de> writes:

>> 2. Building dovecot with clucene and stemmer support for search
>
> Does this really work with nnimap-shell-program? I thought one has to
> use the dovecot daemon for this?

Yes, but setting up a dovecot daemon is not too difficult and it has
other advantages. For instance, I have two email accounts, one is purely
local and the other is synced using offlineimap, and I can easily access
both of them (and search through both of them) by using two dovecot
users.

Best,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7
Monthly Athmospheric CO₂, Mauna Loa Obs. 2016-08: 402.25, 2015-08: 398.93

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  9:51         ` Alan Schmitt
@ 2016-09-30 10:33           ` Rainer M Krug
  2016-09-30 11:50           ` Dave Abrahams
  1 sibling, 0 replies; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30 10:33 UTC (permalink / raw)
  To: Alan Schmitt; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 1089 bytes --]

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> On 2016-09-30 08:59, Rainer M Krug <Rainer@krugs.de> writes:
>
>>> 2. Building dovecot with clucene and stemmer support for search
>>
>> Does this really work with nnimap-shell-program? I thought one has to
>> use the dovecot daemon for this?
>
> Yes,

Interesting - maybe I should try this sometime.
So the nnimap-shell-program stays the same with clucene and stemming?
Where do I have to enable clucene and stemming?


> but setting up a dovecot daemon is not too difficult and it has other
> advantages.

Yup - I am using homebrew on a Mac, and the daemon is just a click
away as I have the config file already.

> For instance, I have two email accounts, one is purely
> local and the other is synced using offlineimap, and I can easily
> access both of them (and search through both of them) by using two
> dovecot users.

Makers sense - and I thought about the same some tome ago.

Cheers,

Rainer

>
> Best,
>
> Alan

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  6:59       ` Rainer M Krug
  2016-09-30  9:51         ` Alan Schmitt
@ 2016-09-30 11:44         ` Dave Abrahams
  2016-09-30 12:56           ` Rainer M Krug
  1 sibling, 1 reply; 26+ messages in thread
From: Dave Abrahams @ 2016-09-30 11:44 UTC (permalink / raw)
  To: Rainer M Krug; +Cc: info-gnus-english


on Thu Sep 29 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> on Fri Sep 23 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>>
>>> Dmitry Alexandrov <321942@gmail.com> writes:
>>>
>>>>> Most discussions on this topic point to
>>>>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>>
>>>>> The name implies that this may be exactly what I want, but the
>>>>> link appears to be dead now.  I get an empty page when I try to
>>>>> view it.
>>>
>>> I am using exactly the same setup, except that I am using
>>>
>>> ,----
>>> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
>>> `----
>>>
>>> to access my local mail and I don't have a dovecot daemon running
>>> permanently.
>>
>> Yes.  Other things that work really well in a setup like that are:
>>
>> 1. Using dovecot's mdbox storage format, which is much more efficient
>
> There is, among others, one reason why I don't use mbox format:
> backup. If I add an email to a folder, the whole mbox file will be
> backed up - in the case of maildir, only the new mail file. This saves
> backup space, especially as I am using a Mac and Time Machine which is
> doing hourly backups.

Not mbox; mdbox.  I've used this setup on a mac with network TM for a
long time.

>> 2. Building dovecot with clucene and stemmer support for search
>
> Does this really work with nnimap-shell-program? I thought one has to
> use the dovecot daemon for this?

Yes, it works.  Feel free to scrape http://github.com/onmsg/onmsg for details
and ask me questions for more.  There's a script in there that I use to
install (most of) this setup with homebrew.

>>
>> 3. Using mbsync from the isync project (scales much better than
>>    offlineimap IMO, though I am about to try dropping it because doveadm
>>    sync is already part of dovecot)
>
> offlineimap works for me - and never change a running system (well -
> that's the theory).

I don't suppose you're syncing any really large folders then?  Among
other things I found doing it all through Python was causing way
much CPU usage on my Macbook pro.

> Interesting - didn't know about doveadm sync - any experiences with
> it?

Not yet.

> How does it compare to isync and offlineimap?

I'm not really in a position to comment yet; I've just skimmed the
output of doveadm help sync.

>>
>>>>
>>>> The Internet Archive Wayback Machine comes to your aid :-)
>>>>
>>>> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>
>>>> (I have no idea whether this page is in fact helpful though.)
>>>
>>> This link is helpful, but it did not actually help with getting the
>>> search to work. There was another discy=ussion abl=out using lucene in
>>> dovecot for the search, but as you, I would prefer notmuch search.
>>>
>>> I have simply defined a shortcut for me to do the search - much
>>> easier. You could even rebiung GG ?
>>>
>>> Cheers,
>>>
>>> Rainer
>>>
>>>>
>>>> _______________________________________________
>>>> info-gnus-english mailing list
>>>> info-gnus-english@gnu.org
>>>> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
-Dave


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  9:51         ` Alan Schmitt
  2016-09-30 10:33           ` Rainer M Krug
@ 2016-09-30 11:50           ` Dave Abrahams
  1 sibling, 0 replies; 26+ messages in thread
From: Dave Abrahams @ 2016-09-30 11:50 UTC (permalink / raw)
  To: info-gnus-english


on Fri Sep 30 2016, Alan Schmitt <alan.schmitt-AT-polytechnique.org> wrote:

> On 2016-09-30 08:59, Rainer M Krug <Rainer@krugs.de> writes:
>
>>> 2. Building dovecot with clucene and stemmer support for search
>>
>> Does this really work with nnimap-shell-program? I thought one has to
>> use the dovecot daemon for this?
>
> Yes, but setting up a dovecot daemon is not too difficult and it has
> other advantages. For instance, I have two email accounts, one is purely
> local and the other is synced using offlineimap, and I can easily access
> both of them (and search through both of them) by using two dovecot
> users.

Sounds complicated.  I just partition the folders of one account and
sync some of them to one place, others to other places.  “Not at all”
could be your other place.

-- 
-Dave


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  1:33       ` myglc2
  2016-09-30  7:04         ` Rainer M Krug
@ 2016-09-30 12:02         ` Dave Abrahams
  2016-09-30 14:28           ` myglc2
  1 sibling, 1 reply; 26+ messages in thread
From: Dave Abrahams @ 2016-09-30 12:02 UTC (permalink / raw)
  To: info-gnus-english


on Thu Sep 29 2016, myglc2 <myglc2-AT-gmail.com> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>>
>> Yes.  Other things that work really well in a setup like that are:
>>
>> 1. Using dovecot's mdbox storage format, which is much more efficient
>>
>> 2. Building dovecot with clucene and stemmer support for search
>>
>> 3. Using mbsync from the isync project (scales much better than
>>    offlineimap IMO,
>
> I saw this too. mbsync (AKA isync) was ~30 times faster in my tests. It
> was also important to build from the git repo because, at least a month
> or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
> out to be important in my gmail sync setup.
>
> Built from the git repo it shows this version info:
>
> g1@g1 ~$ mbsync --version
> isync 1.3.0
>
>> though I am about to try dropping it because doveadm sync is already
>> part of dovecot)
>
> A couple months ago I put a fair amount of effort into trying to use
> dovecot sync. I ended up concluding that mbsync is faster and easier to
> config.

Really!

I always knew it was pretty great, but this is quite surprising given
that doveadm sync has access to dovecot's internals and can store state
across sync runs.  Do you have any further details?

-- 
-Dave



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 11:44         ` Dave Abrahams
@ 2016-09-30 12:56           ` Rainer M Krug
  2016-09-30 17:13             ` Dave Abrahams
  0 siblings, 1 reply; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30 12:56 UTC (permalink / raw)
  To: Dave Abrahams; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 4003 bytes --]

Dave Abrahams <dave@boostpro.com> writes:

> on Thu Sep 29 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>
>> Dave Abrahams <dave@boostpro.com> writes:
>>
>>> on Fri Sep 23 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>>>
>>>> Dmitry Alexandrov <321942@gmail.com> writes:
>>>>
>>>>>> Most discussions on this topic point to
>>>>>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>>>
>>>>>> The name implies that this may be exactly what I want, but the
>>>>>> link appears to be dead now.  I get an empty page when I try to
>>>>>> view it.
>>>>
>>>> I am using exactly the same setup, except that I am using
>>>>
>>>> ,----
>>>> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
>>>> `----
>>>>
>>>> to access my local mail and I don't have a dovecot daemon running
>>>> permanently.
>>>
>>> Yes.  Other things that work really well in a setup like that are:
>>>
>>> 1. Using dovecot's mdbox storage format, which is much more efficient
>>
>> There is, among others, one reason why I don't use mbox format:
>> backup. If I add an email to a folder, the whole mbox file will be
>> backed up - in the case of maildir, only the new mail file. This saves
>> backup space, especially as I am using a Mac and Time Machine which is
>> doing hourly backups.
>
> Not mbox; mdbox.  I've used this setup on a mac with network TM for a
> long time.

OK - sorry.

>
>>> 2. Building dovecot with clucene and stemmer support for search
>>
>> Does this really work with nnimap-shell-program? I thought one has to
>> use the dovecot daemon for this?
>
> Yes, it works.  Feel free to scrape http://github.com/onmsg/onmsg for details
> and ask me questions for more.  There's a script in there that I use to
> install (most of) this setup with homebrew.

Thanks - will look into this.

>
>>>
>>> 3. Using mbsync from the isync project (scales much better than
>>>    offlineimap IMO, though I am about to try dropping it because doveadm
>>>    sync is already part of dovecot)
>>
>> offlineimap works for me - and never change a running system (well -
>> that's the theory).
>
> I don't suppose you're syncing any really large folders then?  Among
> other things I found doing it all through Python was causing way
> much CPU usage on my Macbook pro.

Hm - I am syncing my All Mail folder as well (1.3GB - just checked), so everything double,
I am not using idle, just every two minutes fast sync and every 10
minutes normal sync - is not really that heavy on my Macbook Pro
Retina. OK - If I am waiting for a sync it is not fast - but what is
when you are waiting...

>
>> Interesting - didn't know about doveadm sync - any experiences with
>> it?
>
> Not yet.
>
>> How does it compare to isync and offlineimap?
>
> I'm not really in a position to comment yet; I've just skimmed the
> output of doveadm help sync.


OK.

Thanks for the info,

Rainer

>
>>>
>>>>>
>>>>> The Internet Archive Wayback Machine comes to your aid :-)
>>>>>
>>>>> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>>
>>>>> (I have no idea whether this page is in fact helpful though.)
>>>>
>>>> This link is helpful, but it did not actually help with getting the
>>>> search to work. There was another discy=ussion abl=out using lucene in
>>>> dovecot for the search, but as you, I would prefer notmuch search.
>>>>
>>>> I have simply defined a shortcut for me to do the search - much
>>>> easier. You could even rebiung GG ?
>>>>
>>>> Cheers,
>>>>
>>>> Rainer
>>>>
>>>>>
>>>>> _______________________________________________
>>>>> info-gnus-english mailing list
>>>>> info-gnus-english@gnu.org
>>>>> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30  7:04         ` Rainer M Krug
@ 2016-09-30 14:01           ` myglc2
  2016-09-30 15:04             ` Rainer M Krug
  2016-09-30 17:13             ` Dave Abrahams
  0 siblings, 2 replies; 26+ messages in thread
From: myglc2 @ 2016-09-30 14:01 UTC (permalink / raw)
  To: info-gnus-english

Rainer M Krug <Rainer@krugs.de> writes:

> myglc2 <myglc2@gmail.com> writes:
>
>> was also important to build from the git repo because, at least a month
>> or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
>> out to be important in my gmail sync setup.
>
> Which feature was this, if I may ask? I would also use isync for
> syncing a gmail account.

One of these sync options ...

Create Both
Expunge Both
SyncState *

... sorry I don't recall which.

> Could you share your config so that I can have
> an idea what I am looking at in a change ti isync?

Here is the top of my ~/.mbsyncrc. It includes Gmail config notes and
the config of one "channel" that syncs one Gmail account into ~/Maildir

# usage:
#   mbsync --all --verbose
# ... or for a single channel ...
#   mbsync --verbose --debug cm
#
# With Gmail configured this way ...
#
# > Settings > Labels
#
#     > System labels
#         Inbox      X Show in Imap
#         Starred      Show in Imap
#         Important    Show in Imap
#         Chats        Show in Imap
#         Sent Mail  X Show in Imap
#         Drafts       Show in Imap
#         All Mail   X Show in Imap
#         Spam         Show in Imap
#         Trash        Show in Imap
#
#     > Labels
#         <remove all labels>
#
# > Settings > Forwarding and POP/IMAP
#
#     > When I mark a message in IMAP as deleted:
#       X  Auto-Expung off
#
#     > When a message is marked as deleted and expunged...
#       X  Immediately delete the message forever
#
# ... the 'cm' config below produces a Mairdir store like ...
#
# /home/glc/Maildir/cm:
# drwx------ 6 glc glc 4096 Dec 12 20:14 [Gmail]
# drwx------ 5 glc glc 4096 Dec 12 20:13 INBOX
#
# /home/glc/Maildir/cm/[Gmail]:
# drwx------ 5 glc glc 4096 Dec 12 20:14 All Mail
# drwx------ 5 glc glc 4096 Dec 12 20:14 Sent Mail
# drwx------ 5 glc glc 4096 Dec 12 20:14 Trash
#
# Any "new" Gmail message initially appears in two ~/Maildir folders,
# either in 'cm/INBOX' + 'cm/[Gmail]/All Mail', or 'cm/[Gmail]/Sent
# Mail' + 'cm/[Gmail]/All Mail'. When you delete a message from both
# of the ~/Maildir folders it will be deleted from Gmail.

###### cm START 

IMAPAccount cm
Host imap.gmail.com
User USER@gmail.com
Pass PASSWORD
AuthMechs LOGIN
SSLType IMAPS
CertificateFile /etc/ssl/certs/ca-certificates.crt

IMAPStore cm-remote
Account cm

MaildirStore cm-local
Path ~/Maildir/cm/
Inbox ~/Maildir/cm/INBOX
SubFolders Verbatim

Channel cm
Master :cm-remote:
Slave :cm-local:
Patterns "*"
Create Both
Expunge Both
SyncState *

###### cm END

> How do these two compare in regards to stability (offlineimap has
> e.g. an issue wit suspend on the Mac)?

Don't know, I use it on a server.

- George



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 12:02         ` Dave Abrahams
@ 2016-09-30 14:28           ` myglc2
  2016-09-30 15:06             ` Rainer M Krug
  0 siblings, 1 reply; 26+ messages in thread
From: myglc2 @ 2016-09-30 14:28 UTC (permalink / raw)
  To: info-gnus-english

Dave Abrahams <dave@boostpro.com> writes:

> on Thu Sep 29 2016, myglc2 <myglc2-AT-gmail.com> wrote:
>
>> Dave Abrahams <dave@boostpro.com> writes:
>>
>>> though I am about to try dropping it because doveadm sync is already
>>> part of dovecot)
>>
>> A couple months ago I put a fair amount of effort into trying to use
>> dovecot sync. I ended up concluding that mbsync is faster and easier to
>> config.
>
> Really!
>
> I always knew it was pretty great, but this is quite surprising given
> that doveadm sync has access to dovecot's internals and can store state
> across sync runs.

I was attracted to the idea of dovecot sync for the same reasons. But in
practice I found that the linkage between dovecot internals and dovecot
sync was confusing and the documentation was difficult to follow.

In fairness, I should say I only experimented with dovecot and never
made it my daily driver. A savvy dovecot person might see it
differently.

> Do you have any further details?

I got as far as syncing between Gmail and a local Dovecot service and
watching it work. I was surprised when it was slower than mbsync. The
logging/debugging also seemed less informative.

At about the same time I decided to use ~/Maildir with git backup as my
primary mail archive, so I didn't investigate it further.



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 14:01           ` myglc2
@ 2016-09-30 15:04             ` Rainer M Krug
  2016-09-30 17:13             ` Dave Abrahams
  1 sibling, 0 replies; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30 15:04 UTC (permalink / raw)
  To: myglc2; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 3356 bytes --]

myglc2 <myglc2@gmail.com> writes:

> Rainer M Krug <Rainer@krugs.de> writes:
>
>> myglc2 <myglc2@gmail.com> writes:
>>
>>> was also important to build from the git repo because, at least a month
>>> or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
>>> out to be important in my gmail sync setup.
>>
>> Which feature was this, if I may ask? I would also use isync for
>> syncing a gmail account.
>
> One of these sync options ...
>
> Create Both
> Expunge Both
> SyncState *
>
> ... sorry I don't recall which.
>
>> Could you share your config so that I can have
>> an idea what I am looking at in a change ti isync?
>
> Here is the top of my ~/.mbsyncrc. It includes Gmail config notes and
> the config of one "channel" that syncs one Gmail account into ~/Maildir

Thanks - I will look through it and see what I will do.

Rainer


>
> # usage:
> #   mbsync --all --verbose
> # ... or for a single channel ...
> #   mbsync --verbose --debug cm
> #
> # With Gmail configured this way ...
> #
> # > Settings > Labels
> #
> #     > System labels
> #         Inbox      X Show in Imap
> #         Starred      Show in Imap
> #         Important    Show in Imap
> #         Chats        Show in Imap
> #         Sent Mail  X Show in Imap
> #         Drafts       Show in Imap
> #         All Mail   X Show in Imap
> #         Spam         Show in Imap
> #         Trash        Show in Imap
> #
> #     > Labels
> #         <remove all labels>
> #
> # > Settings > Forwarding and POP/IMAP
> #
> #     > When I mark a message in IMAP as deleted:
> #       X  Auto-Expung off
> #
> #     > When a message is marked as deleted and expunged...
> #       X  Immediately delete the message forever
> #
> # ... the 'cm' config below produces a Mairdir store like ...
> #
> # /home/glc/Maildir/cm:
> # drwx------ 6 glc glc 4096 Dec 12 20:14 [Gmail]
> # drwx------ 5 glc glc 4096 Dec 12 20:13 INBOX
> #
> # /home/glc/Maildir/cm/[Gmail]:
> # drwx------ 5 glc glc 4096 Dec 12 20:14 All Mail
> # drwx------ 5 glc glc 4096 Dec 12 20:14 Sent Mail
> # drwx------ 5 glc glc 4096 Dec 12 20:14 Trash
> #
> # Any "new" Gmail message initially appears in two ~/Maildir folders,
> # either in 'cm/INBOX' + 'cm/[Gmail]/All Mail', or 'cm/[Gmail]/Sent
> # Mail' + 'cm/[Gmail]/All Mail'. When you delete a message from both
> # of the ~/Maildir folders it will be deleted from Gmail.
>
> ###### cm START 
>
> IMAPAccount cm
> Host imap.gmail.com
> User USER@gmail.com
> Pass PASSWORD
> AuthMechs LOGIN
> SSLType IMAPS
> CertificateFile /etc/ssl/certs/ca-certificates.crt
>
> IMAPStore cm-remote
> Account cm
>
> MaildirStore cm-local
> Path ~/Maildir/cm/
> Inbox ~/Maildir/cm/INBOX
> SubFolders Verbatim
>
> Channel cm
> Master :cm-remote:
> Slave :cm-local:
> Patterns "*"
> Create Both
> Expunge Both
> SyncState *
>
> ###### cm END
>
>> How do these two compare in regards to stability (offlineimap has
>> e.g. an issue wit suspend on the Mac)?
>
> Don't know, I use it on a server.
>
> - George
>
>
> _______________________________________________
> info-gnus-english mailing list
> info-gnus-english@gnu.org
> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 14:28           ` myglc2
@ 2016-09-30 15:06             ` Rainer M Krug
  0 siblings, 0 replies; 26+ messages in thread
From: Rainer M Krug @ 2016-09-30 15:06 UTC (permalink / raw)
  To: myglc2; +Cc: info-gnus-english


[-- Attachment #1.1: Type: text/plain, Size: 1759 bytes --]

myglc2 <myglc2@gmail.com> writes:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> on Thu Sep 29 2016, myglc2 <myglc2-AT-gmail.com> wrote:
>>
>>> Dave Abrahams <dave@boostpro.com> writes:
>>>
>>>> though I am about to try dropping it because doveadm sync is already
>>>> part of dovecot)
>>>
>>> A couple months ago I put a fair amount of effort into trying to use
>>> dovecot sync. I ended up concluding that mbsync is faster and easier to
>>> config.
>>
>> Really!
>>
>> I always knew it was pretty great, but this is quite surprising given
>> that doveadm sync has access to dovecot's internals and can store state
>> across sync runs.
>
> I was attracted to the idea of dovecot sync for the same reasons. But in
> practice I found that the linkage between dovecot internals and dovecot
> sync was confusing and the documentation was difficult to follow.
>
> In fairness, I should say I only experimented with dovecot and never
> made it my daily driver. A savvy dovecot person might see it
> differently.
>
>> Do you have any further details?
>
> I got as far as syncing between Gmail and a local Dovecot service and
> watching it work. I was surprised when it was slower than mbsync. The
> logging/debugging also seemed less informative.
>
> At about the same time I decided to use ~/Maildir with git backup as my
> primary mail archive, so I didn't investigate it further.

Ha - git as backup for maildir - sounds perfect. Never thought about
that.

Rainer

>
>
> _______________________________________________
> info-gnus-english mailing list
> info-gnus-english@gnu.org
> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 12:56           ` Rainer M Krug
@ 2016-09-30 17:13             ` Dave Abrahams
  0 siblings, 0 replies; 26+ messages in thread
From: Dave Abrahams @ 2016-09-30 17:13 UTC (permalink / raw)
  To: info-gnus-english


on Fri Sep 30 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> on Thu Sep 29 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>>
>>> Dave Abrahams <dave@boostpro.com> writes:
>>>
>>>> on Fri Sep 23 2016, Rainer M Krug <Rainer-AT-krugs.de> wrote:
>>>>
>>>>> Dmitry Alexandrov <321942@gmail.com> writes:
>>>>>
>>>>>>> Most discussions on this topic point to
>>>>>>> http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>>>>
>>>>>>> The name implies that this may be exactly what I want, but the
>>>>>>> link appears to be dead now.  I get an empty page when I try to
>>>>>>> view it.
>>>>>
>>>>> I am using exactly the same setup, except that I am using
>>>>>
>>>>> ,----
>>>>> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir")))
>>>>> `----
>>>>>
>>>>> to access my local mail and I don't have a dovecot daemon running
>>>>> permanently.
>>>>
>>>> Yes.  Other things that work really well in a setup like that are:
>>>>
>>>> 1. Using dovecot's mdbox storage format, which is much more efficient
>>>
>>> There is, among others, one reason why I don't use mbox format:
>>> backup. If I add an email to a folder, the whole mbox file will be
>>> backed up - in the case of maildir, only the new mail file. This saves
>>> backup space, especially as I am using a Mac and Time Machine which is
>>> doing hourly backups.
>>
>> Not mbox; mdbox.  I've used this setup on a mac with network TM for a
>> long time.
>
> OK - sorry.
>
>>
>>>> 2. Building dovecot with clucene and stemmer support for search
>>>
>>> Does this really work with nnimap-shell-program? I thought one has to
>>> use the dovecot daemon for this?
>>
>> Yes, it works.  Feel free to scrape http://github.com/onmsg/onmsg for details
>> and ask me questions for more.  There's a script in there that I use to
>> install (most of) this setup with homebrew.
>
> Thanks - will look into this.

Warning: it's got a lot of user-specific paths and such in it.  The
onmsg project was once intended for general consumption but I've just
been using it to maintain my personal setup.  So you'll need to
customize it.

>>>> 3. Using mbsync from the isync project (scales much better than
>>>>    offlineimap IMO, though I am about to try dropping it because doveadm
>>>>    sync is already part of dovecot)
>>>
>>> offlineimap works for me - and never change a running system (well -
>>> that's the theory).
>>
>> I don't suppose you're syncing any really large folders then?  Among
>> other things I found doing it all through Python was causing way
>> much CPU usage on my Macbook pro.
>
> Hm - I am syncing my All Mail folder as well (1.3GB - just checked),
> so everything double, I am not using idle, just every two minutes fast
> sync and every 10 minutes normal sync - is not really that heavy on my
> Macbook Pro Retina. OK - If I am waiting for a sync it is not fast -
> but what is when you are waiting...

How's battery life?  

>>> Interesting - didn't know about doveadm sync - any experiences with
>>> it?
>>
>> Not yet.
>>
>>> How does it compare to isync and offlineimap?
>>
>> I'm not really in a position to comment yet; I've just skimmed the
>> output of doveadm help sync.
>
> OK.
>
> Thanks for the info,
>
> Rainer
>
>>
>>>>
>>>>>>
>>>>>> The Internet Archive Wayback Machine comes to your aid :-)
>>>>>>
>>>>>> http://wayback.archive.org/web/20160305042051/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>>>>>>
>>>>>> (I have no idea whether this page is in fact helpful though.)
>>>>>
>>>>> This link is helpful, but it did not actually help with getting the
>>>>> search to work. There was another discy=ussion abl=out using lucene in
>>>>> dovecot for the search, but as you, I would prefer notmuch search.
>>>>>
>>>>> I have simply defined a shortcut for me to do the search - much
>>>>> easier. You could even rebiung GG ?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Rainer
>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> info-gnus-english mailing list
>>>>>> info-gnus-english@gnu.org
>>>>>> https://lists.gnu.org/mailman/listinfo/info-gnus-english

-- 
-Dave



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: nnir-notmuch-engine setup with gnus/offlineimap/dovecot
  2016-09-30 14:01           ` myglc2
  2016-09-30 15:04             ` Rainer M Krug
@ 2016-09-30 17:13             ` Dave Abrahams
  1 sibling, 0 replies; 26+ messages in thread
From: Dave Abrahams @ 2016-09-30 17:13 UTC (permalink / raw)
  To: info-gnus-english


on Fri Sep 30 2016, myglc2 <myglc2-AT-gmail.com> wrote:

> Rainer M Krug <Rainer@krugs.de> writes:
>
>> myglc2 <myglc2@gmail.com> writes:
>>
>>> was also important to build from the git repo because, at least a month
>>> or two ago, the tar.gz at version 1.2.1 was missing a feature that turns
>>> out to be important in my gmail sync setup.
>>
>> Which feature was this, if I may ask? I would also use isync for
>> syncing a gmail account.
>
> One of these sync options ...
>
> Create Both
> Expunge Both
> SyncState *
>
> ... sorry I don't recall which.
>
>> Could you share your config so that I can have
>> an idea what I am looking at in a change ti isync?
>
> Here is the top of my ~/.mbsyncrc. It includes Gmail config notes and
> the config of one "channel" that syncs one Gmail account into ~/Maildir
>
> # usage:
> #   mbsync --all --verbose
> # ... or for a single channel ...
> #   mbsync --verbose --debug cm
> #
> # With Gmail configured this way ...
> #
> # > Settings > Labels
> #
> #     > System labels
> #         Inbox      X Show in Imap
> #         Starred      Show in Imap
> #         Important    Show in Imap
> #         Chats        Show in Imap
> #         Sent Mail  X Show in Imap
> #         Drafts       Show in Imap
> #         All Mail   X Show in Imap
> #         Spam         Show in Imap
> #         Trash        Show in Imap
> #
> #     > Labels
> #         <remove all labels>
> #
> # > Settings > Forwarding and POP/IMAP
> #
> #     > When I mark a message in IMAP as deleted:
> #       X  Auto-Expung off
> #
> #     > When a message is marked as deleted and expunged...
> #       X  Immediately delete the message forever
> #
> # ... the 'cm' config below produces a Mairdir store like ...
> #
> # /home/glc/Maildir/cm:
> # drwx------ 6 glc glc 4096 Dec 12 20:14 [Gmail]
> # drwx------ 5 glc glc 4096 Dec 12 20:13 INBOX
> #
> # /home/glc/Maildir/cm/[Gmail]:
> # drwx------ 5 glc glc 4096 Dec 12 20:14 All Mail
> # drwx------ 5 glc glc 4096 Dec 12 20:14 Sent Mail
> # drwx------ 5 glc glc 4096 Dec 12 20:14 Trash
> #
> # Any "new" Gmail message initially appears in two ~/Maildir folders,
> # either in 'cm/INBOX' + 'cm/[Gmail]/All Mail', or 'cm/[Gmail]/Sent
> # Mail' + 'cm/[Gmail]/All Mail'. When you delete a message from both
> # of the ~/Maildir folders it will be deleted from Gmail.
>
> ###### cm START 
>
> IMAPAccount cm
> Host imap.gmail.com
> User USER@gmail.com
> Pass PASSWORD
> AuthMechs LOGIN
> SSLType IMAPS
> CertificateFile /etc/ssl/certs/ca-certificates.crt
>
> IMAPStore cm-remote
> Account cm
>
> MaildirStore cm-local
> Path ~/Maildir/cm/
> Inbox ~/Maildir/cm/INBOX
> SubFolders Verbatim
>
> Channel cm
> Master :cm-remote:
> Slave :cm-local:
> Patterns "*"
> Create Both
> Expunge Both
> SyncState *
>
> ###### cm END
>
>> How do these two compare in regards to stability (offlineimap has
>> e.g. an issue wit suspend on the Mac)?
>
> Don't know, I use it on a server.

I don't have any issues suspending my macs with isync running.

-- 
-Dave



^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2016-09-30 17:13 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-22 23:00 nnir-notmuch-engine setup with gnus/offlineimap/dovecot Arkady Grudzinsky
2016-09-23  5:21 ` Dmitry Alexandrov
2016-09-23  7:56   ` Rainer M Krug
2016-09-29 21:06     ` Dave Abrahams
2016-09-30  1:33       ` myglc2
2016-09-30  7:04         ` Rainer M Krug
2016-09-30 14:01           ` myglc2
2016-09-30 15:04             ` Rainer M Krug
2016-09-30 17:13             ` Dave Abrahams
2016-09-30 12:02         ` Dave Abrahams
2016-09-30 14:28           ` myglc2
2016-09-30 15:06             ` Rainer M Krug
2016-09-30  6:59       ` Rainer M Krug
2016-09-30  9:51         ` Alan Schmitt
2016-09-30 10:33           ` Rainer M Krug
2016-09-30 11:50           ` Dave Abrahams
2016-09-30 11:44         ` Dave Abrahams
2016-09-30 12:56           ` Rainer M Krug
2016-09-30 17:13             ` Dave Abrahams
2016-09-23  7:58 ` Rainer M Krug
2016-09-23 21:27   ` Adam Sjøgren
2016-09-26 23:51     ` Arkady Grudzinsky
2016-09-27  2:54       ` Adam Sjøgren
2016-09-27  7:27       ` Rainer M Krug
2016-09-28 22:10         ` Arkady Grudzinsky
2016-09-29  6:57           ` Rainer M Krug

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).