Gnus development mailing list
 help / color / mirror / Atom feed
* nnir and notmuch?
@ 2014-09-23  8:40 Rainer M Krug
  2014-09-23 10:15 ` Steinar Bang
  0 siblings, 1 reply; 21+ messages in thread
From: Rainer M Krug @ 2014-09-23  8:40 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

Now that I can search gmane (perfect - love it!) I can't search my local
imap server via notmuch anymore - any suggestions how I can achieve
both via nnir or "G G"?

I can obviously still run notmuch-search, and I also could binf=d it to
a new keyboard shortcut, but I would prefer that "G G" searches, when on
folders from the local maildir, using notmuch. 

Is this possible?

Thanks,

Rainer   


-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23  8:40 nnir and notmuch? Rainer M Krug
@ 2014-09-23 10:15 ` Steinar Bang
  2014-09-23 10:24   ` Rasmus
  2014-09-23 10:48   ` Rainer M Krug
  0 siblings, 2 replies; 21+ messages in thread
From: Steinar Bang @ 2014-09-23 10:15 UTC (permalink / raw)
  To: ding

>>>>> Rainer M Krug <Rainer@krugs.de>:

> Now that I can search gmane (perfect - love it!) I can't search my local
> imap server via notmuch anymore - any suggestions how I can achieve
> both via nnir or "G G"?

One way would be to replace
 (nnimap . imap)
in nnir-method-default-engines with
 (nnimap . notmuch)

But that would mask native imap search for other IMAP servers you might
have.




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:15 ` Steinar Bang
@ 2014-09-23 10:24   ` Rasmus
  2014-09-23 10:50     ` Rainer M Krug
  2014-09-23 10:48   ` Rainer M Krug
  1 sibling, 1 reply; 21+ messages in thread
From: Rasmus @ 2014-09-23 10:24 UTC (permalink / raw)
  To: ding

Steinar Bang <sb@dod.no> writes:

>>>>>> Rainer M Krug <Rainer@krugs.de>:
>
>> Now that I can search gmane (perfect - love it!) I can't search my local
>> imap server via notmuch anymore - any suggestions how I can achieve
>> both via nnir or "G G"?
>
> One way would be to replace
>  (nnimap . imap)
> in nnir-method-default-engines with
>  (nnimap . notmuch)
>
> But that would mask native imap search for other IMAP servers you might
> have.

If Rainer is using nnmaildir it might be less of an issue.

A less elegant way would be to replace G-G.  Something like: 

(defun my-gnus-G-G (NNIR-EXTRA-PARMS &optional SPECS)
  "Chose the right search"
  (interactive "P")
       (let ((backend (car-safe
                       (split-string (or (gnus-group-group-name) "") ":"))))
         (unless (eq "" backend)
           (case backend
             ("nntp+gmane" 
              (gnus-group-make-nnir-group NNIR-EXTRA-PARMS SPECS))
             ("..." (search-using-notmuch))
             (otherwise
              (gnus-group-make-nnir-group NNIR-EXTRA-PARMS SPECS))))))

(define-key gnus-group-mode-map (kbd "G G") 'my-gnus-G-G)

I don't know how well it would work in practice. 

—Rasmus

-- 
May the Force be with you




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:15 ` Steinar Bang
  2014-09-23 10:24   ` Rasmus
@ 2014-09-23 10:48   ` Rainer M Krug
  2014-09-23 11:59     ` Steinar Bang
                       ` (2 more replies)
  1 sibling, 3 replies; 21+ messages in thread
From: Rainer M Krug @ 2014-09-23 10:48 UTC (permalink / raw)
  To: ding


<#secure method=pgpmime mode=sign> 

Steinar Bang <sb@dod.no> writes:

>>>>>> Rainer M Krug <Rainer@krugs.de>:
>
>> Now that I can search gmane (perfect - love it!) I can't search my
>> local imap server via notmuch anymore - any suggestions how I can
>> achieve both via nnir or "G G"?
>
> One way would be to replace
>  (nnimap . imap) in nnir-method-default-engines with
>  (nnimap . notmuch)

I had

,----
|            '(nnimap "Maildir"
|                       (nnimap-stream shell) (nnimap-shell-program
|                       "/usr/local/opt/dovecot/libexec/dovecot/imap -o
|                       mail_location=maildir:$HOME/Maildir")
|                       (nnir-search-engine 'notmuch)))
`----

in my gnus file, but it produced an error message.
I am just trying your suggestion, and I get the following:

,----
| notmuch args: search --format=text --output=files hallo
| Massaging notmuch output...done
| Search produced empty results.
| gnus-group-read-ephemeral-group: Couldn't request group: Search produced empty results.
`----

although the search on the commandline

,----
| notmuch search --format=text --output=files Rainer
`----

yields many results.

So - when setting nnir-method-default-engines, nnir uses notmuch but
does not return any results, while using nnir-search-engine does not
work.

Any suggestions from here? I am by the way using Gnus v5.13 - is notmuch
integrated as an nnir search engine?

>
> But that would mask native imap search for other IMAP servers you
> might have.

No problem for me - I am using offlineimap to get my mail and then only
access the local imap server and no other - at least at the moment.

Cheers,

Rainer

>
>
>

--
Rainer M. Krug 
email: Rainer<at>krugs<dot>de 
PGP: 0x0F52F982



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:24   ` Rasmus
@ 2014-09-23 10:50     ` Rainer M Krug
  0 siblings, 0 replies; 21+ messages in thread
From: Rainer M Krug @ 2014-09-23 10:50 UTC (permalink / raw)
  To: Rasmus; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

Rasmus <rasmus@gmx.us> writes:

> Steinar Bang <sb@dod.no> writes:
>
>>>>>>> Rainer M Krug <Rainer@krugs.de>:
>>
>>> Now that I can search gmane (perfect - love it!) I can't search my local
>>> imap server via notmuch anymore - any suggestions how I can achieve
>>> both via nnir or "G G"?
>>
>> One way would be to replace
>>  (nnimap . imap)
>> in nnir-method-default-engines with
>>  (nnimap . notmuch)
>>
>> But that would mask native imap search for other IMAP servers you might
>> have.
>
> If Rainer is using nnmaildir it might be less of an issue.

I am using local imap, but only local imap - I want to leave the maildir
under the control of dovecot.

>
> A less elegant way would be to replace G-G.  Something like: 

That would most likely work - but as you say less elegant and I would
like to keep to much hacking out of my setup. But I will keep it in
mind and use it if everything else fails.

Rainer

>
> (defun my-gnus-G-G (NNIR-EXTRA-PARMS &optional SPECS)
>   "Chose the right search"
>   (interactive "P")
>        (let ((backend (car-safe
>                        (split-string (or (gnus-group-group-name) "") ":"))))
>          (unless (eq "" backend)
>            (case backend
>              ("nntp+gmane" 
>               (gnus-group-make-nnir-group NNIR-EXTRA-PARMS SPECS))
>              ("..." (search-using-notmuch))
>              (otherwise
>               (gnus-group-make-nnir-group NNIR-EXTRA-PARMS SPECS))))))
>
> (define-key gnus-group-mode-map (kbd "G G") 'my-gnus-G-G)
>
> I don't know how well it would work in practice. 
>
> —Rasmus

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:48   ` Rainer M Krug
@ 2014-09-23 11:59     ` Steinar Bang
  2014-09-23 13:15     ` Andrew Cohen
  2014-09-23 16:14     ` Eric Abrahamsen
  2 siblings, 0 replies; 21+ messages in thread
From: Steinar Bang @ 2014-09-23 11:59 UTC (permalink / raw)
  To: ding

>>>>> Rainer M Krug <Rainer@krugs.de>:

> Any suggestions from here?

Sorry, no.  I just read the nnir.el to see how nnir backends are picked.




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:48   ` Rainer M Krug
  2014-09-23 11:59     ` Steinar Bang
@ 2014-09-23 13:15     ` Andrew Cohen
  2014-09-24  7:18       ` Rainer M Krug
  2014-09-23 16:14     ` Eric Abrahamsen
  2 siblings, 1 reply; 21+ messages in thread
From: Andrew Cohen @ 2014-09-23 13:15 UTC (permalink / raw)
  To: ding

>>>>> "Rainer" == Rainer M Krug <Rainer@krugs.de> writes:

    Rainer> <#secure method=pgpmime mode=sign> Steinar Bang <sb@dod.no>
    Rainer> writes:

    >>>>>>> Rainer M Krug <Rainer@krugs.de>:
    >> 
    >>> Now that I can search gmane (perfect - love it!) I can't search
    >>> my local imap server via notmuch anymore - any suggestions how I
    >>> can achieve both via nnir or "G G"?

    Rainer> I had

    Rainer> ,---- | '(nnimap "Maildir" | (nnimap-stream shell)
    Rainer> (nnimap-shell-program |
    Rainer> "/usr/local/opt/dovecot/libexec/dovecot/imap -o |
    Rainer> mail_location=maildir:$HOME/Maildir") | (nnir-search-engine
    Rainer> 'notmuch))) `----

    Rainer> in my gnus file, but it produced an error message.


"G G" should use whatever search engine you have configured. Setting
nnir-search-engine in the server spec to notmuch will make "G G" use
notmuch for that server. You don't say what  error you received; I think
that you should  use the name of the server rather than a symbol (that
is, try (nnir-search-engine notmuch) without the apostrophe).

You will have to configure variables for notmuch. I don't use the
notmuch engine so I can't help you with this configuration, but you can
read the notmuch docs and maybe someone on the list who does use it can
chime in.

Regards,
Andy




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 10:48   ` Rainer M Krug
  2014-09-23 11:59     ` Steinar Bang
  2014-09-23 13:15     ` Andrew Cohen
@ 2014-09-23 16:14     ` Eric Abrahamsen
  2014-09-24  7:26       ` Rainer M Krug
  2 siblings, 1 reply; 21+ messages in thread
From: Eric Abrahamsen @ 2014-09-23 16:14 UTC (permalink / raw)
  To: ding

Rainer M Krug <Rainer@krugs.de> writes:

> <#secure method=pgpmime mode=sign> 
>
> Steinar Bang <sb@dod.no> writes:
>
>>>>>>> Rainer M Krug <Rainer@krugs.de>:
>>
>>> Now that I can search gmane (perfect - love it!) I can't search my
>>> local imap server via notmuch anymore - any suggestions how I can
>>> achieve both via nnir or "G G"?
>>
>> One way would be to replace
>>  (nnimap . imap) in nnir-method-default-engines with
>>  (nnimap . notmuch)
>
> I had
>
> ,----
> |            '(nnimap "Maildir"
> |                       (nnimap-stream shell) (nnimap-shell-program
> |                       "/usr/local/opt/dovecot/libexec/dovecot/imap -o
> |                       mail_location=maildir:$HOME/Maildir")
> |                       (nnir-search-engine 'notmuch)))
> `----
>
> in my gnus file, but it produced an error message.
> I am just trying your suggestion, and I get the following:
>
> ,----
> | notmuch args: search --format=text --output=files hallo
> | Massaging notmuch output...done
> | Search produced empty results.
> | gnus-group-read-ephemeral-group: Couldn't request group: Search produced empty results.
> `----

For a while I ran notmuch and imap configured like this, and got a lot
of the same problems, because I was a bit confused about how all the
bits and pieces interacted. I think the problem is that notmuch returns
its hits based on filesystem paths. These paths don't necessarily match
up with how imap locates its messages, though -- things like mailboxes
transformations and namespaces can mean that imap sees the location of a
message in a way that doesn't jive with the filesystem path.

If you've got multiple Gnus backends in effect, another awkwardness is
that notmuch will probably return all the hits from all backends,
whereas nnir will attempt to filter out anything that doesn't belong to
the group/server from which you initiated the search.

I don't know all the grisly details, or whether it could be made to
work: when I realized what was happening I just stopped using notmuch as
a nnir engine for nnimap messages. Now I have a Frankenstein search
setup, using notmuch-search by itself, and the imap nnir engine, in
parallel. I'm shifting over to using nnir only, and will eventually get
rid of notmuch.

Not very helpful, I know, but I think there are fundamental
incompatibilities in specifying notmuch as a nnir engine for imap
servers.

Eric




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 13:15     ` Andrew Cohen
@ 2014-09-24  7:18       ` Rainer M Krug
  0 siblings, 0 replies; 21+ messages in thread
From: Rainer M Krug @ 2014-09-24  7:18 UTC (permalink / raw)
  To: Andrew Cohen; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

Andrew Cohen <cohen@bu.edu> writes:

>>>>>> "Rainer" == Rainer M Krug <Rainer@krugs.de> writes:
>
>     Rainer> <#secure method=pgpmime mode=sign> Steinar Bang <sb@dod.no>
>     Rainer> writes:
>
>     >>>>>>> Rainer M Krug <Rainer@krugs.de>:
>     >> 
>     >>> Now that I can search gmane (perfect - love it!) I can't search
>     >>> my local imap server via notmuch anymore - any suggestions how I
>     >>> can achieve both via nnir or "G G"?
>
>     Rainer> I had
>
>     Rainer> ,---- | '(nnimap "Maildir" | (nnimap-stream shell)
>     Rainer> (nnimap-shell-program |
>     Rainer> "/usr/local/opt/dovecot/libexec/dovecot/imap -o |
>     Rainer> mail_location=maildir:$HOME/Maildir") | (nnir-search-engine
>     Rainer> 'notmuch))) `----
>
>     Rainer> in my gnus file, but it produced an error message.
>
>
> "G G" should use whatever search engine you have configured. Setting
> nnir-search-engine in the server spec to notmuch will make "G G" use
> notmuch for that server. You don't say what  error you received; I think
> that you should  use the name of the server rather than a symbol (that
> is, try (nnir-search-engine notmuch) without the apostrophe).

Sounds like an idea - I'll try it out. But I do not assume that that is
working if it is not when notmuch is set as default search engine for
imap (I use only local imap with maildir as backend, which is indexed by
notuch).

>
> You will have to configure variables for notmuch. 

Notmuch from the command line is working - if you refer to parameters
relating to nnir, which ones do I have to set?

> I don't use the notmuch engine so I can't help you with this
> configuration, but you can read the notmuch docs and maybe someone on
> the list who does use it can chime in.

Thanks,

Rainer

>
> Regards,
> Andy
>
>
>

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-23 16:14     ` Eric Abrahamsen
@ 2014-09-24  7:26       ` Rainer M Krug
  2014-09-24  9:41         ` Eric Abrahamsen
  0 siblings, 1 reply; 21+ messages in thread
From: Rainer M Krug @ 2014-09-24  7:26 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 3551 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Rainer M Krug <Rainer@krugs.de> writes:
>
>> <#secure method=pgpmime mode=sign> 
>>
>> Steinar Bang <sb@dod.no> writes:
>>
>>>>>>>> Rainer M Krug <Rainer@krugs.de>:
>>>
>>>> Now that I can search gmane (perfect - love it!) I can't search my
>>>> local imap server via notmuch anymore - any suggestions how I can
>>>> achieve both via nnir or "G G"?
>>>
>>> One way would be to replace
>>>  (nnimap . imap) in nnir-method-default-engines with
>>>  (nnimap . notmuch)
>>
>> I had
>>
>> ,----
>> |            '(nnimap "Maildir"
>> |                       (nnimap-stream shell) (nnimap-shell-program
>> |                       "/usr/local/opt/dovecot/libexec/dovecot/imap -o
>> |                       mail_location=maildir:$HOME/Maildir")
>> |                       (nnir-search-engine 'notmuch)))
>> `----
>>
>> in my gnus file, but it produced an error message.
>> I am just trying your suggestion, and I get the following:
>>
>> ,----
>> | notmuch args: search --format=text --output=files hallo
>> | Massaging notmuch output...done
>> | Search produced empty results.
>> | gnus-group-read-ephemeral-group: Couldn't request group: Search produced empty results.
>> `----
>
> For a while I ran notmuch and imap configured like this, and got a lot
> of the same problems, because I was a bit confused about how all the
> bits and pieces interacted. I think the problem is that notmuch returns
> its hits based on filesystem paths. These paths don't necessarily match
> up with how imap locates its messages, though -- things like mailboxes
> transformations and namespaces can mean that imap sees the location of a
> message in a way that doesn't jive with the filesystem path.

This might be the reason - makes sense. As I want to use dovecot to
access my email stored in maildir (for notmuch, as this is not modifying
thew mails when only used for searching), I think I can not use it this
way. Makes perfect sense.

>
> If you've got multiple Gnus backends in effect, another awkwardness is
> that notmuch will probably return all the hits from all backends,
> whereas nnir will attempt to filter out anything that doesn't belong to
> the group/server from which you initiated the search.

I have only the local imap and gmane.

>
> I don't know all the grisly details, or whether it could be made to
> work: when I realized what was happening I just stopped using notmuch
> as a nnir engine for nnimap messages.

I think I will do the same here.

> Now I have a Frankenstein search setup, using notmuch-search by
> itself, and the imap nnir engine, in parallel. 

Sounds like the best solution - and then just bind notmuch-search to a
keyboard shortcut.

> I'm shifting over to using nnir only, and will eventually get rid of
> notmuch.

I tried the normal search on y imap folders, and it is incredibly slow
(folders with 10 000 and more emails). Also, my archive is split into
years, and a topic can span two years - so the normal imap search is not
an option for me here.

>
> Not very helpful, I know, but I think there are fundamental
> incompatibilities in specifying notmuch as a nnir engine for imap
> servers.

In contrast - quite helpful, as I understood why it is not working to
use notmuch for an imap backend. And as my setup is running nicely, I do
not intend to change it.

Thanks a lot,

Rainer

>
> Eric
>
>
>

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24  7:26       ` Rainer M Krug
@ 2014-09-24  9:41         ` Eric Abrahamsen
  2014-09-24 13:51           ` Adam Sjøgren
  0 siblings, 1 reply; 21+ messages in thread
From: Eric Abrahamsen @ 2014-09-24  9:41 UTC (permalink / raw)
  To: ding

Rainer M Krug <Rainer@krugs.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Rainer M Krug <Rainer@krugs.de> writes:
>>
>>> <#secure method=pgpmime mode=sign> 
>>>
>>> Steinar Bang <sb@dod.no> writes:
>>>
>>>>>>>>> Rainer M Krug <Rainer@krugs.de>:
>>>>
>>>>> Now that I can search gmane (perfect - love it!) I can't search my
>>>>> local imap server via notmuch anymore - any suggestions how I can
>>>>> achieve both via nnir or "G G"?
>>>>
>>>> One way would be to replace
>>>>  (nnimap . imap) in nnir-method-default-engines with
>>>>  (nnimap . notmuch)
>>>
>>> I had
>>>
>>> ,----
>>> |            '(nnimap "Maildir"
>>> |                       (nnimap-stream shell) (nnimap-shell-program
>>> |                       "/usr/local/opt/dovecot/libexec/dovecot/imap -o
>>> |                       mail_location=maildir:$HOME/Maildir")
>>> |                       (nnir-search-engine 'notmuch)))
>>> `----
>>>
>>> in my gnus file, but it produced an error message.
>>> I am just trying your suggestion, and I get the following:
>>>
>>> ,----
>>> | notmuch args: search --format=text --output=files hallo
>>> | Massaging notmuch output...done
>>> | Search produced empty results.
>>> | gnus-group-read-ephemeral-group: Couldn't request group: Search produced empty results.
>>> `----
>>
>> For a while I ran notmuch and imap configured like this, and got a lot
>> of the same problems, because I was a bit confused about how all the
>> bits and pieces interacted. I think the problem is that notmuch returns
>> its hits based on filesystem paths. These paths don't necessarily match
>> up with how imap locates its messages, though -- things like mailboxes
>> transformations and namespaces can mean that imap sees the location of a
>> message in a way that doesn't jive with the filesystem path.
>
> This might be the reason - makes sense. As I want to use dovecot to
> access my email stored in maildir (for notmuch, as this is not modifying
> thew mails when only used for searching), I think I can not use it this
> way. Makes perfect sense.
>
>>
>> If you've got multiple Gnus backends in effect, another awkwardness is
>> that notmuch will probably return all the hits from all backends,
>> whereas nnir will attempt to filter out anything that doesn't belong to
>> the group/server from which you initiated the search.
>
> I have only the local imap and gmane.
>
>>
>> I don't know all the grisly details, or whether it could be made to
>> work: when I realized what was happening I just stopped using notmuch
>> as a nnir engine for nnimap messages.
>
> I think I will do the same here.
>
>> Now I have a Frankenstein search setup, using notmuch-search by
>> itself, and the imap nnir engine, in parallel. 
>
> Sounds like the best solution - and then just bind notmuch-search to a
> keyboard shortcut.
>
>> I'm shifting over to using nnir only, and will eventually get rid of
>> notmuch.
>
> I tried the normal search on y imap folders, and it is incredibly slow
> (folders with 10 000 and more emails). Also, my archive is split into
> years, and a topic can span two years - so the normal imap search is not
> an option for me here.

You can search more than one group with nnir, by marking the desired
groups, or calling it on a topic heading, or on a whole server in the
*Server* buffer.

I just set up a new system with the lucene text indexer, and it works
much better. I have a blog post mostly written about this, and can post
it sooner if you think it might be helpful.

>> Not very helpful, I know, but I think there are fundamental
>> incompatibilities in specifying notmuch as a nnir engine for imap
>> servers.
>
> In contrast - quite helpful, as I understood why it is not working to
> use notmuch for an imap backend. And as my setup is running nicely, I do
> not intend to change it.
>
> Thanks a lot,
>
> Rainer
>
>>
>> Eric
>>
>>
>>




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24  9:41         ` Eric Abrahamsen
@ 2014-09-24 13:51           ` Adam Sjøgren
  2014-09-24 17:40             ` Eric Abrahamsen
  0 siblings, 1 reply; 21+ messages in thread
From: Adam Sjøgren @ 2014-09-24 13:51 UTC (permalink / raw)
  To: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I just set up a new system with the lucene text indexer, and it works
> much better.

Than what?


  Just curious,

    Adam

-- 
 "They didn't care that they'd seen it work in                Adam Sjøgren
  practise, because they already knew it couldn't work   asjo@koldfront.dk
  in theory."




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24 13:51           ` Adam Sjøgren
@ 2014-09-24 17:40             ` Eric Abrahamsen
  2014-09-24 19:34               ` Alan Schmitt
                                 ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Eric Abrahamsen @ 2014-09-24 17:40 UTC (permalink / raw)
  To: ding

asjo@koldfront.dk (Adam Sjøgren) writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I just set up a new system with the lucene text indexer, and it works
>> much better.
>
> Than what?

Than the native imap search, which is god-awfully slow.

To be clear, I use dovecot as a local imap cache. Originally I wasn't
using it as a daemon: mail came in via isync, which called dovecot at
/usr/lib/dovecot/imap. Then gnus called the same executable via
nnimap-shell-program to get the mail. Dovecot only "woke up" when one or
the other program called it.

Nnir search just used whatever search facilities dovecot provided out of
the box. It was slow. I read about integrating lucene into dovecot, and
as far as I could tell it wouldn't work when you were just calling
dovecot's imap executable -- you had to be running a dovecot instance.

So I switched to an always-running dovecot instance, configured the
lucene indexer, and now the exact same nnir searches go like lightning.

That's basically a summary of the blog post :)

E




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24 17:40             ` Eric Abrahamsen
@ 2014-09-24 19:34               ` Alan Schmitt
  2014-09-25  0:08                 ` Eric Abrahamsen
  2014-09-24 19:47               ` Adam Sjøgren
  2014-09-26  7:21               ` Eric S Fraga
  2 siblings, 1 reply; 21+ messages in thread
From: Alan Schmitt @ 2014-09-24 19:34 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]

On 2014-09-25 01:40, Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> To be clear, I use dovecot as a local imap cache. Originally I wasn't
> using it as a daemon: mail came in via isync, which called dovecot at
> /usr/lib/dovecot/imap. Then gnus called the same executable via
> nnimap-shell-program to get the mail. Dovecot only "woke up" when one or
> the other program called it.
>
> Nnir search just used whatever search facilities dovecot provided out of
> the box. It was slow. I read about integrating lucene into dovecot, and
> as far as I could tell it wouldn't work when you were just calling
> dovecot's imap executable -- you had to be running a dovecot instance.
>
> So I switched to an always-running dovecot instance, configured the
> lucene indexer, and now the exact same nnir searches go like lightning.
>
> That's basically a summary of the blog post :)

I'd be really curious to hear more about this. Please let us know when
you publish your post.

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24 17:40             ` Eric Abrahamsen
  2014-09-24 19:34               ` Alan Schmitt
@ 2014-09-24 19:47               ` Adam Sjøgren
  2014-09-26  7:21               ` Eric S Fraga
  2 siblings, 0 replies; 21+ messages in thread
From: Adam Sjøgren @ 2014-09-24 19:47 UTC (permalink / raw)
  To: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>>> much better.

>> Than what?

> Than the native imap search, which is god-awfully slow.

Ah, cool - thanks for the explanation.


  :-),

   Adam

-- 
 "I haven't seen that kind of shit since the bad              Adam Sjøgren
  old NFS days. Apparently my Mac has nostalgia for      asjo@koldfront.dk
  SunOS 4.1.3."




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24 19:34               ` Alan Schmitt
@ 2014-09-25  0:08                 ` Eric Abrahamsen
  0 siblings, 0 replies; 21+ messages in thread
From: Eric Abrahamsen @ 2014-09-25  0:08 UTC (permalink / raw)
  To: ding

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> On 2014-09-25 01:40, Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> To be clear, I use dovecot as a local imap cache. Originally I wasn't
>> using it as a daemon: mail came in via isync, which called dovecot at
>> /usr/lib/dovecot/imap. Then gnus called the same executable via
>> nnimap-shell-program to get the mail. Dovecot only "woke up" when one or
>> the other program called it.
>>
>> Nnir search just used whatever search facilities dovecot provided out of
>> the box. It was slow. I read about integrating lucene into dovecot, and
>> as far as I could tell it wouldn't work when you were just calling
>> dovecot's imap executable -- you had to be running a dovecot instance.
>>
>> So I switched to an always-running dovecot instance, configured the
>> lucene indexer, and now the exact same nnir searches go like lightning.
>>
>> That's basically a summary of the blog post :)
>
> I'd be really curious to hear more about this. Please let us know when
> you publish your post.
>
> Alan

Will do!




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-24 17:40             ` Eric Abrahamsen
  2014-09-24 19:34               ` Alan Schmitt
  2014-09-24 19:47               ` Adam Sjøgren
@ 2014-09-26  7:21               ` Eric S Fraga
  2014-09-26  9:05                 ` Rainer M Krug
  2014-09-26  9:54                 ` Adam Sjøgren
  2 siblings, 2 replies; 21+ messages in thread
From: Eric S Fraga @ 2014-09-26  7:21 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

Eric et al.,

All the proposed solutions require local imap mirroring if I understand
correctly.  This is an appealing approach for a number of
reasons.  However, I have been spectacularly unsuccessful in getting
either offlineimap or isync to synchronise with an MS Exchange
server.  I get filename errors or the transfer simply quits without ever
managing to complete the first synchronisation.  (I should nte that I
haven't tried doing this in over a year now... maybe things have
improved?)  Has anybody here managed it?

Thanks,
eric

-- 
: Eric S Fraga, GnuPG: 0xFFFCF67D
: in Emacs 24.3.1 + Ma Gnus v0.12 + evil-git-0469bee
: BBDB version 3.0.50 (2013-11-16 11:30:49 -0600)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 180 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-26  7:21               ` Eric S Fraga
@ 2014-09-26  9:05                 ` Rainer M Krug
  2014-10-01 18:03                   ` Eric S Fraga
  2014-09-26  9:54                 ` Adam Sjøgren
  1 sibling, 1 reply; 21+ messages in thread
From: Rainer M Krug @ 2014-09-26  9:05 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 2940 bytes --]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Eric et al.,

Hi Eric,

>
> All the proposed solutions require local imap mirroring if I understand
> correctly.  This is an appealing approach for a number of
> reasons.  However, I have been spectacularly unsuccessful in getting
> either offlineimap or isync to synchronise with an MS Exchange
> server.  I get filename errors or the transfer simply quits without ever
> managing to complete the first synchronisation.  (I should nte that I
> haven't tried doing this in over a year now... maybe things have
> improved?)  Has anybody here managed it?

I use offlineimap it for gmail (works), iCloud (works) and a university
accout where they are running ... Exchange server.

Below is the relevant part of my .offlineimaprc file.

My offlineimap synchs to a local dovecot server (lucerne indexing
working!) and puts all folders from this account into a folder named
SUN.

I do the same for my other accounts, so I easily know which ones are
From which account.

But my guess is that the configuration of the Exchange server is probably
different - but if you use imap access at the moment, it should work.

Cheers,

Rainer

--8<---------------cut here---------------start------------->8---
## %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
## %% SUN %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
## %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

[Repository LocalSUN]
type = IMAP
remotehost = localhost
remoteuser = MYUSERNAME
remotepasseval = get_keychain_pass(account="MYUSERNAME")
holdconnectionopen = yes
maxconnections = 3
nametrans = lambda folder: re.sub('^SUN.', '', folder)
folderfilter = lambda folder: folder.startswith('SUN')
keepalive = 60
holdconnectionopen = yes

[Repository RemoteSUN]
type = IMAP
remotehost = imap.sun.ac.za
# port = 143
remoteuser = MYUSERNAME
remotepasseval = get_keychain_pass(account="MYUSERNAME")
######
maxconnections = 1
nametrans = lambda foldername: 'SUN.' + foldername
######
folderfilter = lambda folder: folder in ['INBOX']
## For now:
# readonly = true
# createfolders = false
######
keepalive = 60
holdconnectionopen = yes

[Account SUN]
localrepository =   LocalSUN
remoterepository = RemoteSUN
######
status_backend = sqlite
######
# postsynchook = ionice -c 3 chrt --idle 0 /bin/sh -c "notmuch new"
# postsynchook = ionice -c 3 chrt --idle 0 /bin/sh -c "notmuch new && afew --tag --verbose --new"
######
######
## For daemon mode
# Minutes between syncs
autorefresh = 2
# Number of quick-syncs between autorefreshes. Quick-syncs do not update if the
# only changes were to IMAP flags
quick = 10
--8<---------------cut here---------------end--------------->8---

 
>
> Thanks,
> eric

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-26  7:21               ` Eric S Fraga
  2014-09-26  9:05                 ` Rainer M Krug
@ 2014-09-26  9:54                 ` Adam Sjøgren
  2014-10-01 19:47                   ` Eric S Fraga
  1 sibling, 1 reply; 21+ messages in thread
From: Adam Sjøgren @ 2014-09-26  9:54 UTC (permalink / raw)
  To: ding

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> I have been spectacularly unsuccessful in getting either offlineimap
> or isync to synchronise with an MS Exchange server. I get filename
> errors or the transfer simply quits without ever managing to complete
> the first synchronisation. (I should nte that I haven't tried doing
> this in over a year now... maybe things have improved?) Has anybody
> here managed it?

My workaround is to use Exchange IMAP as a glorified POP3, so I just get
new emails from Exchange, but never manipulate anything on Exchange.

I.e. as described in:

 * http://gnus.org/manual/gnus_179.html#SEC179


  Mvh.

   Adam

-- 
 "You do not know what geometry means, do you?"               Adam Sjøgren
 "Of course I do, it is a type of camel."                asjo@koldfront.dk




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-26  9:05                 ` Rainer M Krug
@ 2014-10-01 18:03                   ` Eric S Fraga
  0 siblings, 0 replies; 21+ messages in thread
From: Eric S Fraga @ 2014-10-01 18:03 UTC (permalink / raw)
  To: ding

On Friday, 26 Sep 2014 at 11:05, Rainer M Krug wrote:
> I use offlineimap it for gmail (works), iCloud (works) and a university
> accout where they are running ... Exchange server.
>
> Below is the relevant part of my .offlineimaprc file.

Thanks.  My configuration was not radically different other than I had
to exclude a large number of folders (ones like Calendar which are not
real email folders) but I'll play around some more when I have a
chance.

-- 
: Eric S Fraga, GnuPG: 0xFFFCF67D
: in Emacs 24.4.50.1 + Ma Gnus v0.12 + evil-git-4382854
: BBDB version 3.1.2 (2014-05-06 11:45:08 -0500)



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: nnir and notmuch?
  2014-09-26  9:54                 ` Adam Sjøgren
@ 2014-10-01 19:47                   ` Eric S Fraga
  0 siblings, 0 replies; 21+ messages in thread
From: Eric S Fraga @ 2014-10-01 19:47 UTC (permalink / raw)
  To: ding

On Friday, 26 Sep 2014 at 11:54, Adam Sjøgren wrote:

[...]

> My workaround is to use Exchange IMAP as a glorified POP3, so I just get
> new emails from Exchange, but never manipulate anything on Exchange.

Ummm, intriguing.  Sort of /back to the future/.  I need to think about
this approach as I am definitely used to having IMAP access to my email
from a number of devices...

thanks,
eric

-- 
: Eric S Fraga, GnuPG: 0xFFFCF67D
: in Emacs 24.4.50.1 + Ma Gnus v0.12 + evil-git-4382854
: BBDB version 3.1.2 (2014-05-06 11:45:08 -0500)



^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2014-10-01 19:47 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-23  8:40 nnir and notmuch? Rainer M Krug
2014-09-23 10:15 ` Steinar Bang
2014-09-23 10:24   ` Rasmus
2014-09-23 10:50     ` Rainer M Krug
2014-09-23 10:48   ` Rainer M Krug
2014-09-23 11:59     ` Steinar Bang
2014-09-23 13:15     ` Andrew Cohen
2014-09-24  7:18       ` Rainer M Krug
2014-09-23 16:14     ` Eric Abrahamsen
2014-09-24  7:26       ` Rainer M Krug
2014-09-24  9:41         ` Eric Abrahamsen
2014-09-24 13:51           ` Adam Sjøgren
2014-09-24 17:40             ` Eric Abrahamsen
2014-09-24 19:34               ` Alan Schmitt
2014-09-25  0:08                 ` Eric Abrahamsen
2014-09-24 19:47               ` Adam Sjøgren
2014-09-26  7:21               ` Eric S Fraga
2014-09-26  9:05                 ` Rainer M Krug
2014-10-01 18:03                   ` Eric S Fraga
2014-09-26  9:54                 ` Adam Sjøgren
2014-10-01 19:47                   ` Eric S Fraga

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).