searching speedup?

searching speedup?

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

Hello all,

I often use G G to search a group.  With most imap servers,
e.g. Google's gmail, this works really well.  However, with Microsoft's
outlook.office365.com exchange server accessed as an imap server, it
takes a very long time (minutes) to satisfy the search
request.  Annoyingly, if I use the web interface for this server,
searches are fast.  It's only through imap that they are slow, it would
seem.

Does anybody have any suggestions on how to search a very slow server?
I would be happy with a search limited to author and subject line if
need be, if it were fast.

Thank you,
eric

-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.6 on Debian buster/sid

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: searching speedup?

[Eric Abrahamsen]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Hello all,
>
> I often use G G to search a group.  With most imap servers,
> e.g. Google's gmail, this works really well.  However, with Microsoft's
> outlook.office365.com exchange server accessed as an imap server, it
> takes a very long time (minutes) to satisfy the search
> request.  Annoyingly, if I use the web interface for this server,
> searches are fast.  It's only through imap that they are slow, it would
> seem.

My guess is that outlook is building an index for the web interface that
isn't available for the IMAP server. Unfortunately you can't do anything
about that -- when issuing an IMAP SEARCH command, only the server knows
if a search index is available or not. It's possible that there is an
index but it's gotten corrupted, but then you'd expect the web interface
to be slow too. It's more likely that they've done one for the web, but
neglected IMAP.

If you have a copy of outlook the MUA on your machine, this link might
(?) be useful:

https://www.stellarinfo.com/blog/fix-outlook-2016-search-problems/

Re: searching speedup?

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1122 bytes --]

On Tuesday,  8 May 2018 at 14:16, Eric Abrahamsen wrote:
> My guess is that outlook is building an index for the web interface that
> isn't available for the IMAP server. Unfortunately you can't do anything
> about that -- when issuing an IMAP SEARCH command, only the server knows
> if a search index is available or not. It's possible that there is an
> index but it's gotten corrupted, but then you'd expect the web interface
> to be slow too. It's more likely that they've done one for the web, but
> neglected IMAP.

This sounds very likely.

> If you have a copy of outlook the MUA on your machine, this link might
> (?) be useful:
>
> https://www.stellarinfo.com/blog/fix-outlook-2016-search-problems/

Thanks.  I don't have Outlook and the web interface does not provide any
links for rebuilding the search index.  I don't think there are search
problems as such, given that the OWA interface is fast.  I think it is
indeed that the imap interface to the server is not properly
supported.  Which is not a surprise.

eric
-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.6 on Debian buster/sid

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: searching speedup?

[Steinar Bang]

>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:

> Eric S Fraga <e.fraga@ucl.ac.uk> writes:

>> I often use G G to search a group.  With most imap servers,
>> e.g. Google's gmail, this works really well.  However, with Microsoft's
>> outlook.office365.com exchange server accessed as an imap server, it
>> takes a very long time (minutes) to satisfy the search
>> request.  Annoyingly, if I use the web interface for this server,
>> searches are fast.  It's only through imap that they are slow, it would
>> seem.

> My guess is that outlook is building an index for the web interface that
> isn't available for the IMAP server. Unfortunately you can't do anything
> about that -- when issuing an IMAP SEARCH command, only the server knows
> if a search index is available or not. It's possible that there is an
> index but it's gotten corrupted, but then you'd expect the web interface
> to be slow too. It's more likely that they've done one for the web, but
> neglected IMAP.

Here's a possible workaround: use davmail
 http://davmail.sourceforge.net

Davmail accesses the Exchange web interface and exposes it as IMAP (and
SMTP and caldav and LDAP).

A theoretical downside to this, is that the IMAP operation here should
be much slower than using the Exchange IMAP directly (since you first
access the folders using the web interface, and then get the IMAP
traffic).  However, if the IMAP support of Exchange is as neglected as
it seems to be, then this may not actually be the case.

And if you're lucky, searching through davmail IMAP would utilize the
FAST search that the web interface enjoys.

Re: searching speedup?

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 495 bytes --]

On Thursday, 10 May 2018 at 10:15, Steinar Bang wrote:

[...]

> Here's a possible workaround: use davmail
>  http://davmail.sourceforge.net

Interesting suggestion.  I've read the documentation and it looks like
it's definitely worth a try.  I cannot imagine it being slower than
Exchange's own imap server.  Thank you.  If/when I get this working,
I'll update here for future reference.

Thanks again,
eric

-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.6 on Debian buster/sid

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: searching speedup?

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On Thursday, 10 May 2018 at 10:15, Steinar Bang wrote:
> Here's a possible workaround: use davmail
>  http://davmail.sourceforge.net

So, I finally found time to play with this.  Essentially worked out of
the box (debian apt install and default server configuration) but I must
admit that I stressed tested it as my exchange inbox has >1/2 million
emails...  davmail was essentially unusable until I set the limit on how
many messages to fetch in each folder:

    davmail.folderSizeLimit=1000

and now it works very well.  I haven't played around with what is a
reasonable value for this setting nor what impact it has on other
aspects of use.  In any case, searching is faster with davmail which was
my main motivation.

So, thank you for this suggestion.

-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.6 on Debian buster/sid

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: searching speedup?

[Steinar Bang]

>>>>> Eric S Fraga <e.fraga@ucl.ac.uk>:

> So, thank you for this suggestion.

You're welcome!

Thanks for giving it such a thorough test! :-)