From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/86521 Path: news.gmane.org!not-for-mail From: myglc2 Newsgroups: gmane.emacs.gnus.general Subject: Re: nnimap backend performances ? Date: Sun, 03 Jan 2016 19:35:54 -0500 Message-ID: <874meukwph.fsf@gmail.com> References: <874mgh1amt.fsf@ericabrahamsen.net> <87twn19hwq.fsf@gmail.com> <87h9iz1789.fsf@ericabrahamsen.net> <87h9iyq7o8.fsf@gmail.com> <871ta08xcq.fsf@ericabrahamsen.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1451867926 22655 80.91.229.3 (4 Jan 2016 00:38:46 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 4 Jan 2016 00:38:46 +0000 (UTC) Cc: ding@gnus.org To: Eric Abrahamsen Original-X-From: ding-owner+M34748@lists.math.uh.edu Mon Jan 04 01:38:34 2016 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from lists1.math.uh.edu ([129.7.128.208]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aFtAP-0004p2-Ti for ding-account@gmane.org; Mon, 04 Jan 2016 01:38:34 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by lists1.math.uh.edu with smtp (Exim 4.85) (envelope-from ) id 1aFt9T-00082C-SB; Sun, 03 Jan 2016 18:37:35 -0600 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by lists1.math.uh.edu with esmtps (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.85) (envelope-from ) id 1aFt9Q-00081s-U7 for ding@lists.math.uh.edu; Sun, 03 Jan 2016 18:37:33 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtps (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1aFt9P-0002Oz-MD for ding@lists.math.uh.edu; Sun, 03 Jan 2016 18:37:32 -0600 Original-Received: from mail-qg0-f51.google.com ([209.85.192.51]) by quimby.gnus.org with esmtps (TLS1.2:RSA_ARCFOUR_SHA1:128) (Exim 4.80) (envelope-from ) id 1aFt9N-0001ll-UF for ding@gnus.org; Mon, 04 Jan 2016 01:37:30 +0100 Original-Received: by mail-qg0-f51.google.com with SMTP id o11so244816719qge.2 for ; Sun, 03 Jan 2016 16:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=K3X8oIuqWgbuFYyO7PaWQHkP7wBKxS9Thix+AuP5J90=; b=JhLAiH16nNHsJu4hFJLeByQ3d2DR66708EgenKtAmSwZ+IfzuOZnzdfjh/yGgYgy+W m3dKTYdw3qhPGtP7UyCaPk30ftkxOPV0Wt5j8HHYjjQ6KYUAAUKDN3GrwO3rFHzYKFBS HacT1pQ0Yk8jTgUQZ/7LL8UYwkbbY8hshyNz/X4oCY4RVttJaNkU+xHdtQyZ6bV0Hul4 AUhLpQ9Gx87eZwgsKF9Cfl7GCMhjXJbVRyaoeK8WeVwgEC0l+o7tJ3rwBsWj4JoOw3jc QneaXA88KBeklfqGULK5E7epIlWksgCVhxd95aaIA43lIXOipXw/LSnWYkmfVM/kaw4y DAfQ== X-Received: by 10.140.242.147 with SMTP id n141mr70600272qhc.2.1451867843413; Sun, 03 Jan 2016 16:37:23 -0800 (PST) Original-Received: from e3a (c-73-167-118-254.hsd1.ma.comcast.net. [73.167.118.254]) by smtp.gmail.com with ESMTPSA id f3sm17103623qge.44.2016.01.03.16.37.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Jan 2016 16:37:22 -0800 (PST) In-Reply-To: <871ta08xcq.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Sat, 02 Jan 2016 11:38:13 +0800") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-Spam-Score: -2.8 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:86521 Archived-At: Eric Abrahamsen writes: >> myglc2 writes: >> >> FWIW, My goal is to sweep multiple gmail accounts into bullet-proof >> archives with tools for searching and clustering several decades of >> work. To get a handle on what is doable, I am experimenting with gnus >> backends and search schemes, mu/mu4e, dovecot, and mbsync. >> >> I would love to hear suggestions of other emacs-compatible solutions I >> should try. > > I might not be too helpful here, unfortunately. I've only ever used > gnus/dovecot/isync. As I said in that post, I think dovecot plus a fts > plugin works great, but I really don't like the search syntax on the > Gnus/nnir side -- I find it very cumbersome. If there were a nicer > syntax for searching, I think I'd be happy with this. Thanks, that is helpful Search syntax is important to me. Lord knows I don't need any new obscure stuff to learn. Doesn't nnir simply pass the search string to the backend. If so, is the syntax you don't like associated with nnimap, IMAP spec, or lucene? I have a toy installation of dovecot running with a small number of messages. Search is very fast but article fetching is surprisingly slow. Have you experienced anything like this? > I could also imagine using nnmaildir -- it seems simpler, at least > conceptually, if all you want is to keep mails archived. In that case > you could probably use notmuch to index the mails, and set nnir to use > that notmuch installation as the search backend. If you don't need > accounts and authentication and everything for a simple local archive, > that should be enough. I'll bet the searching would be even faster, as > well. I currently have mu4e set up, which uses the same indexer (xapian) as notmuch with the same set of messages in Maildir. The search response seems as fast, and article fetch seems much faster. I plan to also set up notmuch and mairix. Then I will have: gnus -- nnir -- nnimap -- dovecot -- maildir lucene messages (index) gnus -- nnir -- notmuch -- maildir xapian messages (index) mu4e -- mu -- maildir xapian messages (index) gnus -- mairix -- maildir flex messages (index) smart group (search cache) Then I can inflate the size of the message stores and compare. Frankly, it is hard to imagine why performance shouldn't be pretty similar across the board. As far as I can tell from reading and other helpful exchanges on the list, the only major difference is that mairix caches a search result in a so-called "smart" permanent group. This is implemented by a maildir folder containing symbolic links to the actual messages. In theory, the second time we preform the exact same search, it should be faster. If we search for the same thing over and over it probably will be faster. If most or our searches are unique, making a cache probably will be slower. If in practice we search for the same term over an over and marix lets us just click on a group containing the result instead of typing the search term, this could indeed be nicer even if getting the search result is slower. As far as syntax, I am assume that mu4e and notmuch use a xapian syntax. Mairix uses the Flex lexical analyzer, so I assume it is different. Meanwhile I am trying to determine if there are other meaningful mairix functional differences. - George