From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 5853 invoked from network); 28 Jun 2021 16:42:31 -0000 Received: from mx1.math.uh.edu (129.7.128.32) by inbox.vuxu.org with ESMTPUTF8; 28 Jun 2021 16:42:31 -0000 Received: from lists1.math.uh.edu ([129.7.128.208]) by mx1.math.uh.edu with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lxuL5-00AUy6-Vi for ml@inbox.vuxu.org; Mon, 28 Jun 2021 11:42:27 -0500 Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by lists1.math.uh.edu with smtp (Exim 4.94) (envelope-from ) id 1lxuL5-005eOy-GI for ml@inbox.vuxu.org; Mon, 28 Jun 2021 11:42:27 -0500 Received: from mx1.math.uh.edu ([129.7.128.32]) by lists1.math.uh.edu with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lxuL3-005eOq-2v for ding@lists.math.uh.edu; Mon, 28 Jun 2021 11:42:25 -0500 Received: from quimby.gnus.org ([95.216.78.240]) by mx1.math.uh.edu with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lxuL1-00AUxn-0T for ding@lists.math.uh.edu; Mon, 28 Jun 2021 11:42:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:Mime-Version:Message-ID:Date:Subject:From:To: Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=RXmjIY9DsVkbVEReVOK6c4YHQhYqusTZZ9EVqjLV6bw=; b=sIBL5s6fW0E4ktuRKKAei3uBzU w/w4Wk2Mg6sfDhK8+/i1+6sofm4qIyA/W+d/URzkmjPQYjzubgP1f0kCO/f+7N4laXWq734b9Ewk2 MsHyrHN3ph11JsP8ko9N1NmBP7Ys7EtghoquwtsIAstqjKxQrQWzE/sJ45RvKpcZ0mwI=; Received: from ciao.gmane.io ([116.202.254.214]) by quimby.gnus.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lxuKt-0004MD-Cj for ding@gnus.org; Mon, 28 Jun 2021 18:42:17 +0200 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1lxuKs-00094s-MR for ding@gnus.org; Mon, 28 Jun 2021 18:42:14 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: ding@gnus.org From: Eric Abrahamsen Subject: Help testing gnus-search fixups Date: Mon, 28 Jun 2021 09:42:08 -0700 Message-ID: <87sg12uejj.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cancel-Lock: sha1:R+alCc/5cw4kMSnZITF8J9yg5og= List-ID: Precedence: bulk --=-=-= Content-Type: text/plain Hi all, I've reworked the way gnus-search parses the output from search utilities like notmuch, which should eliminate some of the bugs and funny behavior that's been reported here. I'd be much happier if I had some more eyes on it -- if you're using notmuch or namazu, please help me give this a whirl! Thanks, Eric --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Rework-gnus-search-indexed-parse-output.patch >From 7f9e962b890bfa0924ca17a7abd8cc83e1b8d2bf Mon Sep 17 00:00:00 2001 From: Eric Abrahamsen Date: Sat, 26 Jun 2021 10:16:19 -0700 Subject: [PATCH 1/1] Rework gnus-search-indexed-parse-output * lisp/gnus/gnus-search.el (gnus-search-indexed-parse-output): Be more careful about matching filesystem paths to Gnus group names; make absolutely sure that we only return valid article numbers. --- lisp/gnus/gnus-search.el | 94 ++++++++++++++++++---------------------- 1 file changed, 42 insertions(+), 52 deletions(-) diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el index 70bde264c1..add34e7e02 100644 --- a/lisp/gnus/gnus-search.el +++ b/lisp/gnus/gnus-search.el @@ -1351,68 +1351,58 @@ gnus-search-run-search (cl-defmethod gnus-search-indexed-parse-output ((engine gnus-search-indexed) server query &optional groups) - (let ((prefix (slot-value engine 'remove-prefix)) - (group-regexp (when groups - (mapconcat - (lambda (group-name) - (mapconcat #'regexp-quote - (split-string - (gnus-group-real-name group-name) - "[.\\/]") - "[.\\\\/]")) - groups - "\\|"))) - artlist vectors article group) + (let ((prefix (or (slot-value engine 'remove-prefix) + "")) + artlist article group) (goto-char (point-min)) + ;; Prep prefix, we want to at least be removing the root + ;; filesystem separator. + (when (stringp prefix) + (setq prefix (file-name-as-directory + (expand-file-name prefix "/")))) (while (not (or (eobp) (looking-at-p "\\(?:[[:space:]\n]+\\)?Process .+ finished"))) (pcase-let ((`(,f-name ,score) (gnus-search-indexed-extract engine))) (when (and f-name (file-readable-p f-name) - (null (file-directory-p f-name)) - (or (null groups) - (and (gnus-search-single-p query) - (alist-get 'thread query)) - (string-match-p group-regexp f-name))) - (push (list f-name score) artlist)))) + (null (file-directory-p f-name))) + (setq group + (replace-regexp-in-string + "[/\\]" "." + (replace-regexp-in-string + "/?\\(cur\\|new\\|tmp\\)?/\\'" "" + (replace-regexp-in-string + "\\`\\." "" + (string-remove-prefix + prefix (file-name-directory f-name)) + nil t) + nil t) + nil t)) + (setq article (file-name-nondirectory f-name) + article + ;; TODO: Provide a cleaner way of producing final + ;; article numbers for the various backends. + (if (string-match-p "\\`[[:digit:]]+\\'" article) + (string-to-number article) + (nnmaildir-base-name-to-article-number + (substring article 0 (string-match ":" article)) + group (string-remove-prefix "nnmaildir:" server)))) + (when (and (numberp article) + (or (null groups) + (member group groups))) + (push (list f-name article group score) + artlist))))) ;; Are we running an additional grep query? (when-let ((grep-reg (alist-get 'grep query))) (setq artlist (gnus-search-grep-search engine artlist grep-reg))) - ;; Prep prefix. - (when (and prefix (null (string-empty-p prefix))) - (setq prefix (file-name-as-directory (expand-file-name prefix)))) - ;; Turn (file-name score) into [group article score]. - (pcase-dolist (`(,f-name ,score) artlist) - (setq article (file-name-nondirectory f-name) - group (file-name-directory f-name)) - ;; Remove prefix. - (when prefix - (setq group (string-remove-prefix prefix group))) - ;; Break the directory name down until it's something that - ;; (probably) can be used as a group name. - (setq group - (replace-regexp-in-string - "[/\\]" "." - (replace-regexp-in-string - "/?\\(cur\\|new\\|tmp\\)?/\\'" "" - (replace-regexp-in-string - "^[./\\]" "" - group nil t) - nil t) - nil t)) - - (push (vector (gnus-group-full-name group server) - (if (string-match-p "\\`[[:digit:]]+\\'" article) - (string-to-number article) - (nnmaildir-base-name-to-article-number - (substring article 0 (string-match ":" article)) - group (string-remove-prefix "nnmaildir:" server))) - (if (numberp score) - score - (string-to-number score))) - vectors)) - vectors)) + ;; Munge into the list of vectors expected by nnselect. + (mapcar (pcase-lambda (`(,_ ,article ,group ,score)) + (vector group article + (if (numberp score) + score + (string-to-number score)))) + artlist))) (cl-defmethod gnus-search-indexed-extract ((_engine gnus-search-indexed)) "Base implementation treats the whole line as a filename, and -- 2.32.0 --=-=-=--