From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/66234 Path: news.gmane.org!not-for-mail From: Justus-bulk@Piater.name Newsgroups: gmane.emacs.gnus.general Subject: nnmaildir support for nnir.el Date: Sun, 03 Feb 2008 14:04:27 +0100 Message-ID: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1202043962 24613 80.91.229.12 (3 Feb 2008 13:06:02 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 3 Feb 2008 13:06:02 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M14725@lists.math.uh.edu Sun Feb 03 14:06:24 2008 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1JLeY2-000367-PF for ding-account@gmane.org; Sun, 03 Feb 2008 14:06:15 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1JLeWw-0004SP-1i; Sun, 03 Feb 2008 07:05:06 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JLeWu-0004S6-G7 for ding@lists.math.uh.edu; Sun, 03 Feb 2008 07:05:04 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1JLeWo-0002GB-8C for ding@lists.math.uh.edu; Sun, 03 Feb 2008 07:05:04 -0600 Original-Received: from serv108.segi.ulg.ac.be ([139.165.32.111]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1JLeWr-0004Xk-00 for ; Sun, 03 Feb 2008 14:05:01 +0100 Original-Received: (qmail 4010 invoked from network); 3 Feb 2008 14:04:26 +0100 Original-Received: from unknown (HELO tool.montefiore.ulg.ac.be) (u193113@[83.186.109.22]) (envelope-sender ) by serv108.segi.ulg.ac.be (qmail-ldap-1.03) with SMTP for ; 3 Feb 2008 14:04:26 +0100 User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (gnu/linux) X-Spam-Score: 0.8 (/) X-Spam-Report: SpamAssassin (3.1.9 2007-02-13) analysis follows Bayesian score: 0.0000 Ham tokens: 0.000-3527--11743h-0s--0d--H*UA:Emacs, 0.000-2991--9958h-0s--0d--H*UA:Gnus, 0.000-2991--9957h-0s--0d--H*u:Gnus, 0.000-2851--9491h-0s--0d--H*u:Emacs, 0.000-2248--7485h-0s--0d--H*UA:linux Spam tokens: 0.963-5212--1389h-59829s--0d--HX-Spam-Relays-External:quimby.gnus.org, 0.963-5212--1389h-59829s--0d--H*RU:quimby.gnus.org, 0.958-1--0h-1s--0d--rolled, 0.949-4315--1806h-55476s--0d--H*Ad:D*gnus.org, 0.940-4210--2294h-58951s--0d--HTo:D*gnus.org Autolearn status: no 1.0 NO_REAL_NAME From: does not include a real name 0.1 FORGED_RCVD_HELO Received: contains a forged HELO -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% [score: 0.0000] 2.3 AWL AWL: From: address is in the auto white-list List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:66234 Archived-At: [Repost from my newly-subscribed address] Hi, I've hacked up some initial nnmaildir support for nnir.el (gnus contrib), for the Swish++ and Namazu backends. During the process, I rolled some redundant code shared by these backends into a new function, which may be useful for other backends as well. Please review and test, and send me your comments. I hope this (or an improved version) can be committed to CVS. See below a diff with respect to the latest CVS version (7.22) for convenient reviewing. For convenient testing, you can download the full file at http://www.montefiore.ulg.ac.be/~piater/test/nnir.el . Thanks, Justus diff -c /local/home/piater/share/emacs/lisp/nnir-orig.el /local/home/piater/share/emacs/lisp/nnir.el *** /local/home/piater/share/emacs/lisp/nnir-orig.el 2008-01-04 15:10:46.200002809 +0100 --- /local/home/piater/share/emacs/lisp/nnir.el 2008-02-02 21:05:29.663526101 +0100 *************** *** 661,667 **** (if (string-match "^\\([^+]+\\)\\+\\(.+\\)$" gname) (format "%s:%s" (match-string 1 gname) (match-string 2 gname)) (concat gname ":"))) ! "native"))) ;; Summary mode commands. --- 663,669 ---- (if (string-match "^\\([^+]+\\)\\+\\(.+\\)$" gname) (format "%s:%s" (match-string 1 gname) (match-string 2 gname)) (concat gname ":"))) ! (format "%s:%s" (car gnus-select-method) (cadr gnus-select-method))))) ;; Summary mode commands. *************** *** 830,835 **** --- 832,865 ---- (nnoo-define-skeleton nnir) + + ;; Helper function currently used by the Swish++ and Namazu backends; + ;; perhaps useful for other backends as well + (defun nnir-compose-result (dirnam article score prefix server) + "Extract the group from dirnam, and create a result vector + ready to be added to the list of search results." + + ;; remove nnir-*-remove-prefix from beginning of dirnam filename + (when (string-match (concat "^" prefix) dirnam) + (setq dirnam (replace-match "" t t dirnam))) + + ;; remove trailing slash and, for nnmaildir, cur/new/tmp + (setq dirnam (substring dirnam 0 (if (string= server "nnmaildir:") -5 -1))) + + ;; eliminate all ".", "/", "\" from beginning. Always matches. + (string-match "^[./\\]*\\(.*\\)$" dirnam) + (setq group (substitute ?. ?/ (match-string 1 dirnam))) ;; "/" -> "." + (setq group (substitute ?. ?\\ group)) ;; "\\" -> "." + + (vector (nnir-group-full-name group server) + (if (string= server "nnmaildir:") + (nnmaildir-base-name-to-article-number + (substring article 0 (string-match ":" article)) + group nil) + (string-to-int article)) + (string-to-int score))) + + ;;; Search Engine Interfaces: ;; freeWAIS-sf interface. *************** *** 939,944 **** --- 969,977 ---- (groupspec (cdr (assq 'group query))) (prefix (nnir-read-server-parm 'nnir-swish++-remove-prefix server)) (artlist nil) + (article-pattern (if (string= server "nnmaildir:") + ":[0-9]+" + "^[0-9]+\\(\\.[a-z0-9]+\\)?$")) (score nil) (artno nil) (dirnam nil) (group nil) ) (when (equal "" qstring) *************** *** 986,992 **** dirnam (file-name-directory (match-string 2))) ;; don't match directories ! (when (string-match "^[0-9]+\\(\\.[a-z0-9]+\\)?$" artno) ;; nnml-use-compressed-files might be any string, but probably this ;; is sufficient. Note that we can't only use the value of ;; nnml-use-compressed-files because old articles might have been --- 1019,1025 ---- dirnam (file-name-directory (match-string 2))) ;; don't match directories ! (when (string-match article-pattern artno) ;; nnml-use-compressed-files might be any string, but probably this ;; is sufficient. Note that we can't only use the value of ;; nnml-use-compressed-files because old articles might have been *************** *** 996,1019 **** ;; maybe limit results to matching groups. (when (or (not groupspec) (string-match groupspec dirnam)) ! ! ;; remove nnir-swish++-remove-prefix from beginning of dirname ! (when (string-match (concat "^" prefix) ! dirnam) ! (setq dirnam (replace-match "" t t dirnam))) ! ! (setq dirnam (substring dirnam 0 -1)) ! ;; eliminate all ".", "/", "\" from beginning. Always matches. ! (string-match "^[./\\]*\\(.*\\)$" dirnam) ! ;; "/" -> "." ! (setq group (substitute ?. ?/ (match-string 1 dirnam))) ! ;; "\\" -> "." ! (setq group (substitute ?. ?\\ group)) ! ! (push (vector (nnir-group-full-name group server) ! (string-to-int artno) ! (string-to-int score)) ! artlist))))) (message "Massaging swish++ output...done") --- 1029,1036 ---- ;; maybe limit results to matching groups. (when (or (not groupspec) (string-match groupspec dirnam)) ! (push (nnir-compose-result dirnam artno score prefix server) ! artlist))))) (message "Massaging swish++ output...done") *************** *** 1207,1212 **** --- 1224,1232 ---- (error "The Namazu backend cannot search specific groups")) (save-excursion (let ( + (article-pattern (if (string= server "nnmaildir:") + ":[0-9]+" + "^[0-9]+$")) (artlist nil) (qstring (cdr (assq 'query query))) (prefix (nnir-read-server-parm 'nnir-namazu-remove-prefix server)) *************** *** 1256,1273 **** article (file-name-nondirectory (match-string 4))) ;; make sure article and group is sane ! (when (and (string-match "^[0-9]+$" article) (not (null group))) ! (when (string-match (concat "^" prefix) group) ! (setq group (replace-match "" t t group))) ! ! ;; remove trailing slash from groupname ! (setq group (substring group 0 -1)) ! ! ;; stuff results into artlist vector ! (push (vector (nnir-group-full-name (substitute ?. ?/ group) server) ! (string-to-int article) ! (string-to-int score)) artlist))) ;; sort artlist by score (apply 'vector --- 1276,1285 ---- article (file-name-nondirectory (match-string 4))) ;; make sure article and group is sane ! (when (and (string-match article-pattern article) (not (null group))) ! (push (nnir-compose-result group article score prefix server) ! artlist))) ;; sort artlist by score (apply 'vector Diff finished. Sat Feb 2 21:20:46 2008