First experimental version of nnglimpse.el, or: Buglet report

First experimental version of nnglimpse.el, or: Buglet report

[Kai Grossjohann]

Hi all,

I have finally gotten around to taking a first shot at implementing
nnglimpse.el, a way of searching your mail from within Gnus if you
use Glimpse.

Or you can read it as a bug report.  Search for the string `Bug?' to
see what it is.  At least I think it's a bug.

Don't forget your asbestos longjohns, this is pre-pre-pre-alpha code!
(If you don't know what to do with the Lisp code, you don't have the
right kind of longjohns, yet ;-)

Limitations/assumptions:

  - Mail backend is (nnml "")
  - Mail is stored in $HOME/Mail
  - Glimpse index is stored in $HOME (uses `-H $HOME' option for
    running glimpse)
  - Pretends to be somewhat customizable, but probably isn't
  - Ugly code

Still, I'm posting this in case someone actually wants to look at it
and make some suggestions.

How would I go about adding the group name the article came from to
the output of nnglimpse?  To the nnglimpse summary buffer?  To the
article buffer?

Those of you who have indexed their mail with Glimpse:  How have you
set up your Glimpse index?  Which files are indexed, where does the
`-H' option point?  What kinds of file names does Glimpse output?
$HOME/Mail/... just like for me?

Those of you who don't use nnml but instead use nnfolder or nnmbox or
nnbabyl:  Have you got any idea how to determine the article number
from the output of Glimpse?

Another idea: I'd like to define a function for the nnglimpse summary
buffer which enters another summary buffer, showing the group the
article is actually coming from.  Since there is this nifty command
gnus-summary-refer-thread it would be quite easy to create a summary
buffer showing just the one article, then invoke this nifty command,
to produce a summary buffer showing the whole thread the article
belongs to.

How does one go about defining a function for nnglimpse summary
buffers only?  How does one go about creating such a `subsidiary'
summary buffer, such that one gets back to the nnglimpse summary
buffer with `q'?

kai
-- 
You ate somebody? -- Just a leg. -- That's terrible! -- Not with mustard.
(Terry Pratchett: Interesting Times)

;;; nnglimpse.el --- search mail with Glimpse
;; Copyright (C) 1998 Kai Großjohann

;; Author: Kai Großjohann <grossjohann@ls6.cs.uni-dortmund.de>
;; Keywords: news, mail, searching, glimpse

;; This file is not part of GNU Emacs.

;; This is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, Inc., 59 Temple Place - Suite 330,
;; Boston, MA 02111-1307, USA.

;;; Commentary:

;;; Code:

;; Setup.
(nnoo-declare nnglimpse)
(nnoo-define-skeleton nnglimpse)
(nnoo-define-basics nnglimpse)

(gnus-declare-backend "nnglimpse" 'mail)

;; Variables.

(defvoo nnglimpse-glimpse-program "glimpse"
  "Name of Glimpse executable.")

(defvoo nnglimpse-glimpse-home (getenv "HOME")
  "Value of `-H' cmd line argument in running Glimpse.")

(defvoo nnglimpse-mail-backend '(nnml "")
  "Backend that is used for storing mail.
Currently, only nnml and nnmh (and compatible) backends are supported.")

;; Internal variables.

(defvoo nnglimpse-current-query nil
  "Internal: stores current query (= group name).")

(defvoo nnglimpse-articles nil
  "Internal: stores search result.")

(defvoo nnglimpse-scratch-buffer (get-buffer-create "*nnglimpse*")
  "Internal: temporary buffer.")

;; Functions.

;; Gnus glue.

(defun gnus-group-make-nnglimpse-group (query)
  (interactive "sGlimpse query: ")
  (gnus-group-read-ephemeral-group query '(nnglimpse "") t nil nil))

(define-key gnus-group-mode-map (kbd "G G")
  'gnus-group-make-nnglimpse-group)

;; Interface functions.

(deffoo nnglimpse-open-server (server &optional definitions)
  ;; just set server variables appropriately
  (nnoo-change-server 'nnglimpse server definitions)
  )

(deffoo nnglimpse-request-group (group &optional server fast)
  "GROUP is the Glimpse query."
  (nnglimpse-possibly-change-server server)
  (setq nnglimpse-articles (nnglimpse-run-glimpse group))
  ;; Result data
  (save-excursion
    (set-buffer nntp-server-buffer)
    (nnheader-insert "211 %d %d %d %s\n"                           ; art nos:
                     (nnglimpse-artlist-length nnglimpse-articles) ; total
                     1                                             ; lowest
                     (nnglimpse-artlist-length nnglimpse-articles) ; highest
                     group)               ; group name
    ))

;; Ignores group argument -- uses nnglimpse-current-query instead.
(deffoo nnglimpse-retrieve-headers (articles &optional group server fetch-old)
  (save-excursion
    (let ((artlist (copy-sequence articles))
          (idx 1)
          (art nil)
          (artitem nil)
          (artgroup nil) (artno nil)
          (artfullgroup nil)
          (novitem nil)
          (novdata nil)
          (foo nil))
      (while (not (null artlist))
        (setq art (car artlist))
        (or (numberp art)
            (nnheader-report
             "nnglimpse-retrieve-headers doesn't grok message ids: %s"
             art))
        (setq artitem (nnglimpse-article-from-artlist nnglimpse-articles art))
        (setq artgroup (nnglimpse-artitem-group artitem))
        (setq artno (nnglimpse-artitem-number artitem))
        (setq artfullgroup (gnus-group-prefixed-name artgroup
                                                     nnglimpse-mail-backend))
        ;; retrieve NOV or HEAD data for this article, transform into
        ;; NOV data and prepend to `novdata'
        (set-buffer nntp-server-buffer)
        (case (setq foo (gnus-retrieve-headers (list artno) artfullgroup nil))
          (nov
           (setq novitem (nnheader-parse-nov)))
          (headers
           (setq novitem (nnheader-parse-head)))
          (t (nnheader-report "Don't support header type %s." foo)))
        ;; replace article number in original group with article number
        ;; in nnglimpse group
        (aset novitem 0 idx)
        (push novitem novdata)
        (setq artlist (cdr artlist))
        (setq idx (1+ idx)))
      (setq novdata (nreverse novdata))
      (set-buffer nntp-server-buffer) (erase-buffer)
      (mapcar 'nnheader-insert-nov novdata)
      'nov)))

;(deffoo nnglimpse-close-server (&optional server)
;  ; ...
;  )

;(deffoo nnglimpse-request-close ()
;  ; ...
;  )

;(deffoo nnglimpse-server-opened (&optional server)
;  ; ...
;  )

;(deffoo nnglimpse-status-message (&optional server)
;  ; ...
;  )

(deffoo nnglimpse-request-article (article
                                   &optional group server to-buffer)
  (save-excursion
    (let* ((artitem (nnglimpse-article-from-artlist nnglimpse-articles
                                                    article))
           (artgroup (nnglimpse-artitem-group artitem))
           (artno (nnglimpse-artitem-number artitem))
           ;; Bug?
           ;; Why must we bind nntp-server-buffer here?  It won't
           ;; work if `buf' is used, say.  (Of course, the set-buffer
           ;; line below must then be updated, too.)
           (nntp-server-buffer (or to-buffer nntp-server-buffer)))
      (set-buffer nntp-server-buffer)
      (erase-buffer)
      (message "Requesting article %d from group %s"
               artno
               (gnus-group-prefixed-name artgroup
                                         nnglimpse-mail-backend))
      (gnus-request-article artno
                            (gnus-group-prefixed-name artgroup
                                                      nnglimpse-mail-backend)
                            nntp-server-buffer)
      (cons artgroup artno))))

;(deffoo nnglimpse-close-group (group &optional server)
;  ; ...
;  )

;(deffoo nnglimpse-request-list (&optional server)
;  ; ...
;  )

;(deffoo nnglimpse-request-post (&optional server)
;  ; ...
;  )

;;; Internal functions

(defun nnglimpse-possibly-change-server (server)
  (unless (and server (nnglimpse-server-opened server))
    (nnglimpse-open-server server)))

(defun nnglimpse-run-glimpse (query)
  "Run given query against glimpse.  Returns a list of (group name, file name)
pairs."
  (save-excursion
    (let ((fnprefix (concat (getenv "HOME") "/Mail/"))
          (artlist nil))
      (setq nnglimpse-current-query query)
      (set-buffer nnglimpse-scratch-buffer)
      (erase-buffer)
      (message "Doing glimpse query %s..." query)
      ;; CCC debug
      (if (string= query "xyzzy")
          [["wimis" 97] ["wimis" 127]]
        (call-process nnglimpse-glimpse-program
                      nil               ; input from /dev/null
                      t                 ; output
                      nil               ; don't redisplay
                      "-H" (getenv "HOME") ; search home dir
                      "-W"              ; match pattern in file
                      "-i" "-l"         ; misc options
                      "-F" fnprefix     ; only mail files
                      query             ; the query, in glimpse format
                      )
        (message "Doing glimpse query %s...done" query)
        (sit-for 0)
        ;; remove superfluous stuff from glimpse output
        (goto-char (point-min))
        (delete-matching-lines "\\.overview~?$")
        (goto-char (point-min))
        (while (re-search-forward (concat "^" fnprefix) nil t)
          (replace-match ""))
        ;; separate group name from article number with \t
        (goto-char (point-min))
        (while (re-search-forward "\\(/\\)[0-9]+$" nil t)
          (replace-match "\t" t t nil 1))
        ;; replace / with . in group names
        (subst-char-in-region (point-min) (point-max) ?/ ?. t)
        ;; massage buffer to contain some Lisp;
        ;; this depends on the artlist encoding internals
        ;; maybe this dependency should be removed?
        (goto-char (point-min))
        (while (not (eobp))
          (insert "[\"")
          (skip-chars-forward "^\t")
          (insert "\" ")
          (end-of-line)
          (insert "]")
          (forward-line 1))
        (insert "])\n")
        (goto-char (point-min))
        (insert "(setq artlist [\n")
        (eval-buffer)
        (sort* artlist
               (function (lambda (x y)
                           (if (string-lessp (nnglimpse-artitem-group x)
                                             (nnglimpse-artitem-group y))
                               t
                             (< (nnglimpse-artitem-number x)
                                (nnglimpse-artitem-number y))))))
        ))))

;; Data type `article list'.

(defun nnglimpse-artlist-length (artlist)
  "Returns number of articles in artlist."
  (length artlist))

(defun nnglimpse-artlist-nth (artlist n)
  "Returns Nth artitem (element) of ARTLIST (first arg).
N starts counting at 0."
  (elt artlist n))

(defun nnglimpse-article-from-artlist (artlist n)
  "Returns element from ARTLIST corresponding to number N.
N starts counting at 1."
  (elt artlist (1- n)))

(defun nnglimpse-artitem-group (artitem)
  "Returns group of artlist item ARTIITEM."
  (elt artitem 0))

(defun nnglimpse-artitem-number (artitem)
  "Returns number of artlist item ARTITEM."
  (elt artitem 1))

(defun nnglimpse-nth-artitem-group (artlist n)
  "Returns group of Nth artitem in ARTLIST."
  (nnglimpse-artitem-group (nnglimpse-artlist-nth artlist n)))

(defun nnglimpse-nth-artitem-number (artlist n)
  "Returns number of Nth artitem in ARTLIST."
  (nnglimpse-artitem-number (nnglimpse-artlist-nth artlist n)))

(defun nnglimpse-artlist-groups (artlist)
  "Returns a list of groups in the given ARTLIST."
  (let ((res nil)
        (with-dups nil))
    ;; extract all groups, maybe with duplicates
    (setq with-dups (mapcar (function (lambda (x)
                                        (nnglimpse-artitem-group x)))
                            artlist))
    ;; remove duplicates
    (mapcar (function (lambda (x)
                        (add-to-list 'res x)))
            with-dups)
    res))

(defun nnglimpse-artlist-for-group (artlist group)
  "Returns a list of all artitems in ARTLIST for GROUP."
  (let ((res nil)
        (idx 0))
    (mapcar (function (lambda (x)
                        (when (string= (nnglimpse-artitem-group x)
                                       group)
                          (push (cons (nnglimpse-artitem-number x)
                                      idx) res)
                          (setq idx (1+ idx)))))
            artlist)
    (nreverse res)))

;; The end.
(provide 'nnglimpse)

Re: First experimental version of nnglimpse.el, or: Buglet report

[Lars Magne Ingebrigtsen]

Kai Grossjohann <grossjohann@amaunet.cs.uni-dortmund.de> writes:

> How would I go about adding the group name the article came from to
> the output of nnglimpse?  To the nnglimpse summary buffer?  To the
> article buffer?

You could alter the Subject headers, for instance, or do what
nnvirtual does -- makes the first entry in the Xrefs line the group
from whence the article came.

> How does one go about defining a function for nnglimpse summary
> buffers only?

Well, you could do

(add-hook 'gnus-summary-mode-hook 'nnglimpse-setup)

or something, and then that function would check whether the current
group is an nnglimpse group, and if so, do its thing.

> How does one go about creating such a `subsidiary' summary buffer,
> such that one gets back to the nnglimpse summary buffer with `q'?

What you want is ephemeral groups.  When exiting from these, you
return to where you were when you started.  `C-d' does this with
digests, for instance, and you could use that command as the model.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: First experimental version of nnglimpse.el, or: Buglet report

[Kai Grossjohann]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > You could alter the Subject headers, for instance, or do what
  > nnvirtual does -- makes the first entry in the Xrefs line the group
  > from whence the article came.

Stupid question: what happens when you do that (prepend the group to
Xrefs)?  I mean, apart from the fact that the thus modified Xrefs
header will be shown in the *Article* buffer.

Or are you suggesting to add the the information to the NOV data only?
What good would that do?  Or to both the NOV data and the article
content?

As you can see, I have no idea what Xrefs is supposed to be for.  I
also have never tried to depend on it.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: First experimental version of nnglimpse.el, or: Buglet report

[Lars Magne Ingebrigtsen]

Kai Grossjohann <grossjohann@amaunet.cs.uni-dortmund.de> writes:

> Stupid question: what happens when you do that (prepend the group to
> Xrefs)?

Nothing much.

> As you can see, I have no idea what Xrefs is supposed to be for.  I
> also have never tried to depend on it.

Xrefs are for marking crossposted articles as read.  But if one knows
that the backend has ordered the Xrefs header in a particular way,
then one could use that knowledge.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: First experimental version of nnglimpse.el, or: Buglet report

[Nelson Jose dos Santos Ferreira]

On 22 Jul 1998 15:35:38 +0200, Kai Grossjohann (aka "Kai"),
regarding 'Re: First experimental version of nnglimpse.el, or: Buglet report', said: 

    >>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
    >> You could alter the Subject headers, for instance, or do what
    >> nnvirtual does -- makes the first entry in the Xrefs line the group
    >> from whence the article came.

    Kai> Stupid question: what happens when you do that (prepend the group to
    Kai> Xrefs)?  I mean, apart from the fact that the thus modified Xrefs
    Kai> header will be shown in the *Article* buffer.

Well I for one followed the Fine Manual suggestions on scoring and
have a rule for exponential negative scoring on excessive
cross-posting which counts the number of elements in the Xref
header. So one would have to have a way to ignore that on the rule.


[...]

Best regards,

	Nelson
---
Nelson Jose dos Santos Ferreira |  .  .  .  | INESC/PT-Servicos 
Systems Integrator              | |. /  / . | Lisbon,PORTUGAL
mailto:Nelson.Ferreira@inesc.pt |.||\| | /  | Phone:+351 (1) 3100050
                                | .|<:/  \. | Fax  :+351 (1) 3100008