nnweb still in use ?

nnweb still in use ?

[dme]

Does anyone still use nnweb ?

I'm trying to add Google support (see attached), but it seems very
temperamental (and not just because of my bad HTML parsing).

Index: nnweb.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnweb.el,v
retrieving revision 6.1
diff -c -r6.1 nnweb.el
*** nnweb.el	2000/11/01 01:21:46	6.1
--- nnweb.el	2000/11/12 22:52:24
***************
*** 56,62 ****
  (defvoo nnweb-type 'dejanews
    "What search engine type is being used.
  Valid types include `dejanews', `dejanewsold', `reference',
! and `altavista'.")
  
  (defvar nnweb-type-definition
    '((dejanews
--- 56,62 ----
  (defvoo nnweb-type 'dejanews
    "What search engine type is being used.
  Valid types include `dejanews', `dejanewsold', `reference',
! `altavista' and `google'.")
  
  (defvar nnweb-type-definition
    '((dejanews
***************
*** 84,89 ****
--- 84,95 ----
       (search . nnweb-altavista-search)
       (address . "http://www.altavista.digital.com/cgi-bin/query")
       (id . "/cgi-bin/news?id@%s")
+      (identifier . identity))
+     (google
+      (article . ignore)
+      (map . nnweb-google-create-mapping)
+      (search . nnweb-google-search)
+      (address . "http://www.google.com/search")
       (identifier . identity)))
    "Type-definition alist.")
  
***************
*** 677,682 ****
--- 683,754 ----
         ("r" . "")
         ("d0" . "")
         ("d1" . "")))))
+   (setq buffer-file-name nil)
+   t)
+ 
+ ;;;
+ ;;; Google
+ ;;;
+ 
+ (defun nnweb-google-create-mapping ()
+   "Perform the search and create an number-to-url alist."
+   (save-excursion
+     (set-buffer nnweb-buffer)
+     (erase-buffer)
+     (let ((part 0))
+       (when (funcall (nnweb-definition 'search) nnweb-search part)
+ 	(let ((i 0)
+ 	      (more t)
+ 	      (case-fold-search t)
+ 	      (active (or (cadr (assoc nnweb-group nnweb-group-alist))
+ 			  (cons 1 0)))
+ 	      subject date from id group
+ 	      map url)
+ 	  (while more
+ 	    ;; Go through all the article hits on this page.
+ 	    (goto-char (point-min))
+ 	    (while (re-search-forward 
+ 		    "^<p><A HREF=\\([^>]+\\)>\\(..*\\)</A>"
+ 		    nil t)
+ 	      (setq
+ 	       url (match-string 1)
+ 	       subject (match-string 2)
+ 	       date "Jan 1 00:00:00 0000"
+ 	       group ""
+ 	       id (concat "<" (match-string 1) ">")
+ 	       from "unknown"
+ 	       )
+ 	      (incf i)
+ 	      (unless (nnweb-get-hashtb url)
+ 		(push
+ 		 (list
+ 		  (incf (cdr active))
+ 		  (make-full-mail-header
+ 		   (cdr active) (concat  "(" group ") " subject) from date
+ 		   id nil 0 0 url))
+ 		 map)
+ 		(nnweb-set-hashtb (cadar map) (car map))))
+ 	    ;; See if we want more.
+ 	    (when (or (not nnweb-articles)
+ 		      (>= i nnweb-max-hits)
+ 		      (not (funcall (nnweb-definition 'search)
+ 				    nnweb-search (incf part))))
+ 	      (setq more nil)))
+ 	  ;; Return the articles in the right order.
+ 	  (setq nnweb-articles
+ 		(sort (nconc nnweb-articles map) 'car-less-than-car)))))))
+ 
+ (defun nnweb-google-search (search &optional part)
+   (url-insert-file-contents
+    (concat
+     (nnweb-definition 'address)
+     "?"
+     (nnweb-encode-www-form-urlencoded
+      `(
+        ("as_qt" . "w")
+        ("as_q" . ,search)
+        ("num" . "100")
+        ))))
    (setq buffer-file-name nil)
    t)
  

dme.

Re: nnweb still in use ?

[Eric Marsden]

>>>>> "dm" == dme  <dme@dme.org> writes:

  dm> Does anyone still use nnweb ?
  dm> I'm trying to add Google support (see attached), but it seems very
  dm> temperamental (and not just because of my bad HTML parsing).

I use watson.el for this: it works with google and deja.com and sends
queries in parallel. Maintaining this type of program in the face of
HTML changes is a bit of a pain, unfortunately.

   <URL:http://www.chez.com/emarsden/downloads/watson.el>

-- 
Eric Marsden                          <URL:http://www.laas.fr/~emarsden/>