Re: nndejanews -- http help needed

[William Perry <wmperry@aventail.com>]

Lars Magne Ingebrigtsen writes:
>I thought it would be way neat do write this backend, so I have.
>Well, I've written the skeleton, but since I know nothing about the
>internals of w3, I wonder whether someone could point me in the right
>direction.  I need functions for:
>
>1) Fetching an URL and waiting for the result

(url-insert-file-contents "URL")

>2) Fetching an URL and returning immediately (must have that async
>   prefetch, you know) 

  w/this function, 'callback' will be called with 'data' when the URL has
been retrieved.

(defun gnus-url-retrieve-asynch (url callback &rest data)
  (let (
	(url-request-method "GET")
	(old-asynch url-be-asynchronous)
	(url-request-data nil)
	(url-request-extra-headers nil)
	(url-working-buffer (generate-new-buffer-name " *dejanews*")))
    (setq-default url-be-asynchronous t)
    (save-excursion
      (set-buffer (get-buffer-create url-working-buffer))
      (setq url-current-callback-data data
	    url-be-asynchronous t
	    url-current-callback-func callback)
      (url-retrieve src))
    (setq-default url-be-asynchronous old-asynch)))

>3) Decoding HTML entities -- >



>4) Encoding a string into an URL

(url-hexify-string string)

>5) Doing a POST form thingie

  Well, the forms stuff expects widgets to be passed in (Emacs-W3 now uses
Per's widget stuff), so you'll have to fake it out a little bit.  Try:

(defun gnus-encode-www-form-urlencoded (pairs)
  (mapconcat 
    (function
      (lambda (data)
        (concat (w3-form-encode-xwfu (car data)) "="
                (w3-form-encode-xwfu (cdr data))))) pairs "&"))

  If you are using the 'get' method, just tack this onto the end of the URL
with a "?", and you should be good.  For posting, let-bind url-request-data
to what's returned by this, and url-request-method to 'POST', and
url-request-extra-headers to something like
 '(("Content-type" . "application/x-www-form-urlencoded"))

  then fetch the HTTP URL.

>Well, that should tide me over, I think...
>
>Interface-wise, I though this would work like this:  
>
>1) The user types a magic command (`G N') and a search criteria
>2) Gnus fetches the "headers" for all the articles that match -- which
>means sniffing the search results (and "next") and stuff
>3) A normal summary buffer is created
>4) The user reads the articles
>
>Simple, eh?

  Sounds very very useful.  Perhaps the base funcitonality should be in W3,
ie: a function to retrieve a search page, and then return a list of all the
headers to documents contained in the results?

-Bill P.