Fetching by MID: wget vs. curl (was: Support fuer das neue groups.google.com)

[Reiner Steib <reinersteib+gmane@imap.cc>]

[ Redirecting this from de.comm.software.gnus to ding.
  See http://www.google.de/groups?as_umsgid=m2hdc64h9e.fsf@seki.fr&hl=en
  for the rest of the thread. ]

On Tue, Sep 27 2005, Sébastien Kirche wrote:

[ Summary: Fetching an article by MID doesn't work for Sébastien when
  using curl or the internal url.el ]

> It does not work here, possibly  due to a 302 http redirection (document
> has moved) from the Google server.
>
> I tried to fetch a  post from fr.comp.applications.emacs dated from Wed,
> 21 Apr 2004 01:29:01 +0200
>
> (MID: <plop87isfu1b5u.fsf@gnu-rox.org>) that one can see it is available
> From the google advanced search form.

With wget, it works for me:

(setq
 ;; for testing:
 gnus-refer-article-method
 '((nnweb "google" (nnweb-type google)))
 ;; Use wget instead of url.el (for nnweb-type google), seems to work more
 ;; reliable.
 mm-url-use-external t
 mm-url-program 'wget)

,----
| From: Xavier Maillard <z...@gnu-rox.org>
| Subject: Re: un mode pour maintenir un dictionnaire/index/repertoire ?
| Newsgroups: fr.comp.applications.emacs
| Date: Wed, 21 Apr 2004 01:29:01 +0200
| Message-ID: <plop87isfu1b5u.fsf@gnu-rox.org>
`----

> Here are some of my settings :
> ,----[ C-h v gnus-refer-article-method RET ]
> | gnus-refer-article-method is a variable defined in `gnus'.
> | Its value is 
> | (current
> |  (nntp "news.gmane.org")
> |  (nnweb "google"
> | 		(nnweb-type google)))
> `----
[...]
> ,----[ C-h v mm-url-program RET ]
> | mm-url-program is a variable defined in `mm-url'.
> | Its value is "curl"
> `----
>
> ,----[ C-h v mm-url-arguments RET ]
> | mm-url-arguments is a variable defined in `mm-url'.
> | Its value is ("--silent")
> `----
[...]
> I wonder if it could be a trick from google to prevent external queries ?

On the command line, I also get the redirection when using curl, even
when faking the User-Agent like mm-url does with wget:

$ mid='plop87isfu1b5u.fsf@gnu-rox.org'
$ curl --silent --user-agent mm-url \
  "http://www.google.com/groups?as_umsgid=$mid&hl=en&dmode=source"

With wget, everything is okay:

$ wget --user-agent=mm-url -q -O - \
  "http://www.google.com/groups?as_umsgid=$mid&hl=en&dmode=source"

I don't know what makes the difference between wget and curl.


[ order of quoting changed ]
> Erf.   Gmane    seems   to   be    broken   to   let   me    post   into
> gmane.emacs.gnus.general. I can read but not post. Although it is ok for
> several  other mailing  lists.  And  i have  already  subscribed to  the
> list...

You need a valid email address for Gmane and confirm the query.  I
have successfully posted some messages to ding via news.gmane.org
today (and I'm not even subscribed to ding).  See also the Gmane FAQ.
If all else fails, ask on gmane.discuss.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/