* Gnus and googlegroups search
@ 2010-09-28 5:42 Richard Riley
2010-09-28 6:29 ` CHENG Gao
0 siblings, 1 reply; 12+ messages in thread
From: Richard Riley @ 2010-09-28 5:42 UTC (permalink / raw)
To: ding
Is there any easy way or plugin for gnus that would enable one to create
a temporary search result group from google-groups? It would be nice to
create a bunch of posts to usenet from a google groups search and be
able to process them etc in Gnus.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 5:42 Gnus and googlegroups search Richard Riley
@ 2010-09-28 6:29 ` CHENG Gao
2010-09-28 10:00 ` Lars Magne Ingebrigtsen
0 siblings, 1 reply; 12+ messages in thread
From: CHENG Gao @ 2010-09-28 6:29 UTC (permalink / raw)
To: ding
*On Tue, 28 Sep 2010 07:42:49 +0200
* Also sprach Richard Riley <rileyrg@googlemail.com>:
> Is there any easy way or plugin for gnus that would enable one to create
> a temporary search result group from google-groups? It would be nice to
> create a bunch of posts to usenet from a google groups search and be
> able to process them etc in Gnus.
Seems nnweb is what you need - search messages from google or gmane groups.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 6:29 ` CHENG Gao
@ 2010-09-28 10:00 ` Lars Magne Ingebrigtsen
2010-09-28 11:50 ` Steinar Bang
0 siblings, 1 reply; 12+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-09-28 10:00 UTC (permalink / raw)
To: ding
CHENG Gao <chenggao@cyberhut.org> writes:
> Seems nnweb is what you need - search messages from google or gmane groups.
I tried it a few weeks back -- the Google search no longer worked.
It works by web-scraping the results, I think, which is notoriously
unreliable. Doesn't Google have a search API? Rewriting nnweb to use
that instead would probably be a good idea.
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 10:00 ` Lars Magne Ingebrigtsen
@ 2010-09-28 11:50 ` Steinar Bang
2010-09-28 12:01 ` Steinar Bang
0 siblings, 1 reply; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 11:50 UTC (permalink / raw)
To: ding
>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:
> It works by web-scraping the results, I think, which is notoriously
> unreliable. Doesn't Google have a search API? Rewriting nnweb to use
> that instead would probably be a good idea.
There is an AJAX REST API, but I only found documentation showing it
from the JS viewpoint, not documenting the actual HTTP calls made, and
their responses:
http://code.google.com/intl/no-NO/apis/ajaxsearch/documentation/
Maybe the mappings from the API to what's sent over the wire are obvious
to JS-heads...?
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 11:50 ` Steinar Bang
@ 2010-09-28 12:01 ` Steinar Bang
2010-09-28 12:11 ` Steinar Bang
2010-09-29 15:05 ` Lars Magne Ingebrigtsen
0 siblings, 2 replies; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 12:01 UTC (permalink / raw)
To: ding
>>>>> Steinar Bang <sb@dod.no>:
>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:
>> It works by web-scraping the results, I think, which is notoriously
>> unreliable. Doesn't Google have a search API? Rewriting nnweb to use
>> that instead would probably be a good idea.
> There is an AJAX REST API, but I only found documentation showing it
> from the JS viewpoint, not documenting the actual HTTP calls made, and
> their responses:
> http://code.google.com/intl/no-NO/apis/ajaxsearch/documentation/
> Maybe the mappings from the API to what's sent over the wire are obvious
> to JS-heads...?
Something here:
http://blog.programmableweb.com/2008/04/21/googles-rest-api-for-search/
Basically you do a GET HTTP request, and get a response with a JSON
body.
A request URL looks like this:
http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Paris%20Hilton
You also need to set the referer header. To what it should be set from
a client application, I don't know. I also don't know if it needs to
DNS match the machine the calling application is done on.
Perhaps it would be neccessary to proxy the queries through
eg. http://www.gnus.org or something?
Hm... let me try...
$ curl -e http://www.gnus.org "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Paris%20Hilton"
{"responseData": {"results":[{"GsearchResultClass":"GwebSearch","unescapedUrl":"http://en.wikipedia.org/wiki/Paris_Hilton","url":"http://en.wikipedia.org/wiki/Paris_Hilton","visibleUrl":"en.wikipedia.org","cacheUrl":"http://www.google.com/search?q\u003dcache:TwrPfhd22hYJ:en.wikipedia.org","title":"\u003cb\u003eParis Hilton\u003c/b\u003e - Wikipedia, the free encyclopedia","titleNoFormatting":"Paris Hilton - Wikipedia, the free encyclopedia","content":"\u003cb\u003eParis\u003c/b\u003e Whitney \u003cb\u003eHilton\u003c/b\u003e (born February 17, 1981) is an American socialite, heiress, media personality, model, singer, author, fashion designer and actress . \u003cb\u003e...\u003c/b\u003e"},{"GsearchResultClass":"GwebSearch","unescapedUrl":"http://www.tmz.com/person/paris-hilton/","url":"http://www.tmz.com/person/paris-hilton/","visibleUrl":"www.tmz.com","cacheUrl":"http://www.google.com/search?q\u003dcache:t-eZlX9vZUYJ:www.tmz.com","title":"\u003cb\u003eParis Hilton\u003c/b\u003e | TMZ.com","titleNoFormatting":"Paris Hilton| TMZ.com","content":"After Japan turned away \u003cb\u003eParis Hilton\u003c/b\u003e -- because of her guilty plea to cocaine possession -- the heiress hightailed it back to the U.S. of A ... arriving \u003cb\u003e...\u003c/b\u003e"},{"GsearchResultClass":"GwebSearch","unescapedUrl":"http://www.parishiltonzone.com/","url":"http://www.parishiltonzone.com/","visibleUrl":"www.parishiltonzone.com","cacheUrl":"http://www.google.com/search?q\u003dcache:EXq0MGRNiNoJ:www.parishiltonzone.com","title":"\u003cb\u003eParis Hilton\u003c/b\u003e Zone | \u003cb\u003eParis Hilton\u003c/b\u003e Pictures, Pics, Photos","titleNoFormatting":"Paris Hilton Zone | Paris Hilton Pictures, Pics, Photos","content":"8000+ new \u003cb\u003eParis Hilton\u003c/b\u003e pictures, Paris wallpaper, lyrics, audio, video,daily Paris pics \u0026amp; news."},{"GsearchResultClass":"GwebSearch","unescapedUrl":"http://www.imdb.com/name/nm0385296/","url":"http://www.imdb.com/name/nm0385296/","visibleUrl":"www.imdb.com","cacheUrl":"http://www.google.com/search?q\u003dcache:1i34KkqnsooJ:www.imdb.com","title":"\u003cb\u003eParis Hilton\u003c/b\u003e","titleNoFormatting":"Paris Hilton","content":"Socialite \u003cb\u003eParis Hilton\u003c/b\u003e was born on February 17, 1981 in New York City... Visit IMDb for Photos, Filmography, Discussions, Bio, News, Awards, Agent, \u003cb\u003e...\u003c/b\u003e"}],"cursor":{"pages":[{"start":"0","label":1},{"start":"4","label":2},{"start":"8","label":3},{"start":"12","label":4},{"start":"16","label":5},{"start":"20","label":6},{"start":"24","label":7},{"start":"28","label":8}],"estimatedResultCount":"19500000","currentPageIndex":0,"moreResultsUrl":"http://www.google.com/search?oe\u003dutf8\u0026ie\u003dutf8\u0026source\u003duds\u0026start\u003d0\u0026hl\u003den\u0026q\u003dParis+Hilton"}}, "responseDetails": null, "responseStatus": 200} $
Nah... the referer can be faked.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:01 ` Steinar Bang
@ 2010-09-28 12:11 ` Steinar Bang
2010-09-28 12:14 ` Steinar Bang
2010-09-28 12:15 ` Julien Danjou
2010-09-29 15:05 ` Lars Magne Ingebrigtsen
1 sibling, 2 replies; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 12:11 UTC (permalink / raw)
To: ding
Emacs lisp JSON parser:
http://cvs.savannah.gnu.org/viewvc/*checkout*/emacs/lisp/json.el?root=emacs
(hm... is it actually _in_ GNU Emacs I wonder? And from what version?)
Linked to from here (parsers for different languages at the bottom):
http://www.json.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:11 ` Steinar Bang
@ 2010-09-28 12:14 ` Steinar Bang
2010-09-28 12:22 ` Steinar Bang
2010-09-28 12:15 ` Julien Danjou
1 sibling, 1 reply; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 12:14 UTC (permalink / raw)
To: ding
>>>>> Steinar Bang <sb@dod.no>:
> Emacs lisp JSON parser:
> http://cvs.savannah.gnu.org/viewvc/*checkout*/emacs/lisp/json.el?root=emacs
> (hm... is it actually _in_ GNU Emacs I wonder? And from what version?)
Not in emacs 22.3, at least, according to locate-library.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:11 ` Steinar Bang
2010-09-28 12:14 ` Steinar Bang
@ 2010-09-28 12:15 ` Julien Danjou
2010-09-28 12:30 ` Steinar Bang
1 sibling, 1 reply; 12+ messages in thread
From: Julien Danjou @ 2010-09-28 12:15 UTC (permalink / raw)
To: ding
[-- Attachment #1: Type: text/plain, Size: 282 bytes --]
On Tue, Sep 28 2010, Steinar Bang wrote:
> (hm... is it actually _in_ GNU Emacs I wonder? And from what
> version?)
It's in Emacs 23.2 at least. I already use it for the Google Maps
extension.
--
Julien Danjou
// ᐰ <julien@danjou.info> http://julien.danjou.info
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:14 ` Steinar Bang
@ 2010-09-28 12:22 ` Steinar Bang
0 siblings, 0 replies; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 12:22 UTC (permalink / raw)
To: ding
>>>>> Steinar Bang <sb@dod.no>:
>> Emacs lisp JSON parser:
>> http://cvs.savannah.gnu.org/viewvc/*checkout*/emacs/lisp/json.el?root=emacs
>> (hm... is it actually _in_ GNU Emacs I wonder? And from what version?)
> Not in emacs 22.3, at least, according to locate-library.
Says here (search for "json.el"), "Included with Emacs as of 2008-02-21":
http://edward.oconnor.cx/elisp/
The above URL also links on to "Introducing json.el":
http://edward.oconnor.cx/2006/03/json.el
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:15 ` Julien Danjou
@ 2010-09-28 12:30 ` Steinar Bang
0 siblings, 0 replies; 12+ messages in thread
From: Steinar Bang @ 2010-09-28 12:30 UTC (permalink / raw)
To: ding
>>>>> Julien Danjou <julien@danjou.info>:
> On Tue, Sep 28 2010, Steinar Bang wrote:
>> (hm... is it actually _in_ GNU Emacs I wonder? And from what
>> version?)
> It's in Emacs 23.2 at least. I already use it for the Google Maps
> extension.
And Emacs 22 users can get version 1.2 of json.el easily from ELPA:
http://tromey.com/elpa/
(git gnus won't run on emacs 21 anymore, so there is no point in
supporting that. Personally I will be stuck with emacs 22 on at least
my UNE (Ubuntu Netbook Edition) netbook, until the issues with opening
emacs 23 are resolved...)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-28 12:01 ` Steinar Bang
2010-09-28 12:11 ` Steinar Bang
@ 2010-09-29 15:05 ` Lars Magne Ingebrigtsen
2010-09-29 18:11 ` Steinar Bang
1 sibling, 1 reply; 12+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-09-29 15:05 UTC (permalink / raw)
To: ding
Steinar Bang <sb@dod.no> writes:
> Basically you do a GET HTTP request, and get a response with a JSON
> body.
Looks good. Now somebody just has to implement the nnweb thing, since
json.el is in Emacs already...
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Gnus and googlegroups search
2010-09-29 15:05 ` Lars Magne Ingebrigtsen
@ 2010-09-29 18:11 ` Steinar Bang
0 siblings, 0 replies; 12+ messages in thread
From: Steinar Bang @ 2010-09-29 18:11 UTC (permalink / raw)
To: ding
>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:
> Steinar Bang <sb@dod.no> writes:
>> Basically you do a GET HTTP request, and get a response with a JSON
>> body.
> Looks good. Now somebody just has to implement the nnweb thing, since
> json.el is in Emacs already...
Please make sure it still builds on emacs22 (which doesn't have it
included). It is easily retrieved using ELPA if one _does_ need it, but
it should build without, I think.
(The netbook I'm writing this on needs to use emacs22 (Ubuntu Netbook
Edition), because when I try to open emacs23 it looks very weird. It's
the wrong size, has an unreadable font, and can't get keyboard focus)
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2010-09-29 18:11 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-09-28 5:42 Gnus and googlegroups search Richard Riley
2010-09-28 6:29 ` CHENG Gao
2010-09-28 10:00 ` Lars Magne Ingebrigtsen
2010-09-28 11:50 ` Steinar Bang
2010-09-28 12:01 ` Steinar Bang
2010-09-28 12:11 ` Steinar Bang
2010-09-28 12:14 ` Steinar Bang
2010-09-28 12:22 ` Steinar Bang
2010-09-28 12:15 ` Julien Danjou
2010-09-28 12:30 ` Steinar Bang
2010-09-29 15:05 ` Lars Magne Ingebrigtsen
2010-09-29 18:11 ` Steinar Bang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).