nngoogle

nngoogle

[Lars Magne Ingebrigtsen]

Lately I've been thinking a bit about whether to create nngoogle.  (My
ISPs news server has now been down for *one* *whole* *hour*!!)  We
have nnweb, of course, bug I'm thinking more along the lines of being
able to subscribe to nngoogle:comp.lang.lisp as a "normal" group, and
get new articles and stuff.

I've looked over the groups.google.com interface, and we can get "raw"
articles, so that's ok, but the problem seems to be that there's no
way to get the number of articles in a group, so that nngoogle would
know when to try to fetch new articles, etc.  The only way seems to be
to fetch ...?group=comp.lang.lisp and see whether there's any new
articles there, and maintain a message-id/nngoogle map.

Anybody have any comments on this before I start hacking?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nngoogle

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've looked over the groups.google.com interface, and we can get "raw"
> articles, so that's ok, but the problem seems to be that there's no
> way to get the number of articles in a group, so that nngoogle would
> know when to try to fetch new articles, etc.  The only way seems to be
> to fetch ...?group=comp.lang.lisp and see whether there's any new
> articles there, and maintain a message-id/nngoogle map.

Actually, it's even worse, now that I've taken a closer look.  You
only get a thread view, so you have to fetch not only the thread view,
but also every single thread to find the Message-ID's of the new
messages.  So `M-g' on such a group would take quite a while.

Is there some other interface I'm not finding? 

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nngoogle

[Lars Magne Ingebrigtsen]

Ami Fischman <usenet@fischman.org> writes:

> The search interface?  You can ask for the most recent 100 articles in a
> group (or articles posted between two dates, sorted by date), and that
> should give you the exploded (unthreaded) view, no?

I tried that, and if I query it for a group name, it gives me the
thread-overview page.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nngoogle

[Ami Fischman]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
> Is there some other interface I'm not finding? 

The search interface?  You can ask for the most recent 100 articles in a
group (or articles posted between two dates, sorted by date), and that
should give you the exploded (unthreaded) view, no?

-- 
  Ami Fischman
  usenet@fischman.org

Re: nngoogle

[Lars Magne Ingebrigtsen]

I've basically done most of nngoogle, and then I discover that the
appoach I've taken doesn't give me the information I need.  Like,
arggh, dude.

The only way to get something that can be used to generate the
.overview file is do download all the articles, and that's not very
nice.  I had hoped I would only have to download the thread
overviews.  (The "left frame".)

Unless I've overlooked something.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nngoogle

[Ami Fischman]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I tried that, and if I query it for a group name, it gives me the
> thread-overview page.

The trick is to make it give you a "search result" set, instead of a group
view.  For example, search for 
"comp.lsi.cad alt.thisgroupreallydoesntexistihope" (without the quotes) in
the group field, and you'll get article listings.  Example query:

http://groups.google.com/groups?as_q=&num=100&as_scoring=d&btnG=Google+Search&as_epq=&as_oq=&as_eq=&as_ugroup=comp.lsi.cad+alt.doesntexistatallgroup&as_usubject=&as_uauthors=&as_umsgid=&lr=&as_drrb=q&as_qdr=&as_mind=12&as_minm=5&as_miny=1981&as_maxd=21&as_maxm=1&as_maxy=2002

> I've basically done most of nngoogle, and then I discover that the
> appoach I've taken doesn't give me the information I need.  Like,
> arggh, dude.
> 
> The only way to get something that can be used to generate the
> .overview file is do download all the articles, and that's not very
> nice.  I had hoped I would only have to download the thread
> overviews.  (The "left frame".)
> 
> Unless I've overlooked something.

Really?  What info are you missing?   Near as I can tell the previous query
gives you everything you need (date, authorname, subject, messageid).  Only
thing that could be viewed as missing is authoremail, but is that really
required?

-- 
  Ami Fischman
  usenet@fischman.org

Re: nngoogle

[Lars Magne Ingebrigtsen]

Ami Fischman <usenet@fischman.org> writes:

> The trick is to make it give you a "search result" set, instead of a group
> view.  For example, search for 
> "comp.lsi.cad alt.thisgroupreallydoesntexistihope" (without the quotes) in
> the group field, and you'll get article listings.  Example query:
>
> http://groups.google.com/groups?as_q=&num=100&as_scoring=d&btnG=Google+Search&as_epq=&as_oq=&as_eq=&as_ugroup=comp.lsi.cad+alt.doesntexistatallgroup&as_usubject=&as_uauthors=&as_umsgid=&lr=&as_drrb=q&as_qdr=&as_mind=12&as_minm=5&as_miny=1981&as_maxd=21&as_maxm=1&as_maxy=2002

Cool!

> Really?  What info are you missing?   Near as I can tell the previous query
> gives you everything you need (date, authorname, subject, messageid).  Only
> thing that could be viewed as missing is authoremail, but is that really
> required?

No, that trick gives me what I need.  The only detail is that it
doesn't give me threading.  

Oh, well.  nngoogle for the scrap heap -- nnweb already has this
covered.  :-/

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Boldifying (was Re: nngoogle)

[Martin Thornquist]

[ Lars Magne Ingebrigtsen ]

> Lately I've been thinking a bit about whether to create nngoogle.  (My
> ISPs news server has now been down for *one* *whole* *hour*!!)  We

Er, Lars, that boldifying regexp isn't quite perfect yet, it seems. In
the above, *hour* doesn't get boldified for me.

(CVS as of 0530 CET last night.)


Martin
-- 
"An ideal world is left as an exercise to the reader."
                                                 -Paul Graham, On Lisp

Re: Boldifying (was Re: nngoogle)

[Lars Magne Ingebrigtsen]

Martin Thornquist <martint@ifi.uio.no> writes:

>> Lately I've been thinking a bit about whether to create nngoogle.  (My
>> ISPs news server has now been down for *one* *whole* *hour*!!)  We
>
> Er, Lars, that boldifying regexp isn't quite perfect yet, it seems. In
> the above, *hour* doesn't get boldified for me.

Darn.  I forgot to include ! and ? as sentence-ending characters.
Should be fixed in CVS now.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen