Re: Does nnweb with Google work any more?

[David Engster <deng@randomsample.de>]

Lars Magne Ingebrigtsen writes:
> David Engster <deng@randomsample.de> writes:
>
>> No, they just don't want to be crawled. A simple "-A foobar" will make
>> it work. Also, adding "&output=gplain" will give raw text.
>
> Oh, nice.  :-)
>
> curl -A foobar 'http://groups.google.com/group/rec.arts.sf.written/msg/eeb018dcf3c1688e?dmode=source&output=gplain'
>
> works fine.
>
> Then the only question is how to get from the Message-ID to the Google
> ID.  Let's see...  the first URL had this snippet in the HTML:
>
> Michael Stemper wrote: In article&lt;9rt27vF38...@mid.individual.net&gt;, <b>...</b></span><br><span class="a">http://groups.google.com/g/0897fef7/t/d00e330e9c82797a/d/eeb018dcf3c1688e</span>
>
> Will there only be one of these URLs in the output?

No idea. Maybe it would be safer to snarf the q=#eeb018... anchor from the
title's target:

<div class="g" align=left><a href="http://www.google.com/url?url=http://groups.google.com/g/0897fef7/t/d00e330e9c82797a/d/eeb018dcf3c1688e%3Fq%3D%23eeb018dcf3c1688e&amp;ei=WM1gT-vtOdTw_Aah-IjTDg&amp;sa=t&amp;ct=res&amp;cd=1&amp;source=groups&amp;usg=AFQjCNGv6gXQ4vTjK4dTlhZQfwmCOamYKw"
target="" dir=ltr>

Anyway, it's a pretty boring scavenger hunt.

-David