nnrss lacks basic crosspost detection; should use guid

nnrss lacks basic crosspost detection; should use guid

[jidanni]

Alas,
http://www.coolloud.org.tw/tag/新移民/feed
http://www.coolloud.org.tw/tag/移民工/feed
have the same item,
<guid isPermaLink="false">35873 at http://www.coolloud.org.tw</guid>
with the same date even, but nnrss is not smart enough to mark it read
globally if read in one group.
Part of the problem is that nnrss totally ignores the guid, and
instead insists on making its own Message-IDs,
Message-ID: <828@=?utf-8?B?6KiC6Zax6Ium5YueX+enu+awkeW3pV/nmoTmlofnq6A=?=.nnrss>
Message-ID: <73@=?utf-8?B?6KiC6Zax6Ium5YueX+aWsOenu+awkV/nmoTmlofnq6A=?=.nnrss>
Decoding, <828@訂閱苦勞_移民工_的文.nnrss>, we see the encoded part
was merely the group description.

If the Message-ID just used the guid, perhaps crossposts could be
detected, and marked as read globally. Only if there is no guid, then
fall back to the current scheme.

Re: nnrss lacks basic crosspost detection; should use guid

[jidanni]

Here's how the guid works:
http://diveintomark.org/archives/2004/05/28/howto-atom-id

Re: nnrss lacks basic crosspost detection; should use guid

[Kevin Ryde]

jidanni@jidanni.org writes:
>
> <guid isPermaLink="false">35873 at http://www.coolloud.org.tw</guid>

What do the specs say about the scope of a non-permalink guid?
I recently struck these two feeds

    http://rss.weather.com.au/sa/adelaide
    http://rss.weather.com.au/nt/darwin

which have the same guid (non-permalink), but different content.  (They
update every 15 minutes or something, the guid changes with such
updates, but to the same new value for the two locations.)

It made me suspect non-permalink guid is only for the given feed url,
apparently not univeral nor even throughout the given server.  (I jigged
up Message-ID from my rss2leafnode program accordingly, but couldn't
find the right bit of the specs to confirm or deny.)

Re: nnrss lacks basic crosspost detection; should use guid

[Russ Allbery]

Kevin Ryde <user42@zip.com.au> writes:

> It made me suspect non-permalink guid is only for the given feed url,
> apparently not univeral nor even throughout the given server.  (I jigged
> up Message-ID from my rss2leafnode program accordingly, but couldn't
> find the right bit of the specs to confirm or deny.)

I'm fairly certain that's correct.  One of the feeds I generate uses a
timestamp for the guid -- it's unique for that particular feed, but it's
definitely not going to be globally unique.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: nnrss lacks basic crosspost detection; should use guid

[jidanni]

RA> I'm fairly certain that's correct.  One of the feeds I generate uses a
RA> timestamp for the guid -- it's unique for that particular feed, but it's
RA> definitely not going to be globally unique.

Well, it turns out the only time a server can use <guid
isPermaLink="true"> is when the guid fully matches a vaild URL.

I.e., the RSS standard has glued the Usenet etc. concept of unique
Message-ID together with the concept of permanent URI.

So now there is no way to say, "yes, message A in the feed X I
provide, is the same as message A in feed Y that I provide", unless
one is also willing to give a permanent URI. I suppose. See
http://core.trac.wordpress.org/ticket/9280

Anyway, maybe nnrss should have a switch:
(set nnrss-assume-all-guids-are-like-usenet-message-ids t)

Re: nnrss lacks basic crosspost detection; should use guid

[Kevin Ryde]

jidanni@jidanni.org writes:
>
> (set nnrss-assume-all-guids-are-like-usenet-message-ids t)

Obviously that's too specific, but a hook that allowed manipulations
after parse and before message building might be worthwhile.  In yahoo
news feeds for example the "link" field can be un-redirected and used as
an ID for when an article appears in multiple feeds.  (Dunno why that
link isn't already a permalink guid, maybe it's not quite permanent or
something.)