GroupLens (gnus-gl.el)

GroupLens (gnus-gl.el)

[Ted Zlatanov]

The ideas behind GroupLens seem useful, even though the service has
shut down.  Is there any possibility of something similar to GroupLens
coming back to Gnus?

Ted

three nnmaildir questions: naming, numbering, and splitting

[David Wuertele]

I am a new nnmaildir user.  Here is my relevant .gnus:

(setq gnus-secondary-select-methods
      '((nnmaildir ""
	 (directory "~/mail/maildirs")
	 (create-directory ".")
	 )
       (nntp "news.gmane.org")))

I have fetchmail deliver directly into the various maildirs in
~/mail/maildirs/*, and I'm seeing weird (wrong) unread article counts,
like this:

       425: nnmaildir:dave@example.com 

even if I have no unread mail in that group.  When I enter such a
group, the number quickly reverts to zero and the cursor just moves
down in the group buffer to the next group.  Ideas?

Second question: I would like to rename that group.  Since I have a
lot of nnmaildir: groups, I'd like to say 'j' to jump, then type
'dave@' and do completion.  But as it stands I have to type
'nnmaildir:dave@' then do completion.  Is there a way to make this
quicker?

Third question: I'm also wondering wondering what the best strategy
for splitting is.

1.  I want to continue to use nnmaildir as the select method

2.  I don't want to split into different maildirs (since they don't do
    crossposting elegantly).

Any suggestions?
Dave

Re: three nnmaildir questions: naming, numbering, and splitting

[Paul Jarc]

David Wuertele <dave-gnus@bfnet.com> wrote:
>       '((nnmaildir ""
> 	 (directory "~/mail/maildirs")
> 	 (create-directory ".")

That's broken.  Currently, create-directory and directory must not be
the same.  With your configuration, group creation and deletion will
break.  Other things might work ok.

> I'm seeing weird (wrong) unread article counts, like this:
>
>        425: nnmaildir:dave@example.com
>
> even if I have no unread mail in that group.  When I enter such a
> group, the number quickly reverts to zero and the cursor just moves
> down in the group buffer to the next group.  Ideas?

This happens (with any backend) when there are holes in the article
range; Gnus ignores the backend's article count and makes up its own
guess based on the highest and lowest article numbers.  nnmaildir used
to be even worse off than other backends in this area, but I think I
fixed that.  To check, load this defun, eval the following call, and
post the result:

(defun report-nnmaildir-group (pgname)
  (let* ((gname (gnus-group-real-name pgname))
         (server-name (gnus-group-real-prefix pgname))
         (method (gnus-server-to-method server-name))
         (server-address (nth 1 method))
         (group (nnmaildir--prepare server-address gname))
         (server nnmaildir--cur-server)
         (max (nnmaildir--group-maxnum server group))
         (min (nnmaildir--grp-min group))
         (count (nnmaildir--grp-count group)))
    (list 'pgname pgname 'max max 'min min 'count count)))

(report-nnmaildir-group "nnmaildir:dave@example.com")

> Second question: I would like to rename that group.  Since I have a
> lot of nnmaildir: groups, I'd like to say 'j' to jump, then type
> 'dave@' and do completion.  But as it stands I have to type
> 'nnmaildir:dave@' then do completion.  Is there a way to make this
> quicker?

Not that I know of.  This isn't specific to nnmaildir; it's the same
with all non-primary select methods, using any backend.  Of course,
you can write a new Lisp command to do what you want.

> Third question: I'm also wondering wondering what the best strategy
> for splitting is.
>
> 1.  I want to continue to use nnmaildir as the select method
>
> 2.  I don't want to split into different maildirs (since they don't do
>     crossposting elegantly).

What specific inelegance are you concerned about?


paul

Re: three nnmaildir questions: naming, numbering, and splitting

[David Wuertele]

>> '((nnmaildir ""
>> (directory "~/mail/maildirs")
>> (create-directory ".")

Paul> That's broken.  Currently, create-directory and directory must
Paul> not be the same.  With your configuration, group creation and
Paul> deletion will break.  Other things might work ok.

So I have to create another directory, link all my maildirs into it,
and then call that create-directory?

Paul> (report-nnmaildir-group "nnmaildir:dave@example.com")

(pgname "nnmaildir:dave@example.com" max 5398 min 2489 count 2485)

>> Third question: I'm also wondering wondering what the best strategy
>> for splitting is.
>> 
>> 1.  I want to continue to use nnmaildir as the select method
>> 
>> 2.  I don't want to split into different maildirs (since they don't do
>> crossposting elegantly).

Paul> What specific inelegance are you concerned about?

The fact that you can't deliver to two different maildirs without
making a file copy.  I want the attributes of the mail I've read to be
maintained between crosspostings.

Dave

Re: three nnmaildir questions: naming, numbering, and splitting

[Paul Jarc]

David Wuertele <dave-gnus@bfnet.com> wrote:
> So I have to create another directory, link all my maildirs into it,
> and then call that create-directory?

"create-directory" is where the maildir directories live.  "directory"
contains symlinks pointing to the maildirs.  But if you don't care
about group creation or deletion, you can just remove
"create-directory" altogether, and "directory" can contain either
maildirs directly or symlinks.

> Paul> (report-nnmaildir-group "nnmaildir:dave@example.com")
>
> (pgname "nnmaildir:dave@example.com" max 5398 min 2489 count 2485)

Gnus's count= max-min+1 = 5398-2489+1 = 2910.
Missing articles= 2910-2485 = 425.

Which is exactly how many article Gnus mistakenly thinks are there.
So this is the same problem other backends have.  To fix it, you can
move all the articles to a temporary group and back again, so they
will all have new, sequential article numbers.  There's also an easier
way with nnmaildir, if you're not using the cache or agent: exit Gnus,
and then: rm -rf ~/mail/maildirs/dave@example.com/.nnmaildir/{num,nov}
The next time you start Gnus, it will be very slow to start, but I'd
guess moving the articles between groups would be even slower.

> you can't deliver to two different maildirs without making a file
> copy.

If they're on the same filesystem, you can link an article from one
maildir into another.  Whether any particular delivery agent actually
does this is another matter.  (nnmaildir tries to, for B c, but Gnus
unwittingly interferes, so it doesn't always work.)


paul

Re: three nnmaildir questions: naming, numbering, and splitting

[Kai Großjohann]

David Wuertele <dave-gnus@bfnet.com> writes:

> Second question: I would like to rename that group.  Since I have a
> lot of nnmaildir: groups, I'd like to say 'j' to jump, then type
> 'dave@' and do completion.  But as it stands I have to type
> 'nnmaildir:dave@' then do completion.  Is there a way to make this
> quicker?

See the variable gnus-group-jump-to-group-prompt.  There you can tell
Gnus to fill-in "nnmaildir:" into the `j' prompt.
-- 
Ambibibentists unite!

Re: GroupLens (gnus-gl.el)

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> The ideas behind GroupLens seem useful, even though the service has
> shut down.  Is there any possibility of something similar to GroupLens
> coming back to Gnus?

I think GroupLens is a really good idea, and it would be nice if
something like that were to make a return, but it's a major project.
Especially the bits where you have to moderate the moderators, so
that the scoring pool isn't poisoned.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: GroupLens (gnus-gl.el)

[Kai Großjohann]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I think GroupLens is a really good idea, and it would be nice if
> something like that were to make a return, but it's a major project.
> Especially the bits where you have to moderate the moderators, so
> that the scoring pool isn't poisoned.

Is something like this necessary?  It seems that wikis work even
without such moderation.

What do the "submit a message to flag it as spam" folks do?  Do they
protect against legitimate messages flagged as spam?

I'm kinda hoping that Somebody would find that Gmane is a good place
for such a server :-)
-- 
A turnip curses Elvis

Re: GroupLens (gnus-gl.el)

[Lars Magne Ingebrigtsen]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> What do the "submit a message to flag it as spam" folks do?  Do they
> protect against legitimate messages flagged as spam?

People running collaborative spam services are quite inconvenienced
by people poisoning the well.  For instance, I read somewhere that
all the RISKS digests end up being tagged as spam, since that's very
widely circulated mailing list, and people (through ill will or not)
end up tagging those as spam.

So it's a problem once the number of participants grow beyond a
certain set.

(See also the problems Slashdot has had with moderation, and their
increasingly complex scoring system with karma points and
metamoderation.) 

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: GroupLens (gnus-gl.el)

[Alex Schroeder]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> For instance, I read somewhere that all the RISKS digests end up
> being tagged as spam, since that's very widely circulated mailing
> list, and people (through ill will or not) end up tagging those as
> spam.

Hm.  I only remember RISKS being filtered because of "naughty words"
-- since they talk about the risks of identity theft, hacks, viruses,
etc., their vocabulary is bound to trigger such filters...

Alex.

Re: GroupLens (gnus-gl.el)

[Lars Magne Ingebrigtsen]

Alex Schroeder <alex@emacswiki.org> writes:

> Hm.  I only remember RISKS being filtered because of "naughty words"
> -- since they talk about the risks of identity theft, hacks, viruses,
> etc., their vocabulary is bound to trigger such filters...

I was following the Vipul's Razor mailing list for a while, and there
was some talk about RISKS being submitted to the service, I think.
(And Razor doesn't do phrase matches.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: GroupLens (gnus-gl.el)

[Alex Schroeder]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> Is something like this necessary?  It seems that wikis work even
> without such moderation.

The problem is that wikis don't use scoring (like slashdot and kuro5in
do), because wikizens believe that voting and scoring is evil.  It is
always better to write some text.  But when it comes to spam, this
does not help for various reasons.

Alex.

Re: GroupLens (gnus-gl.el)

[Raja R Harinath]

Hi,

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> The ideas behind GroupLens seem useful, even though the service has
>> shut down.  Is there any possibility of something similar to GroupLens
>> coming back to Gnus?
>
> I think GroupLens is a really good idea, and it would be nice if
> something like that were to make a return, but it's a major project.
> Especially the bits where you have to moderate the moderators, so
> that the scoring pool isn't poisoned.

But, GroupLens is not supposed to be a moderation system.  It is
supposed to find people who appear to have opinions similar to yours,
and weights the score based on that "closeness" criterion.

It is thus less likely to be poisoned -- unless you are poisonous too :-)

- Hari
PS: Even though GroupLens is/was a project @cs.umn.edu, I'm not part
    of that group

-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu

Re: GroupLens (gnus-gl.el)

[Lars Magne Ingebrigtsen]

Raja R Harinath <harinath@cs.umn.edu> writes:

> But, GroupLens is not supposed to be a moderation system.  It is
> supposed to find people who appear to have opinions similar to yours,
> and weights the score based on that "closeness" criterion.
>
> It is thus less likely to be poisoned -- unless you are poisonous too :-)

It's a more subtle tool than, er, "flat" moderation, but it's still
pretty easy to poison (unless you use some kind of authorization
system, or keep it "in the family").  Hacking up a system that would
inject a gazillion fake profiles that would make it likely that your
important MAKE VIAGRA NOW messages rate highly isn't particularly
difficult.  Probably.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: GroupLens (gnus-gl.el)

[Ted Zlatanov]

On Sun, 02 Feb 2003, larsi@gnus.org wrote:
> It's a more subtle tool than, er, "flat" moderation, but it's still
> pretty easy to poison (unless you use some kind of authorization
> system, or keep it "in the family").  Hacking up a system that would
> inject a gazillion fake profiles that would make it likely that your
> important MAKE VIAGRA NOW messages rate highly isn't particularly
> difficult.  Probably.

So let's build an authorization system, using the standard "put this
number in a noisy image and make the user type it back" technique.
You'll have to register through it, and the system could also require
a PGP message signature every time you submit a score (not for the
computational burden, but because spammers are not likely to bother
with it at that point).  It will be difficult to set up, and that will
be a Good Thing IMO.

Ted

Re: GroupLens (gnus-gl.el)

[Simon Josefsson]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sun, 02 Feb 2003, larsi@gnus.org wrote:
>> It's a more subtle tool than, er, "flat" moderation, but it's still
>> pretty easy to poison (unless you use some kind of authorization
>> system, or keep it "in the family").  Hacking up a system that would
>> inject a gazillion fake profiles that would make it likely that your
>> important MAKE VIAGRA NOW messages rate highly isn't particularly
>> difficult.  Probably.
>
> So let's build an authorization system, using the standard "put this
> number in a noisy image and make the user type it back" technique.

It is simple to break most, if not all, currently used automated
turing tests...

> You'll have to register through it, and the system could also require
> a PGP message signature every time you submit a score

This could work, but only if users trust the signatures out of band,
which for many computer savvy people is already possible.  Getting
your key signed by computer savvy people wherever you are is not that
difficult (think debian).  Abusers could be locked out by placing
their keys on a blacklist, threshold signed by people that think they
should be locked out.  Then other users can chose which blacklist to
use and which threhshold to require (i.e., how many signatures of
implicitly trusted people required to black list someone).

All this assumes there are more good, and actively interested, users
than bad users, which I believe is as good as you can get (byzantine
models etc).

> It will be difficult to set up, and that will be a Good Thing IMO.

That rules out world domination.  If we assume there are more good
people than bad people in the world, making it easy enough for
everyone to use would improve the system.  (And if we make the
opposite assumption, then the good users could simply negate the votes
on reading, if they don't tell the bad users. :-))

Re: GroupLens (gnus-gl.el)

[Ted Zlatanov]

On Sun, 02 Feb 2003, jas@extundo.com wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:

>> So let's build an authorization system, using the standard "put
>> this number in a noisy image and make the user type it back"
>> technique.
> 
> It is simple to break most, if not all, currently used automated
> turing tests...

An image of a word/number with random noise and a random font is
pretty hard to recognize automatically.  But anyhow, this is just to
apply to be a contributor - you have to be voted in next, see below :)

>> You'll have to register through it, and the system could also
>> require a PGP message signature every time you submit a score
> 
> This could work, but only if users trust the signatures out of band,
> which for many computer savvy people is already possible.  Getting
> your key signed by computer savvy people wherever you are is not
> that difficult (think debian).  Abusers could be locked out by
> placing their keys on a blacklist, threshold signed by people that
> think they should be locked out.  Then other users can chose which
> blacklist to use and which threhshold to require (i.e., how many
> signatures of implicitly trusted people required to black list
> someone).
> 
> All this assumes there are more good, and actively interested, users
> than bad users, which I believe is as good as you can get (byzantine
> models etc).

Those who used to frequent BBS systems will remember there used to be
new user voting systems.  New users would have to get a number of
votes from current users to be accepted.  Furthermore, the sysop could
delete the account if he felt like it.

New users had to read the greeting messages carefully, and spend more
than a few minutes answering a detailed questionnaire.  They had to
show some intelligence, some wit, and some awareness of the purpose of
the board.  Thus, their reading and writing skills were a factor
(unless they simply knew a lot of current members).

The equivalent for Gnus moderation would be a user moderation group.
Let's say group Kabal is started by me.  I would tell Gnus users:
point your Gnus to http://kabal.org/gnus-lens.  There, they would find
the current moderator list, and they would set up GnusLens (how about
that name instead of GroupLens?) to consult that web site for article
scores through some HTTP low-bandwidth mechanism.

I would promote a few other people to be Kabal meta-moderators, and
they would also have the authority to give Kabal contributors scores,
to blacklist contributors, etc.  Furthermore, while it may take 20
regular group contributors to approve or reject a new contributor,
meta-moderators could do it on their own.

Picking the Kabal user group would mean you accept me and my friends
as good judges of character, not as good article scorers necessarily.

>> It will be difficult to set up, and that will be a Good Thing IMO.
> 
> If we assume there are more good people than bad people in the
> world, making it easy enough for everyone to use would improve the
> system.  (And if we make the opposite assumption, then the good
> users could simply negate the votes on reading, if they don't tell
> the bad users. :-))

I don't know about good/bad user ratios.  I think making it hard to
sign up (in a read-the-FAQ way, not in a IQ-test way) simply means
that meta-moderators and regular contributors will have less people to
vote for.

This is getting a bit off-track for Gnus, but it's all probably
necessary if we want to create a GnusLens system (GnusLoupe? I don't
know what a good name would be, but I'm pretty sure GroupLens is
trademarked according to their web site).  Writing the code to run
such a web site is definitely a major endeavor, and the site would
generate a lot of CPU load and network traffic.

Ted

Re: GroupLens (gnus-gl.el)

[Raja R Harinath]

Hi,

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Raja R Harinath <harinath@cs.umn.edu> writes:
>
>> But, GroupLens is not supposed to be a moderation system.  It is
>> supposed to find people who appear to have opinions similar to yours,
>> and weights the score based on that "closeness" criterion.
>>
>> It is thus less likely to be poisoned -- unless you are poisonous too :-)
>
> It's a more subtle tool than, er, "flat" moderation, but it's still
> pretty easy to poison 

But the subtlety is of the same kind that makes bayesian filtering
work.  The larger your corpus of ratings, the more resilient to
poisoning.  The poisoner has to put in more work to get to you, and
he'll only get a small payoff for all that effort (fewer people
reached for amount of effort expended).

> (unless you use some kind of authorization system, or keep it "in
> the family").  

I think the issues here are

  * make sure that submitted ratings are credited to right person
    (simple PGP signatures)
  * make sure that the database maintainer is trustworthy 
    (unlike Amazon.com ;-)

Bogus data will be easier to ignore, since it'll be hard for that data
to be "close to" good data.

> Hacking up a system that would inject a gazillion fake profiles that
> would make it likely that your important MAKE VIAGRA NOW messages
> rate highly isn't particularly difficult.  Probably.

But, if none of them is "close", that's not a problem.  And, each time
you come near a poisoned profile, you can distance yourself from it
with your own ratings.

- Hari
-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu