new spam functionality added

new spam functionality added

[Ted Zlatanov]

I just added spam-split: a function to be used in nnmail-split-fancy,
for instance.  It outputs the value of smap-split-group if a message
is sent from a blackhole relay, or if the sender is blacklisted.  The
whitelist is checked before the blacklist is consulted.

I also made some small fixes here and there to the source of spam.el,
nothing major.

query-dns from dns.el didn't work for me, and I couldn't figure out
the exact problem.  That caused problems with the blackhole check.
Who's in charge of dns.el?

I'm ramping up after 2 months of 10% use of my right hand, so I'll try
to keep up with the spam work but I'm still pretty slow.  Sorry for
the delay, and feel free to suggest improvements.  Look in the ding
archives for previous discussions on this topic, where Lars and I have
mentioned the plans for the spam.el library.

Thanks
Ted

Re: new spam functionality added

[Scott A Crosby]

On Wed, 31 Jul 2002 15:24:34 -0400, Ted Zlatanov <tzz@lifelogs.com> writes:

> I'm ramping up after 2 months of 10% use of my right hand, so I'll try
> to keep up with the spam work but I'm still pretty slow.  Sorry for
> the delay, and feel free to suggest improvements.  Look in the ding
> archives for previous discussions on this topic, where Lars and I have
> mentioned the plans for the spam.el library.

Why reimplement the wheel? We can link to an external program (like
spamassassin) and have it do all the real work for us. (Thats what my
current setup is). Then just patternmatch on the headers it adds to
the email.

Scott

Re: new spam functionality added

[Ted Zlatanov]

On 31 Jul 2002, scrosby@cs.rice.edu wrote:
> Why reimplement the wheel? We can link to an external program (like
> spamassassin) and have it do all the real work for us. (Thats what
> my current setup is). Then just patternmatch on the headers it adds
> to the email.

That's fine as a DIY option.  Pure Lisp is more portable, easier to
extend and customize by novice users, and integrates better with the
rest of Gnus.  I think this would be an especially important feature
on platforms like Windows, or in situations where a user can't easily
insert headers in mail received (IMAP on an inaccessible server, for
instance).  I may be wrong of course, but Lars was the one who started
the spam.el work, so maybe he should weigh in.

Ted

Re: new spam functionality added

[Simon Josefsson]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> On Wed, 31 Jul 2002 15:24:34 -0400, Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> I'm ramping up after 2 months of 10% use of my right hand, so I'll try
>> to keep up with the spam work but I'm still pretty slow.  Sorry for
>> the delay, and feel free to suggest improvements.  Look in the ding
>> archives for previous discussions on this topic, where Lars and I have
>> mentioned the plans for the spam.el library.
>
> Why reimplement the wheel? We can link to an external program (like
> spamassassin) and have it do all the real work for us. (Thats what my
> current setup is). Then just patternmatch on the headers it adds to
> the email.

How to hook up to external programs is already described in the manual
(suggest improvements if something is missing).  Reimplementing it
allows for greater flexibility, although I guess much work will be
needed until it can supersede spamassassin.  It is also more
lightweight than calling external programs.

My only suggestion is: texi documentation ;-)

Another thought on documentation: if spam.el documentation is added,
and the NoCeM node is moved, to the "Thwarting Email Spam" node, I
think that node becomes long enough, and important enough, to warrant
moving it up to the top-level menu.  Opinions?

Re: new spam functionality added

[Josh Huber]

Simon Josefsson <jas@extundo.com> writes:

> Another thought on documentation: if spam.el documentation is added,
> and the NoCeM node is moved, to the "Thwarting Email Spam" node, I
> think that node becomes long enough, and important enough, to
> warrant moving it up to the top-level menu.  Opinions?

Sure, perhaps we should include something about TMDA?

http://cvs.sf.net/cgi-bin/viewcvs.cgi/~checkout~/tmda/tmda/contrib/tmda.el

I'm in the middle of writing up some documentation to put on
my.gnus.org, and converting it into texi for the manual would be
simple.  What do you think?

-- 
Josh Huber

Re: new spam functionality added

[Scott A Crosby]

On Wed, 31 Jul 2002 16:25:53 -0400, Josh Huber <huber+dated+1028578998.eb7ca9@alum.wpi.edu> writes:

> Simon Josefsson <jas@extundo.com> writes:
> 
> Sure, perhaps we should include something about TMDA?
> 

Please don't.. TMDA is tragedy of the commons. It only helps one
person by putting extra work and effort upon everyone else. If
everyone used it, things will turn to crap.

BTW, whats the archive address for this list?

Scott

Re: new spam functionality added

[Josh Huber]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Please don't.. TMDA is tragedy of the commons. It only helps one
> person by putting extra work and effort upon everyone else. If
> everyone used it, things will turn to crap.

I disagree.  With all of TMDA's facilities for tagging messages which
expire, keyword addresses and sender addresses most people don't even
know you're using it. (apart from a funny looking dated email
address).

In practice, over the past 2 weeks the only messages which have
appeared in my pending queue have been spams.  It's worked 100%, with
0 false positives.  I used an initial seed whitelist based on my
outbox and a few other sources, and it's been working quite well.

What don't you like about it?

> BTW, whats the archive address for this list?

Well, it's archived on nntp+quimby.gnus.org:gnus.ding, which is where
I read/post.

have a nice day,

-- 
Josh Huber

Re: new spam functionality added

[Stainless Steel Rat]

* Josh Huber <huber+dated+1028579883.ab0915@alum.wpi.edu>  on Wed, 31 Jul 2002
| What don't you like about it?

We is disliked about TMDA is that it punnishes legitimate users while doing
nothing to actually stop spam.

-- 
Rat <ratinox@peorth.gweep.net>    \ Ingredients of Happy Fun Ball include an
Minion of Nathan - Nathan says Hi! \ unknown glowing substance which fell to
PGP Key: at a key server near you!  \ Earth, presumably from outer space.
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[Stainless Steel Rat]

* Stainless Steel Rat <ratinox@peorth.gweep.net>  on Wed, 31 Jul 2002
| We is disliked about TMDA is that it punnishes legitimate users while doing
  ^^
Ahem.  "What is disliked".

-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[Josh Huber]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> We is disliked about TMDA is that it punnishes legitimate users
> while doing nothing to actually stop spam.

In practice though, 99% of your mail is either whitelisted or sent to
a dated address.

As for stopping spam, of course it does.  It stops spam from getting
into your inbox.  Do you actually have a suggestion for something
which will stop spam for good?

I suspect it's something which will never go away.  Even if the burden
of transmission shifted from the receiver to the sender, it will only
get worse.  Look at at your US mail for an example of that!

-- 
Josh Huber

Re: new spam functionality added

[Paul Jarc]

Josh Huber <huber+dated+1028581649.ce9fdf@alum.wpi.edu> wrote:
> Even if the burden of transmission shifted from the receiver to the
> sender, it will only get worse.  Look at at your US mail for an
> example of that!

I think I get more junk email than junk paper mail.  Anyway, if
senders had to store pending messages, receivers could develop ways of
recognizing them and could avoid downloading the messages, even if
they couldn't avoid the notification that there was a message to be
downloaded.  So there would be at least some traffic savings.


paul

Re: new spam functionality added

[David Masterson]

>>>>> Paul Jarc writes:

> Josh Huber <huber+dated+1028581649.ce9fdf@alum.wpi.edu> wrote:

>> Even if the burden of transmission shifted from the receiver to the
>> sender, it will only get worse.  Look at at your US mail for an
>> example of that!

> I think I get more junk email than junk paper mail.  

You're lucky then.  Often the real mail is hidden in the junk mail, so
I have to sort through it anyway just to make sure I'm not throwing
the good stuff out.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Frank Schmitt]

Josh Huber <huber+dated+1028581649.ce9fdf@alum.wpi.edu> writes:

> I suspect it's something which will never go away.  Even if the burden
> of transmission shifted from the receiver to the sender, it will only
> get worse.  Look at at your US mail for an example of that!

I'm glad I live in Germany, here you can place a sign on your post box
saying "No advertisements please" and you are rid of it. Hell, I wished it
was the same for my E-Mail.

But what really enrages me isn't so much the fact that I get
Advertisements via E-Mail but the fact, that's so senseless. I have the
opinion that if you want money you've got to work for it, so the "Make
money fast" stuff doesn't interest me. I'm satisfied with both my sex
life in general and the length of my most precious part in special, so I
also have no interest in "We've done it! We cracked a real porn
dialer..." or "Meet young horny women" or "Three inches in only four
weeks". And *_I DON'T SPEAK KOREAN!!!_*. This really enrages me. Some
stupid dumpasses, send thousands of E-Mails to people who can't even
read them. It costs me money, it costs my provider money, it slows down
the net and what for? They'll probably have a hit ratio of about
1:100,000. And big parts of the European and US spam enters the net in
Korea, too since there seems to be hundreds of with broken servers which
act as open gateways, with abuse departments who answer "English speak
not, understanding not". And a big part of this machines stand in
schools. Isn't that braindead? I think I've got to stop before I become
really angry, but one thing I promise to all spammers out there:
If I ever get hold of one you bloody bastards, you'll find out what
Pfählen is.

-- 
One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them
In the Land of Mordor where the Shadows lie.

Re: new spam functionality added

[Josh Huber]

Frank Schmitt <usereplyto@Frank-Schmitt.net> writes:

> I'm glad I live in Germany, here you can place a sign on your post
> box saying "No advertisements please" and you are rid of it. Hell, I
> wished it was the same for my E-Mail.

Wow, that's so nice!

I get about a 2:1 ratio of junk mail EVERY DAY in the US mail.
Typically it's constructed in such a way where you can't tell it's
junk until you open the envelope.  Quite frustrating.

-- 
Josh Huber

Re: new spam functionality added

[Harry Putnam]

Josh Huber <huber+dated+1028589381.274799@alum.wpi.edu> writes:

> Frank Schmitt <usereplyto@Frank-Schmitt.net> writes:
>
>> I'm glad I live in Germany, here you can place a sign on your post
>> box saying "No advertisements please" and you are rid of it. Hell, I
>> wished it was the same for my E-Mail.
>
> Wow, that's so nice!
>
> I get about a 2:1 ratio of junk mail EVERY DAY in the US mail.
> Typically it's constructed in such a way where you can't tell it's
> junk until you open the envelope.  Quite frustrating.

I was considering that phenomena one day.  I live in a 7 unit
apartment building.  Even that small number, in one week, accumulates a
very large pile of junk mail.  And that is only what gets left in the
common bin.  Not what people collect from there personal boxes.

I began to wonder what kind of tonnage would be involved if one were
to measure a weeks worth of that crap across the entire US.
It a bit frightening... Seems it should be stopped by law somehow.

It's one small source (compared to all trash) that could be done away
with easily.

Re: new spam functionality added

[Scott A Crosby]

On Thu, 01 Aug 2002 10:38:43 -0700, Harry Putnam <reader@newsguy.com> writes:

> Josh Huber <huber+dated+1028589381.274799@alum.wpi.edu> writes:
> 
> I was considering that phenomena one day.  I live in a 7 unit
> apartment building.  Even that small number, in one week, accumulates a
> very large pile of junk mail.  And that is only what gets left in the
> common bin.  Not what people collect from there personal boxes.
> 
> I began to wonder what kind of tonnage would be involved if one were
> to measure a weeks worth of that crap across the entire US.
> It a bit frightening... Seems it should be stopped by law somehow.
> 
> It's one small source (compared to all trash) that could be done away
> with easily.

Advertising does serve a purpose. It does tell you of goods and
services that you may be interested. For example, a Qwest phone call
made me realize that I should look around at long distance
plans.... Which reduced my phone bill by a factor of 3!

I also just moved 1700 miles 2 months ago. Again, advertising
circulars were useful; they told me about local places I may be
interested in going to.

Advertising by itself isn't bad.. Unwanted communication wasting your
time is.

Scott

Re: new spam functionality added

[Harry Putnam]

Scott A Crosby <scrosby@cs.rice.edu> writes:

>> It's one small source (compared to all trash) that could be done away
>> with easily.
>
> Advertising does serve a purpose. It does tell you of goods and
> services that you may be interested. For example, a Qwest phone call
> made me realize that I should look around at long distance
> plans.... Which reduced my phone bill by a factor of 3!
>
> I also just moved 1700 miles 2 months ago. Again, advertising
> circulars were useful; they told me about local places I may be
> interested in going to.
>
> Advertising by itself isn't bad.. Unwanted communication wasting your
> time is.

Heartily disagree!  No reason whatever for advertising.  That is, not
any good ones anyway.  One can always find what they need to know
about.  especially with the advent of google ... hehe.

Advertising is just some pinhead pushing his line of bull for
profit.  Not a nice little neighborhood helper at all.

Maybe once in a while it turns up something you happen to want at the
moment but those 170,000 other copies didn't.  I think it represents
a really deficient system, absolute wastefullness etc etc.

Can you really argue the 100s, maybe thousands of tons of paper per week
is some how `ok'?  Some of it may find a good use wrapping fresh
fish. Or in what few outhouse remain in USA.

Re: new spam functionality added

[Per Abrahamsen]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Advertising does serve a purpose. It does tell you of goods and
> services that you may be interested. 

Yep.  In fact, www.jatak.com has been a success in Denmark.  It is a
service that allows users to ask for advertisement in email, and
specify their interests as well what format they prefer.

Advertisers then pay www.jatak.com to distribute the advertising to
customers known to be interested.

Advertersing is both useful and popular, when done right.  It is only
the unsolisticated kind that stinks.

Re: new spam functionality added

[Stainless Steel Rat]

* Josh Huber <huber+dated+1028581649.ce9fdf@alum.wpi.edu>  on Wed, 31 Jul 2002
| In practice though, 99% of your mail is either whitelisted or sent to
| a dated address.

It is that remaining 1% of legitimate mail being bounced that I find
unacceptable.

| As for stopping spam, of course it does.  It stops spam from getting
| into your inbox.  Do you actually have a suggestion for something
| which will stop spam for good?

Hashcash.  Not X-Hashcash, real hashcash.

-- 
Rat <ratinox@peorth.gweep.net>    \ Warning: pregnant women, the elderly, and
Minion of Nathan - Nathan says Hi! \ children under 10 should avoid prolonged
PGP Key: at a key server near you!  \ exposure to Happy Fun Ball.
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[Scott A Crosby]

On Wed, 31 Jul 2002 21:25:04 -0400, Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> Hashcash.  Not X-Hashcash, real hashcash.

Unlikely. If the hashcash is redeemable for cash, then either it has
to be about $.0001, or each email will include a built-in 90% profit
margin for the recipient. (And I don't want to know how that profit
margin would be, err, exploited, in unique and interesting ways by a
moneymaking enterprise.)


Scott

Re: new spam functionality added

[Stainless Steel Rat]

* Scott A Crosby <scrosby@cs.rice.edu>  on Wed, 31 Jul 2002
| Unlikely. If the hashcash is redeemable for cash, then either it has
| to be about $.0001,

Er... no.  There is money directly involved.  In brief, hashcash works like
this:

You want to send me mail.  You compose your message and hand it off to your
MTA for delivery.  Your MTA contacts my MTA.  My MTA generates a random
string (a random number) and mashes it through SHA-1.  My MTA gives your
MTA that string and says, "find me another string that has an SHA-1 hash
with N bits of collision".  Your MTA spends a few seconds generating random
strings and mashing them through SHA-1 until it generates a hash that has N
similar bits.  Your MTA gives me that string.  My MTA validates it, and
then accepts your message.

The idea is to make every sending MTA spend a bit of CPU time doing
something in a proveable way before accepting their messages.  Spammers
simply will not be able to survive because it will take too long for their
rogue spamware MTAs to push out their junk.

-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[David Masterson]

>>>>> Stainless Steel Rat writes:

[...with respect to HashCash...]

> The idea is to make every sending MTA spend a bit of CPU time doing
> something in a proveable way before accepting their messages.
> Spammers simply will not be able to survive because it will take too
> long for their rogue spamware MTAs to push out their junk.

But it assumes that all (or at least a majority of) MTAs adopt this
(as well as being well-configured to prevent anonymous forwarding).

In the meantime...?

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Stainless Steel Rat]

* David Masterson <dmaster@synopsys.com>  on Thu, 01 Aug 2002
| But it assumes that all (or at least a majority of) MTAs adopt this
| (as well as being well-configured to prevent anonymous forwarding).

Agreed, that is the showstopper.

| In the meantime...?

Spamassassin or at least the DCC.

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[Florian Weimer]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> The idea is to make every sending MTA spend a bit of CPU time doing
> something in a proveable way before accepting their messages.

This would be the death of mailing lists, wouldn't it?

(I know that serving mailing lists is currently bound by disk I/O, but
that's not the point.)

Re: new spam functionality added

[Russ Allbery]

Florian Weimer <fw@deneb.enyo.de> writes:
> Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

>> The idea is to make every sending MTA spend a bit of CPU time doing
>> something in a proveable way before accepting their messages.

> This would be the death of mailing lists, wouldn't it?

All the descriptions I've heard of schemes of this type require
whitelisting legitimate mailing lists.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: new spam functionality added

[Simon Josefsson]

Florian Weimer <fw@deneb.enyo.de> writes:

> Stainless Steel Rat <ratinox@peorth.gweep.net> writes:
>
>> The idea is to make every sending MTA spend a bit of CPU time doing
>> something in a proveable way before accepting their messages.
>
> This would be the death of mailing lists, wouldn't it?

Pretty much so, I think.  That's why I think hashcash needs to be
computed at the machine that sends the original mail; X-Hashcash that
is.  The current implementation isn't very good though, as it stalls
Emacs while computing hashcash.

Hashcash with ~< 25 bits is just a toy, you need more bits to make it
costly for spammers.

Re: new spam functionality added

[Stainless Steel Rat]

* Simon Josefsson <jas@extundo.com>  on Sat, 03 Aug 2002
| Pretty much so, I think.  That's why I think hashcash needs to be
| computed at the machine that sends the original mail; X-Hashcash that
| is.  The current implementation isn't very good though, as it stalls
| Emacs while computing hashcash.

X-Hashcash has problems in that there is no way to force spammers to use
it, and any implementation is nontransparent to users.

| Hashcash with ~< 25 bits is just a toy, you need more bits to make it
| costly for spammers.

Collision size in a proper hashcash implementation is "scoreable", so that
known, trusted sources need a very small collision while spamhauses require
much more.  Consider: if $MAILSOURCE requires a collision that takes 6
seconds on average to calculate, that limits the number of messages it can
send to $MYISP to 14400 per day.

Legitimate mail sources, like ISPs with enforced anti-spam policies, will
get low scores, requiring less than 0.25 seconds of calculation time,
perhaps only 12-16 bits.  ISPs who ignore spam complaints and allow it to
be sent from their servers (gblx-cough-cough) will need 27-30 bits,
effectively choking them off until they clean up their acts.  At least,
that is how my personal hashcash system would be configured; YMMV.

-- 
Rat <ratinox@peorth.gweep.net>    \ Ingredients of Happy Fun Ball include an
Minion of Nathan - Nathan says Hi! \ unknown glowing substance which fell to
PGP Key: at a key server near you!  \ Earth, presumably from outer space.
       That and five bucks will get you a small coffee at Starbucks.

hashcash (was Re: new spam functionality added)

[Simon Josefsson]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> * Simon Josefsson <jas@extundo.com>  on Sat, 03 Aug 2002
> | Pretty much so, I think.  That's why I think hashcash needs to be
> | computed at the machine that sends the original mail; X-Hashcash that
> | is.  The current implementation isn't very good though, as it stalls
> | Emacs while computing hashcash.
>
> X-Hashcash has problems in that there is no way to force spammers to use
> it

My MTA could refuse to receive mail without X-Hashcash.  Same level as
"real" hashcash gives.

> , and any implementation is nontransparent to users.

In the same way MIME and SMTP is nontransparent -- the MUA will have
to support it.  I don't grok how this is a big problem.

> | Hashcash with ~< 25 bits is just a toy, you need more bits to make it
> | costly for spammers.
>
> Collision size in a proper hashcash implementation is "scoreable", so that
> known, trusted sources need a very small collision while spamhauses require
> much more.  Consider: if $MAILSOURCE requires a collision that takes 6
> seconds on average to calculate, that limits the number of messages it can
> send to $MYISP to 14400 per day.

How do you differentiate between trusted sources and spamhauses?  This
sounds like lots of configuration work and maintainance.  It isn't
difficult to fake the source.

> Legitimate mail sources, like ISPs with enforced anti-spam policies, will
> get low scores, requiring less than 0.25 seconds of calculation time,
> perhaps only 12-16 bits.  ISPs who ignore spam complaints and allow it to
> be sent from their servers (gblx-cough-cough) will need 27-30 bits,
> effectively choking them off until they clean up their acts.  At least,
> that is how my personal hashcash system would be configured; YMMV.

MMV.  With that setup you will not be much better off with using
todays publicly available black/white-lists.  Spammers would exploit
known-good spammers until they get blacklisted, and then move on to
another victim ISP.  I would require anyone wanting to send mail to me
to use a reasonable level of hashcash which would exclude spammers.
Then even if a spammer hacked a known-good ISP, I wouldn't accept
their spam blindly.  And hacking known-good ISPs isn't difficult.

Re: hashcash (was Re: new spam functionality added)

[Stainless Steel Rat]

* Simon Josefsson <jas@extundo.com>  on Sat, 03 Aug 2002
| My MTA could refuse to receive mail without X-Hashcash.  Same level as
| "real" hashcash gives.

That is not the same thing as forcing the sender to use it.  That is
nothing more than rejecting mail based on an abstract black list.


| In the same way MIME and SMTP is nontransparent -- the MUA will have
| to support it.  I don't grok how this is a big problem.

They are transparent to the user.


| How do you differentiate between trusted sources and spamhauses?  This
| sounds like lots of configuration work and maintainance.

Hardly.  Scoring can be based on a variety of criteria, ranging from volume
to votes from users.  Different sites will require different criteria, but
there is no reason why they cannot be automated.

| It isn't difficult to fake the source.

It is difficult to fake your IP, and there are relatively simple ways of
detecting such attempts.

-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[David Masterson]

>>>>> Stainless Steel Rat writes:

> * Josh Huber <huber+dated+1028581649.ce9fdf@alum.wpi.edu>  on Wed,
> 31 Jul 2002

> | In practice though, 99% of your mail is either whitelisted or sent
> | to a dated address.

> It is that remaining 1% of legitimate mail being bounced that I find
> unacceptable.

Assuming that the remaining 1% is all "legitimate" is reaching...

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Stainless Steel Rat]

* David Masterson <dmaster@synopsys.com>  on Thu, 01 Aug 2002
| > It is that remaining 1% of legitimate mail being bounced that I find
| > unacceptable.

| Assuming that the remaining 1% is all "legitimate" is reaching...

Is it?  Do you have any conclusive proof that the remaining 1% of mail that
TDMA loses is not legitimate?

-- 
Rat <ratinox@peorth.gweep.net>    \ Caution: Happy Fun Ball may suddenly
Minion of Nathan - Nathan says Hi! \ accelerate to dangerous speeds.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: new spam functionality added

[Scott A Crosby]

On Wed, 31 Jul 2002 16:41:30 -0400, Josh Huber <huber+dated+1028579883.ab0915@alum.wpi.edu> writes:

> Scott A Crosby <scrosby@cs.rice.edu> writes:
> 
> > Please don't.. TMDA is tragedy of the commons. It only helps one
> > person by putting extra work and effort upon everyone else. If
> > everyone used it, things will turn to crap.
> 
> I disagree.  With all of TMDA's facilities for tagging messages which
> expire, keyword addresses and sender addresses most people don't even
> know you're using it. (apart from a funny looking dated email
> address).
> 
> In practice, over the past 2 weeks the only messages which have
> appeared in my pending queue have been spams.  It's worked 100%, with
> 0 false positives.  I used an initial seed whitelist based on my
> outbox and a few other sources, and it's been working quite well.

Because its tragedy of the commons, I bitbucket any TMDA user. (and if
I start getting too many of em, I'll make a public blacklist of em.)

> 
> What don't you like about it?
> 

Well, Jack Twilly phrased one of my problems most elegantly. :) TMDA
''works'' by pushing work onto everyone else.. You know, tragedy of
the commons.

Here's a post I did a while ago on why I don't like it, or any other
scheme requiring autoreply-crap for communication.

++
   No.. Think of it carefully.. TMDA works by polluting everyone
   else. By forcing everyone else you contact to do extra work. This
   is tragedy of the commons.

   Imagine a world where everyone uses it (or something similar), but,
   say, 10% have it misconfigured. This is a world with mailing lists.

   Mailing list maintance functions (including initial requests to
   subscribe, or confirmation requests from web-maintance.) either get
   accepted automatically, (direct route for spam!), or force the
   mailing list admin to deal with the automated 'please reply to me'
   messages.. Which they'll ignore, then they'll still get users
   asking why email subscriptions don't work.

   Mailing list messages... Post to a mailing list the first time and
   potentially get tens, hundreds, even thousands of 'please reply to
   me' messages. Hey, they only take a second each to deal with!

   Now, imagine there's a daemon that autoreplies to such 'please
   reply to me' messages.. Well, just forge the spam to appear to come
   from a legitimate user, and guess what, the bounces go to them, and
   their client helpfully 'authenticates' the spam.. (The daemon can't
   be configured to record every email sent and only autoreply to
   autoreplies to emails the user actually sent. Many times people
   will use many systems and email servers, but only one email
   address.)

   For more fun, you may even get mail loops of 'please reply to me'
   messages.

   Now, in the above examples, you can eliminate this undesireable
   behavior by automatically accepting, unchecked, mailing list
   maintance messages, or autoreply messages, or a blanket opening for
   mailing list messages... However, spam can be easily forged to
   appear to be a maintance message or an autoreply message.

   Under the assumption that there *will* be misconfigured clients,
   they'll have to deal with mailing lists that they don't know
   about. either by spamming posters to the list (unacceptable), or
   filtering them out into a seperate folder that the user will have
   to manually check.

   In all cases, if the 'please reply to me' messages are mechanically
   replyable, then a daemon will be created to deal with that trash
   automatically, and most users will use it. (So, spammers can forge
   their email to come from almost any user, and the daemon of the
   forged address will reply.) Or, those messages can be used to
   indicate that an email address is live. (Send a message to someone
   using TMDA, confirm that they use TMDA, now you know you can forge
   spam from that address and their daemon will authenticate it for
   you for free!)

   Of course the other option here is to spam from legitimate hosts
   that have been cracked by today's IIS/outlook worm. (Or one of the
   30,000 *STILL* infected code-red machines.) The cracked systems run
   email servers and reply automatically.

   Now, if the 'please reply to me' messages are NOT mechanically
   replyable, then we've saturated the internet with an even larger
   amount of trash and mail pollution that has to be dealt with on a
   message-by-message basis. (As per the above scenario's.)

   In any case. TMDA is not a solution, its a problem.

   TMDA and any other scheme that requires such automated response to
   all sent emails is tragedy of the commons. There's no better
   example. It superficially helps the user, to the detriment of
   everyone else. Ergo, it will proliferate and everyone will be even
   worse off.

++


> Well, it's archived on nntp+quimby.gnus.org:gnus.ding, which is where
> I read/post.

Thanks!

Scott

Re: new spam functionality added

[Paul Jarc]

Scott A Crosby <scrosby@cs.rice.edu> wrote:
>    Mailing list maintance functions (including initial requests to
>    subscribe, or confirmation requests from web-maintance.) either get
>    accepted automatically, (direct route for spam!), or force the
>    mailing list admin to deal with the automated 'please reply to me'
>    messages..

A well-behaved subscriber would add the MLM address to their whitelist
before subscribing.  List admins can feel free to drop confirmation
requests from poorly-behaved subscribers.

>    Mailing list messages... Post to a mailing list the first time and
>    potentially get tens, hundreds, even thousands of 'please reply to
>    me' messages.

I'm fairly certain that's false.  TMDA sends its confirmation requests
to the envelope sender, not From:, I think.  MLMs rewrite the envelope
sender.

>    Now, imagine there's a daemon that autoreplies to such 'please
>    reply to me' messages..

Actually, the replies can be automated, even safely, if the
confirmation request includes the entire original message, or a
checksum of it, so the autoconfirmer can verify that the user really
sent the original message.

>    Well, just forge the spam to appear to come from a legitimate
>    user, and guess what, the bounces go to them, and their client
>    helpfully 'authenticates' the spam..

That's why there shouldn't be badly implemented autoconfirmers.  But
are there any?  Anyway, this is not an argument against TMDA itself.

>    (The daemon can't be configured to record every email sent and
>    only autoreply to autoreplies to emails the user actually sent.

Sure it can.  But recording entire emails themselves wouldn't be the
best way to do it.

>    Many times people will use many systems and email servers, but
>    only one email address.)

A checksum scheme could still work in such situations.

>    For more fun, you may even get mail loops of 'please reply to me'
>    messages.

If the confirmation request is sent from a dated address which does
not itself require confirmation, there is no loop.

Of course, if everyone starts using TMDA, spammers will quickly start
guessing dated addresses.  But then dated addresses will just evolve
to be a hash of the date and a secret instead of just the date.

>    Under the assumption that there *will* be misconfigured clients,
>    they'll have to deal with mailing lists that they don't know
>    about. either by spamming posters to the list (unacceptable), or
>    filtering them out into a seperate folder that the user will have
>    to manually check.

I'm not sure what you mean.

>    Of course the other option here is to spam from legitimate hosts
>    that have been cracked by today's IIS/outlook worm. (Or one of the
>    30,000 *STILL* infected code-red machines.) The cracked systems run
>    email servers and reply automatically.

Cracked systems can be abused even without TMDA in the picture.

>    TMDA and any other scheme that requires such automated response to
>    all sent emails is tragedy of the commons.

TMDA doesn't require that unless you configure it that way.


paul

Re: new spam functionality added

[Josh Huber]

prj@po.cwru.edu (Paul Jarc) writes:

> Of course, if everyone starts using TMDA, spammers will quickly
> start guessing dated addresses.  But then dated addresses will just
> evolve to be a hash of the date and a secret instead of just the
> date.

Just thought I'd mention this -- TMDA already does this. (hash with
secret)  The hash is included with dated, sender and keyword addresses
so you can't just make them up and have them work.

-- 
Josh Huber

Re: new spam functionality added

[Josh Huber]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Because its tragedy of the commons, I bitbucket any TMDA user. (and
> if I start getting too many of em, I'll make a public blacklist of
> em.)

Do you actually get a lot of confirmation requests?

> [snip lengthly example]

I can see how you might think something like this would happen.  There
are a few flaws in your example.  (unless you assume many people have
misconfigured TMDA installations...but then why not assume they have
misconfigured MTA installations which spew duplicate messages all over
the place (or allow unrestricted relaying of messages)).

Most of your concerns are answered in the FAQ on the tmda site.

1. You will not get confirmation requests for posting to a mailing
list which has subscribers using TMDA.  A confirmation request (if
any) would go to the *envelope sender* not the original sender (you).
Of course, no confirmation request will be sent because you would not
subscribe to a mailing list with TMDA using a bare (non-tagged)
address.  Get it?  The problem you describe with mailing lists just
does not happen.

2. "loops of please reply to me" -- this does not happen.  Answer in
the FAQ:

http://tmda.net/faq.cgi?req=show&file=faq04.012.htp

3. Setting up an auto-responder typically means the spammer has
provided a valid return address, which is very rare.  In the case they
have, it's easy to find who it is and get their connection shut off.

You really speak of absolute worst-case behavior in your complaint,
which isn't very fair.  As an example, one of the authors of TMDA
mentions in the FAQ that approximately 6% of his correspondence is
asked for confirmation.

I agree that making it harder than hitting reply is a bit much.  I've
seen one instance where the recipient has you go to his web site and
fill out a CGI form describing the image displaed on the web page.
This is pretty sick, and going much too far IMO.

Anyway, my initial suggestion to put some text about TMDA in the
manual was just that -- a suggestion.  I really don't want to start a
flame war here :)

Have a nice day,

-- 
Josh Huber

Re: new spam functionality added

[Paul Jarc]

Josh Huber <huber+dated+1028582805.6306a1@alum.wpi.edu> wrote:
> 3. Setting up an auto-responder typically means the spammer has
> provided a valid return address, which is very rare.

Scott's talking about spammers taking advantage of an existing
(badly-written, hypothetical) autoconfirmer, not setting up their own.
Such existing autoconfirmers would be set up by people who don't want
to be bothered by confirmation requests, and who don't care to take
the time to find a well-written one.


paul

Re: new spam functionality added

[Josh Huber]

prj@po.cwru.edu (Paul Jarc) writes:

> Scott's talking about spammers taking advantage of an existing
> (badly-written, hypothetical) autoconfirmer, not setting up their
> own.  Such existing autoconfirmers would be set up by people who
> don't want to be bothered by confirmation requests, and who don't
> care to take the time to find a well-written one.

Ah, okay.  That makes more sense.  I really can't immagine
autoresponders which make no attempt at verifying what they're doing
though.  (well, that's not true, but I'd *hope* daemons like this
would be shut down quickly!)

-- 
Josh Huber

Re: new spam functionality added

[Paul Jarc]

Josh Huber <huber+dated+1028584847.72d15f@alum.wpi.edu> wrote:
> I really can't immagine autoresponders which make no attempt at
> verifying what they're doing though.  (well, that's not true, but
> I'd *hope* daemons like this would be shut down quickly!)

If they weren't, they could still only be used once.  After a spam
gets through because of an autoconfirmer, that address would get added
to everyone's blacklist.


paul

Re: new spam functionality added

[Scott A Crosby]

On Wed, 31 Jul 2002 17:47:15 -0400, Josh Huber <huber+dated+1028582805.6306a1@alum.wpi.edu> writes:

> Scott A Crosby <scrosby@cs.rice.edu> writes:
> 
> > Because its tragedy of the commons, I bitbucket any TMDA user. (and
> > if I start getting too many of em, I'll make a public blacklist of
> > em.)
> 
> Do you actually get a lot of confirmation requests?
> 

Never, for which I'm glad.. But as its a bad idea in principal, I
refuse to support it in any way.

> > [snip lengthly example]
> 
> I can see how you might think something like this would happen.  There
> are a few flaws in your example.  (unless you assume many people have
> misconfigured TMDA installations...but then why not assume they have
> misconfigured MTA installations which spew duplicate messages all over
> the place (or allow unrestricted relaying of messages)).

TMDA is an end-user system, there will be misconfigurations of it, and
bad reimplementations.. Hell, I got a vacation message from a message
to this mailing list only an hour ago... After 20 years, you'd think
they'd not be writing broken vacation programs anymore.

> Most of your concerns are answered in the FAQ on the tmda site.
> 
> 1. You will not get confirmation requests for posting to a mailing
> list which has subscribers using TMDA.  A confirmation request (if
> any) would go to the *envelope sender* not the original sender (you).

That either harasses the mailing list admin, gets posted accidently, or
gets dropped (in which case, the user is likely to harass the mailing
list admin for why the subscription request failed.)

> Of course, no confirmation request will be sent because you would not
> subscribe to a mailing list with TMDA using a bare (non-tagged)
> address.  Get it?  The problem you describe with mailing lists just
> does not happen.

Sure.. I could configure TMDA to not act like an idiot. (well, mostly,
I'd still screw up a little and harass some mail admins), but I'm not
a typical user.

> 2. "loops of please reply to me" -- this does not happen.  Answer in
> the FAQ:
> 
> http://tmda.net/faq.cgi?req=show&file=faq04.012.htp

   ''If X uses his common sense, this won't happen.''

And what if X isn't aware of all the subtle issues? Which is what
you'd expect from most people and implementations out there.


Or, this is interesting:

   ''headers 'X-Delivery-Agent:.*TMDA' ok''

to auto-allow-TMDA messages through. So, start putting that on all of your email.

> You really speak of absolute worst-case behavior in your complaint,

This is not unfair.. 20 years later, they're still implementing bad
vacation programs.

> which isn't very fair.  As an example, one of the authors of TMDA
> mentions in the FAQ that approximately 6% of his correspondence is
> asked for confirmation.

I don't know.... if 6% of my outgoing email required confirmation, I'd
be pretty annoyed. Autoreplying to one or two TMDA's a day costs me a
lot more time than skimming my spam folder for accidental false
positives.

> I agree that making it harder than hitting reply is a bit much.  I've
> seen one instance where the recipient has you go to his web site and
> fill out a CGI form describing the image displaed on the web page.
> This is pretty sick, and going much too far IMO.

Bitbucket em. :)  Or better yet, put them in a public blacklist of 'morons'.

> Anyway, my initial suggestion to put some text about TMDA in the
> manual was just that -- a suggestion.  I really don't want to start a
> flame war here :)

I should apologize.. I've just been reading too much about people
lauding TMDA as being great. However, I think its a fundamentally bad
idea; a perfect example of tragedy of the commons.

Scott

Re: new spam functionality added

[Josh Huber]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> TMDA is an end-user system, there will be misconfigurations of it,
> and bad reimplementations.. Hell, I got a vacation message from a
> message to this mailing list only an hour ago... After 20 years,
> you'd think they'd not be writing broken vacation programs anymore.

I agree, but you can hardly blame TMDA for bad re-implementations,
right?  (btw, I got the same vacation message!)

> Sure.. I could configure TMDA to not act like an idiot. (well,
> mostly, I'd still screw up a little and harass some mail admins),
> but I'm not a typical user.

I see where you're coming from here.  Something like TDMA does require
some effort and know-how to configure and use.  It will be interesting
to see what really happens over the next few years with junk email.
For the moment, I think *I'm* using TMDA in a responsible manner, if I
feel like it's too much burden on others I'll stop using it.

>    ''If X uses his common sense, this won't happen.''
>
> And what if X isn't aware of all the subtle issues? Which is what
> you'd expect from most people and implementations out there.

Well, okay.  I really see what you're saying here.  I think a tool
like TMDA is pretty powerful and used incorrectly it could be an
incredible pain in the ass to everyone.  (see for example all the
poorly designed virus sanners spamming mailing lists out there)

> Or, this is interesting:
>
>    ''headers 'X-Delivery-Agent:.*TMDA' ok''
>
> to auto-allow-TMDA messages through. So, start putting that on all
> of your email.

Personally, I don't think this is a good solution and wouldn't
recommend using this kind of a rule with TMDA, but it is available.

> I don't know.... if 6% of my outgoing email required confirmation,
> I'd be pretty annoyed. Autoreplying to one or two TMDA's a day costs
> me a lot more time than skimming my spam folder for accidental false
> positives.

I agree, 6% is a lot.  Of course, not too much information is there
wrt this figure.  the 6% may include requesting confirmations from
spam as well.

> I should apologize.. I've just been reading too much about people
> lauding TMDA as being great. However, I think its a fundamentally
> bad idea; a perfect example of tragedy of the commons.

You keep saying that, but I'm not so sure I agree.  Personally, I'm
going to wait and see.  The tragedy of the commons seems to rely on
the fact that there are limited resources, and once these resources
are used, they are no longer available.  I don't think this is the
case with bandwidth.  I really do see what you're saying, honestly :)

I guess I'm willing to wait and see if it will work, rather then just
writing it off as terrible without even trying it.  As I've said
before, it's worked for me quite well so far but I have yet to see how
well it will scale.

Perhaps we should stop spamming the list with this discussion since it
has very little to do with the development of Gnus :)

(btw, please respect my Mail-Copies-To header)

ttyl,

-- 
Josh Huber

Re: new spam functionality added

[Paul Jarc]

Josh Huber <huber+dated+1028587986.de2d4e@alum.wpi.edu> wrote:
> The tragedy of the commons seems to rely on the fact that there are
> limited resources, and once these resources are used, they are no
> longer available.  I don't think this is the case with bandwidth.

It is the case with bandwidth, but more importantly, it is the case
with the time humans spend replying to confirmation requests.  But a
well-written autoconfirmer would take care of that.


paul

Re: new spam functionality added

[Alan Shutko]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> TMDA is an end-user system, there will be misconfigurations of it, and
> bad reimplementations.. 

Even if it weren't end-user, there would be misconfigurations.  I've
been on two mailing lists so far hit by digiportals authorization
messages.  The auth requested goes to the right place, but the auth
granted spams the list.

Sure, their software is broken... what do you want to bet that some
TMDA software will be broken too?  I mean, there are still systems
which don't send errors to the envelope sender.

-- 
Alan Shutko <ats@acm.org> - In a variety of flavors!
If money can't buy happiness, I guess you'll just have to rent it.

Re: new spam functionality added

[David Masterson]

>>>>> Alan Shutko writes:

> Scott A Crosby <scrosby@cs.rice.edu> writes:

>> TMDA is an end-user system, there will be misconfigurations of it,
>> and bad reimplementations..

> Even if it weren't end-user, there would be misconfigurations.

All complex systems suffer these problems for a time.  Then someone
figures out how to improve the configuration process and the problems
die down.  Eventually, Linux distributions could automatically setup a
basic TMDA environment (ie. purely "dated" addresses) and provide
tools to adjust the configuration for specific issues.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Josh Huber]

David Masterson <dmaster@synopsys.com> writes:

> All complex systems suffer these problems for a time.  Then someone
> figures out how to improve the configuration process and the
> problems die down.  Eventually, Linux distributions could
> automatically setup a basic TMDA environment (ie. purely "dated"
> addresses) and provide tools to adjust the configuration for
> specific issues.

Right, even if you don't request confirmation on your usual bare
address, using dated addresses which only ask for confirmation when
they have expired would be a huge win for posting on lists and usenet.

ttyl,

-- 
Josh Huber

Re: new spam functionality added

[Scott A Crosby]

On Thu, 01 Aug 2002 15:33:52 -0400, Josh Huber <huber+dated+1028662344.a3e4f6@alum.wpi.edu> writes:

> David Masterson <dmaster@synopsys.com> writes:
> 
> > All complex systems suffer these problems for a time.  Then someone
> > figures out how to improve the configuration process and the
> > problems die down.  Eventually, Linux distributions could
> > automatically setup a basic TMDA environment (ie. purely "dated"
> > addresses) and provide tools to adjust the configuration for
> > specific issues.
> 
> Right, even if you don't request confirmation on your usual bare
> address, using dated addresses which only ask for confirmation when
> they have expired would be a huge win for posting on lists and usenet.
> 

For mailing lists and usenet. However, if I want a a bunch of 'date
limited' email addresses, I just subscribe to hundreds of mailing
lists and use those addresses for a week.

And, web pages can't really support 'date-limited' addresses. Nor can
resume's. Nor do address books. (Hey Cynbe, you said you were working
on XXX and didn't know how to do YYY, well, my friend at CMU at
$UNUSABLE_ADDRESS knows how to use YYY.)

So, little benefit, and a lot of hassle put upon everyone else. 

Scott

Re: new spam functionality added

[Paul Jarc]

Scott A Crosby <scrosby@cs.rice.edu> wrote:
> And, web pages can't really support 'date-limited' addresses. Nor can
> resume's. Nor do address books.

How do you mean?

> (Hey Cynbe, you said you were working on XXX and didn't know how to
> do YYY, well, my friend at CMU at $UNUSABLE_ADDRESS knows how to use
> YYY.)

It isn't unusable; it just takes more effort to use.


paul

Re: new spam functionality added

[Jack Twilley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>>>>> "Scott" == Scott A Crosby <scrosby@cs.rice.edu> writes:

Scott> And, web pages can't really support 'date-limited'
Scott> addresses. Nor can resume's. Nor do address books.

>>>>> "Paul" == Paul Jarc <prj@po.cwru.edu> writes:

Paul> How do you mean?

Here's an example bbdb entry:

Josh Huber
            net: huber+dated+1028662344.a3e4f6@alum.wpi.edu, huber+dated+1028589381.274799@alum.wpi.edu, huber+dated+1028587986.de2d4e@alum.wpi.edu, huber+dated+1028584847.72d15f@alum.wpi.edu, huber+dated+1028582805.6306a1@alum.wpi.edu, huber+dated+1028584680.506e13@alum.wpi.edu, huber+dated+1028581649.ce9fdf@alum.wpi.edu, huber+dated+1028581168.a22834@alum.wpi.edu, huber+dated+1028579883.ab0915@alum.wpi.edu, huber+dated+1028578998.eb7ca9@alum.wpi.edu, huber+dated+1028397034.e3bd13@alum.wpi.edu, huber+dated+1028396536.89ee0c@alum.wpi.edu, huber+dated+1028149268.fd896c@alum.wpi.edu, huber+dated+1028146621.950d1c@alum.wpi.edu, huber+dated+1027973678.42aed0@alum.wpi.edu, huber+dated+1027963120.b29c3b@alum.wpi.edu, huber+dated+1027880117.821a32@alum.wpi.edu, huber+dated+1027879739.c41f0b@alum.wpi.edu, huber+dated+1027792404.fd7d58@alum.wpi.edu, huber+dated+1027627644.0404c4@alum.wpi.edu, huber@alum.wpi.edu, huber@mclx.com
    last-access: 2002-03-28
     newsgroups: comp.mail.mutt
                 gnu.emacs.gnus
                 gnus.ding

Jack.
(I think this is what he means.)
- -- 
Jack Twilley
jmt at twilley dot org
http colon slash slash www dot twilley dot org slash tilde jmt slash
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (FreeBSD)

iD8DBQE9SbOrGPFSfAB/ezgRAsXwAKC7EhSgYVRtX0t6zysMALut7m3AUwCg1eHD
12nB+gwpVfthyVBzoUmkysQ=
=75CN
-----END PGP SIGNATURE-----

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Jack Twilley <jmt+usenet@twilley.org> wrote:
> Josh Huber
>             net: huber+dated+1028662344.a3e4f6@alum.wpi.edu, huber+dated+1028589381.274799@alum.wpi.edu,[...]

So one technology will adapt to a new one.  There's no reason address
books *can't* deal with dated addresses; they just happen not to at
the moment.  "We were here first" is not an effective argument.


paul

Re: TMDA (was: new spam functionality added)

[Scott A Crosby]

On Thu, 01 Aug 2002 18:23:29 -0400, prj@po.cwru.edu (Paul Jarc) writes:

> Jack Twilley <jmt+usenet@twilley.org> wrote:
> > Josh Huber
> >             net: huber+dated+1028662344.a3e4f6@alum.wpi.edu, huber+dated+1028589381.274799@alum.wpi.edu,[...]
> 
> So one technology will adapt to a new one.  There's no reason address
> books *can't* deal with dated addresses; they just happen not to at
> the moment.  "We were here first" is not an effective argument.

How does one identify what noisy suffixes are and are not supposed to
be stripped off? Contrasting above:

   huber+dated+1028662344.a3e4f6@alum.wpi.edu
   jmt+usenet@twilley.org

Not to mention the dozen other TMDA-like schemes.

Do what you want.. I'll just score TMDA users appropriately.

Scott

Re: TMDA (was: new spam functionality added)

[Josh Huber]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> How does one identify what noisy suffixes are and are not supposed
> to be stripped off? Contrasting above:
>
>    huber+dated+1028662344.a3e4f6@alum.wpi.edu
>    jmt+usenet@twilley.org

Well, in both cases why bother storing the extention part of the
address?  Perhaps you might like:

tmda-normalize-address from tmda.el?

http://cvs.sf.net/cgi-bin/viewcvs.cgi/~checkout~/tmda/tmda/contrib/tmda.el

Works for me.

> Do what you want.. I'll just score TMDA users appropriately.

Don't you think you're overreacting a little bit?  You just come of
like an ass with comments like that.  But, obviously, do what *you*
want to.

-- 
Josh Huber

Re: TMDA (was: new spam functionality added)

[Scott A Crosby]

On Thu, 01 Aug 2002 19:29:55 -0400, Josh Huber <huber+dated+1028676430.c7f341@alum.wpi.edu> writes:

> Scott A Crosby <scrosby@cs.rice.edu> writes:
> 
> > How does one identify what noisy suffixes are and are not supposed
> > to be stripped off? Contrasting above:
> >
> >    huber+dated+1028662344.a3e4f6@alum.wpi.edu
> >    jmt+usenet@twilley.org
> 
> Well, in both cases why bother storing the extention part of the
> address?  Perhaps you might like:
> 

Because I don't know what address they may want.. I use a variant of
split-fancy-with-parent[1] to put messages into folders, others may do it
by having modulated From: addresses to go with different projects/topics.

> Don't you think you're overreacting a little bit?  You just come of
> like an ass with comments like that. 

Perhaps.. Perhaps not.. But yes, that was rude to state, my apologies.

Scott



[1]
(defun my-split-fancy-with-parent ()
  "Do a split-with-parent, however, ignore the result if it wants to put it in sent-mail. This way, we put followups in the same group, however, we never put followups into sent-mail."

  (let ((my-split (nnmail-split-fancy-with-parent)))
    ;;(message "Doing-my-split-fancy-with-parent")
    ;;(message my-split)
    (if (or (null my-split) (string-match "sent-mail" my-split))
	nil
      my-split)))

Re: new spam functionality added

[Ted Zlatanov]

On 01 Aug 2002, dmaster@synopsys.com wrote:
> All complex systems suffer these problems for a time.  Then someone
> figures out how to improve the configuration process and the
> problems die down.  Eventually, Linux distributions could
> automatically setup a basic TMDA environment (ie. purely "dated"
> addresses) and provide tools to adjust the configuration for
> specific issues.

Can you and anyone else interested in TMDA follow up separately on an
appropriate mailing list, or at least change the subject line and
references?  I'm pretty sure the discussion has nothing to do anymore
with my original message about spam.el and dns.el.

Getting back to topic, if someone could check why query-dns in dns.el
is not working, I'd appreciate it.  There doesn't seem to be an owner
for the library, since no one has responded to my original inquiry.

Thanks
Ted

Re: new spam functionality added

[Paul Jarc]

Ted Zlatanov <tzz@lifelogs.com> wrote:
> Getting back to topic, if someone could check why query-dns in dns.el
> is not working, I'd appreciate it.  There doesn't seem to be an owner
> for the library, since no one has responded to my original inquiry.

Lars seems to have done the most work on it.


paul

Re: new spam functionality added

[Simon Josefsson]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Getting back to topic, if someone could check why query-dns in dns.el
> is not working, I'd appreciate it.  There doesn't seem to be an owner
> for the library, since no one has responded to my original inquiry.

What does "not working" mean?  Limited testing seems to work:

(query-dns "www.extundo.com" 'A)
"217.13.230.178"

Re: new spam functionality added

[Ted Zlatanov]

On Thu, 01 Aug 2002, jas@extundo.com wrote:
> What does "not working" mean?  Limited testing seems to work:
> 
> (query-dns "www.extundo.com" 'A)
> "217.13.230.178"

It took me a while to find the problem, because it only showed up on
the first invocation of query-dns, and only sporadically at that.  You
should have spam.el loaded.

The symptom: grab a known relay from
http://spamcop.net/w3m?action=inprogress&type=relay

Open a fresh Emacs.  Load spam.el.

Add the relay to the Received: header of a message (I wrote the
message buffer to a file, I don't think that made a difference).

Invoke spam-check-blackholes with that message or file buffer active.

You will often get this error:

Debugger entered--Lisp error: (args-out-of-range 1 3)
  query-dns("100.191.110.198.dev.null.dk")
  spam-check-blackholes()
  eval((spam-check-blackholes))
  eval-expression((spam-check-blackholes) nil)
  call-interactively(eval-expression)

The second and further times query-dns is invoked, it works every
time.  I'm not sure if this is a problem with my personal DNS setup,
with the spam.el invocation of query-dns, or with dns.el itself.

Thanks
Ted

Re: new spam functionality added

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

>>>>> In <m33ct6pcm9.fsf@heechee.beld.net>
>>>>>	Ted Zlatanov <tzz@lifelogs.com> wrote:

> Debugger entered--Lisp error: (args-out-of-range 1 3)
>   query-dns("100.191.110.198.dev.null.dk")

I don't know what has occurred, but does the following patch
help?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 364 bytes --]

--- dns.el~	2002-04-30 02:01:11 +0000
+++ dns.el	2002-08-23 02:40:55 +0000
@@ -328,7 +328,7 @@
 	(delete-process process))
       (when tcp-p
 	(goto-char (point-min))
-	(delete-region (point) (+ (point) 2)))
+	(delete-region (point) (min (+ (point) 2) (point-max))))
       (unless (zerop (buffer-size))
 	(let ((result (dns-read (buffer-string))))
 	  (if fullp

Re: new spam functionality added

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

On Fri, 23 Aug 2002, yamaoka@jpl.org wrote:
> 
>>>>>> In <m33ct6pcm9.fsf@heechee.beld.net>
>>>>>>	Ted Zlatanov <tzz@lifelogs.com> wrote:
> 
>> Debugger entered--Lisp error: (args-out-of-range 1 3)
>>   query-dns("100.191.110.198.dev.null.dk")
> 
> I don't know what has occurred, but does the following patch
> help?

No, I got the same error for a different IP after patching dns.el.

The first time I run spam-check-blackholes:

Debugger entered--Lisp error: (args-out-of-range 1 3)
  query-dns("101.233.166.195.relays.ordb.org")
  spam-check-blackholes()
  eval((spam-check-blackholes))
  eval-expression((spam-check-blackholes) nil)
  call-interactively(eval-expression)

The second time it works fine.

I'm attaching the message I used - just put it in a buffer, load
spam.el, and run spam-check-blackholes.

Thanks
Ted


[-- Attachment #2: spam message sample --]
[-- Type: text/plain, Size: 5108 bytes --]

Return-Path: <bug-cfengine-admin@gnu.org>
Delivered-To: lifelogs-lifelogs:com-tzz@lifelogs.com
X-Envelope-To: tzz@lifelogs.com
Received: (qmail 68524 invoked by uid 3080); 3 Aug 2002 13:47:26 -0000
Delivered-To: lifelogs-lifelogs:com-tzz-lists@lifelogs.com
Received: (qmail 68521 invoked from network); 3 Aug 2002 13:47:25 -0000
Received: from fencepost.gnu.org (199.232.76.164)
  by saghotta.pair.com with SMTP; 3 Aug 2002 13:47:25 -0000
Received: from localhost ([127.0.0.1] helo=fencepost.gnu.org)
	by fencepost.gnu.org with esmtp (Exim 3.35 #1 (Debian))
	id 17azFT-0000nF-00; Sat, 03 Aug 2002 09:47:15 -0400
Received: from [195.166.233.101] (helo=mail.com)
	by fencepost.gnu.org with smtp (Exim 3.35 #1 (Debian))
	id 17azEd-0000il-00
	for <bug-cfengine@prep.ai.mit.edu>; Sat, 03 Aug 2002 09:46:24 -0400
From: "kailash" <kailashh@mail.com>
To: <bug-cfengine@prep.ai.mit.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Reply-To: "kailash" <kailashh@mail.com>
X-Priority: 1 (Highest)
Content-Transfer-Encoding: 8bit
Message-Id: <E17azEd-0000il-00@fencepost.gnu.org>
Subject: (no subject)
Sender: bug-cfengine-admin@gnu.org
Errors-To: bug-cfengine-admin@gnu.org
X-BeenThere: bug-cfengine@gnu.org
X-Mailman-Version: 2.0.11
Precedence: bulk
List-Help: <mailto:bug-cfengine-request@gnu.org?subject=help>
List-Post: <mailto:bug-cfengine@gnu.org>
List-Subscribe: <http://mail.gnu.org/mailman/listinfo/bug-cfengine>,
	<mailto:bug-cfengine-request@gnu.org?subject=subscribe>
List-Id: Bug reports for GNU cfengine <bug-cfengine.gnu.org>
List-Unsubscribe: <http://mail.gnu.org/mailman/listinfo/bug-cfengine>,
	<mailto:bug-cfengine-request@gnu.org?subject=unsubscribe>
List-Archive: <http://mail.gnu.org/pipermail/bug-cfengine/>
Date: Sat, 3 Aug 2002 14:45:58 -0700



CONFIDENTIAL U.K. FAX NUMBER: 44-870-1309342

Dear Sir,

Pardon me for starting this great proposal this way as it might surprise you. My name is Yuvraj
Kailash a native of Nepal and attorney to the late king of Nepal who died as a result of loss in
temper caused by an argument between him and his family, as they were against him marrying his
fiancée .You were recommended by a close confidant. Who assured us of your capability and
reliability to assist us. We seek the assistance of someone who is genuinely interested in
entering into a business relationship with long term focus, and willingness of a kind heart to
help. 

If you are conversant with events in Nepal in May/June last year 2001, (June 1 2001), the media
was filled with reports that Crown Prince Dipendra of Nepal had shot and killed several members of
the Nepalese Royal Family, among whom were his parents King Birenda and Queen Aiswarya, before
trying to kill himself.

Prior to the late kings death (Prince Dipendra), he kept aside a secret amount of money, for his
fiancée (Miss Divyani Rana) worth over  US$30,000,000.00(Thirty million United States Dollars).
This was made in the heat of misunderstanding between him and members of his family. The money was
kept away in such a way to prevent any member of his family from having any access or trace to the
funds, which was just a part of his wealth, this was like a gift to his fiancée and a further
affirmation to her that he was surly going to marry her, even though, his family was against their
marriage. We have kept this a secret for sometime now and feel this is the right time to execute
this transaction.

The death of my late boss, left us stranded, we have therefore been searching for a genuine and
reliable person or company of trust and with whom the Nepalese Royal Family have never had any
previous, personal or business relationship with. That person will assist in the transfer and
business re-investment of this money. We cannot do it alone due to our present social status. We
also intend to give you a reasonable share of the funds for your assistance. These funds have been
hidden away, known only to his fiancée and myself. We would want you to safeguard this fund from
being clamped down by any body or agencies.

As soon as we receive your letter of acceptance/acknowledgement, I shall give you more highlights
on this transaction.

THIS IS A RISK FREE TRANSACTION AND MUST NOT BE REVILED TO ANYONE NOT EVEN YOUR LAWYER, ATTORNEY,
FAMILY, FRIENDS ETC. If you are capable and trust worthy and can keep to our instructions to make
the above Proposal a reality.
Then CONTACT US IMMEDIATELY ONLY THROUGH OUR CONFIDENTIAL U.K FAX NUMBER: 44-870-1309342 AS
INDICATED ABOVE 

DO NOT REPLY US BY EMAIL. Please indicate in your reply your PERSONAL/CONFIDENTIAL TELEPHONE
NUMBERS, CELLULAR OR MOBILE NUMBER, CONFIDENTIAL FAX NUMBER AND YOUR CONFIDENTIAL EMAIL ADDRESS
when replying.

ONCE AGAIN, PLEASE DO NOT REPLY BY EMAIL. Your  Reply Should be by Fax only to 44-870-1309342 
(United kingdom)



GOD BLESS YOU.

Yuvraj Kailash




_______________________________________________
Bug-cfengine mailing list
Bug-cfengine@gnu.org
http://mail.gnu.org/mailman/listinfo/bug-cfengine

Re: new spam functionality added

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Open a fresh Emacs.  Load spam.el.
>
> Add the relay to the Received: header of a message (I wrote the
> message buffer to a file, I don't think that made a difference).
>
> Invoke spam-check-blackholes with that message or file buffer active.
>
> You will often get this error:
>
> Debugger entered--Lisp error: (args-out-of-range 1 3)
>   query-dns("100.191.110.198.dev.null.dk")

Remove dns.elc (so that an uncompiled dns.el will be loaded), and
repeat.  That should give you a better backtrace.  I can't really
imagine why it fails the first time...  *imagines a bit*  Well, ok,
it might be timing out the first time, but there's code to protect
against that...

Anyway, the new backtrace should help.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new spam functionality added

[Ted Zlatanov]

OK, let's start with the basics.  dns.elc is removed, everything is
coming out of dns.el which is from the latest CVS Gnus.  I did
spam-check-blackholes on your message, and query-dns was the origin of
the trouble.

(query-dns "244.224.91.80.bl.spamcop.net")

produces:

Debugger entered--Lisp error: (file-error "connection failed" "connection refused" "192.168.2.1" "dns")
  open-network-stream("dns" #<buffer  *temp*> "192.168.2.1" "domain")
  (if (fboundp (quote make-network-process)) (make-network-process :name "dns" :coding (quote binary) :buffer (current-buffer) :host server :service "domain" :type (quote datagram)) (open-network-stream "dns" (current-buffer) server "domain"))
  (let ((server ...) (coding-system-for-read ...) (coding-system-for-write ...)) (if (fboundp ...) (make-network-process :name "dns" :coding ... :buffer ... :host server :service "domain" :type ...) (open-network-stream "dns" ... server "domain")))
  (dns-make-network-process (car dns-servers))
  (let ((process ...) (tcp-p ...) (step 100) (times ...) (id ...)) (process-send-string process (dns-write ... tcp-p)) (while (and ... ...) (accept-process-output process 0 step) (decf times step)) (ignore-errors (delete-process process)) (when tcp-p (goto-char ...) (delete-region ... ...)) (unless (zerop ...) (let ... ...)))
  (save-current-buffer (set-buffer temp-buffer) (let (... ... ... ... ...) (process-send-string process ...) (while ... ... ...) (ignore-errors ...) (when tcp-p ... ...) (unless ... ...)))
  (with-current-buffer temp-buffer (let (... ... ... ... ...) (process-send-string process ...) (while ... ... ...) (ignore-errors ...) (when tcp-p ... ...) (unless ... ...)))
  (unwind-protect (with-current-buffer temp-buffer (let ... ... ... ... ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (let ((temp-buffer ...)) (unwind-protect (with-current-buffer temp-buffer ...) (and ... ...)))
  (with-temp-buffer (let (... ... ... ... ...) (process-send-string process ...) (while ... ... ...) (ignore-errors ...) (when tcp-p ... ...) (unless ... ...)))
  (let (default-enable-multibyte-characters) (with-temp-buffer (let ... ... ... ... ... ...)))
  (mm-with-unibyte-buffer (let (... ... ... ... ...) (process-send-string process ...) (while ... ... ...) (ignore-errors ...) (when tcp-p ... ...) (unless ... ...)))
  query-dns("244.224.91.80.bl.spamcop.net")
  eval((query-dns "244.224.91.80.bl.spamcop.net"))
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp)

I'm behind a firewall, that's why my DNS server is 192.168.2.1
locally.  It works from the command line:

tzz@heechee> dig 244.224.91.80.bl.spamcop.net

; <<>> DiG 9.2.1 <<>> 244.224.91.80.bl.spamcop.net
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 53479
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;244.224.91.80.bl.spamcop.net.  IN      A

;; Query time: 50 msec
;; SERVER: 192.168.2.1#53(192.168.2.1)
;; WHEN: Sun Dec 29 21:21:30 2002
;; MSG SIZE  rcvd: 46

tzz@heechee> host 244.224.91.80.bl.spamcop.net
Host 244.224.91.80.bl.spamcop.net not found: 3(NXDOMAIN)

I think I'm doing this correctly.  Emacs 21.2.1, latest CVS Gnus.  I
have very little knowledge of the Emacs make-network-process calls and
socket handling so I couldn't debug further, sorry.  The arguments to
make-network-process look reasonable, though.  Could it be using TCP
instead of UDP for some strange reason?

Ted

Re: new spam functionality added

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Debugger entered--Lisp error: (file-error "connection failed" "connection refused" "192.168.2.1" "dns")
>   open-network-stream("dns" #<buffer  *temp*> "192.168.2.1" "domain")

Right.  I've now added extra error handling code around this call.

> I'm behind a firewall, that's why my DNS server is 192.168.2.1
> locally.

But it's strange that it fails the first time, but not the second
time.  

> Could it be using TCP instead of UDP for some strange reason?

Yes, it's using TCP for you.  Newer versions of Emacs support UDP
network connections, but dns.el will fall back on TCP connections if
you have an older Emacs.

If the following returns non-nil, you have UDP support:

(fboundp 'make-network-process)

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new spam functionality added

[Ted Zlatanov]

On Mon, 30 Dec 2002, larsi@gnus.org wrote:
>> I'm behind a firewall, that's why my DNS server is 192.168.2.1
>> locally.
> 
> But it's strange that it fails the first time, but not the second
> time.  

I can't verify if that problem is gone, since I can't do regular DNS
queries to begin with.

>> Could it be using TCP instead of UDP for some strange reason?
> 
> Yes, it's using TCP for you.  Newer versions of Emacs support UDP
> network connections, but dns.el will fall back on TCP connections if
> you have an older Emacs.
> 
> If the following returns non-nil, you have UDP support:
> 
> (fboundp 'make-network-process)

It returns nil.  I'm using Emacs 21.2, which is the latest non-pretest
version available from the Emacs web site.  Can we do something for
the people without a pretest version, such as optionally invoking the
nslookup/host/dig programs?  It seems like a reasonable compromise in
dns.el for those without UDP support.

If that's not possible, maybe spam.el is the place to add that kludge?

Ted

Re: new spam functionality added

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> It returns nil.  I'm using Emacs 21.2, which is the latest non-pretest
> version available from the Emacs web site.  Can we do something for
> the people without a pretest version, such as optionally invoking the
> nslookup/host/dig programs?

Well, you could open the firewall for TCP dns connections.  :-)

> It seems like a reasonable compromise in dns.el for those without
> UDP support.

I think that would be inappropriate for dns.el.

> If that's not possible, maybe spam.el is the place to add that kludge?

Sure.  There's a dig.el (in the Gnus distribution) that does much the
same as dns.el, although with a different interface.  spam.el could
have variables you could set to say whether to use dns.el or dig.el.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new spam functionality added

[Ted Zlatanov]

On Mon, 30 Dec 2002, larsi@gnus.org wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
> 
>> It returns nil.  I'm using Emacs 21.2, which is the latest
>> non-pretest version available from the Emacs web site.  Can we do
>> something for the people without a pretest version, such as
>> optionally invoking the nslookup/host/dig programs?
> 
> Well, you could open the firewall for TCP dns connections.  :-)

I was under the impression that I didn't need to accept inbound
connections for TCP DNS lookups.  I guess I do - you learn something
new every day...

>> If that's not possible, maybe spam.el is the place to add that
>> kludge?
> 
> Sure.  There's a dig.el (in the Gnus distribution) that does much
> the same as dns.el, although with a different interface.  spam.el
> could have variables you could set to say whether to use dns.el or
> dig.el.

Cool, I can do:

(dig-invoke "244.224.91.80.bl.spamcop.net")
[switch to the *dig output* buffer]
check if (dig-extract-rr "244.224.91.80.bl.spamcop.net") is nil

There doesn't seem to be a way to do all of these in one shot in
dig.el; should I write the corresponding code in spam.el or should
dig.el be extended to have a dig-dns-query function analogous to
dns-query in dns.el?

Ted

Re: new spam functionality added

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> There doesn't seem to be a way to do all of these in one shot in
> dig.el; should I write the corresponding code in spam.el or should
> dig.el be extended to have a dig-dns-query function analogous to
> dns-query in dns.el?

I think the latter would be best.  If the interface to this
dig-dns-query (or perhaps just `dig-query') was the same as
`dns-query', that would be nice -- the user could then just switch
around these functions at will.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new spam functionality added

[Ted Zlatanov]

On Mon, 30 Dec 2002, larsi@gnus.org wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
> 
>> There doesn't seem to be a way to do all of these in one shot in
>> dig.el; should I write the corresponding code in spam.el or should
>> dig.el be extended to have a dig-dns-query function analogous to
>> dns-query in dns.el?
> 
> I think the latter would be best.  If the interface to this
> dig-dns-query (or perhaps just `dig-query') was the same as
> `dns-query', that would be nice -- the user could then just switch
> around these functions at will.

I still can't use dns.el with my firewall, I suspect it simply does
not like TCP DNS queries.  Strange.

I only need to check whether the following looks OK to you before
adding it to dig.el:

(defun dig-query (domain &optional
		   query-type query-class query-option dig-option server)
  "Query addresses of a DOMAIN using dig, by calling `dig-invoke' and `dig-extract-rr'.
Optional arguments are passed to `dig-invoke' and `dig-extract-rr'.  Returns nil for a nonexistent domain."
(let ((buffer (dig-invoke domain query-type query-class query-option dig-option server)))
  (when buffer
      (switch-to-buffer buffer)
      (setq digger (dig-extract-rr domain query-type query-class))
      (kill-buffer buffer)
      digger)))

It seems to work just fine here for existent and nonexistent domains.
I don't want to do the extra work of converting the DIG response into
the query-dns format, it's fine as a string for the purposes of
spam.el which only cares if it's nil or not.

Ted

Re: new spam functionality added

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I only need to check whether the following looks OK to you before
> adding it to dig.el:

Yup; looks OK to me...

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

spam.el now supports blackholes by default

[Ted Zlatanov]

I added the spam-use-dig variable (t by default) that tells spam.el to
use dig.el instead of internal DNS checks.

dig.el got the query-dig function, which will actually check an IP
address.

spam-check-blackholes was extended to support the spam-use-dig
variable.

I tested pretty extensively and found no problems with the new
functionality.  I enabled the blackhole check by default in spam.el
(spam-use-blackholes).  Does anyone think it should be off by default?

I also updated the manual with the information on blackholes.

Let me know if you find bugs or documentation errors.

Ted

Re: spam.el now supports blackholes by default

[Matt Armstrong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I enabled the blackhole check by default in spam.el
> (spam-use-blackholes).  Does anyone think it should be off by
> default?

Yes.  blackhole lists seem to die relatively frequently when compared
to the useful lifetime of a stable emacs distribution.  This makes it
likely that the list would be out of date soon after emacs shipped.
There is a precedent for a defunct list being changed to always return
"yes" to all SPAM queries (to try to get people to quit using it),
which would compound the problem.

The result would be that everybody would have to change the default to
get things to work (bad), rather than changing from the default to get
extra features (good).

E.g. default spam-blackhole-servers has rbl.maps.vix.com which is
defunct (according to http://mail-abuse.org/rbl/usage.html).

Re: spam.el now supports blackholes by default

[Ted Zlatanov]

On Mon, 30 Dec 2002, matt@lickey.com wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
> 
>> I enabled the blackhole check by default in spam.el
>> (spam-use-blackholes).  Does anyone think it should be off by
>> default?
> 
> Yes.  blackhole lists seem to die relatively frequently when
> compared to the useful lifetime of a stable emacs distribution.
> This makes it likely that the list would be out of date soon after
> emacs shipped.  

> E.g. default spam-blackhole-servers has rbl.maps.vix.com which is
> defunct (according to http://mail-abuse.org/rbl/usage.html).

Fixed in spam.el, thanks.  That's something we can't control, so I
guess we just have to keep spam-blackhole-servers updated as the RBL
list changes.

> The result would be that everybody would have to change the default
> to get things to work (bad), rather than changing from the default
> to get extra features (good).

I think enabling spam.el is already changing from the default (since
you add spam-split to your split rules).  Users may expect something
as useful as blackhole checks enabled by default.  But it's simple
enough to set spam-use-blackholes to t, so I'll set it to disabled.

I updated the manual as well.

> There is a precedent for a defunct list being changed to always
> return "yes" to all SPAM queries (to try to get people to quit using
> it), which would compound the problem.

That seems like an easy choice for administrators, but a bad one for
the users.  Oh well.

Ted

spam.el proposed group parameters

[Ted Zlatanov]

I'd like to add the following group parameters to gnus.el, but I have
very little knowledge of the accepted practices for group parameters.

Also, I'll have to define a 'spam' variable group and move all the
existing variables to it in spam.el, so ignore the "variable-group
nnmail-expire" piece for now.

The idea is that these group parameters will work in parallel with the
regular spam.el variables.  But I'm not sure if I should let the
system-wide variables override the group parameters or vice versa.

For instance, say a user enabled spam-process-with-ifile for a group,
but also has spam-use-ifile set to nil.  What's the accepted Gnus
behavior in this case?  Am I approaching the problem the wrong way?
Should I eliminate the system-wide variables and only use group
parameters?

The existing examples of gnus-define-group-parameter didn't show the
code I needed, and I'm not sure how to use that function.  I would
like to use choices instead of simple booleans, so for instance
spam-process-with-ifile and spam-process-with-bogofilter would just be
two choices in the list spam-process-backends, and the group type
would be 'ham or 'spam.  That would be better than booleans, I think.
So how do I use gnus-define-group-parameter to define a list of
symbolic tags, and then later in other code check whether a tag is in
the current list?  Any help is greatly appreciated.

Thanks
Ted

------------------------------------------------------------------------

;; group parameters for spam processing added by Ted Zlatanov <tzz@lifelogs.com>
(gnus-define-group-parameter
 spam
 :type bool
 :function gnus-group-spam-p
 :function-document
 "Check whether GROUP is designated as a spam group or not."
 :variable gnus-spam-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically mark new articles as spam on
summary entry.  If non-nil, this should be a regexp that should match
all groups in which to do automatic spam tagging.  This only makes
sense for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Spam" t)
 :parameter-document
 "All articles that are unread will be marked as spam on summary entry.")

(gnus-define-group-parameter
 ham
 :type bool
 :function gnus-group-ham-p
 :function-document
 "Check whether GROUP is designated as a ham (non-spam) group or not."
 :variable gnus-ham-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process new articles as ham
(non-spam) on summary exit.  If non-nil, this should be a regexp that
should match all groups in which to do automatic ham tagging.  This
only makes sense for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Not Spam" t)
 :parameter-document
 "All articles that are not marked as spam will be processed as such at summary exit.")

(gnus-define-group-parameter
 spam-process
 :type bool
 :function gnus-group-spam-process-p
 :function-document
 "Check whether GROUP articles will be processed as spam at summary exit."
 :variable gnus-spam-process-newsgroups
 :variable-default t
 :variable-document
 "*Groups in which to automatically process spam articles on
summary exit.  If non-nil, this should be a regexp that should match
all groups in which to do automatic spam processing.  This only makes
sense for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Spam" t)
 :parameter-document
 "Articles that are marked as spam will be processed on summary exit.")

(gnus-define-group-parameter
 spam-process-with-ifile
 :type bool
 :function gnus-group-spam-process-with-ifile-p
 :function-document
 "Check whether GROUP articles will be processed as spam with ifile at summary exit."
 :variable gnus-spam-process-with-ifile-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process spam articles with ifile
on summary exit.  If non-nil, this should be a regexp that should
match all groups in which to do automatic spam processing with ifile.
This only makes sense for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Spam With Ifile" t)
 :parameter-document
 "Articles that are marked as spam will be processed with ifile on summary exit.")

(gnus-define-group-parameter
 spam-process-with-bogofilter
 :type bool
 :function gnus-group-spam-process-with-bogofilter-p
 :function-document
 "Check whether GROUP articles will be processed as spam with bogofilter at summary exit."
 :variable gnus-spam-process-with-bogofilter-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process spam articles with
bogofilter on summary exit.  If non-nil, this should be a regexp that
should match all groups in which to do automatic spam processing with
bogofilter.  This only makes sense for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Spam With Bogofilter" t)
 :parameter-document
 "Articles that are marked as spam will be processed with bogofilter on summary exit.")

(gnus-define-group-parameter
 spam-process-with-blacklist
 :type bool
 :function gnus-group-spam-process-with-blacklist-p
 :function-document
 "Check whether GROUP articles will be processed as spam, adding senders to the blacklist at summary exit."
 :variable gnus-spam-process-with-blacklist-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process spam articles, adding
senders to the blacklist on summary exit.  If non-nil, this should be
a regexp that should match all groups in which to do automatic spam
processing with the blacklist.  This only makes sense for mail
groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Spam With Blacklist" t)
 :parameter-document
 "Articles that are marked as spam will be processed with blacklist on summary exit.")

(gnus-define-group-parameter
 spam-process-ham-with-whitelist
 :type bool
 :function gnus-group-spam-process-ham-with-whitelist-p
 :function-document
 "Check whether GROUP articles will be processed as ham, adding senders to the whitelist at summary exit."
 :variable gnus-spam-process-ham-with-whitelist-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process ham (non-spam) articles,
adding senders to the whitelist on summary exit.  If non-nil, this
should be a regexp that should match all groups in which to do
automatic ham processing with the whitelist.  This only makes sense
for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Ham With Whitelist" t)
 :parameter-document
 "Ham (non-spam) articles will be processed with whitelist on summary exit.")

(gnus-define-group-parameter
 spam-process-ham-with-BBDB
 :type bool
 :function gnus-group-spam-process-ham-with-BBDB-p
 :function-document
 "Check whether GROUP articles will be processed as ham, adding senders to the BBDB at summary exit."
 :variable gnus-spam-process-ham-with-BBDB-newsgroups
 :variable-default nil
 :variable-document
 "*Groups in which to automatically process ham (non-spam) articles,
adding senders to the BBDB on summary exit.  If non-nil, this
should be a regexp that should match all groups in which to do
automatic ham processing with the BBDB.  This only makes sense
for mail groups."
 :variable-group nnmail-expire
 :variable-type '(choice (const nil)
		 regexp)
 :parameter-type '(const :tag "Group Contents Are Processed As Ham With BBDB" t)
 :parameter-document
 "Ham (non-spam) articles will be processed with BBDB on summary exit.")

Re: spam.el proposed group parameters

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> For instance, say a user enabled spam-process-with-ifile for a group,
> but also has spam-use-ifile set to nil.  What's the accepted Gnus
> behavior in this case?

Group parameters take precedence over global variables.

> Am I approaching the problem the wrong way?  Should I eliminate the
> system-wide variables and only use group parameters?

No, having both are fine.

> The existing examples of gnus-define-group-parameter didn't show the
> code I needed, and I'm not sure how to use that function.  I would
> like to use choices instead of simple booleans, so for instance
> spam-process-with-ifile and spam-process-with-bogofilter would just be
> two choices in the list spam-process-backends, and the group type
> would be 'ham or 'spam.  That would be better than booleans, I think.
> So how do I use gnus-define-group-parameter to define a list of
> symbolic tags, and then later in other code check whether a tag is in
> the current list?  Any help is greatly appreciated.

I don't think there's any code in the `gnus-define-group-parameter'
macro for doing that, but extending it to support, say, a `choice' in
addition to the other types should be doable.  After all, the macro
basically defines a `defcustom' (that has support for `choice') and a
function... 

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: spam.el now supports blackholes by default

[luis fernandes]

    matt> Yes.  blackhole lists seem to die relatively frequently
    matt> when compared to the useful lifetime of a stable emacs
    matt> distribution.  This makes it likely that the list would be
    matt> out of date soon after emacs shipped.  There is a precedent
    matt> for a defunct list being changed to always return "yes" to
    matt> all SPAM queries (to try to get people to quit using it),
    matt> which would compound the problem.

Could we host a black-hole list on gnus.org and have Gnus check for
updates (weekley, or monthly) and download new lists (or just the
diffs) if the master list has been updated?

Re: spam.el now supports blackholes by default

[Ted Zlatanov]

On Sun, 5 Jan 2003, elf@ee.ryerson.ca wrote:
> Could we host a black-hole list on gnus.org and have Gnus check for
> updates (weekley, or monthly) and download new lists (or just the
> diffs) if the master list has been updated?

I would be OK with that, as a user preference (it would be an option
when you customize the blackhole servers list to get it from
gnus.org).  The update frequency could also be customized there.

Ted

Re: spam.el now supports blackholes by default

[Lars Magne Ingebrigtsen]

luis fernandes <elf@ee.ryerson.ca> writes:

> Could we host a black-hole list on gnus.org and have Gnus check for
> updates (weekley, or monthly) and download new lists (or just the
> diffs) if the master list has been updated?

A meta-blackhole list?  Sure.  It could be put under CVS so it can be
updated easily, too.

But perhaps this is more of a project suitable for my.gnus.org? 

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: new spam functionality added

[Jason R. Mastaler]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Because its tragedy of the commons, I bitbucket any TMDA user. 

While I don't feel such animosity is warranted, I've heard it before,
and understand where it's coming from.  Unwillingness to accept the
severity of the spam problem, and the fact that the Internet isn't the
place it used to be.  Like those who refuse to lock their front doors
despite the changing climate of their neighborhood, because they never
have before, and don't want to hear what the neighbors are saying.

Unfortunately, boycotting TMDA users will do nothing to help the
problem.  You aren't teaching anyone a lesson with such behavior.

Would it make you feel better if I uninstalled TMDA and went back to
an unsavory e-mail experience?  How about putting your energy into
something productive such as working on a real solution to the problem
instead of hurting those who have found some temporary relief?

I never said TMDA was a solution to spam, it's morphine for the cancer
patient.  It alleviates some of my suffering, while I keep my eye on
the prize, which is a real "cure".

> (and if I start getting too many of em, I'll make a public blacklist
> of em.)

Who is going to respect a "public blacklist" filled with the names of
those whom Scott has philosophical differences with.  Please.

Re: new spam functionality added

[Russ Allbery]

Jason R Mastaler <jason@mastaler.com> writes:

> While I don't feel such animosity is warranted, I've heard it before,
> and understand where it's coming from.  Unwillingness to accept the
> severity of the spam problem, and the fact that the Internet isn't the
> place it used to be.

It's not unwillingness to accept anything.  It's refusal to jump through
other people's hoops so that they can receive my mail.  Spam sucks.  I
deal with it.  Other people can too.  There are a lot of different tactics
that can work.  Making people go to more effort to send you mail isn't one
that I'm willing to support by extending that effort.

> Unfortunately, boycotting TMDA users will do nothing to help the
> problem.  You aren't teaching anyone a lesson with such behavior.

If you use TMDA in a configuration that requires me confirm my mail when I
send it to you, then you simply won't receive mail from me.  I am of the
somewhat egotistical opinion that in general you'll lose more from not
receiving mail from me than I'll lose in not being able to send mail to
you, since I'm more frequently answering questions than asking them.

If you can live with simply cutting off communication with people like me
as acceptable lossage in keeping away the spam, then more power to you.
If you ever want to get mail from me, you'll write to me with a repliable
address; requiring confirmation renders the e-mail address unrepliable as
far as I'm concerned and I will treat it accordingly.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: new spam functionality added

[Jason R. Mastaler]

Russ Allbery <rra@stanford.edu> writes: 
 
> It's refusal to jump through other people's hoops so that they can
> receive my mail.

Do you have the same objections to confirming a mailing list
subscription?  What's the difference?

> Spam sucks.  I deal with it.  Other people can too.

Misery loves company.

> There are a lot of different tactics that can work.

Work, yes, but not nearly well enough.  Ibuprofen won't cut it; I need
morphine.

> If you can live with simply cutting off communication with people
> like me as acceptable lossage in keeping away the spam, then more
> power to you.

No, that's certainly not acceptable.  However, senders who won't
confirm their messages merely out of spite are in the vast minority,
so it's not really a problem.  TMDA has ways around these situations
anyway.

> If you ever want to get mail from me, you'll write to me with a
> repliable address; requiring confirmation renders the e-mail address
> unrepliable as far as I'm concerned and I will treat it accordingly.

Of course.  Have you actually taken a look at TMDA, or are you just
making assumptions here?  With TMDA, I'll always write to you with a
repliable address.  This is one of the reasons another poster in this
thread said that the majority of correspondents won't even notice it.
I totally agree, doing otherwise would be extremely rude.

Re: new spam functionality added

[Russ Allbery]

Jason R Mastaler <jason-exp-1028953774.6ffe92@mastaler.com> writes:
> Russ Allbery <rra@stanford.edu> writes:

>> It's refusal to jump through other people's hoops so that they can
>> receive my mail.

> Do you have the same objections to confirming a mailing list
> subscription?

No.

> What's the difference?

That's a service that I'm requesting, and the confirmation protects *me*
against forged subscriptions to mailing lists.  It is also a one-time cost
to pay for an ongoing stream of mail, whereas with individual
confirmations I have to keep confirming my mail to individuals I only mail
once.

> Of course.  Have you actually taken a look at TMDA, or are you just
> making assumptions here?

I'm not making any assumptions.  My message spelled out in great detail
specifically what I was objecting to.  If you're using TMDA in some other
configuration, then I'm not objecting to that, am I?

I have encountered people with TMDA configured precisely along the lines
that I've previously described.  I don't confirm my mail to them.  It's a
waste of my time, which I consider valuable and refuse to allow be wasted
in that fashion.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Russ Allbery <rra@stanford.edu> wrote:
> I have encountered people with TMDA configured precisely along the lines
> that I've previously described.  I don't confirm my mail to them.  It's a
> waste of my time, which I consider valuable and refuse to allow be wasted
> in that fashion.

If a reliable autoconfirmer were written, would you consider it a
waste of time to set it up for your mailbox?

Jason, you might want to consider adding such a component to TMDA.
Messages I send would get a header field added containing
hash(secret+hash(message)).  When I send a message to you, the TMDA
confirmation request would contain at least hash(message) as computed
by you and hash(secret+hash(message)) as contained in my message, if
not the whole message itself, which I could use to compute
hash(message) if you didn't include it.  Given that, a program
receiving my mail can verify the the hash in the original message was
computed using my secret, and can send a confirmation and drop/reroute
the confirmation request.


paul

Re: TMDA (was: new spam functionality added)

[Russ Allbery]

Paul Jarc <prj@po.cwru.edu> writes:
> Russ Allbery <rra@stanford.edu> wrote:

>> I have encountered people with TMDA configured precisely along the
>> lines that I've previously described.  I don't confirm my mail to them.
>> It's a waste of my time, which I consider valuable and refuse to allow
>> be wasted in that fashion.

> If a reliable autoconfirmer were written, would you consider it a waste
> of time to set it up for your mailbox?

Hm.  Maybe.  But I'd at least think about it.

I think one of the requirements I'd have there would be to be able to
specify somehow that the confirmation messages went to some other address
of my choosing.  That way, I wouldn't have to trust the autoconfirmer with
my regular mail.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Russ Allbery <rra@stanford.edu> wrote:
> I think one of the requirements I'd have there would be to be able to
> specify somehow that the confirmation messages went to some other address
> of my choosing.  That way, I wouldn't have to trust the autoconfirmer with
> my regular mail.

Currently, you can do that by tweaking your envelope sender.  Maybe
TMDA could be made to inspect your message header and use a different
address if one were specified there for its use.


paul

Re: TMDA (was: new spam functionality added)

[Per Abrahamsen]

prj@po.cwru.edu (Paul Jarc) writes:

> If a reliable autoconfirmer were written, would you consider it a
> waste of time to set it up for your mailbox?

If a reliable autoconfirmer were written, some spammers would start
using it.  We need some amount manual intervention for the system to
scale.

But I don't consider the problem with people who will refuse to spend
a second or two to confirm their existence for significant.  I lose
mail today because by filters (automatic and manual) are too
agressive, and I can't afford the time to use less agressive filters:

- Today, I can lose mail from anybody, through no fault of theirs.
  Even if it is something that it is very important to them that I
  see.

- With TMDA, I will only lose mail from people who deliberately decide
  that getting their message through isn't worth two seconds of their
  tine.

It seem much more fair to lose mail from the later category than the
former.

The problem that does frighten me is whether non-technical people will
understand the request for a confirmation message.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Per Abrahamsen <abraham@dina.kvl.dk> wrote:
> If a reliable autoconfirmer were written, some spammers would start
> using it.

In doing so, they would give up their anonymity.  We'd be able to
find them and report them (and blacklist them).


paul

Re: TMDA (was: new spam functionality added)

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Per Abrahamsen <abraham@dina.kvl.dk> wrote:
>> If a reliable autoconfirmer were written, some spammers would start
>> using it.
>
> In doing so, they would give up their anonymity.  We'd be able to
> find them and report them (and blacklist them).

Is this the problem?  In theory I can track down IP's on the net
today, but tracking down spammers this way doesn't seem like a
solution that prevents all spam in practice, as spammers uses hacked
or incorrectly configured machines.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Simon Josefsson writes:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Per Abrahamsen <abraham@dina.kvl.dk> wrote:
>>> If a reliable autoconfirmer were written, some spammers would
>>> start using it.

>> In doing so, they would give up their anonymity.  We'd be able to
>> find them and report them (and blacklist them).

> Is this the problem?  In theory I can track down IP's on the net
> today, but tracking down spammers this way doesn't seem like a
> solution that prevents all spam in practice, as spammers uses hacked
> or incorrectly configured machines.

When the spammer assumes that no one can reply to his message, then
all these things can be spoofed in the email.  All you really know is
the hostname and IP address of the last relayer of the email which is
something that spammers rely upon to maintain their anonymity.  If the
spammer decides to put in an autoconfirmer to get around TMDA, then he
has to supply a good "return address" in his SPAM which now makes him
traceable.  Even if it's only traceable back to a compromised machine,
that machine can be blacklisted and the sysadmin informed to fix it.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* David Masterson <dmaster@synopsys.com>  on Mon, 05 Aug 2002
| something that spammers rely upon to maintain their anonymity.  If the
| spammer decides to put in an autoconfirmer to get around TMDA, then he
| has to supply a good "return address" in his SPAM which now makes him
| traceable.  Even if it's only traceable back to a compromised machine,
| that machine can be blacklisted and the sysadmin informed to fix it.

"winatial9077@hotmail.com" is not what I would call "easilly traceable".  A
spammer only needs it for one round of automatically responding to TMDA
approvals.

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[Russ Allbery]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:
> * David Masterson <dmaster@synopsys.com>  on Mon, 05 Aug 2002
> | something that spammers rely upon to maintain their anonymity.  If the
> | spammer decides to put in an autoconfirmer to get around TMDA, then he
> | has to supply a good "return address" in his SPAM which now makes him
> | traceable.  Even if it's only traceable back to a compromised machine,
> | that machine can be blacklisted and the sysadmin informed to fix it.

> "winatial9077@hotmail.com" is not what I would call "easilly traceable".  A
> spammer only needs it for one round of automatically responding to TMDA
> approvals.

Given the volume of spam runs, if any appreciable number of people use
TMDA that's a heck of a lot of traffic.  Probably enough that Hotmail
would close the mailbox down before they got a chance to confirm all those
messages.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* Russ Allbery <rra@stanford.edu>  on Mon, 05 Aug 2002
| Given the volume of spam runs, if any appreciable number of people use
| TMDA that's a heck of a lot of traffic.  Probably enough that Hotmail
| would close the mailbox down before they got a chance to confirm all
| those messages.

That is a big "if" because "appreciable" cannot be easilly defined.
Hotmail gets a -lot- of traffic.  The Hotmail account that I had (and let
expire) got more spam in a day than my regular mailbox gets legitimate mail
in a day.  That includes mailing lists (Gnus, Newton Talk, and several
others of moderate to high volume).  I am not sure that TMDA confirmations
would ammount to anything significant enough for Hotmail to proactively
shut off the spammer.

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Stainless Steel Rat writes:

> * David Masterson <dmaster@synopsys.com>  on Mon, 05 Aug 2002

> | something that spammers rely upon to maintain their anonymity.  If
> | the spammer decides to put in an autoconfirmer to get around TMDA,
> | then he has to supply a good "return address" in his SPAM which
> | now makes him traceable.  Even if it's only traceable back to a
> | compromised machine, that machine can be blacklisted and the
> | sysadmin informed to fix it.

> "winatial9077@hotmail.com" is not what I would call "easilly
> traceable".  A spammer only needs it for one round of automatically
> responding to TMDA approvals.

Sure, but now you (the receiver of spam) are sure that the email came
from hotmail.com and you have (a list of) email addresses that were
used to do the spamming.  Thru some of the recent legislation (like,
IIRC, the anti-spamming law in California), I think you (or a class
action group) have reasonable cause to sue Hotmail to either:

* find out who the spammer is and sue them -and/or-
* force Hotmail to prevent anonymous account generation

-- 
David Masterson
dsm@rawbw.com

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* David Masterson <dsm@rawbw.com>  on Mon, 05 Aug 2002
| * find out who the spammer is and sue them -and/or-
| * force Hotmail to prevent anonymous account generation

... crushing any chance at confidental access to on-line cancer and rape
support groups, for example.

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Stainless Steel Rat writes:

> * David Masterson <dsm@rawbw.com>  on Mon, 05 Aug 2002
> | * find out who the spammer is and sue them -and/or-
> | * force Hotmail to prevent anonymous account generation

> ... crushing any chance at confidental access to on-line cancer and rape
> support groups, for example.

The needs of the many...

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> David Masterson writes:
>>>>> Stainless Steel Rat writes:
>> * David Masterson <dsm@rawbw.com>  on Mon, 05 Aug 2002
>> | * find out who the spammer is and sue them -and/or-
>> | * force Hotmail to prevent anonymous account generation

>> ... crushing any chance at confidental access to on-line cancer and
>> rape support groups, for example.

> The needs of the many...

Also, if needed, a system could be set up for this purpose that allow
people with these accounts to send limited numbers of email within a
given time period, thus making the system unattractive to spammers.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* David Masterson <dmaster@synopsys.com>  on Tue, 06 Aug 2002
| Also, if needed, a system could be set up for this purpose that allow
| people with these accounts to send limited numbers of email within a
| given time period, thus making the system unattractive to spammers.

I like the idea of just making the system unattractive to spammers without
making it unattractive to Joe Average.

-- 
Rat <ratinox@peorth.gweep.net>    \ Warning: pregnant women, the elderly, and
Minion of Nathan - Nathan says Hi! \ children under 10 should avoid prolonged
PGP Key: at a key server near you!  \ exposure to Happy Fun Ball.
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[Lloyd Zusman]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> * David Masterson <dmaster@synopsys.com>  on Tue, 06 Aug 2002
> | Also, if needed, a system could be set up for this purpose that allow
> | people with these accounts to send limited numbers of email within a
> | given time period, thus making the system unattractive to spammers.
>
> I like the idea of just making the system unattractive to spammers without
> making it unattractive to Joe Average.

Agreed ... and for that reason, I putting my two cents in to support
Hashcash.

And I also support TMDA and others of its ilk ... because in my
experience, very few "Joe Average" type folks that I know have had a
problem with the couple extra keystrokes or mouseclicks that are
necessary to send the initial confirmation message.  In fact, several of
my "Joe Average" correspondents have written back to tell me that after
being introduced to TMDA via my email account, they are off to try to
install it and use it themselves.

In the ideal email world that I'm envisioning today, there would be some
sort of combination of hashcash and confirmation.  Most people I know
would consider this to be a small and quite acceptable price to pay for
spam reduction.

In the case of a the small minority who are dead set against email
confirmation (and yes, in my experience so far, it IS a small minority),
I'd be happy to pre-whitelist those people if I really want to
communicate with them ... many of them are vocal about their opposition
to email confirmations, which makes it easy for me to identify a lot of
them before I communicate.

There will still be the occasional person who wants to initiate contact,
but who is morally opposed to my confirmation system, and who therefore
decides not to follow through once he or she discovers that I'm using
such a system.  Even in that case, I can see the initial email that this
person sent, since I keep a "pending" queue that I review once or twice
a day.  And if I want to respond to that person, TMDA easily allows me
to "release" his or her email and to whitelist the return address.

So I can still respond to that person, who then has the choice as to
whether he or she wants to continue communicating with me, without any
need for confirmation.

I believe that only the most die-hard anti-email-confirmation zealots
would refuse to continue communicating with someone with whom they
originally initiated contact, from whom they received a confirmation
request that they ignored, and from whom they subsequently received a
gracious reply anyway, with an explanation that confirmation is no
longer necessary for them, because they have been whitelisted.  And I
wouldn't lose a whole lot of sleep if an person like that would refuse
to continue the email conversation that he or she originally initiated
with me, and which I voluntarily chose to continue anyway, under these
circumstances.

So for me (and I believe for lots of other "Joe Average" folks), a
TMDA-like system is useful and desirable.

And as I mentioned, that in combination with hashcash would go a long
way towards significantly reducing spam.


-- 
 Lloyd Zusman
 ljz@asfast.com

Hashcash (was: TMDA)

[Lars Magne Ingebrigtsen]

(To sum up my feelings on the issue -- I recognize that other people
feel that they have no other option but to use a challenge-response
service to deal with mail from people they don't know, and that's OK
by me.  I do feel, however, that if I were to do so, I would be
slightly rude, since I haven't gotten to the point of desperation
where spam is concerned.  But people are different.)

Lloyd Zusman <ljz@asfast.com> writes:

> Agreed ... and for that reason, I putting my two cents in to support
> Hashcash.

Which brings me to what I wanted to write about.  I think hashcash is
a brilliant idea, as it requires no centralized functionality, and
people can just switch over to using hashcash when they want, and
other people can choose to recognize hashcash when they want, or
not.  For instance, I could easily see me putting in a mail split
rule that says "if this is a hashcashed message, put it in nnml:misc,
and don't do any further tests for spamminess.

So I want to make Gnus support X-Hashcash.  I see that there's a
hashcash.el in the contrib directory, written by Paul E. Foley.
Would it be possible to assign the copyright for this to the FSF?

There doesn't seem to be any code in there to verify the hashcash
payment.  Does anybody have anything for doing that?

Also, the posting code seems to be synchronous.  I think it would be
nice to put the hashcash generation in the background, and postpone
the sending until the message has been hashcashed.  Has anybody done
any work in that area?  Could, perhaps, the delayed nndrafts code be
used for that?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Hashcash

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> There doesn't seem to be any code in there to verify the hashcash
> payment.  Does anybody have anything for doing that?

I lobbied for this in SpamAssassin, and there are patches available,
but nothing seems to happen.  They seemed to want to wait for a
standard to emerge, which seemed very backwards to what SA is all
about to me, but anyway.

I noticed that Lars updated Gnus' hashcash.el from upstream that has
verify functions, but it seems XEmacs specific, and it didn't work in
incomming mail buffers with RFC 822 body separators.  Patch attached;
Paul do you think it is OK?

> Also, the posting code seems to be synchronous.  I think it would be
> nice to put the hashcash generation in the background, and postpone
> the sending until the message has been hashcashed.  Has anybody done
> any work in that area?  Could, perhaps, the delayed nndrafts code be
> used for that?

Sounds like a good idea.  We'd might as well write a complete elisp
implementation of it, since it justs loop over a SHA-1 computation.

--- hashcash.el.~1.6~	Thu Jan  2 17:52:32 2003
+++ hashcash.el	Thu Jan  2 19:28:44 2003
@@ -37,11 +37,19 @@
   "*The default minimum number of bits to accept on incoming payments."
   :type 'integer)
 
-(defcustom hashcash-accept-resources `((,(user-mail-address) nil))
+(defcustom hashcash-accept-resources `((,(if (fboundp 'user-mail-address)
+					     (user-mail-address)
+					   user-mail-address) . nil))
   "*An association list mapping hashcash resources to payment amounts.
 Resources named here are to be accepted in incoming payments.  If the
 corresponding AMOUNT is NIL, the value of `hashcash-default-accept-payment'
-is used instead.")
+is used instead."
+  :type '(repeat (cons (string)
+		       (choice (const :tag "Default" nil)
+			       (const :tag "20 bits" 20)
+			       (const :tag "25 bits" 25)
+			       (const :tag "30 bits" 30)
+			       (integer)))))
 
 (defcustom hashcash "/usr/local/bin/hashcash"
   "*The path to the hashcash binary.")
@@ -93,11 +101,19 @@
 
 (defun hashcash-check-payment (token str val)
   "Check the validity of a hashcash payment."
-  (zerop (call-process hashcash nil nil nil "-c"
+  (zerop (save-excursion
+	   (set-buffer (get-buffer-create " *hashcash verify*"))
+	   (erase-buffer)
+	   (insert (format "%s -c -d -f %s -b %s -r %s %s\n"
+			   hashcash hashcash-double-spend-database
+			   (number-to-string val) str token))
+	   ;; hashcash uses isatty() to set -quiet flag, so we don't
+	   ;; see anything. process-connection-type doesn't help.
+	   (call-process hashcash nil t nil "-c"
 		       "-d" "-f" hashcash-double-spend-database
 		       "-b" (number-to-string val)
 		       "-r" str
-		       token)))
+			 token))))
 
 ;;;###autoload
 (defun hashcash-insert-payment (arg)
@@ -112,14 +128,16 @@
 ;;;###autoload
 (defun hashcash-verify-payment (token &optional resource amount)
   "Verify a hashcash payment"
-  (let ((key (cadr (split-string-by-char token ?:))))
+  (let ((key (caddr (split-string token ":"))))
     (cond ((null resource)
 	   (let ((elt (assoc key hashcash-accept-resources)))
-	     (and elt (hashcash-check-payment token (car elt)
-			(or (cadr elt) hashcash-default-accept-payment)))))
+	     (and elt (hashcash-check-payment
+		       token (car elt)
+		       (or (cdr elt) hashcash-default-accept-payment)))))
 	  ((equal token key)
 	   (hashcash-check-payment token resource
-				(or amount hashcash-default-accept-payment)))
+				   (or amount
+				       hashcash-default-accept-payment)))
 	  (t nil))))
 
 ;;;###autoload
@@ -159,7 +177,8 @@
 					   hashcash-default-accept-payment)))
     (save-excursion
       (goto-char (point-min))
-      (search-forward mail-header-separator)
+      (or (search-forward (concat "\n" mail-header-separator "\n") nil t)
+	  (search-forward-regexp "[^:]+:\\([^\n]\\|\n[ \t]\\)+\n\n" nil t))
       (beginning-of-line)
       (let ((end (point))
 	    (ok nil))

Re: Hashcash

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@extundo.com> writes:

> I lobbied for this in SpamAssassin, and there are patches available,
> but nothing seems to happen.  They seemed to want to wait for a
> standard to emerge, which seemed very backwards to what SA is all
> about to me, but anyway.

Yup.  :-)

>> Also, the posting code seems to be synchronous.  I think it would be
>> nice to put the hashcash generation in the background, and postpone
>> the sending until the message has been hashcashed.  Has anybody done
>> any work in that area?  Could, perhaps, the delayed nndrafts code be
>> used for that?
>
> Sounds like a good idea.  We'd might as well write a complete elisp
> implementation of it, since it justs loop over a SHA-1 computation.

But if we do it in Emacs Lisp, will we be able to do it
asynchronously?  Hm, perhaps -- is it just running an external SHA-1
program repeatedly?  Then it can be done with process sentinels, I
guess...

Then sending could basically be:

1) Put the message in the draft buffer

2) Compute the payment in the background

3) Insert the payment and send it

If the user quits Gnus/Emacs before the payment has been computed,
the message will remain in the draft group, which might be annoying,
but probably not a major problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Hashcash

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>>> Also, the posting code seems to be synchronous.  I think it would be
>>> nice to put the hashcash generation in the background, and postpone
>>> the sending until the message has been hashcashed.  Has anybody done
>>> any work in that area?  Could, perhaps, the delayed nndrafts code be
>>> used for that?
>>
>> Sounds like a good idea.  We'd might as well write a complete elisp
>> implementation of it, since it justs loop over a SHA-1 computation.
>
> But if we do it in Emacs Lisp, will we be able to do it
> asynchronously?  Hm, perhaps -- is it just running an external SHA-1
> program repeatedly?  Then it can be done with process sentinels, I
> guess...

Calling external SHA1 will be too slow (it loops many times), but
maybe looping a elisp sha1 in the idle-timer could work (every loop is
short and it can be resumed fast).  Perhaps the simplest thing is to
disable hashcash in Gnus if the external "hashcash" binary isn't
installed, and to enable it if it is present, though.

Re: Hashcash

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@extundo.com> writes:

> Calling external SHA1 will be too slow (it loops many times), but
> maybe looping a elisp sha1 in the idle-timer could work (every loop is
> short and it can be resumed fast).

Is there a SHA1 implementation in Emacs Lisp?  Hm, yes, I remember
some talk about that...

> Perhaps the simplest thing is to disable hashcash in Gnus if the
> external "hashcash" binary isn't installed, and to enable it if it
> is present, though.

Yup.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* prj@po.cwru.edu (Paul Jarc)  on Mon, 05 Aug 2002
| In doing so, they would give up their anonymity.

Bah.  Spammers use throwaway accounts.  So what if they lose the anonymity
of a single Hotmail address.  Nothing stops them from randomly generating
hundreds of others.

| We'd be able to find them and report them (and blacklist them).

Reporting doesn't much work anymore.  Too many ISPs simply ignore spam
complaints.

Blacklisting throwaway addresses is pointless when spammers can generate
all the Hotmail and Yahoo addresses they need.

-- 
Rat <ratinox@peorth.gweep.net>    \ Caution: Happy Fun Ball may suddenly
Minion of Nathan - Nathan says Hi! \ accelerate to dangerous speeds.
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Stainless Steel Rat writes:

> * prj@po.cwru.edu (Paul Jarc)  on Mon, 05 Aug 2002
> | In doing so, they would give up their anonymity.

> Bah.  Spammers use throwaway accounts.  So what if they lose the
> anonymity of a single Hotmail address.  Nothing stops them from
> randomly generating hundreds of others.

Do Hotmail (or Yahoo) accounts allow autoconfirmers?

> | We'd be able to find them and report them (and blacklist them).

> Reporting doesn't much work anymore.  Too many ISPs simply ignore
> spam complaints.

Thus blacklisting.

> Blacklisting throwaway addresses is pointless when spammers can
> generate all the Hotmail and Yahoo addresses they need.

But they have to work harder to do it.  If spammers do as you say to
"get around" TMDA, the user simply blacklists them with the first spam
to get through.  So spam, if not eliminated, is cut down, but the work
of the spammer goes up.

Of course, you could also do things to prevent auto-confirmers from
working (like random confirmation messages and rotating confirm email
addresses), but then the regular people can't hide TMDA behind an
auto-confirmer either.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: TMDA (was: new spam functionality added)

[Stainless Steel Rat]

* David Masterson <dmaster@synopsys.com>  on Mon, 05 Aug 2002
| Do Hotmail (or Yahoo) accounts allow autoconfirmers?

Automatic confirmation does not yet exist with TMDA, to my knowledge.  This
is all hypothetical stuff.


| Thus blacklisting.

Doesn't work with throwaway addresses.  They just pick a new one.  Black
lists do not work without constant and continuous human intervention.


| But they have to work harder to do it.

BS.  One guy needs to program around it for all spammers to benefit.

| If spammers do as you say to "get around" TMDA, the user simply
| blacklists them with the first spam to get through.  So spam, if not
| eliminated, is cut down, but the work of the spammer goes up.

That won't work because the hotmail or yahoo address used for the first
confirmation is never used again.  It is a one-shot, throwaway address.
Putting "spammer@hotmail.com" into your blacklist.  will not stop anything
from "iamnotaspammer@hotmail.com" from getting through.

| Of course, you could also do things to prevent auto-confirmers from
| working

Which defeats the purpose of automatic confirmation, by the way.

| (like random confirmation messages and rotating confirm email addresses),
| but then the regular people can't hide TMDA behind an auto-confirmer
| either.

All of which puts more work on the user.  Bad idea.

-- 
Rat <ratinox@peorth.gweep.net>    \ Warning: pregnant women, the elderly, and
Minion of Nathan - Nathan says Hi! \ children under 10 should avoid prolonged
PGP Key: at a key server near you!  \ exposure to Happy Fun Ball.
       That and five bucks will get you a small coffee at Starbucks.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Stainless Steel Rat writes:

> * David Masterson <dmaster@synopsys.com>  on Mon, 05 Aug 2002
> | Do Hotmail (or Yahoo) accounts allow autoconfirmers?

> Automatic confirmation does not yet exist with TMDA, to my
> knowledge.  This is all hypothetical stuff.

So?  If systems like Hotmail and Yahoo that allow anonymous account
generation do not provide a means to use auto-confirmers, then
spammers can't use those systems to combat TMDA.

> | But they have to work harder to do it.

> BS.  One guy needs to program around it for all spammers to benefit.

Talk about hypothetical...

> Putting "spammer@hotmail.com" into your blacklist.  will not stop
> anything from "iamnotaspammer@hotmail.com" from getting through.

In the current scenario, spammers have an advantage in that they are
completely anonymous.  Since all the headers in the spam can be
spoofed, the most you know is the last system to relay the spam to
you.  Basically, you can't trace who sent you the email.

In the TMDA scenario, spammers now have to put some sort of (TBD)
auto-confirmer into place to "get the spam through".  This means that
the "From" or "Reply-to" address on their email now has to be valid so
that TMDA can send confirmation email to their auto-confirmer.
They've now left a set of fingerprints on the email that could come
back to haunt them through legal proceedings.  Not everyone is likely
to "take them to court", but the possibility goes way up under TMDA
(it doesn't have to be an individual, it could be a class action
against the spammer *OR* his ISP).

TMDA might not be a complete *technical* defense against SPAM, but, in
concert with a legal defense, it changes the playing field on the
spammer such that they may not want to play anymore.  At worst, it
should force ISPs to look at holes in their security or face
blacklisting by a class of users.

-- 
David Masterson
dsm@rawbw.com

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

David Masterson <dsm@rawbw.com> wrote:
> In the TMDA scenario, spammers now have to put some sort of (TBD)
> auto-confirmer into place to "get the spam through".

Or abuse a compromised system - but those will always be a problem
anyway.

> This means that the "From" or "Reply-to" address on their email now
> has to be valid so that TMDA can send confirmation email to their
> auto-confirmer.

Actually, it's the envelope sender, which may or may not be easy for
the recipient to look at, depending on all the software standing
between them and their mail.  Usually it's not too hard to get at if
you want it, I think, but you do have to know that that's what you're
looking for.  Header fields can still be forged.


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> David Masterson <dsm@rawbw.com> wrote:

>> This means that the "From" or "Reply-to" address on their email now
>> has to be valid so that TMDA can send confirmation email to their
>> auto-confirmer.

> Actually, it's the envelope sender,

That's bad.  It should be the actual From: header used in the
message.  Is this configurable in TMDA?

> which may or may not be easy for the recipient to look at, depending
> on all the software standing between them and their mail.

Or it could disappear completely.

I just found an email message from a cousin of mine in the SPAM
folder, when checking it for false positives.  I let all messages not
adressed directly to me, or to a mailing list I subscribe to, "drop
through" into a spam folder, which I check occasionally. His email
program had in its infinite wisdom (billware...) decided to use the
envelope address, instead of the From: address.  And the envelope
address wasn't in the list of official addresses in the filter.

By your description, all TMDA confirmation messages will end up in my
spam folder, and not be read until the next time 

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
>>>>>> prj@po.cwru.edu (Paul Jarc):
>> Actually, it's the envelope sender,
>
> That's bad.  It should be the actual From: header used in the
> message.

Why?

> Is this configurable in TMDA?

I don't this so, but check the website if you really want to know.

> I let all messages not adressed directly to me, or to a mailing list
> I subscribe to, "drop through" into a spam folder, which I check
> occasionally. His email program had in its infinite wisdom
> (billware...) decided to use the envelope address, instead of the
> From: address.  And the envelope address wasn't in the list of
> official addresses in the filter.

Sounds like your configuration was a little broken.  (Yes, his mailer
was too.)


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> Steinar Bang <sb@dod.no> wrote:

>> That's bad.  It should be the actual From: header used in the
>> message.

> Why?

Because that's the address I would like to validate, because it is the
address my MUA uses when responding.

I would like the whitelist to allow messages based on their envelope
address though, to allow messsages from mailing lists to reach me.

> Sounds like your configuration was a little broken.

That's debatable, and in any case it was the default exim config of
debian. 

The default config of debian, and other linuxen I have seen, is to have
username@FQDN as the envelope address, and that was what it was at the
time.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
>> Steinar Bang <sb@dod.no> wrote:
>>> That's bad.  It should be the actual From: header used in the
>>> message.
...
> Because that's the address I would like to validate, because it is the
> address my MUA uses when responding.

As long as your MUA uses the same set of addresses every time, and as
long as you read messages sent to all those addresses, there's no
reason to care which address someone else's TMDA knows about.

> The default config of debian, and other linuxen I have seen, is to have
> username@FQDN as the envelope address, and that was what it was at the
> time.

That's fine, but then it's up to you to make sure you see messages
sent to that address.  Otherwise you won't see bounces, etc.


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> Steinar Bang <sb@dod.no> wrote:
>>> Steinar Bang <sb@dod.no> wrote:
>>>> That's bad.  It should be the actual From: header used in the
>>>> message.
> ...
>> Because that's the address I would like to validate, because it is the
>> address my MUA uses when responding.

> As long as your MUA uses the same set of addresses every time, and as
> long as you read messages sent to all those addresses, there's no
> reason to care which address someone else's TMDA knows about.

Ah, we misunderstand each other.

I was thinking about setting up TMDA for myself, and reply to From: is
what I want from it.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
> I was thinking about setting up TMDA for myself, and reply to From: is
> what I want from it.

I don't understand.  Your TMDA wouldn't send messages to you.  Do you
mean you would want your TMDA to reply to the From addresses in
incoming messages to you?  That would make your TMDA different from
everyone else's, so it wouldn't be what they expect, thus not what
they're prepared for, thus not what they want.


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> ... Do you mean you would want your TMDA to reply to the From
> addresses in incoming messages to you? 

Yes.  What's in the From: field is what I would like to have
validated.  I don't care if the envelope address is valid or not,
since that's not what Gnus uses when I press `r'.

> That would make your TMDA different from everyone else's, so it
> wouldn't be what they expect, thus not what they're prepared for,
> thus not what they want.

What does what other people want have to do with how I set up my own
TMDA?

Besides, most people don't know what envelope addresses are, so why
would they care, one way or another?

Re: TMDA (was: new spam functionality added)

[Matt Armstrong]

Steinar Bang <sb@dod.no> writes:

>>>>>> prj@po.cwru.edu (Paul Jarc):
>
>> David Masterson <dsm@rawbw.com> wrote:
>
>>> This means that the "From" or "Reply-to" address on their email now
>>> has to be valid so that TMDA can send confirmation email to their
>>> auto-confirmer.
>
>> Actually, it's the envelope sender,
>
> That's bad.  It should be the actual From: header used in the
> message.  Is this configurable in TMDA?

All types of automatic replies should go to the envelope sender.
Failure to do that is how you end up with auto-replies getting sent to
mailing lists, etc.


>> which may or may not be easy for the recipient to look at,
>> depending on all the software standing between them and their mail.
>
> Or it could disappear completely.
>
> I just found an email message from a cousin of mine in the SPAM
> folder, when checking it for false positives.  I let all messages
> not adressed directly to me, or to a mailing list I subscribe to,
> "drop through" into a spam folder, which I check occasionally. His
> email program had in its infinite wisdom (billware...) decided to
> use the envelope address, instead of the From: address.  And the
> envelope address wasn't in the list of official addresses in the
> filter.
>
> By your description, all TMDA confirmation messages will end up in
> my spam folder, and not be read until the next time

Probably best to fix your envelope sender.

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> Matt Armstrong <matt@lickey.com>:

> Steinar Bang <sb@dod.no> writes:

> All types of automatic replies should go to the envelope sender.

Not _all_ types of automatic replies.

Bounces should go to the envelope sender, but I can't offhand think of
any others that should.

> Failure to do that is how you end up with auto-replies getting sent
> to mailing lists, etc.

A proper auto-replier (I'm guessing you mean a vacation-like program
here?) shouldn't be sending a notification in the first place, if the
incoming message isn't addressed to the recipient.

If an auto-replier in its infinite wisdom decides to send a
notification anyway, it shouldn't send it to the envelope address,
which in this case is the list manager.  It should send it to the
address in the From: field, which is supposed to be the address of the
person that sent the message to the list.

> Probably best to fix your envelope sender.

The envelope sender in question, was the default envelope configuraton
for exim in debian woody.  AFAIR it was the same as the default
envelope in SuSE 6.2, ie. username@FQDN.

Some people would debate that this is the correct value for the
envelope sender.

Re: TMDA (was: new spam functionality added)

[Matt Armstrong]

Steinar Bang <sb@dod.no> writes:

>>>>>> Matt Armstrong <matt@lickey.com>:
>
>> Steinar Bang <sb@dod.no> writes:
>
>> All types of automatic replies should go to the envelope sender.
>
> Not _all_ types of automatic replies.
>
> Bounces should go to the envelope sender, but I can't offhand think
> of any others that should.

This isn't an authoritative reference, but this draft gives (I think)
good arguments to the contrary:

http://www.ietf.org/internet-drafts/draft-moore-auto-email-response-00.txt


>> Failure to do that is how you end up with auto-replies getting sent
>> to mailing lists, etc.
>
> A proper auto-replier (I'm guessing you mean a vacation-like program
> here?) shouldn't be sending a notification in the first place, if
> the incoming message isn't addressed to the recipient.

Agreed -- this is suggested in the above mentioned draft too.


> If an auto-replier in its infinite wisdom decides to send a
> notification anyway, it shouldn't send it to the envelope address,
> which in this case is the list manager.  It should send it to the
> address in the From: field, which is supposed to be the address of
> the person that sent the message to the list.

I disagree with you here.  The list admin is in a better position to
kick the offending user off the list or otherwise fix the problem.

The current debian 'vacation' program replies to the envelope sender,
and I assume it has been that way for a long time.  So there is some
historical precedent as well.


>> Probably best to fix your envelope sender.
>
> The envelope sender in question, was the default envelope
> configuraton for exim in debian woody.  AFAIR it was the same as the
> default envelope in SuSE 6.2, ie. username@FQDN.
>
> Some people would debate that this is the correct value for the
> envelope sender.

I assumed the envelope sender you were using wasn't actually reaching
you.

If so, I'd call that a configuration error.  I think it is reasonable
to expect the envelope sender to be valid -- the SMTP bounce and
delivery status mechanisms depend on it.

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> Matt Armstrong <matt@lickey.com>:

>> The envelope sender in question, was the default envelope
>> configuraton for exim in debian woody.  AFAIR it was the same as
>> the default envelope in SuSE 6.2, ie. username@FQDN.

>> Some people would debate that this is the correct value for the
>> envelope sender.

> I assumed the envelope sender you were using wasn't actually
> reaching you.

Well, I don't count myself among "some people"...:-)

But for one of my machines, username@FQDN will reach me, only to be
filtered into my SPAM folder.  For my laptop nomade machine,
username@FQDN is almost never correct.

But I've changed the envelope on outgoing messages now.  The only
emails I get to the envelope address, are billware users that put me
in their address books with this address.

> If so, I'd call that a configuration error.  I think it is
> reasonable to expect the envelope sender to be valid -- the SMTP
> bounce and delivery status mechanisms depend on it.

I tend to agree on that, but the only workaround that suits my setup
(multiple mail drops, where I have no personal control over the MTAs),
was to make the envelope address be the same as the From field.

I don't know if this is the correct behaviour, but it is the most
correct behaviour I have been able to think of.  I guess this is the
same behaviour as mail programs on Win32, that sends their mail
through an SMTP server, gets?

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
> the only workaround that suits my setup (multiple mail drops, where
> I have no personal control over the MTAs), was to make the envelope
> address be the same as the From field.

There's nothing wrong with that.  What your envelope sender should be
is entirely up to you, as long as messages to that address reach you.


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> Steinar Bang <sb@dod.no> wrote:
>> the only workaround that suits my setup (multiple mail drops, where
>> I have no personal control over the MTAs), was to make the envelope
>> address be the same as the From field.

> There's nothing wrong with that. 

It can introduce mail loops.  Ie. it has...;-)

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>>>>> prj@po.cwru.edu (Paul Jarc):
>> Steinar Bang <sb@dod.no> wrote:

>>> the only workaround that suits my setup (multiple mail drops,
>>> where I have no personal control over the MTAs), was to make the
>>> envelope address be the same as the From field.

>> There's nothing wrong with that. 

> It can introduce mail loops.  Ie. it has...;-)

I have to correct myself.  Those mail loops were related to a procmail
setup setting my address on that machine as the envelope address for
all forwarded mail (happens when procmail runs without root
priviledges, ie. when run from a ~/.forward file).

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> Matt Armstrong <matt@lickey.com>:

> Steinar Bang <sb@dod.no> writes:
>>>>>>> Matt Armstrong <matt@lickey.com>:
>> 
>>> Steinar Bang <sb@dod.no> writes:

>>> All types of automatic replies should go to the envelope sender.

>> Not _all_ types of automatic replies.

>> Bounces should go to the envelope sender, but I can't offhand think
>> of any others that should.

> This isn't an authoritative reference, but this draft gives (I
> think) good arguments to the contrary:

> http://www.ietf.org/internet-drafts/draft-moore-auto-email-response-00.txt

But I think I can use that document to justify that TMDA should send
its responses to the From field address:

The above document states that:

   - For responses sent by Personal Responders, the From field SHOULD
     contain the name of the recipient and an address chosen by the
     recipient to be recognizable to correspondents. Normally this
     would be the same address that was used to send mail to that
     recipient.

And since the From field is what I would like to validate, because
that's the email address I plan to use to respond, that's the one I
would like TMDA to verify for me.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
> And since the From field is what I would like to validate, because
> that's the email address I plan to use to respond, that's the one I
> would like TMDA to verify for me.

Think of it as validating a person, not an address.  As long as the
person is validated, why do you care which address is used for the
validation?


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> Think of it as validating a person, not an address.  As long as the
> person is validated, why do you care which address is used for the
> validation?

I want the address used when I do `r' to be a valid, working address.

Mainly because I'm tired of editing "deletethis" or "ihatespam" out of
the From addresses of bounced messages, before doing a resend (or not,
as the case may be).

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Steinar Bang <sb@dod.no> wrote:
>>>>>> prj@po.cwru.edu (Paul Jarc):
>> As long as the person is validated, why do you care which address
>> is used for the validation?
>
> I want the address used when I do `r' to be a valid, working address.
>
> Mainly because I'm tired of editing "deletethis" or "ihatespam" out of
> the From addresses of bounced messages, before doing a resend (or not,
> as the case may be).

Nothing on your end can stop people from sending you messages like
that.  If you started using a TMDA that sent confirmation requests to
the From address, you'd just see bounced confirmation requests for
those nonexistent addresses instead of the original messages - or if
for some reason you didn't see those bounces, those messages would
silently disappear, without any indication to the sender as to why.


paul

Re: TMDA (was: new spam functionality added)

[Steinar Bang]

>>>>> prj@po.cwru.edu (Paul Jarc):

> Nothing on your end can stop people from sending you messages like
> that.  If you started using a TMDA that sent confirmation requests
> to the From address, you'd just see bounced confirmation requests
> for those nonexistent addresses instead of the original messages -
> or if for some reason you didn't see those bounces,

I would configure away the TMDA bounce reports if possible.

> those messages would silently disappear, without any indication to
> the sender as to why.

I can live with that.

If they want to reach me, they should use a working sender address.

Re: TMDA (was: new spam functionality added)

[David Masterson]

>>>>> Per Abrahamsen writes:

> prj@po.cwru.edu (Paul Jarc) writes:

>> If a reliable autoconfirmer were written, would you consider it a
>> waste of time to set it up for your mailbox?

> If a reliable autoconfirmer were written, some spammers would start
> using it.  We need some amount manual intervention for the system to
> scale.

If they do, then they will have to leave their "fingerprints" on the
SPAM, thus making them easier to trace.

> The problem that does frighten me is whether non-technical people will
> understand the request for a confirmation message.

I guess it depends on how friendly the confirmation message is.  Of
course, you'll have to watch that the confirmation message isn't
assasinated by an overzealous SpamAssassin.

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: TMDA (was: new spam functionality added)

[Scott A Crosby]

On Mon, 05 Aug 2002 19:38:07 +0200, Per Abrahamsen <abraham@dina.kvl.dk> writes:

> But I don't consider the problem with people who will refuse to spend
> a second or two to confirm their existence for significant.  I lose
> mail today because by filters (automatic and manual) are too
> agressive, and I can't afford the time to use less agressive filters:
> 
> - Today, I can lose mail from anybody, through no fault of theirs.
>   Even if it is something that it is very important to them that I
>   see.
> 
> - With TMDA, I will only lose mail from people who deliberately decide
>   that getting their message through isn't worth two seconds of their
>   tine.

No, you lose email from anyone who is aware that you have TMDA and
doesn't want to have to deal with your autoreply messages.


However, if you (purposely or inadvertently) *hide* the fact that you
use TMDA. Then you're exploiting the fact that most people will
respond out of a psychological need to not feel like they wasted their
time. Many people will spend an extra 10 seconds to avoid having their
email deleted unread (and thus wasting the results of several minutes
work). 


I fall into the first catagory. I'll be happy as long as I can detect
users of such autoreply robots.[1]

Scott


[1] The following score rules seem to detect some TMDA-containing
emails. They will not work on a mailing list that has reply-to
munging. Alter the scores to suit.

(("extra"
  (".*[-+]\\(dated\\|exp\\|sender\\|keyword\\)[-+]" -2001 nil r "Reply-To"))
 ("from"
  ("[-+]\\(dated\\|exp\\|sender\\|keyword\\)[-+]" -2001 nil r)))

These split-fancy rules attempt to detect an auto-reply.
	 ("X-Delivery-Agent" "TMDA" "junk.tmda")
	 ("Delivery-Agent" "TMDA" "junk.tmda")

Advice for fixing/cleaning these up or enhancing these rules is
welcome. I have had limited exposure to TMDA autoreplies.

Re: TMDA (was: new spam functionality added)

[Per Abrahamsen]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> However, if you (purposely or inadvertently) *hide* the fact that you
> use TMDA. Then you're exploiting the fact that most people will
> respond out of a psychological need to not feel like they wasted their
> time. Many people will spend an extra 10 seconds to avoid having their
> email deleted unread (and thus wasting the results of several minutes
> work). 

It is still *their* choice.  I have no obligation to read any email
send to me.  By not using TMDA, I make it *my* choice, by
accidentially junking their messages.  By using TMDA, it become
*their* choice.  I do not care for what *their* reasons will be for
making *their* choice.

I no, I have no intension of munging my headers or signature in order
to accomodate the people who believe that their message is worth
several minuttes of their time to write, but not an additional 10
seconds to ensure it is read.  These people have a problem I refuse to
make mine, or force on third parties by munging headers.

This is similar to the current situation, where I do not publish my
blacklist rules, so people who write me will not know whether they
will be filtered.  

> These split-fancy rules attempt to detect an auto-reply.
> 	 ("X-Delivery-Agent" "TMDA" "junk.tmda")
> 	 ("Delivery-Agent" "TMDA" "junk.tmda")

This is reasonable.  This just put you in the same position like
someone from Taiwan who would write me today.  He would not be read,
because of my filters.  The difference is that you choose not to be
read deliberately, while the Taiwanese hardly had a choice of where to
be born.

Re: TMDA (was: new spam functionality added)

[Scott A Crosby]

On Tue, 06 Aug 2002 12:10:34 +0200, Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Scott A Crosby <scrosby@cs.rice.edu> writes:
> 
> > However, if you (purposely or inadvertently) *hide* the fact that you
> > use TMDA. Then you're exploiting the fact that most people will
> > respond out of a psychological need to not feel like they wasted their
> > time. Many people will spend an extra 10 seconds to avoid having their
> > email deleted unread (and thus wasting the results of several minutes
> > work). 
> 
> It is still *their* choice.  I have no obligation to read any email
> send to me.  By not using TMDA, I make it *my* choice, by
> accidentially junking their messages.  By using TMDA, it become
> *their* choice.  I do not care for what *their* reasons will be for
> making *their* choice.
> 

So, why not give them the choice when you send messages out?

By informing them that you use TMDA, it is their choice whether to
respond to your origional message. They may then priortize their time
as they choose.[1]

> I no, I have no intension of munging my headers or signature in order
> to accomodate the people who believe that their message is worth
> several minuttes of their time to write, but not an additional 10
> seconds to ensure it is read.  These people have a problem I refuse to
> make mine, or force on third parties by munging headers.

You are more than willing to place your problem upon others with
robot-reply messages. Why not advertise that fact? That way you not
only get messages from people willing to reply to your robot, but the
only people who will ever write to you are those who are aware of the
invconvenience, and willing to accept it in order to talk to you?

This is the best of both worlds. I, and everyone else, gets to
prioritize my time and avoid unwanted inconvenience. And you only get
messages from people who are willing to accept the inconvenience to
talk to you.

So, why hide your use of TMDA? 

IMHO, having TMDA advertise itself sounds like an excellent
feature. Furthermore, I suspect that it would alleviate many of the
complaints about it. When people send an email, they don't like having
an unsolicited robot-reply; with this feature, they'll know to expect
it.


Scott

[1] In my case, away from being inconvenienced by robot-reply
messages. However, others may consider this professionalism and be
more likely to read such messages.

Re: TMDA (was: new spam functionality added)

[Per Abrahamsen]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> By informing them that you use TMDA, it is their choice whether to
> respond to your origional message. 

It is their choice in any case.

> You are more than willing to place your problem upon others with
> robot-reply messages.

If you have a problem with robot-reply messages, you have a problem.
Not me.  Stop trying to make your problems mine.

BTW: robot-reply messages are almost as old as email, think
"vacation".  Should we have an "X-vacation-user" header as well?

> So, why hide your use of TMDA? 

I won't hide it.  I just won't advertise it.  I don't advertise my
line of work, political or sexual orientation either.  Even though
these are issues some people will have problems with.
 
I do advertise my religion, but that is my choise.

Re: new spam functionality added

[clemens fischer]

                            Re: new spam functionality added

>   From: Scott A Crosby <scrosby@cs.rice.edu>
>   Date: 31 Jul 2002 16:07:23 -0500

>   Now, imagine there's a daemon that autoreplies to such 'please
>   reply to me' messages.. Well, just forge the spam to appear to come
>   from a legitimate user, and guess what, the bounces go to them, and
>   their client helpfully 'authenticates' the spam.. (The daemon can't
>   be configured to record every email sent and only autoreply to
>   autoreplies to emails the user actually sent. Many times people

well, i use a system where each of my (normal) emails is signed using a
cryptographic hash.  the resulting number is placed into a header of my
emails (don't look for it in this posting, it is processed by lynx).

then, when i get back the message for authentication, i recompute the value
and compare it to the value in the headers.  we use this system on the
"bernstein" lists at cr.yp.to, where a program called "qsecretary" verifies
every posting as an anti-spam measure.  the program for these automatic
verifications is called "pymsgauth" (a python script).

but what we're really looking for is a better internet-mail infrastructure,
right?  one where the sender is responsible for storing the messages, not
the receiver as it is with todays SMTP.  this infrastructure could be like
discussed in the IM2000 working group.  it would make spamming in general
next to impossible, at least too expensive to be even considered.

clemens

Re: new spam functionality added

[Per Abrahamsen]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Please don't.. TMDA is tragedy of the commons. It only helps one
> person by putting extra work and effort upon everyone else. If
> everyone used it, things will turn to crap.

Uh, the whitelist features would make the world a much better place.
I'd certainly be willing to offer an extra "reply" the first time I
mail a new person, in exchange for getting rid of the spam.

The mangled, temporary Usenet from adresses are pure evil though.
They break all the nice odd features that depend on each person having
a rarely changing id, such as bbdb, google profiles, welcome letters,
and whitelist moderated newsgroups.

Although if they are used as Reply-To instead of From, they are not
that harmful.

Re: new spam functionality added

[Jack Twilley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>>>>> "Josh" == Josh Huber <huber+dated+1028578998.eb7ca9@alum.wpi.edu> writes:

[...]

Josh> Sure, perhaps we should include something about TMDA?

Yes, something recommending against it, and perhaps a chunk of code
that reverses the damage TMDA causes would be nice.  Feel free to
write something up for BBDB too that automatically strips the crap
- From addresses before adding them to the database.

Jack.
(annoyed by TMDA users.)
- -- 
Jack Twilley
jmt at twilley dot org
http colon slash slash www dot twilley dot org slash tilde jmt slash
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (FreeBSD)

iD8DBQE9SEy9GPFSfAB/ezgRAoBpAJ90C+ei9gPQj4jNPqZDYHwLia96ZACgmppu
YIv80WjQRoqE+JDQSSRb3I8=
=Uh4A
-----END PGP SIGNATURE-----

Re: new spam functionality added

[Josh Huber]

Jack Twilley <jmt+usenet@twilley.org> writes:

> Yes, something recommending against it

Well, I won't be doing this, because I recommend it :)

> and perhaps a chunk of code that reverses the damage TMDA causes
> would be nice.  Feel free to write something up for BBDB too that
> automatically strips the crap From addresses before adding them to
> the database.

This is already included in my tmda.el:

(setq bbdb-canonicalize-net-hook 'tmda-normalize-address)

> (annoyed by TMDA users.)

Why?

-- 
Josh Huber

Re: new spam functionality added

[Simon Josefsson]

Jack Twilley <jmt+usenet@twilley.org> writes:

>>>>>> "Josh" == Josh Huber <huber+dated+1028578998.eb7ca9@alum.wpi.edu> writes:
>
> [...]
>
> Josh> Sure, perhaps we should include something about TMDA?
>
> Yes, something recommending against it, and perhaps a chunk of code
> that reverses the damage TMDA causes would be nice.

I also think Gnus should include code that reverts the damage caused.
One thing that annoys me is that the attribution lines are messed up.
Would it make sense to remove all occurances of

\+dated\+[0-9]+\.[a-z0-9]+

in the attribution line?  Would there be false positives?  Perhaps
this causes more damage than it helps though.

Re: new spam functionality added

[David Masterson]

>>>>> Simon Josefsson writes:

> Jack Twilley <jmt+usenet@twilley.org> writes:

>>>>>>> "Josh" == Josh Huber
>>>>>>> <huber+dated+1028578998.eb7ca9@alum.wpi.edu> writes:

Josh> Sure, perhaps we should include something about TMDA?

>> Yes, something recommending against it, and perhaps a chunk of code
>> that reverses the damage TMDA causes would be nice.

> I also think Gnus should include code that reverts the damage caused.
> One thing that annoys me is that the attribution lines are messed up.
> Would it make sense to remove all occurances of

> \+dated\+[0-9]+\.[a-z0-9]+

> in the attribution line?  Would there be false positives?  Perhaps
> this causes more damage than it helps though.

I think that's "[-+]dated\+[0-9+\.[A-Za-z0-9]+".

However, I don't understand what you mean by "damage caused"?

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Simon Josefsson]

Josh Huber <huber+dated+1028578998.eb7ca9@alum.wpi.edu> writes:

> Simon Josefsson <jas@extundo.com> writes:
>
>> Another thought on documentation: if spam.el documentation is added,
>> and the NoCeM node is moved, to the "Thwarting Email Spam" node, I
>> think that node becomes long enough, and important enough, to
>> warrant moving it up to the top-level menu.  Opinions?
>
> Sure, perhaps we should include something about TMDA?
>
> http://cvs.sf.net/cgi-bin/viewcvs.cgi/~checkout~/tmda/tmda/contrib/tmda.el
>
> I'm in the middle of writing up some documentation to put on
> my.gnus.org, and converting it into texi for the manual would be
> simple.  What do you think?

Personally I don't like it, but perhaps carefully worded informative
documentation doesn't hurt.  Functions for dealing with the corrupted
addresses (for matching purposes) could be included, perhaps.

(Assuming TMDA contains the part which bounces mail, asking the sender
to answer with a cookie to deliver the mail.  That's annoying, and I
haven't understood how two people not on each other's whitelist could
ever communicate with each other if they both uses the scheme.  And it
looks ugly.)

Re: new spam functionality added

[David Masterson]

>>>>> Simon Josefsson writes:

> (Assuming TMDA contains the part which bounces mail, asking the sender
> to answer with a cookie to deliver the mail.  That's annoying, and I
> haven't understood how two people not on each other's whitelist could
> ever communicate with each other if they both uses the scheme.

Three ways that I can think of:

1. The first person sends the second person a "dated" email address
   which allows the second person to respond within (say) 5 days
   without a cookie request.

2. The first person can generate a special email address for the
   second person that will never ask for a cookie (if it's abused, the
   address can be dropped).

3. The first person adds the second person to his whitelist before
   sending the original message.

the second person doesn't reply within the timeout period, then he's
probably never going to respond.  On the off chance that he was gone
for a long vacation, then he'll be inconvenienced by the cookie
reply, but that's a one time thing and should be relatively
infrequent.

> And it looks ugly.)

I'll accept ugly if works as advertised.  Currently, though, it's only
really useful on systems processing their own mail (as opposed to POP
or IMAP off another server).  Properly configured, it does look like
it will eliminate SPAM from your Inbox, but your computer is still
processing the SPAM.  It looks like a good option for highly connected
email servers to offer to their customers (HotMail+TMDA??).

-- 
David Masterson                David DOT Masterson AT synopsys DOT com
Sr. R&D Engineer               Synopsys, Inc.
Software Engineering           Sunnyvale, CA

Re: new spam functionality added

[Alan Shutko]

David Masterson <dmaster@synopsys.com> writes:

> Three ways that I can think of:

Each way depends on the first sender knowing that the other person is
also using TMDA.  How would you know that?

-- 
Alan Shutko <ats@acm.org> - In a variety of flavors!
Virgin wool: Wool from slow sheep.

Re: new spam functionality added

[Paul Jarc]

Alan Shutko <ats@acm.org> wrote:
> David Masterson <dmaster@synopsys.com> writes:
>> Three ways that I can think of:
>
> Each way depends on the first sender knowing that the other person is
> also using TMDA.  How would you know that?

Nope.  All those methods will do no harm if the other person does not
use TMDA.  The first sender can (and would be wise to) use them
regardless.


paul

Re: new spam functionality added

[Simon Josefsson]

David Masterson <dmaster@synopsys.com> writes:

>>>>>> Simon Josefsson writes:
>
>> (Assuming TMDA contains the part which bounces mail, asking the sender
>> to answer with a cookie to deliver the mail.  That's annoying, and I
>> haven't understood how two people not on each other's whitelist could
>> ever communicate with each other if they both uses the scheme.
>
> Three ways that I can think of:
>
> 1. The first person sends the second person a "dated" email address
>    which allows the second person to respond within (say) 5 days
>    without a cookie request.

This will sort of work, but it requires a UI for the first person to
specify the time delay.  Sometimes I send mail I know wont be read for
or replied to for many months, and having it default to 3 months will
allow too much spam.  E.g., bug reports.  And if the message ends up
in public before this time, you will get spam to it and the system
fails.

> 2. The first person can generate a special email address for the
>    second person that will never ask for a cookie (if it's abused, the
>    address can be dropped).

This doesn't work if you don't know the second person.  E.g., on
mailing lists and UseNet.  It deteriorates into using temporary email
addresses all the time, which breaks address books etc.

> 3. The first person adds the second person to his whitelist before
>    sending the original message.

Again doesn't work on mailing lists.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> David Masterson <dmaster@synopsys.com> writes:
>> 1. The first person sends the second person a "dated" email address
>>    which allows the second person to respond within (say) 5 days
>>    without a cookie request.
...
> And if the message ends up in public before this time, you will get
> spam to it and the system fails.

For some values of "fails"; stopping some spam is better than stopping
none.

>> 2. The first person can generate a special email address for the
>>    second person that will never ask for a cookie (if it's abused, the
>>    address can be dropped).
>
> This doesn't work if you don't know the second person.  E.g., on
> mailing lists and UseNet.

The problem (posed by yourself) was how two *people* can communicate
if they both use TMDA and are not on each others' whitelists.  Usenet
can't use TMDA.  Mailing lists can, but in that case, you'd use a
wide-open envelope sender address, for the mailing list to sends its
confirmation request to, and your normal, TMDA-protected From address.
You'll get the MLM's confirmation request through your wide-open
address; you'll confirm your message; the MLM will rewrite the
envelope sender before distributing your message; other subscribers
won't see it.

>> 3. The first person adds the second person to his whitelist before
>>    sending the original message.
>
> Again doesn't work on mailing lists.

What specific failure do you have in mind?


paul

Re: TMDA (was: new spam functionality added)

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> David Masterson <dmaster@synopsys.com> writes:
>>> 1. The first person sends the second person a "dated" email address
>>>    which allows the second person to respond within (say) 5 days
>>>    without a cookie request.
> ...
>> And if the message ends up in public before this time, you will get
>> spam to it and the system fails.
>
> For some values of "fails"; stopping some spam is better than stopping
> none.

Of course.

>>> 2. The first person can generate a special email address for the
>>>    second person that will never ask for a cookie (if it's abused, the
>>>    address can be dropped).
>>
>> This doesn't work if you don't know the second person.  E.g., on
>> mailing lists and UseNet.
>
> The problem (posed by yourself) was how two *people* can communicate
> if they both use TMDA and are not on each others' whitelists.  

Yes, and many times when I start to communicate with someone is that I
post to a mailing list or UseNet group and some other person (unknown
to me beforehand) replies to me personally.

>>> 3. The first person adds the second person to his whitelist before
>>>    sending the original message.
>>
>> Again doesn't work on mailing lists.
>
> What specific failure do you have in mind?

I post to the group Y, person X replies to me personally.  I can't
white list him, so I must bounce back a cookie request.  Lots of
cookies if you have many short email relationships.

Re: TMDA (was: new spam functionality added)

[Per Abrahamsen]

Simon Josefsson <jas@extundo.com> writes:

> Yes, and many times when I start to communicate with someone is that I
> post to a mailing list or UseNet group and some other person (unknown
> to me beforehand) replies to me personally.

This was quite common on Usenet 10 years ago, but has become
increasingly rare with the rise of Spam.  In fact, an increasing
number of new (post-August) users consider all private replies to
Usenet messages abuse.

I hope widespread use of TMDA-like whitelists will counter that
saddening trend, so that you once again will be able to contact
strangers you have meet on the net, without getting your message
treated like spam.

Today email is for most part a secret id only shared by a select group
of friends, quickly discared when it leaks out of that circle.  A
single extra message the first time you send mail to a new person seem
to be a small price for restoring email to the universal and open
medium it once was.  

And I have no sympathy for those who claim that they can handle having
an open email adress without relying whitelists, and from that
conclude that everybody else must be able to handle the same.  People
have different amounts of time and different amount of skill for
automatic and manual filtering.

Without whitelist filtering, keeping the email handle secret is,
tradically, really the best option for the majority of people.

Re: TMDA (was: new spam functionality added)

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> I post to the group Y, person X replies to me personally.  I can't
> white list him, so I must bounce back a cookie request.

He whitelists you before writing to you personally, so there's no
loop.

> Lots of cookies if you have many short email relationships.

Lots of TCP packets, too.  We don't really care about those because
they aren't pushed in our faces.  Cookies would be the same, given an
autoconfirmer.


paul

Re: new spam functionality added

[Ted Zlatanov]

On Wed, 31 Jul 2002, jas@extundo.com wrote:
> My only suggestion is: texi documentation ;-)

I'll be glad to work on that when spam.el has solidified a bit.  Right
now it needs more work (and testing).

> Another thought on documentation: if spam.el documentation is added,
> and the NoCeM node is moved, to the "Thwarting Email Spam" node, I
> think that node becomes long enough, and important enough, to
> warrant moving it up to the top-level menu.  Opinions?

I'm OK either way (leave as is or reparent the doc node).

Thanks
Ted