spam.el: asking about removing spam-use-BBDB

spam.el: asking about removing spam-use-BBDB

[Ted Zlatanov]

I'd like to remove the spam-use-BBDB processor in spam.el.

I believe it's hardly used and after testing it for a while, I believe
it's not as good as I thought originally.

Let me know if you disagree.

Thanks
Ted

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

> I'd like to remove the spam-use-BBDB processor in spam.el.
> I believe it's hardly used and after testing it for a while, I believe
> it's not as good as I thought originally.

I use bbdb as a whitelist and so set spam-use-BBDB-exclusive to t.  Works exactly as I want it to.  If there's an alternative way to use a whitelist I am not afraid to change.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

>> I'd like to remove the spam-use-BBDB processor in spam.el.
>> I believe it's hardly used and after testing it for a while, I believe
>> it's not as good as I thought originally.

> I use bbdb as a whitelist and so set spam-use-BBDB-exclusive to t.
> Works exactly as I want it to.  If there's an alternative way to use a
> whitelist I am not afraid to change.

Since updating, I've run into a couple problems.  First one I think I figured out, but I wonder if it might still have some relevance - I was seeing undefined function errors.  I fixed that by recompiling with a running Emacs that had bbdb loaded.  However, now it seems that the bbdb matching function is no longer matching addresses that are clearly in my bbdb.  Here's a snippets from *Messages*:

gnus-registry-post-process-group: no matches for sender 'info@hfcu.org'.
gnus-registry-post-process-group: no matches for recipients 'david <david.goldberg6@verizon.net>'.
spam-split: calling the spam-check-BBDB function
Wrote /home/dsg/.Mail/spam/22071

info@hfcu.org is definitely in my bbdb.

Would I just be better off exporting the bbdb to a whitelist file?

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: spam.el: asking about removing spam-use-BBDB

[Eric Abrahamsen]

On Sun, Oct 23 2011, Dave Goldberg wrote:

>>> I'd like to remove the spam-use-BBDB processor in spam.el.
>>> I believe it's hardly used and after testing it for a while, I believe
>>> it's not as good as I thought originally.
>
>> I use bbdb as a whitelist and so set spam-use-BBDB-exclusive to t.
>> Works exactly as I want it to.  If there's an alternative way to use a
>> whitelist I am not afraid to change.
>
> Since updating, I've run into a couple problems. First one I think I
> figured out, but I wonder if it might still have some relevance - I
> was seeing undefined function errors. I fixed that by recompiling with
> a running Emacs that had bbdb loaded. However, now it seems that the
> bbdb matching function is no longer matching addresses that are
> clearly in my bbdb. Here's a snippets from *Messages*:

It was probably `bbdb-search-simple' that was undefined: that function
disappeared between BBDB versions 2 and 3, and spam.el is still using
it. It's possible that by recompiling you ended up with a mish-mash of
BBDB versions: what does `bbdb-version' report?

Eric

-- 
GNU Emacs 24.0.90.1 (i686-pc-linux-gnu, GTK+ Version 2.24.4)
 of 2011-10-06 on pelletnNo Gnus v0.18

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

> On Sun, Oct 23 2011, Dave Goldberg wrote:
>>>> I'd like to remove the spam-use-BBDB processor in spam.el.
>>>> I believe it's hardly used and after testing it for a while, I believe
>>>> it's not as good as I thought originally.
>> 
>>> I use bbdb as a whitelist and so set spam-use-BBDB-exclusive to t.
>>> Works exactly as I want it to.  If there's an alternative way to use a
>>> whitelist I am not afraid to change.
>> 
>> Since updating, I've run into a couple problems. First one I think I
>> figured out, but I wonder if it might still have some relevance - I
>> was seeing undefined function errors. I fixed that by recompiling with
>> a running Emacs that had bbdb loaded. However, now it seems that the
>> bbdb matching function is no longer matching addresses that are
>> clearly in my bbdb. Here's a snippets from *Messages*:

> It was probably `bbdb-search-simple' that was undefined: that function
> disappeared between BBDB versions 2 and 3, and spam.el is still using
> it. It's possible that by recompiling you ended up with a mish-mash of
> BBDB versions: what does `bbdb-version' report?

No.  I use gnus from git, Emacs 23.1.1 from Ubuntu, and bbdb 2.36 from sourceforge.  There is no bbdb 3 on my system; I still use XEmacs at work and as far as I can tell there's been no progress or even interest in porting bbdb 3 to XEmacs.  I share my bbdb file between home and work and trying to manage that with the different formats is simply not worth my while.  While I'm interested in the work going on in 3.X, 2.36 has been meeting my needs to date.

I'm guessing that one of the changes to spam.el wrapped the bbdb autoloads in eval-when-compile (I wish I were sufficiently familiar with git as I am with cvs and svn so I could more easily check) and since the emacs run out of make in my checkout directory doesn't include bbdb, or at least my full load path, the autoloads aren't in the elc.  I ran into a similar situation with eudc, requiring me to maintain a local copy of eudcb-bbdb.el.

I have, on occasion, seen a situation in which replies from someone on my whitelist to someone else who is not end up in spam due to the split-with-parent functionality, but that is not the case this time as even root level messages (from a threading perspective, sorry for not being able to think of a better term) are mysteriously being sent to spam.  Even weirder, in the same split, I've seen one message from a sender properly filed to inbox and another sent to spam.

Anyway, Ted did say he wanted to eliminate the bbdb engine, and I'm not worth keeping it around since all I need is a whitelist.  I notice that spam.el has a whitelist capability.  It's trivial to turn bbdb into a list of email addresses using bbdb-display-layout-alist; I'll give that a try.

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: spam.el: asking about removing spam-use-BBDB

[Richard Riley]

david.goldberg6@verizon.net (Dave Goldberg) writes:

>
> Anyway, Ted did say he wanted to eliminate the bbdb engine, and I'm not worth
> keeping it around since all I need is a whitelist.  I notice that spam.el has a
> whitelist capability.  It's trivial to turn bbdb into a list of email addresses
> using bbdb-display-layout-alist; I'll give that a try.


Eliminate bbdb from what? From core? (fwiw, I migrated to bbdb 3 and it
works well with a couple of tweaks).

Re: spam.el: asking about removing spam-use-BBDB

[Ted Zlatanov]

On Mon, 24 Oct 2011 11:21:22 +0200 Richard Riley <rileyrg@googlemail.com> wrote: 

RR> david.goldberg6@verizon.net (Dave Goldberg) writes:
>> 
>> Anyway, Ted did say he wanted to eliminate the bbdb engine, and I'm not worth
>> keeping it around since all I need is a whitelist.  I notice that spam.el has a
>> whitelist capability.  It's trivial to turn bbdb into a list of email addresses
>> using bbdb-display-layout-alist; I'll give that a try.


RR> Eliminate bbdb from what? From core? (fwiw, I migrated to bbdb 3 and it
RR> works well with a couple of tweaks).

No, just the BBDB backend from spam.el.  But I'm keeping it.

Ted

Re: spam.el: asking about removing spam-use-BBDB

[Ted Zlatanov]

On Sun, 23 Oct 2011 10:18:01 +0800 Eric Abrahamsen <eric@ericabrahamsen.net> wrote: 

EA> It was probably `bbdb-search-simple' that was undefined: that function
EA> disappeared between BBDB versions 2 and 3, and spam.el is still using
EA> it. It's possible that by recompiling you ended up with a mish-mash of
EA> BBDB versions: what does `bbdb-version' report?

I thought I pushed out a fix for that.  Hmm.  It's annoying because
I've already gone to BBDB v3 so I can't test v2 easily.

On Sun, 23 Oct 2011 22:22:18 -0400 david.goldberg6@verizon.net (Dave Goldberg) wrote: 

DG> Anyway, Ted did say he wanted to eliminate the bbdb engine, and I'm
DG> not worth keeping it around since all I need is a whitelist.  I notice
DG> that spam.el has a whitelist capability.  It's trivial to turn bbdb
DG> into a list of email addresses using bbdb-display-layout-alist; I'll
DG> give that a try.

I'll support the BBDB backend to spam.el.  I just need to fix the
`bbdb-search-simple' bug, it should work fine otherwise.

Ted

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

> On Sun, 23 Oct 2011 10:18:01 +0800 Eric Abrahamsen
> <eric@ericabrahamsen.net> wrote:

EA> It was probably `bbdb-search-simple' that was undefined: that function
EA> disappeared between BBDB versions 2 and 3, and spam.el is still using
EA> it. It's possible that by recompiling you ended up with a mish-mash of
EA> BBDB versions: what does `bbdb-version' report?

> I thought I pushed out a fix for that.  Hmm.  It's annoying because
> I've already gone to BBDB v3 so I can't test v2 easily.

I'll be glad to be your guinea pig for that.  As I said, I have no intention of moving to bbdb 3 until either it supports XEmacs, or I find time to migrate off it at work.  I have a lot of custom code there that, initially anyway, doesn't port directly to GNU Emacs.

> On Sun, 23 Oct 2011 22:22:18 -0400 david.goldberg6@verizon.net (Dave
> Goldberg) wrote:

DG> Anyway, Ted did say he wanted to eliminate the bbdb engine, and I'm
DG> not worth keeping it around since all I need is a whitelist.  I notice
DG> that spam.el has a whitelist capability.  It's trivial to turn bbdb
DG> into a list of email addresses using bbdb-display-layout-alist; I'll
DG> give that a try.

> I'll support the BBDB backend to spam.el.  I just need to fix the
> `bbdb-search-simple' bug, it should work fine otherwise.

Well like I said, since a recent update, it's almost having the opposite effect as what I want with many messages from people in .bbdb going into the spam group.  I'm more than happy to try to help debug this, but I don't know where to begin. 

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

>> I'll support the BBDB backend to spam.el.  I just need to fix the
>> `bbdb-search-simple' bug, it should work fine otherwise.

> Well like I said, since a recent update, it's almost having the
> opposite effect as what I want with many messages from people in .bbdb
> going into the spam group.  I'm more than happy to try to help debug
> this, but I don't know where to begin.

Found it.

The recent change to spam.el does this:

              (if (fboundp 'bbdb-search)
                  (bbdb-search (bbdb-records) who) ;; v3
                (bbdb-search-simple nil who)) ;; v2

So it's assuming that if bbdb 2.* is in use, (fboundp 'bbdb-search) should return nil.  C-h f bbdb-search gives this:

   bbdb-search is a Lisp macro in `bbdb-com.el'.

   (bbdb-search records &optional name company net notes phone)
   [...]

I don't know if a macro is supposed to cause fboundp to return t or if I've somehow completely screwed up my emacs (23.1.1), or if there's some difference in bbdb 2.36 (sourceforge CVS) from what you expected to find,  but the above explains why bbdb look ups are not working - the search is not set up right.  bbdb-search-simple is a part of my bbdb, so for now, my trivial fix is:

(if (fboundp 'bbdb-search-simple)
    (bbdb-search-simple nil who) ;; v2
  (bbdb-search (bbdb-records) who)) ;; v3

Obviously I have no clue if that would continue to work as desired with bbdb v3, though.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: spam.el: asking about removing spam-use-BBDB

[Ted Zlatanov]

On Tue, 01 Nov 2011 21:14:29 -0400 david.goldberg6@verizon.net (Dave Goldberg) wrote: 

>>> I'll support the BBDB backend to spam.el.  I just need to fix the
>>> `bbdb-search-simple' bug, it should work fine otherwise.

>> Well like I said, since a recent update, it's almost having the
>> opposite effect as what I want with many messages from people in .bbdb
>> going into the spam group.  I'm more than happy to try to help debug
>> this, but I don't know where to begin.

DG> my trivial fix is:

DG> (if (fboundp 'bbdb-search-simple)
DG>     (bbdb-search-simple nil who) ;; v2
DG>   (bbdb-search (bbdb-records) who)) ;; v3

DG> Obviously I have no clue if that would continue to work as desired with bbdb v3, though.

It doesn't work exactly right, unfortunately.  I'll look at it when I
get a chance.  I'm really sorry for the inconvenience.

Ted

Re: spam.el: asking about removing spam-use-BBDB

[Ted Zlatanov]

On Thu, 03 Nov 2011 17:45:13 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Tue, 01 Nov 2011 21:14:29 -0400 david.goldberg6@verizon.net (Dave Goldberg) wrote: 
>>>> I'll support the BBDB backend to spam.el.  I just need to fix the
>>>> `bbdb-search-simple' bug, it should work fine otherwise.

>>> Well like I said, since a recent update, it's almost having the
>>> opposite effect as what I want with many messages from people in .bbdb
>>> going into the spam group.  I'm more than happy to try to help debug
>>> this, but I don't know where to begin.

DG> my trivial fix is:

DG> (if (fboundp 'bbdb-search-simple)
DG> (bbdb-search-simple nil who) ;; v2
DG> (bbdb-search (bbdb-records) who)) ;; v3

DG> Obviously I have no clue if that would continue to work as desired with bbdb v3, though.

TZ> It doesn't work exactly right, unfortunately.  I'll look at it when I
TZ> get a chance.  I'm really sorry for the inconvenience.

I comitted a fix that removes the whole `bbdb-search*' issue.  It also
avoids the BBDB autoloads.  Please test and let me know.

Simplest test to verify it pick knownemail@knownhost.com out of BBDB,
then:

(spam-exists-in-BBDB-p "knownemail@knownhost.com")

The fundamental issue is with the `bbdb-search' defmacro and
compile-time detection of BBDB.  I'm not good enough with that part of
Emacs to tell what's up, but I think I've taken the right (and faster)
approach.

Ted

Re: spam.el: asking about removing spam-use-BBDB

[Dave Goldberg]

> On Thu, 03 Nov 2011 17:45:13 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

> I comitted a fix that removes the whole `bbdb-search*' issue.  It also
> avoids the BBDB autoloads.  Please test and let me know.

> Simplest test to verify it pick knownemail@knownhost.com out of BBDB,
> then:

> (spam-exists-in-BBDB-p "knownemail@knownhost.com")

> The fundamental issue is with the `bbdb-search' defmacro and
> compile-time detection of BBDB.  I'm not good enough with that part of
> Emacs to tell what's up, but I think I've taken the right (and faster)
> approach.

It works, but only if I compile spam.el with bbdb loaded.  spam.elc created by the Makefile doesn't pick it up.  I get this backtrace when trying the test:

Debugger entered--Lisp error: (invalid-function bbdb-gethash)
  bbdb-gethash("david.goldberg6@verizon.net")
  spam-exists-in-BBDB-p("david.goldberg6@verizon.net")
  eval((spam-exists-in-BBDB-p "david.goldberg6@verizon.net"))
  eval-expression((spam-exists-in-BBDB-p "david.goldberg6@verizon.net") nil)
  call-interactively(eval-expression nil nil)

When compiled in my running emacs with bbdb loaded, the test returns the bbdb entry for that address.

I have to wonder if this is because I do not have bbdb installed in a particular location that gets picked up by emacs -batch, which i understand does not load .emacs.  When I used to use xemacs (still do at work, but don't use spam.el there because all filtering is on the server) I installed bbdb in a local "package" directory tree in my home directory.  Is there an equivalent to that with GNU Emacs I should be using?

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net