Regarding spam.el

Regarding spam.el

[Jonas Steverud]

1. Is gnus-group-*-exit-processor-bogofilter obsolete? If so, it
   should be better marked as such in the info file. Same for all
   similar obsolete variables.

2. In "Spam ELisp Package Sequence of Events": Second half of fifth
       paragraph: The `spam-autodetect' and
       `spam-autodetect-methods'... Yes, what about them? First half
       of the paragrapg is clear, then a bunch of parameters and
       varaibles comes out of the blue. What are they used for? What
       can they be set to? The reader only gets confused. Either a
       reference to a info node or explaination text is needed.
       The paragraph after explains it but not very clear. An example
       woul be very nice.

3. There are several group parameters and variables mentioned in the
   info file and it is not always obvious which is which. A separate
   page/node or at the end of one of the nodes with a list of all
   variables and the corresponding parameters with examples would be a
   good idea.

4. How do I tell Gnus that nnfolder:Spam is my spam group and that all
   other groups are ham? There a number of references to a
   spam-contents variable/parameter but I got all confused trying to
   figure out what to set it to. And where. Several parts of the info
   nodes only lists what parameters and variables there are but not
   what I set them to, what values that are valid.

5. The following paragraph from "Spam ELisp Package Global Variables"
   is typical for the entire documentation:

   "When you leave a _spam_ group, all spam-marked articles are marked
as expired after processing with the spam processor.  This is not done
for _unclassified_ or _ham_ groups.  Also, any *ham* articles in a spam
group will be moved to a location determined by either the
`ham-process-destination' group parameter or a match in the
`gnus-ham-process-destinations' variable, which is a list of regular
expressions matched with group names (it's easiest to customize this
variable with `customize-variable gnus-ham-process-destinations').  Each
newsgroup specification has the format (REGEXP PROCESSOR) in a standard
Lisp list, if you prefer to customize the variable manually.  The
ultimate location is a group name or names.  If the
`ham-process-destination' parameter is not set, ham articles are left
in place.  If the `spam-mark-ham-unread-before-move-from-spam-group'
parameter is set, the ham articles are marked as unread before being
moved."

   a. The author gets sidetracked by his own willingness to explain
      and help the reader, e.g. the "(it's easiest to customize...)"
      part. It is not uncommon that when the author explains what
      something is, he then tells the reader how to set the
      variable/parameter ("use M-x customize-..." or "...G c or G p as
      usual"). These lengthy side notes cuts up the flow of the text
      and starts to annoy the reader when he reads it for the
      umpteenth time. Better to state in the beginning of the Spam
      nodes that these fascillities can be used and when a reminder is
      needed, add a short reminder that doesn't side track the reader.

   b. The explaination of (gnus-)ham-process-destinations is not
      complete and even confusing. It is a list of regexp matched with
      group names. In the next sentance it is declared as a (REGEXP
      PROCESSOR). That is not a list of regexps, it is a list of lists
      that contains a regexp and a PROCESSOR. What this PROCESSOR is
      not explained.

   c. The given information is not consistent with information given a
      bit further down the page:

   "When you leave a _ham_ or _unclassified_ group, all *spam* articles
are moved to a location determined by either the
`spam-process-destination' group parameter or a match in the
`gnus-spam-process-destinations' variable, which is a list of regular
expressions matched with group names (it's easiest to customize this
variable with `customize-variable gnus-spam-process-destinations').
Each newsgroup specification has the repeated format (REGEXP GROUP) and
they are all in a standard Lisp list, if you prefer to customize the
variable manually.  The ultimate location is a group name or names.  If
the `spam-process-destination' parameter is not set, the spam articles
are only expired.  The group name is fully qualified, meaning that if
you see `nntp:servername' before the group name in the group buffer
then you need it here as well."

      Here the parameter is a list of (REGEXP GROUP). What is the
      REGEXP matched on?

   d. In the example section gives the following example:
      ((spam-process-destination . "nnimap+mail.lifelogs.com:train"))
      When did a string become a (REGEXP PROCESSOR) or (REGEXP GROUP)?
      Same goes for
      (ham-process-destination "nnimap+mail.lifelogs.com:mail"
                               "nnimap+mail.lifelogs.com:trainham")

Summary:
 The entire documentation needs to be fleshed out with examples and
 a better flow of text. At some places it is more or less just a long
 list of variable and parameter names which is as penetrable as a
 brick wall.

I'm willing to help with this, but I need to understand how it works
before I can do anything about it.

-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )

Re: Regarding spam.el

[Russ Allbery]

Jonas Steverud <tvrud@bredband.net> writes:

> 1. Is gnus-group-*-exit-processor-bogofilter obsolete? If so, it
>    should be better marked as such in the info file. Same for all
>    similar obsolete variables.

Well, at least in 5.10.6, it doesn't appear to be possible to do without
them.  I have the following in my initialization file:

 '(gnus-parameters
   '(("^nnml\\+sent:"
      (gnus-use-scoring nil)
      (gnus-show-threads nil))
     ("^nnml:"
      (gnus-use-scoring nil))
     ("^nnml:lists\\."
      (gnus-use-scoring t))
     ("^nnml:mail\\.\\(spam.*\\|misc\\)"
      (spam-contents gnus-group-spam-classification-spam)
      (spam-process (gnus-group-spam-exit-processor-bogofilter
                     gnus-group-ham-exit-processor-bogofilter)))
     ("^nnml:\\(mail\\.\\(eyrie\\|rra\\|bounces\\|root\\)\\|project\\..*\\)"
      (gnus-use-scoring t)
      (spam-contents gnus-group-spam-classification-ham)
      (spam-process (gnus-group-spam-exit-processor-bogofilter
                     gnus-group-ham-exit-processor-bogofilter)))
     ("^nnml:work\\.\\(personal\\|ls\\|news.*\\)"
      (spam-contents gnus-group-spam-classification-ham)
      (spam-process (gnus-group-spam-exit-processor-bogofilter
                     gnus-group-ham-exit-processor-bogofilter)))))

If I replace those instances of gnus-group-*-exit-processor-bogofilter
with '(spam spam-use-bogofilter) and the like as described in the manual,
exit processing no longer happens and nothing gets registered with
bogofilter.  I tried a few variations with and without the ' and the ()s
and couldn't get it to work.

Maybe I'm missing something?

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: Regarding spam.el

[Jonas Steverud]

Russ Allbery <rra@stanford.edu> writes:

> Well, at least in 5.10.6, it doesn't appear to be possible to do without
> them.  I have the following in my initialization file:

Sorry, I should have mentioned that I use No Gnus v0.1 nowadays. Not
that it matters according to your comment below.

[...]
> If I replace those instances of gnus-group-*-exit-processor-bogofilter
> with '(spam spam-use-bogofilter) and the like as described in the manual,
> exit processing no longer happens and nothing gets registered with
> bogofilter.

So what you're saying is that the documentation is *ahead* of the
code, and not, as usually is the case, lagging? What a strange world
it is... ;-)

-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )

Re: Regarding spam.el

[Ted Zlatanov]

On Wed, 10 Mar 2004, tvrud@bredband.net wrote:

> Russ Allbery <rra@stanford.edu> writes:
> 
>> If I replace those instances of
>> gnus-group-*-exit-processor-bogofilter with '(spam
>> spam-use-bogofilter) and the like as described in the manual, exit
>> processing no longer happens and nothing gets registered with
>> bogofilter.
> 
> So what you're saying is that the documentation is *ahead* of the
> code, and not, as usually is the case, lagging? What a strange world
> it is... ;-)

As the saying goes, if the code and docs disagree, they are both
wrong :)

Ted

Re: Regarding spam.el

[Ted Zlatanov]

On Tue, 09 Mar 2004, rra@stanford.edu wrote:

> Jonas Steverud <tvrud@bredband.net> writes:
> 
>> 1. Is gnus-group-*-exit-processor-bogofilter obsolete? If so, it
>>    should be better marked as such in the info file. Same for all
>>    similar obsolete variables.
> 
> Well, at least in 5.10.6, it doesn't appear to be possible to do
> without them.
[...]
> If I replace those instances of
> gnus-group-*-exit-processor-bogofilter with '(spam
> spam-use-bogofilter) and the like as described in the manual, exit
> processing no longer happens and nothing gets registered with
> bogofilter.  I tried a few variations with and without the ' and the
> ()s and couldn't get it to work.

No, this is what should work.  The spam-group-processor-p and
spam-group-processor-multiple-p do this, and if you can check the
former to see if you can trace the bug, it would be helpful.  I'll
take a look as well when I get a chance.

Ted

Re: Regarding spam.el

[Ted Zlatanov]

On Tue, 09 Mar 2004, tvrud@bredband.net wrote:

> 1. Is gnus-group-*-exit-processor-bogofilter obsolete? If so, it
>    should be better marked as such in the info file. Same for all
>    similar obsolete variables.

Those variables are obsolete, but I'm keeping them until the next
major release of Gnus.  The ({spam|ham} spam-use-BACKEND) format will
replace the processor variables.  How should these deprecations be
marked, in bulk (a single note) or individually?  I'll add them.

> 2. In "Spam ELisp Package Sequence of Events": Second half of fifth
>        paragraph: The `spam-autodetect' and
>        `spam-autodetect-methods'... Yes, what about them? 

That's a good catch.  The sentence ran on for so long, I forgot it
was supposed to say something :)  Fixed.

> First half of the paragrapg is clear, then a bunch of parameters and
> varaibles comes out of the blue. 

Well yes, I have to introduce them at some point :)

> What are they used for? 

I think that's explained clearly.  Feel free to suggest alternate wording.

> What can they be set to?

That's what the customize interface is for.  I don't want to explain
every possible option - as you noticed, the information is very dense
already.

> The reader only gets confused. Either a reference to a info node or
> explaination text is needed.  The paragraph after explains it but
> not very clear. An example woul be very nice.

My setup in "Spam ELisp Package Configuration Examples" uses
spam-autodetect.  I welcome any other fixes to the docs - pointers,
more examples, rewordings.

> 
> 3. There are several group parameters and variables mentioned in the
>    info file and it is not always obvious which is which. A separate
>    page/node or at the end of one of the nodes with a list of all
>    variables and the corresponding parameters with examples would be
>    a good idea.

I'm not sure at all what you mean here.

> 4. How do I tell Gnus that nnfolder:Spam is my spam group and that
>    all other groups are ham? There a number of references to a
>    spam-contents variable/parameter but I got all confused trying to
>    figure out what to set it to. And where. Several parts of the
>    info nodes only lists what parameters and variables there are but
>    not what I set them to, what values that are valid.

Have you tried the customize interface to the
gnus-spam-newsgroup-contents variable?  You just need to set
nnfolder:Spam as 'spam, and AFTERWARDS in the list you need to set .*
as 'ham.

Using `G c' on a group/topic will show you all the spam-related
parameters.  Each one has a corresponding global variable, as defined
in gnus.el (but I don't know of a way to link the variable
customization to the parameter customization).

> 5. The following paragraph from "Spam ELisp Package Global
>    Variables" is typical for the entire documentation:
> 
>    "When you leave a _spam_ group, all spam-marked articles are
>    marked as expired after processing with the spam processor.  This
>    is not done for _unclassified_ or _ham_ groups.  Also, any *ham*
>    articles in a spam group will be moved to a location determined
>    by either the `ham-process-destination' group parameter or a
>    match in the `gnus-ham-process-destinations' variable, which is a
>    list of regular expressions matched with group names (it's
>    easiest to customize this variable with `customize-variable
>    gnus-ham-process-destinations').  Each newsgroup specification
>    has the format (REGEXP PROCESSOR) in a standard Lisp list, if you
>    prefer to customize the variable manually.  The ultimate location
>    is a group name or names.  If the `ham-process-destination'
>    parameter is not set, ham articles are left in place.  If the
>    `spam-mark-ham-unread-before-move-from-spam-group' parameter is
>    set, the ham articles are marked as unread before being moved."
> 
>    a. The author gets sidetracked by his own willingness to explain
>       and help the reader, e.g. the "(it's easiest to customize...)"
>       part. It is not uncommon that when the author explains what
>       something is, he then tells the reader how to set the
>       variable/parameter ("use M-x customize-..." or "...G c or G p
>       as usual"). These lengthy side notes cuts up the flow of the
>       text and starts to annoy the reader when he reads it for the
>       umpteenth time. Better to state in the beginning of the Spam
>       nodes that these fascillities can be used and when a reminder
>       is needed, add a short reminder that doesn't side track the
>       reader.

A lot of readers jump to the section of interest.  I think it's fair
to expect that, and to provide help there.  If you would like to
suggest an alternate way to accomodate both careful and
jump-to-point-of-interest readers, I'm all ears.

>    b. The explaination of (gnus-)ham-process-destinations is not
>       complete and even confusing. It is a list of regexp matched
>       with group names. In the next sentance it is declared as a
>       (REGEXP PROCESSOR). That is not a list of regexps, it is a
>       list of lists that contains a regexp and a PROCESSOR. What
>       this PROCESSOR is not explained.

I added a fix for this error, thank you.

>    c. The given information is not consistent with information given
>       a bit further down the page:
> 
>    "When you leave a _ham_ or _unclassified_ group, all *spam*
>    articles are moved to a location determined by either the
>    `spam-process-destination' group parameter or a match in the
>    `gnus-spam-process-destinations' variable, which is a list of
>    regular expressions matched with group names (it's easiest to
>    customize this variable with `customize-variable
>    gnus-spam-process-destinations').  Each newsgroup specification
>    has the repeated format (REGEXP GROUP) and they are all in a
>    standard Lisp list, if you prefer to customize the variable
>    manually.  The ultimate location is a group name or names.  If
>    the `spam-process-destination' parameter is not set, the spam
>    articles are only expired.  The group name is fully qualified,
>    meaning that if you see `nntp:servername' before the group name
>    in the group buffer then you need it here as well."
> 
>       Here the parameter is a list of (REGEXP GROUP). What is the
>       REGEXP matched on?

Reread the explanations of gnus-{spam,ham}-process-destinations and
see if they make more sense after my fixes.

>    d. In the example section gives the following example:
>       ((spam-process-destination .
>       "nnimap+mail.lifelogs.com:train")) When did a string become a
>       (REGEXP PROCESSOR) or (REGEXP GROUP)?  Same goes for
>       (ham-process-destination "nnimap+mail.lifelogs.com:mail"
>       "nnimap+mail.lifelogs.com:trainham")

That's the group/topic parameter, whereas the paragraphs above were
talking about the gnus-{spam,ham}-process-destinations variables.
With the group/topic parameters, the regexp is implied and we only
need the group name(s).

> Summary:
>  The entire documentation needs to be fleshed out with examples and
>  a better flow of text. At some places it is more or less just a
>  long list of variable and parameter names which is as penetrable as
>  a brick wall.

I agree.  spam.el is very complex, and I would love to a) improve the
documentation, and b) write a new user customization interface to it.

> I'm willing to help with this, but I need to understand how it works
> before I can do anything about it.

Thanks for the offer.

There is one reason why I'm not good at documenting spam.el: I know it
too well.  Where someone else might say, "wait, why is this variable
mentioned here" or "what does that parameter mean?" I can miss those
spots.  So it's not intentional that the documentation is confusing,
and I want to make it better.

If you are interested in helping, I'll be glad to incorporate your
changes into the manual.

Thanks
Ted

Re: Regarding spam.el

[Jonas Steverud]

Ted Zlatanov <tzz@lifelogs.com> writes:


(I only saw your mails recently due to an accident with
gnus-split-with-parent,  which is ridiculous since I was keeping an eye
out for them. :-/)

> On Tue, 09 Mar 2004, tvrud@bredband.net wrote:
>
>> 1. Is gnus-group-*-exit-processor-bogofilter obsolete? If so, it
>>    should be better marked as such in the info file. Same for all
>>    similar obsolete variables.
>
> Those variables are obsolete, but I'm keeping them until the next
> major release of Gnus.  The ({spam|ham} spam-use-BACKEND) format will
> replace the processor variables.  How should these deprecations be
> marked, in bulk (a single note) or individually?  I'll add them.

Individually IMHO.

Maybe something like this:

 - Variabel: gnus-group-spam-exit-processor-bogofilter (obsolete)

That way the user can skip the entire description and not find out
afterwards that it is obsolete. When I read it I know some variables
are obsolete and since I don't remember which I have to scan downward
each time I reach a variable to see if it is obsolete or not. It
definitely reduces the reading speed and adds a bit of confusion.

> I think that's explained clearly.  Feel free to suggest alternate wording.

See below.

>> What can they be set to?
>
> That's what the customize interface is for.  I don't want to explain
> every possible option - as you noticed, the information is very dense
> already.

Yes, and I think that is part of the problem - the information is very
dense. Writing documentation is not a contest in writing with as few
words as possible (not in any way claiming that you participate in
such a contest ;-) ), it is actually a contest in writing more words
then you think is necessary. There are extremes off course as any
university student can testify about, some authors are payed by the
page number. I've written quite a few documents in my time (does ISO
9000 and ISO 9001 ring a bell?) and in my experience, there
are numerous occasions when one think that "this is enough" - and that
is exactly not the case. One should always write ten per cent more
then one think is necessary.

What the documentation really needs is more words.

E.g. the sequence of events needs a list of the stages at the beginning,
i.e. 1) Stage A, 2) Stage B etc., and then each stage is describes in
a sub topic of its own if they become lengthy or at least under a
headline. OTOH the nodes in a info file are very short so it shouldn't
be a problem to have a node for each stage.


>> The reader only gets confused. Either a reference to a info node or
>> explaination text is needed.  The paragraph after explains it but
>> not very clear. An example woul be very nice.
>
> My setup in "Spam ELisp Package Configuration Examples" uses
> spam-autodetect. 

Yes, and that is very good! Maybe include a pointer to that section?

In general, the examples in "this is my setup" are usually quite
complex (which is good!) but to understand the concept they need to be
very simple.

I.e. include a very simple example where you describe the
variable/concept: "To do Foo, set the bar variable to X, i.e. add the
following line of code to the group parameter: (BAR . X)" where Foo is
a very simple thing. Have a look at the examples for e.g. splitting
mail, they are usually very simple but very descriptive.

Sorry, but I have no specific suggestions.


>> 3. There are several group parameters and variables mentioned in the
>>    info file and it is not always obvious which is which. A separate
>>    page/node or at the end of one of the nodes with a list of all
>>    variables and the corresponding parameters with examples would be
>>    a good idea.
>
> I'm not sure at all what you mean here.

What I tried to say is that in some places a number of parameters and
variables is introduced ("blah X blah Y blah Z blah") but it is not
clear if X is a parameter or a variable. If X is a parameter, is then
Y and Z it as well or can it be so that X and Z are parameters and Y
is a variable? If X is a parameter, what is then the corresponding
global variable (because it is mentioned that for each parameter,
there is a variable, but no hint of naming scheme)?

E.g. from the sequence of events:
------
   If a spam is found in any group (this can be changed to only non-spam
groups with `spam-move-spam-nonspam-groups-only'), it is processed by
the active `spam-processors' (*note Spam ELisp Package Global
Variables::) when the group is exited.  Furthermore, the spam is moved
------

Is spam-move-spam-nonspam-groups-only a parameter or a variable? I
have to look at the example to
know. spam-move-spam-nonspam-groups-only is only mentioned twice,
above and in the example. I will not be able to read the docs with C-h v 
unless I load the spam.el package. Which I probably won't do before I
understood that I have to do that and how I will be able to do it. I
will not load it before I've understood what will happen when I do. If
you catch my drift (or however the expression goes).

> Have you tried the customize interface [...]

Nope, I prefer the old fashioned way of typing Lisp one-liners on a
neural interface... ;-)

Besides, my aim with this is to "debug" the manual, to try to find
the parts which confuses me. Using the customize interface is not
exactly cheating, but isn't exactly right either.

(And I only used the customize interface once before for JDEE and I
can't say that I know how to handle it.)


> A lot of readers jump to the section of interest.

Yes, I am like that myself. But when one reads something for the first
time it is a good idea to start on the first page and read then in
sequence. You have to know that something is of interest before you
can jump there, so to speak. ;-)

> Reread the explanations of gnus-{spam,ham}-process-destinations and
> see if they make more sense after my fixes.

Which are in CVS I expect. I've decided that I will use the released
versions of No Gnus, so I will see the new text when 0.02 is released.

>>    d. In the example section gives the following example:
>>       ((spam-process-destination .
>>       "nnimap+mail.lifelogs.com:train")) When did a string become a
>>       (REGEXP PROCESSOR) or (REGEXP GROUP)?  Same goes for
>>       (ham-process-destination "nnimap+mail.lifelogs.com:mail"
>>       "nnimap+mail.lifelogs.com:trainham")
>
> That's the group/topic parameter, whereas the paragraphs above were
> talking about the gnus-{spam,ham}-process-destinations variables.

See my comment above about distinguishing between parameters and variables.

> There is one reason why I'm not good at documenting spam.el: I know it
> too well.  Where someone else might say, "wait, why is this variable
> mentioned here" or "what does that parameter mean?" I can miss those
> spots.  So it's not intentional that the documentation is confusing,
> and I want to make it better.

That is way I send these emails. I want to help with that, to point
out the places that needs to be fleshed out. I might not be able with
the text to write yet since I need to understand more of it before,
but I hope my questions and points helps you understand how others
read the documentation.

(I better add a disclaimer at once: If I ever sounds like I'm ranting,
 that is not my intention. :-) )

Enjoy.

/Jonas 
-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )