Agent as cache

Agent as cache

[ShengHuo ZHU]

I'll check in some codes which enable Gnus to cache NOV of any method
covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
and patches are welcome.

ShengHuo

Re: Agent as cache

[Bjørn Mork]

ShengHuo ZHU <zsh@cs.rochester.edu> writes:

> I'll check in some codes which enable Gnus to cache NOV of any method
> covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
> it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
> and patches are welcome.

I get "End of file during parsing" when I try to enter my nnimap INBOX
with a number prefix (e.g. "1 0 [enter]"):

Retrieving newsgroup: nnimap+imap.nextra.no:INBOX...
nnimap: Updating info for nnimap+imap.nextra.no:INBOX...done
Fetching headers for nnimap+imap.nextra.no:INBOX...
gnus-agent-braid-nov: 
End of file during parsing

Giving a large enough number, I am able to enter the group and view
already read articles. But the newest articles are missing from the
summary.  This group have some ticked articles, and probably lots of
holes in the numbering due to mail splitting and total-expire. I guess
these holes may cause problems. I don't see anything similar in other
groups.

(setq gnus-agent-cache nil) fixes the problem


Bjørn
-- 
Every unpleasant person is a peasant, right?  

Re: Agent as cache

[Harry Putnam]

ShengHuo ZHU <zsh@cs.rochester.edu> writes:

> I'll check in some codes which enable Gnus to cache NOV of any method
> covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
> it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
> and patches are welcome.
>
> ShengHuo

ShengHuo, what benefit will this provide?
What is the practical aim of this move?

Re: Agent as cache

[ShengHuo ZHU]

Harry Putnam <reader@newsguy.com> writes:

> ShengHuo ZHU <zsh@cs.rochester.edu> writes:
>
>> I'll check in some codes which enable Gnus to cache NOV of any method
>> covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
>> it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
>> and patches are welcome.
>>
>> ShengHuo
>
> ShengHuo, what benefit will this provide?
> What is the practical aim of this move?

The benefit is saving time and bandwidth, i.e. Gnus use the copy of
NOVs or articles in the agent if they are downloaded.

ShengHuo

Re: Agent as cache

[ShengHuo ZHU]

Bjørn Mork <bmork@dod.no> writes:

> ShengHuo ZHU <zsh@cs.rochester.edu> writes:
>
>> I'll check in some codes which enable Gnus to cache NOV of any method
>> covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
>> it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
>> and patches are welcome.
>
> I get "End of file during parsing" when I try to enter my nnimap INBOX
> with a number prefix (e.g. "1 0 [enter]"):
>
> Retrieving newsgroup: nnimap+imap.nextra.no:INBOX...
> nnimap: Updating info for nnimap+imap.nextra.no:INBOX...done
> Fetching headers for nnimap+imap.nextra.no:INBOX...
> gnus-agent-braid-nov: 
> End of file during parsing
>
> Giving a large enough number, I am able to enter the group and view
> already read articles. But the newest articles are missing from the
> summary.  This group have some ticked articles, and probably lots of
> holes in the numbering due to mail splitting and total-expire. I guess
> these holes may cause problems. I don't see anything similar in other
> groups.

Probably. Could you send me the backtrace and the article numbers (the
first column) in ~/News/agent/nnimap/imap.nextra.no/INBOX/.overview ?

ShengHuo

Re: Agent as cache

[Kai Großjohann]

ShengHuo ZHU <zsh@cs.rochester.edu> writes:

> Harry Putnam <reader@newsguy.com> writes:
>
>> ShengHuo ZHU <zsh@cs.rochester.edu> writes:
>>
>>> I'll check in some codes which enable Gnus to cache NOV of any method
>>> covered by gnus-agent.  Some behaviors of gnus-agent may be buggy.  If
>>> it bothers you, (setq gnus-agent-cache nil).  Of course, bug reports
>>> and patches are welcome.
>>
>> ShengHuo, what benefit will this provide?
>> What is the practical aim of this move?
>
> The benefit is saving time and bandwidth, i.e. Gnus use the copy of
> NOVs or articles in the agent if they are downloaded.

Ah, so it doesn't only pertain to the NOV data, but also to the
articles?  And this means that Gnus takes the Agentized copy of the
data even when online?  Way cool.

kai
-- 
Simplification good!  Oversimplification bad!  (Larry Wall)

Re: Agent as cache

[ShengHuo ZHU]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

[...]

> Ah, so it doesn't only pertain to the NOV data, but also to the
> articles?  And this means that Gnus takes the Agentized copy of the
> data even when online?  Way cool.

Yes. One more thing to clarify: when you read a group online
(plugged), Gnus saves the NOV data, but not the articles.  So, when
you go offline, you can still read the summary you've seen though you
didn't explicitly fetch the group. But the articles you read are not
saved in the agent directories.

ShengHuo

Re: Agent as cache

[Kai Großjohann]

ShengHuo ZHU <zsh@cs.rochester.edu> writes:

> But the articles you read are not saved in the agent directories.

I wonder what users will be screaming for next :-)

kai
-- 
Simplification good!  Oversimplification bad!  (Larry Wall)

Anyone using DCC to block spam? How to config Gnus based on X-DCC-* ?

[Chris Shenton]

I recently started trying to use DCC (Distributed Checksum
Clearinghouse, http://www.rhyolite.com/anti-spam/dcc/) to block spam. 
I've configged my MTA to pass my mail through the "dccproc" which adds
a header to the mail based on the counts of potential spam the
clearinghouse has seen.  Seems to be working well, some obvious spam
showed up today with this DCC header: 

  X-DCC-Etherboy-Metrics: thanatos.shenton.org 1002; env_From=1 From=1
	Subject=1 Message-ID=1 Received=1 Body=many Fuz1=many

indicating the same body was received by "many" DCC users, though the
From, Subject, and Message-ID were all different. Another DCC header:

  X-DCC-wanadoo-be-Metrics: thanatos.shenton.org 1016; env_From=1 From=1
	Subject=many Message-ID=1 Received=1 Body=many Fuz1=many

indicates less clever spam with the exact same Subject, as well as
Body.


So my question is how can I get Gnus to nuke or filter based on this
line? I realize I could do something like:

(setq nnmail-split-methods
      '(
	("in.spam"		"^X-DCC-.*Body=many")
        ...))

But seems that more complicated decisions would require truly hairy
regexps. 

More immediately is how to examine the entire header, even though it
runs across multiple lines.  Or is Gnus smart enough to grab the
entire header somewhere I can play with it? 

Any suggestions? 

Thanks.

Re: Agent as cache

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> ShengHuo ZHU <zsh@cs.rochester.edu> writes:
>
>> But the articles you read are not saved in the agent directories.
>
> I wonder what users will be screaming for next :-)

Agentized article to be teated like mail messages ... he he.

Re: Anyone using DCC to block spam? How to config Gnus based on X-DCC-* ?

[Kai Großjohann]

Chris Shenton <chris@shenton.org> writes:

> More immediately is how to examine the entire header, even though it
> runs across multiple lines.  Or is Gnus smart enough to grab the
> entire header somewhere I can play with it? 

Gnus converts each header to a single line before
nnmail-split-methods is invoked.  So ".*" does what you need.

kai
-- 
Simplification good!  Oversimplification bad!  (Larry Wall)

Re: Anyone using DCC to block spam? How to config Gnus based on X-DCC-* ?

[Daniel Pittman]

On 11 Jan 2002, Chris Shenton wrote:
> I recently started trying to use DCC (Distributed Checksum 
> Clearinghouse, http://www.rhyolite.com/anti-spam/dcc/) to block 
> spam.  [...]  X-DCC-Etherboy-Metrics: thanatos.shenton.org 1002; 
> env_From=1 From=1 Subject=1 Message-ID=1 Received=1 Body=many 
> Fuz1=many
>
> indicating the same body was received by "many" DCC users, 
> though the From, Subject, and Message-ID were all different.  
> Another DCC header:
>
>   X-DCC-wanadoo-be-Metrics: thanatos.shenton.org 1016; 
>   env_From=1 From=1 Subject=many Message-ID=1 Received=1 
>   Body=many Fuz1=many
>
> indicates less clever spam with the exact same Subject, as well 
> as Body.

Ah, how suitably clever. It actually uses a different header for 
each
record. What a /useful/ feature. (Ahem. I feel better now.)

> So my question is how can I get Gnus to nuke or filter based on 
> this line? I realize I could do something like:
>
> (setq nnmail-split-methods '( ("in.spam" "^X-DCC-.*Body=many") 
> ...))
>
> But seems that more complicated decisions would require truly 
> hairy regexps.

Well, that works for splitting, and it's going to be a PITA to get 
it to
match. IIRC, you can use a function that gets called to do some of 
the
splitting, though, which may well be easier. Check the info.

...personally, though, I would tend to use this header in scoring 
and
mark up or down based on the appropriate bits of the header, in
all.SCORE.

        Daniel

--
Love is like Hearts. You want to follow suit, but you don't want
to have the lead.  -- Dustin Fisher

Re: Anyone using DCC to block spam? How to config Gnus based on X-DCC-* ?

[Chris Shenton]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Gnus converts each header to a single line before
> nnmail-split-methods is invoked.  So ".*" does what you need.

Beauty, this will do what I need, at least for simple regexp matches.

Many thanks!

Howto on using DCC with qmail and Gnus [was: Anyone using DCC to block spam? How to config Gnus based on X-DCC-* ?]

[Chris Shenton]

Jack Twilley <jmt@twilley.org> writes:

> If you were to write up a web page with pointers to DCC and including
> your .gnus lines to support DCC, that would do wonders to help Gnus
> users like me from considering using it. :-)

OK, I did. :-)

http://chris.shenton.org/sysadm/antispam/dcc-qmail-gnus.html

It's a quick-n-dirty approach but seems to be working fine.  It can be
made more "serious" later if this is inadequate.

I hope this helps.