[Q] stripping banners at the start of messages

[Q] stripping banners at the start of messages

[Steinar Bang]

Platform: Gnus v5.8.8 (CVS checkout), XEmacs 20.4, linux

Some mailing lists have started putting those annoying banners at the
start of the messages instead of at the end.

I have tried getting rid of banners like this:

-------------------------- eGroups Sponsor -------------------------~-~>
Tired of searching for an apartment?  Try Viva!  It's fast and free.
Now serving Dallas, Houston, Austin, Phoenix, Denver and Las Vegas.
Rent by September 30th - get $100 cash!
http://click.egroups.com/1/9154/16/_/461700/_/968288860/
---------------------------------------------------------------------_->

with the following setting in the group parameters of the mail group
handling the list:

 (banner . "\n-------------------------- eGroups Sponsor -------------------------~-~>\n.*\n.*\n.*\n.*\n---------------------------------------------------------------------_->\n")

Is it me who has gotten the syntax of the banner setting wrong?  Or
doesn't banner suppresion work on banners not at the end of the
messages? 

Thanx!


- Steinar

Re: [Q] stripping banners at the start of messages

[Emerick Rogul]

Steinar Bang writes:

: Platform: Gnus v5.8.8 (CVS checkout), XEmacs 20.4, linux
: Some mailing lists have started putting those annoying banners at the
: start of the messages instead of at the end.

: I have tried getting rid of banners like this:

: -------------------------- eGroups Sponsor -------------------------~-~>
: Tired of searching for an apartment?  Try Viva!  It's fast and free.
: Now serving Dallas, Houston, Austin, Phoenix, Denver and Las Vegas.
: Rent by September 30th - get $100 cash!
: http://click.egroups.com/1/9154/16/_/461700/_/968288860/
---------------------------------------------------------------------_-: 

: with the following setting in the group parameters of the mail group
: handling the list:

:  (banner . "\n-------------------------- eGroups Sponsor -------------------------~-~>\n.*\n.*\n.*\n.*\n---------------------------------------------------------------------_->\n")

: Is it me who has gotten the syntax of the banner setting wrong?  Or
: doesn't banner suppresion work on banners not at the end of the
: messages? 

I use the "banner" setting to get rid of the eGroups Sponsor messages
too, and it seems to work fine whether they're at the beginning or end
of the message.  Be aware that eGroups periodically changes the format
of the message (how annoying...), so you may need to tweak it every
once in a while.  I distinctly remember that when they moved it to the
top they also changed the format a little, which tricked me into
thinking that "banner" wouldn't work if it wasn't at the end of a
message.

Try building your banner up line by line instead of all at once,
too.  You may see an error in your regex.  Here's the regex I use for
my eGroups group:

(banner . "^--------------------------------------------------------------------<e|-\n\\(.\\|\n\\)*--------------------------------------------------------------------|e>-\n"))

-Emerick
-- 
-------------------------------------------------------------------------
Emerick Rogul        /\/  "i wouldn't want to live in a world that had
emerick@cs.bu.edu    /\/   seen my genitalia. you've got no trump card."
-------------------------------------------------------- janeane garofalo

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Steinar Bang <sb@metis.no> writes:

> I have tried getting rid of banners like this:
> 
> -------------------------- eGroups Sponsor -------------------------~-~>
> Tired of searching for an apartment?  Try Viva!  It's fast and free.
> Now serving Dallas, Houston, Austin, Phoenix, Denver and Las Vegas.
> Rent by September 30th - get $100 cash!
> http://click.egroups.com/1/9154/16/_/461700/_/968288860/
> ---------------------------------------------------------------------_->
> 
> with the following setting in the group parameters of the mail group
> handling the list:
> 
>  (banner . "\n-------------------------- eGroups Sponsor -------------------------~-~>\n.*\n.*\n.*\n.*\n---------------------------------------------------------------------_->\n")


I'm having exactly the same problem (eGroups banner).  I can get rid
of one line okay (by customizing the group and putting 
-------------------------- eGroups Sponsor -------------------------~-~>\n
as the banner, but multiple-lines don't work.  Oddly, though, the code
in article-strip-banner which deletes the banner is

(while (re-search-forward banner nil t)
	      (delete-region (match-beginning 0) (match-end 0)))

and if I edit the article and eval this via M-: and the explicit
banner, it all goes.  Is there some way I should be escaping newlines
in the regexp?

-- 
. Paul Stevenson . Reasearch Fellow . University of Surrey .
I dream in beige

Re: [Q] stripping banners at the start of messages

[Kai Großjohann]

On 25 Sep 2000, Paul Stevenson wrote:

> Is there some way I should be escaping newlines in the regexp?

Maybe you need to type C-q C-j to enter a newline character, rather
than "\n"?

kai
-- 
I like BOTH kinds of music.

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Jonas Steverud <d4jonas@dtek.chalmers.se> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > On 25 Sep 2000, Paul Stevenson wrote:
> > 
> > > Is there some way I should be escaping newlines in the regexp?
> > 
> > Maybe you need to type C-q C-j to enter a newline character, rather
> > than "\n"?
> 
> Nope, see my previuos post today.

Interestingly, I was originally trying to use the regexp for the
banner via group customization, where the literal \n didn't work.
Having used Jonas' fine regexp (with \n not C-q C-j), inserted in the
Group parameter buffer, I notice it shows up in the customization
buffer as if C-q C-j had be tapped.  Hmm.

-- 
. Paul Stevenson . Reasearch Fellow . University of Surrey .
I dream in beige

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Jonas Steverud <d4jonas@dtek.chalmers.se> writes:

> Side note: I have never used the customization - I do things in pure
> elisp. It might be so that in customization you have to do C-q C-j but
> in elisp \n is sufficent. Sorry for my misunderstanding.

I would probably use the parameter setting more often if it were bound
to a key combination like the customization is! (or is it? - the menu
suggests not)

-- 
. Paul Stevenson . Reasearch Fellow . University of Surrey .
I dream in beige

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Jonas Steverud <d4jonas@dtek.chalmers.se> writes:

> [...]
> 
> G p  on the group in the group buffer.

Mmm. That makes sense.  Ought someone with CVS access put the key
combination on the menu?

-- 
. Paul Stevenson . Reasearch Fellow . University of Surrey .
I dream in beige

Re: [Q] stripping banners at the start of messages

[Kai Großjohann]

On 26 Sep 2000, Jonas Steverud wrote:

> Side note: I have never used the customization - I do things in pure
> elisp. It might be so that in customization you have to do C-q C-j
> but in elisp \n is sufficent. Sorry for my misunderstanding.

When the Lisp reader reads a string, \n is converted to a newline
character, and the regex engine then sees the newline character.

But strings entered into Customize aren't passed through the Lisp
reader first, so you need to input the newline character directly.

kai
-- 
I like BOTH kinds of music.

Re: [Q] stripping banners at the start of messages

[Toby Speight]

0> In article <vafitrjuqmb.fsf@lucy.cs.uni-dortmund.de>,
0> Kai Großjohann <URL:mailto:Kai.Grossjohann@CS.Uni-Dortmund.DE> ("Kai") wrote:

Kai> But strings entered into Customize aren't passed through the
Kai> Lisp reader first, so you need to input the newline character
Kai> directly.

Unless you use Customize's "Show as Lisp expression" feature.  :-)

Re: [Q] stripping banners at the start of messages

[Matt Pharr]

Jonas Steverud <d4jonas@dtek.chalmers.se> writes:
> My regexp which works well (for the time being...) is
> 
>  (banner . "^[ ,-,_,-]+eGroups Sponsor\\(.*\n\\)\\(.*\n\\)\\(.*\n\\)\\(.*\n\\)\\(.*\n\\)?\\(.*\n\\)?[ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-][ ,-,_,-].*\n")

Since there's a lot of arcane magic in figuring out these multi-line
regexps (I'd never be able to put together magic like the above, much as I
hate the egroups banners), I've got a suggested patch that I just cooked
up.

The basic idea is that in addition to the old "banner" group parameter,
there are now new banner-start and banner-end parameters.  A new
article-treat-banner function (see below) also looks for these two new
parameters; if both are present, it will nuke all text between the start of
the first regexp and the end of the second one.  This makes for much easier
configurability.

For example, for eGroups lists, I've got these set like this:

((banner-start . "^----.*eGroups")
 (banner-end . "^----.*>"))

That's all it takes.

What do people think about this?  I'm also no major elisp hacker, so if I
coded it up poorly, I'm happy to hear about it.  If this seems to be a
generally useful thing, I'll write up some documentation for the manual and
commit it all to the CVS repository.

-matt

*** gnus-art.el.orig	Mon Sep 25 12:47:09 2000
--- gnus-art.el	Mon Sep 25 12:56:57 2000
***************
*** 769,775 ****
  
  (defcustom gnus-treat-strip-banner t
    "Strip banners from articles.
! The banner to be stripped is specified in the `banner' group parameter.
  Valid values are nil, t, `head', `last', an integer or a predicate.
  See the manual for details."
    :group 'gnus-article-treat
--- 769,776 ----
  
  (defcustom gnus-treat-strip-banner t
    "Strip banners from articles.
! The banner to be stripped is specified in the `banner', and/or `banner-start' 
! and `banner-end' group parameters.
  Valid values are nil, t, `head', `last', an integer or a predicate.
  See the manual for details."
    :group 'gnus-article-treat
***************
*** 1649,1673 ****
  	     (match-beginning 0) (match-end 0) 'pem)))))))
  
  (defun article-strip-banner ()
!   "Strip the banner specified by the `banner' group parameter."
    (interactive)
    (save-excursion
      (save-restriction
        (let ((inhibit-point-motion-hooks t)
  	    (banner (gnus-group-get-parameter gnus-newsgroup-name 'banner))
  	    (gnus-signature-limit nil)
! 	    buffer-read-only beg end)
! 	(when banner
! 	  (article-goto-body)
! 	  (cond
! 	   ((eq banner 'signature)
! 	    (when (gnus-article-narrow-to-signature)
! 	      (widen)
! 	      (forward-line -1)
! 	      (delete-region (point) (point-max))))
! 	   ((stringp banner)
! 	    (while (re-search-forward banner nil t)
! 	      (delete-region (match-beginning 0) (match-end 0))))))))))
  
  (defun article-babel ()
    "Translate article using an online translation service."
--- 1650,1683 ----
  	     (match-beginning 0) (match-end 0) 'pem)))))))
  
  (defun article-strip-banner ()
!   "Strip the banner specified by the `banner', and/or `banner-start' and `banner-end' group parameters."
    (interactive)
    (save-excursion
      (save-restriction
        (let ((inhibit-point-motion-hooks t)
  	    (banner (gnus-group-get-parameter gnus-newsgroup-name 'banner))
+ 	    (banner-start (gnus-group-get-parameter gnus-newsgroup-name 'banner-start))
+ 	    (banner-end (gnus-group-get-parameter gnus-newsgroup-name 'banner-end))
  	    (gnus-signature-limit nil)
! 	    buffer-read-only beg)
! 		(when banner
! 		  (article-goto-body)
! 		  (cond
! 		   ((eq banner 'signature)
! 			(when (gnus-article-narrow-to-signature)
! 			  (widen)
! 			  (forward-line -1)
! 			  (delete-region (point) (point-max))))
! 		   ((stringp banner)
! 			(while (re-search-forward banner nil t)
! 			  (delete-region (match-beginning 0) (match-end 0))))))
! 		(when (and banner-start banner-end (stringp banner-start) 
! 				   (stringp banner-end))
! 		  (article-goto-body)
! 		  (while (re-search-forward banner-start nil t)
! 			(setq beg (match-beginning 0))
! 			(if (re-search-forward banner-end nil t)
! 				(delete-region beg (match-end 0)))))))))
  
  (defun article-babel ()
    "Translate article using an online translation service."


-- 
Matt Pharr                              <URL:http://graphics.stanford.edu/~mmp>
===============================================================================
In a cruel and evil world, being cynical can allow you to get some
entertainment out of it. --Daniel Waters

Re: [Q] stripping banners at the start of messages

[Kai Großjohann]

On 25 Sep 2000, Matt Pharr wrote:

> Since there's a lot of arcane magic in figuring out these multi-line
> regexps (I'd never be able to put together magic like the above,
> much as I hate the egroups banners), I've got a suggested patch that
> I just cooked up.

Well, \\(.\\|\n\\)* matches a sequence of zero or more characters,
including newline characters.  Hence, "x\\(.\\|\n\\)*y" matches from
the first x to the last y in a message.

The problems start when you have two banners in a message, one at the
beginning and one at the end.  If both start with x and end with y,
the above regex will do the wrong thing.

kai
-- 
I like BOTH kinds of music.

Re: [Q] stripping banners at the start of messages

[Steve Harris]

I was able to strip these eGroups banners (one at the beginning _and_
one at the end) by noting that the a simple newline properly delimits
the banner you're struggling to kill. Do we really need a "start" and
"end" regex pair for this?

(banner . "^---+ eGroups Sponsor\\(.+\n\\)+\n\\|^\nWant out\\? bmw\\(.\\|\n\\)*")

-- 
Steven E. Harris        :: sharris   @primus.com
Primus                  :: http://www.primus.com

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Steve Harris <sharris@primus.com> writes:

> I was able to strip these eGroups banners (one at the beginning _and_
> one at the end) by noting that the a simple newline properly delimits
> the banner you're struggling to kill. Do we really need a "start" and
> "end" regex pair for this?
> 
> (banner . "^---+ eGroups Sponsor\\(.+\n\\)+\n\\|^\nWant out\\? bmw\\(.\\|\n\\)*")

Is it possible to have multiple banner parameters?  I tried this
yesterday and found that it sort of worked depending on the order I
specified them, although I never quite figured out what was going
on...

-- 
. Paul Stevenson . Reasearch Fellow . University of Surrey .
I dream in beige

Re: [Q] stripping banners at the start of messages

[Kai Großjohann]

On 26 Sep 2000, Paul Stevenson wrote:

> Is it possible to have multiple banner parameters?  I tried this
> yesterday and found that it sort of worked depending on the order I
> specified them, although I never quite figured out what was going
> on...

You can use "\\|" in a regex to match multiple things.

Emacs evaluates alternatives in order.  Given the string "axxbb", the
regex "a..b\\|a...b" matches a different string than "a...b\\|a..b".
I think.  You can use this to fine-tune what should be matched by your
regex. 

kai
-- 
I like BOTH kinds of music.

Re: [Q] stripping banners at the start of messages

[Toby Speight]

0> In article <i5k8zsgwn2g.lawbreaker@gromit.ph.surrey.ac.uk>,
0> Paul Stevenson <URL:mailto:p.stevenson@surrey.ac.uk> ("Paul") wrote:

Paul> I'm having exactly the same problem (eGroups banner).

Mine looks like

(banner . "^[ 	\n]*-------------------+\\( eGroups Sponsor -+\\)?....\n\\([^-\n].*\n+\\)+-------------------+....[ 	\n]*\n\\|^To unsubscribe from .*\n.*unsubscribe@egroups.com\n*")

But I think I could simplify it by noticing that the ad banner contains
no empty lines and is followed by an empty line (IOW, it is delimited
by \n\n):

(banner . "^[ 	\n]*-------------------+\\( eGroups Sponsor -+\\)?....\n\\(.+\n\\)+\n*\\|^To unsubscribe from .*\n.*unsubscribe@egroups.com\n*")


Something I'd really like to see is per-article banners based on headers
such as author or Received, to strip out some of those annoying company
disclaimers, in addition to per-group banners for mailing lists etc.

(And maybe it would let me specify egroups banner in one place only,
instead of having to change each groups parameters every time they
change it - the groups don't have a common parent topic).

Re: [Q] stripping banners at the start of messages

[ShengHuo ZHU]

Toby Speight <streapadair@gmx.net> writes:

> (And maybe it would let me specify egroups banner in one place only,
> instead of having to change each groups parameters every time they
> change it - the groups don't have a common parent topic).

A new feature, customizing gnus-article-banner-alist, allows this.

For example,
        
(setq gnus-article-banner-alist
      '((egroups . "^[ \t\n]*-------------------+\\( eGroups Sponsor -+\\)?....\n\\(.+\n\\)+")))

And set the parameter banner of all egroups group as symbol egroups. 

ShengHuo

Re: [Q] stripping banners at the start of messages

[Paul Stevenson]

Toby Speight <streapadair@gmx.net> writes:

> Something I'd really like to see is per-article banners based on headers
> such as author or Received, to strip out some of those annoying company
> disclaimers, in addition to per-group banners for mailing lists etc.

Hmmm, per-header treatment could be extended to looking at the mail
agent and then using knowledge of its quoting mechanism to restore
reply-before-quoted-text emails to the one true way.  Oooh, that'd be
nice. Unrelated to banners, of course, and would just be a washing
function.  Maybe I'll give it a go.