nnimap-split-fancy: regexp ".*" matches less than expected

nnimap-split-fancy: regexp ".*" matches less than expected

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 1842 bytes --]

Hi,

Consider the following (somewhat contrived) "fancy" splitting method:

  (nnimap-split-fancy
   (| ("cc" "\\(gnucash-user\\)@gnucash.org" "list.\\1")
      "INBOX"))

Consider also the attached example email (this is an email that was
recently sent to the public gnucash-user@gnucash.org list), which
contains the following header:

  Cc: gnucash-user <gnucash-user@gnucash.org>

When I put the above nnimap-split-fancy declaration into my nnimap
declaration in my ~/.gnus.el, how will Gnus split the example email?
Let's ask Gnus!  "M-x gnus-summary-respool-query" (B q) says:

  This message would go to list.gnucash-user

Also, "M-x gnus-summary-respool-trace" (B t) says:

  ("cc" "\\(gnucash-user\\)@gnucash.org" "list.\\1")
  "list.\\1"

OK, good.  That's what I expected.  Now let's change the declaration to
this:

  (nnimap-split-fancy
   (| ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
      "INBOX"))

This is the same as before, but I've replaced "gnucash-user" with ".*".
I expected the same result as before, but Gnus surprised me.  Instead,
gnus-summary-respool-query says:

  This message would go to list.user

and gnus-summary-respool-trace says:

  ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
  "list.\\1"

In this case, why does the ".*" match "user" instead of "gnucash-user"?
I've read the Gnus manual ((gnus) Splitting Mail, and similar sections).
I've read the elisp manual ((elisp) Regular Expressions).  I've even
experimented on the raw message headers using "M-x regexp-builder".  My
understanding is that "gnucash-user" is present in the string being
matched against, and that ".*" is a greedy regexp which matches any
character repeated zero or more times.  I don't understand how ".*" can
possibly match just the string "user".  What am I missing?

-- 
Chris

[-- Attachment #1.2: example.txt --]
[-- Type: text/plain, Size: 7527 bytes --]

From nobody Fri Apr  6 22:06:45 2018
Delivered-To: cmmarusich@gmail.com
Received: by 2002:a19:6818:0:0:0:0:0 with SMTP id d24-v6csp2304563lfc;
        Sun, 1 Apr 2018 20:57:04 -0700 (PDT)
X-Google-Smtp-Source: AIpwx48CCtl3hmRaQ6QRkqT9e9BwJndzYhL+IFUolIZDVFgiiYNDzFk4WeDLIRoGTeRK5N4yrWg7
X-Received: by 10.237.47.1 with SMTP id l1mr12188951qtd.29.1522641424634;
        Sun, 01 Apr 2018 20:57:04 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1522641424; cv=none;
        d=google.com; s=arc-20160816;
        b=S3KPZAQxA5PsSH/+xYbeddjxkbd+0afBJ9mnZWP6pDjISVbK6ZicY93Hi6Ww1jO3wm
         Hp4gBSGSmpsrH1crtNyzpSBbbJIGH71acjMjcVdqTNhx930y032L9KgVREoJvFU8k59i
         JGEDsaVhNoczJOwevz06vC71wXvOZctSYrbXGTqF4iCtLBJmDfYP5E2D790KjP0AvwwL
         jr29UbXhQMkoAU7utSDCqXRBHxQUYC9VuvK5SBjiuuDFEwM06c1qCW2LlHh0XyN5IcDb
         KV7kBlcAQpTif/p2pUrRAacI1ogeo6y1f8SI6wjzN3zR2FJ7/QzAxNuCmzs+Ur9GiO10
         N1kA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=sender:errors-to:content-transfer-encoding:cc:list-subscribe
         :list-help:list-post:list-archive:list-unsubscribe:list-id
         :precedence:to:references:message-id:date:in-reply-to:from:subject
         :mime-version:dkim-signature:dkim-filter:delivered-to
         :arc-authentication-results;
        bh=PsqK4vfsgdiy0Q4jVGUH3oQFF6p/fOk6zxaZF1n1KJA=;
        b=xoJkGZRn1M4g50K6GnMTjz9AmYvx7CpRx42gSMaK2LjtbFg3xvlDCa9kRBQJGwGf4e
         0VS9yYH7Be5nDYDUO64NMR4h0CAxFVVjMimAampu0qWPFZx6O2VP55i3qIZsreGhIy4c
         PPcZd2EWH58otECUUyVJwZrT6tIelzce9CVsqUUocMO73Ffq4AdRKES2oNOAQCabkiL0
         Iw1z3xjXgzOK+LL0JQEHiK58eSm6jsHu7mc7+28UibZsLqdjyxwM0ZpQ2xiuFQ6eXjo2
         GndVkAmz5pU7vgRqgSgCJ3phboxwryfeiA989Jc9lZYRaouvtNmVD+jWMr68aDAC+j5D
         Tsrw==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=neutral (body hash did not verify) header.i=@umich.edu header.s=relay-2016-05-12 header.b=GL6//b+W;
       spf=pass (google.com: domain of gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org designates 204.107.200.65 as permitted sender) smtp.mailfrom=gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=umich.edu
Return-Path: <gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org>
Received: from code.gnucash.org (code.gnucash.org. [204.107.200.65])
        by mx.google.com with ESMTPS id y83si3079193qka.369.2018.04.01.20.57.03
        for <cmmarusich@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Sun, 01 Apr 2018 20:57:04 -0700 (PDT)
Received-SPF: pass (google.com: domain of gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org designates 204.107.200.65 as permitted sender) client-ip=204.107.200.65;
Authentication-Results: mx.google.com;
       dkim=neutral (body hash did not verify) header.i=@umich.edu header.s=relay-2016-05-12 header.b=GL6//b+W;
       spf=pass (google.com: domain of gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org designates 204.107.200.65 as permitted sender) smtp.mailfrom=gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=umich.edu
Received: from code.gnucash.org (localhost [IPv6:::1])
	by code.gnucash.org (Postfix) with ESMTP id 7A2A5AC277
	for <cmmarusich@gmail.com>; Sun,  1 Apr 2018 23:56:36 -0400 (EDT)
X-Virus-Status: Clean
X-Virus-Scanned: clamav-milter 0.99.2 at code.gnucash.org
X-Original-To: gnucash-user@gnucash.org
Delivered-To: gnucash-user@gnucash.org
Received: from tombraider.mr.itd.umich.edu (smtp.mail.umich.edu
 [141.211.12.86])
 by code.gnucash.org (Postfix) with ESMTP id 0AC47A9693
 for <gnucash-user@gnucash.org>; Sun,  1 Apr 2018 23:55:28 -0400 (EDT)
DKIM-Filter: OpenDKIM Filter v2.11.0 code.gnucash.org 0AC47A9693
Authentication-Results: code.gnucash.org;
 dkim=pass (2048-bit key) header.d=umich.edu header.i=@umich.edu
 header.b="GL6//b+W"
X-Virus-Status: Clean
X-Virus-Scanned: clamav-milter 0.99.2 at code.gnucash.org
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=umich.edu;
 s=relay-2016-05-12; t=1522641273;
 bh=1Z81v1IhSgPUNFTag8trjMmarqsxHG1nDhGGlvLgPu8=;
 h=Subject:From:In-Reply-To:Date:Cc:References:To;
 b=GL6//b+WQ90TCoi8jq3dOmxre2vNrFr6FcAyUSb9t2VEb97Sc5LP9G+mbhTgVVCkY
 f2R5GgXmhPBedwcTc1XtP6lP3/ihRFza4Sjd4taoydM1geOX/noOL5pVGmCndRtkFz
 zA0lALRu/MPQK5DLr8ZREshnIulqKrJlfocHGa4ZH1RWDgR4PoAoKa53VfFDk3bAVO
 zqeVuEPhZhLJsKtkm6dE1oMNTlfBm93Vv7ZZjaJnXCFrmEuKTPnb73Q2McGDYJ/9/O
 mb7lnJPh1EWX33TP6WSZSZJ5fPOONCZj6W6hBYvxBIalQrOtDlA6LHtvmo9liY2ei9
 64dWgmM+hhQEQ==
Authentication-Results: tombraider.mr.itd.umich.edu;
 iprev=pass policy.iprev=173.15.232.226 (bayswater.msalexander.com);
 auth=pass smtp.auth=mta
Received: FROM bayswater.msalexander.com (bayswater.msalexander.com
 [173.15.232.226])
 By tombraider.mr.itd.umich.edu ID 5AC1A979.ADD37.10060; Authuser mta;
 Sun, 01 Apr 2018 23:54:33 -0400
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: Fast reactions to 2.7.8
From: Mike Alexander <mta@umich.edu>
In-Reply-To: <4996487.NFLN1pfjtV@legolas.kobaltwit.lan>
Date: Sun, 1 Apr 2018 23:54:02 -0400
Message-Id: <E939E02F-5983-49C1-9B47-D1EBD8B7BFC6@umich.edu>
References: <1522169078.1885292.1317906360.64CE1A6F@webmail.messagingengine.com>
 <4996487.NFLN1pfjtV@legolas.kobaltwit.lan>
To: Geert Janssens <geert.gnucash@kobaltwit.be>
X-Mailer: Apple Mail (2.3273)
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,HTML_MESSAGE,UNPARSEABLE_RELAY autolearn=ham
 autolearn_force=no version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on code.gnucash.org
X-Content-Filtered-By: Mailman/MimeDel 2.1.21
X-BeenThere: gnucash-user@gnucash.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: General Accounting Discussions <gnucash-user.gnucash.org>
List-Unsubscribe: <https://lists.gnucash.org/mailman/options/gnucash-user>,
 <mailto:gnucash-user-request@gnucash.org?subject=unsubscribe>
List-Archive: <http://lists.gnucash.org/pipermail/gnucash-user/>
List-Post: <mailto:gnucash-user@gnucash.org>
List-Help: <mailto:gnucash-user-request@gnucash.org?subject=help>
List-Subscribe: <https://lists.gnucash.org/mailman/listinfo/gnucash-user>,
 <mailto:gnucash-user-request@gnucash.org?subject=subscribe>
Cc: gnucash-user <gnucash-user@gnucash.org>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org
Sender: "gnucash-user" <gnucash-user-bounces+cmmarusich=gmail.com@gnucash.org>

> On Mar 31, 2018, at 8:37 AM, Geert Janssens <geert.gnucash@kobaltwit.be> wrote:
> 
> * On linux there's a second default theme called "Adwaita-dark" which should 
> also give you a dark themed gnucash. Unfortunately this doesn't work on 
> Windows. I suppose it's not included or improperly configured there ? On OS X 
> I haven't tried this.

This is packaged in the app bundle on MacOS and seems to work ok.

     Mike
 
_______________________________________________
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Andreas Schwab]

On Apr 06 2018, Chris Marusich <cmmarusich@gmail.com> wrote:

> This is the same as before, but I've replaced "gnucash-user" with ".*".
> I expected the same result as before, but Gnus surprised me.  Instead,
> gnus-summary-respool-query says:
>
>   This message would go to list.user
>
> and gnus-summary-respool-trace says:
>
>   ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
>   "list.\\1"
>
> In this case, why does the ".*" match "user" instead of "gnucash-user"?

I think that happens because the actual regexp used for searching is
"\\(\\(FIELD\\):.*\\)\\<\\(VALUE\\)\\>", note the embedded ".*".

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1592 bytes --]

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Apr 06 2018, Chris Marusich <cmmarusich@gmail.com> wrote:
>
>> This is the same as before, but I've replaced "gnucash-user" with ".*".
>> I expected the same result as before, but Gnus surprised me.  Instead,
>> gnus-summary-respool-query says:
>>
>>   This message would go to list.user
>>
>> and gnus-summary-respool-trace says:
>>
>>   ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
>>   "list.\\1"
>>
>> In this case, why does the ".*" match "user" instead of "gnucash-user"?
>
> I think that happens because the actual regexp used for searching is
> "\\(\\(FIELD\\):.*\\)\\<\\(VALUE\\)\\>", note the embedded ".*".

That would explain it.  Thank you!

It would be nice if the Gnus manual described in a little more detail
how the regexp defined in the "fancy" split method will be used.
Without knowledge that a ".*" was put in front of my regexp, the results
I was seeing made no sense.  I would submit a patch for the manual
myself, but I'm still just a Gnus beginner, so I can't (yet!) speak with
much authority on the subject.

By the way, where is this regexp defined in the Gnus source?  I ran "M-x
rgrep" with the following regexp in the Emacs 25.3 source but found no
results:

"\\\\\\\\(\\\\\\\\(.+?\\\\\\\\):\\.\\*\\\\\\\\)\\\\\\\\<\\\\\\\\(.+?\\\\\\\\)\\\\\\\\>"

"M-x regexp-builder" tells me that this regexp should match the regexp
string you mentioned above.  If you know of a better way for me to find
it in the source, I'd love to know!

Again, thank you for the help.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Andreas Schwab]

On Apr 07 2018, Chris Marusich <cmmarusich@gmail.com> wrote:

> By the way, where is this regexp defined in the Gnus source?

See nnmail-split-it.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3364 bytes --]

Hello again!

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Apr 06 2018, Chris Marusich <cmmarusich@gmail.com> wrote:
>
>> This is the same as before, but I've replaced "gnucash-user" with ".*".
>> I expected the same result as before, but Gnus surprised me.  Instead,
>> gnus-summary-respool-query says:
>>
>>   This message would go to list.user
>>
>> and gnus-summary-respool-trace says:
>>
>>   ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
>>   "list.\\1"
>>
>> In this case, why does the ".*" match "user" instead of "gnucash-user"?
>
> I think that happens because the actual regexp used for searching is
> "\\(\\(FIELD\\):.*\\)\\<\\(VALUE\\)\\>", note the embedded ".*".

I've tried to figure out a good way to accomplish what I want, but I'm
stuck.  I was hoping to come up with a "fancy" splitting method that
would split the email into groups like this:

  foo@example-lists.com -> list.foo
  foo-bar@example-lists.com -> list.foo-bar
  foo-bar_baz@example-lists.com -> list.foo-bar_baz

I currently am in a situation where I need to filter a LOT of groups
like this, and I do not know a priori what their exact names will be.  I
could just observe the list names on emails I receive over time and add
individual rules explicitly for those list names as I discover them, but
if possible, I'd rather just craft a simple regexp to capture all of
them up front and be done with it.

How can I accomplish that?  Basically, whatever comes before the @, I
want to put it in the group the name.  Because the greedy ".*" is
foisted upon me, it eats up pretty much all of the characters in the
list name.  If instead of ".*" the embedded regexp used ".*?"
(non-greedy), I think it would be easy to accomplish what I want with a
regexp like this:

  "\\([[:graph:]]+\\)@example-lists.com"

However, that doesn't work when the greedy ".*" precedes my regexp,
since it eats up most of what I want to capture.  What to do?

As long as it isn't possible for me to change the regexp that precedes
my own, I think the only solution is for me to embed a sentinel regexp
at the start of my own regexp, to stop the greedy ".*" from going any
farther.  Thankfully, many of the lists I need to filter begin with a
specific string that does not repeat itself later in the list name, call
it "myprefix".  They all look kind of like this:

  myprefix-foo@example-lists.com
  myprefix-foo-bar@example-lists.com
  myprefix-foo-bar_baz@example-lists.com

Thanks to this happy coincidence, I suppose I can get by for now by
defining a single regexp like this in my "fancy" splitting method:

  "\\(myprefix-[[:graph:]]+\\)@example-lists.com"

But I'm curious: would there be a way to accomplish the same thing, even
when the list names are not entirely predictable?

Stepping back a moment, I also wonder: am I going about this the wrong
way?  Maybe my goal SHOULDN'T be to filter the list traffic like this.
Maybe the right thing to do in this situation is actually to learn to
use Gnus' "scoring" feature.  Maybe I should just dump all my mail into
a single group, and use the scoring feature to automatically hide the
messages that I don't care about.

The opinions of Gnus gurus would be very welcome.  As a famous Martian
once said, "I am but an egg."  Thank you again for the help!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Andreas Schwab]

On Apr 08 2018, Chris Marusich <cmmarusich@gmail.com> wrote:

> I've tried to figure out a good way to accomplish what I want, but I'm
> stuck.  I was hoping to come up with a "fancy" splitting method that
> would split the email into groups like this:
>
>   foo@example-lists.com -> list.foo
>   foo-bar@example-lists.com -> list.foo-bar
>   foo-bar_baz@example-lists.com -> list.foo-bar_baz

Does it help to add the special characters as word characters in
nnmail-split-fancy-syntax-table?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: nnimap-split-fancy: regexp ".*" matches less than expected

[Dan Christensen]

On Apr  8, 2018, Chris Marusich <cmmarusich@gmail.com> wrote:

> Andreas Schwab <schwab@linux-m68k.org> writes:
>
>> On Apr 06 2018, Chris Marusich <cmmarusich@gmail.com> wrote:
>>
>>> This is the same as before, but I've replaced "gnucash-user" with ".*".
>>> I expected the same result as before, but Gnus surprised me.  Instead,
>>> gnus-summary-respool-query says:
>>>
>>>   This message would go to list.user
>>>
>>> and gnus-summary-respool-trace says:
>>>
>>>   ("cc" "\\(.*\\)@gnucash.org" "list.\\1")
>>>   "list.\\1"
>>>
>>> In this case, why does the ".*" match "user" instead of "gnucash-user"?
>>
>> I think that happens because the actual regexp used for searching is
>> "\\(\\(FIELD\\):.*\\)\\<\\(VALUE\\)\\>", note the embedded ".*".
>
> I've tried to figure out a good way to accomplish what I want, but I'm
> stuck.

Does it help to put a literal space character at the start of your
regexp, like " \\(.*\\)@gnucash.org"?

I'm not sure what "\\<" and "\\>" are doing in the constructed regexp,
but if they are matching literal "<" and ">" characters, then this
probably won't work.

Dan