Problems with gnus-registry

Problems with gnus-registry

[Jake Colman]

I am starting to use gnus-registry so that emails are filed in the same
folder as its parent.  Unfortunately, it does not seem to be working.
Hopefuly, I am simply just misusing it in some way.

These settings appear in my .gnus file:

-----------------------------------------------

(setq gnus-registry-max-entries 2500
      gnus-registry-use-long-group-names nil)

(gnus-registry-initialize)

(setq nnimap-split-fancy
      '(|
        ; some up-front splitting
        (: gnus-registry-split-fancy-with-parent)

-----------------------------------------------

I will typically send an email such that, using gcc-self, it wil be filed
into a specified group.  I would expect that gnus-registry would auto-file
any reply to that email into the same group as its parent.  Instead, it gets
filed using the other rules contained in nnimap-split-fancy.

Similarly, I will sometimes manually copy an unread email into a specific
group.  I will then go to that group, and reply to that email.  Again
subsequent replies are not filed with its parent.

Finally, incoming emails might be auto-filtered into a specific group via
nnimap-split-fancy.  I will then go to that group, and read and and reply to
that email.  Again subsequent replies are not filed with its parent.

Should gnus-registry handle these cases as I am expecting?  If so, what do I
need to do to make it work?

Any help would be greatly appreciated.

-- 
Jake Colman
Director of Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Fri, 18 Jan 2008 10:21:53 -0500 Jake Colman <colman@ppllc.com> wrote: 

Jake> I am starting to use gnus-registry so that emails are filed in the same
Jake> folder as its parent.  Unfortunately, it does not seem to be working.
Jake> Hopefuly, I am simply just misusing it in some way.

Jake> These settings appear in my .gnus file:

Jake> -----------------------------------------------

Jake> (setq gnus-registry-max-entries 2500
Jake>       gnus-registry-use-long-group-names nil)

Jake> (gnus-registry-initialize)

Jake> (setq nnimap-split-fancy
Jake>       '(|
Jake>         ; some up-front splitting
Jake>         (: gnus-registry-split-fancy-with-parent)

Hi Jake,

this looks OK.  Can you send me your registry contents for a particular
message that didn't get split properly, and the message itself?  Also
the stuff in *messages* (make sure you increase gnus-verbose before you
run it) printed by the registry would be helpful.

Jake> I will typically send an email such that, using gcc-self, it wil be filed
Jake> into a specified group.  I would expect that gnus-registry would auto-file
Jake> any reply to that email into the same group as its parent.  Instead, it gets
Jake> filed using the other rules contained in nnimap-split-fancy.

This should work.  You can look in the registry file (after you save it)
to check what's stored.  Also you can do

M-: (gethash "ID" gnus-registry-hashtb)

where ID is the message ID (as the registry remembers it, with brackets
and all) to see the exact registry info about a message.  In fact, do
this M-: before anything else to be sure the registry is loaded and
knows about your messages.

Jake> Similarly, I will sometimes manually copy an unread email into a specific
Jake> group.  I will then go to that group, and reply to that email.  Again
Jake> subsequent replies are not filed with its parent.

This should definitely be noticed by the registry.  Again, I need to see
the registry contents and the message that didn't get filed right, plus
the *messages* if possible.

Jake> Finally, incoming emails might be auto-filtered into a specific group via
Jake> nnimap-split-fancy.  I will then go to that group, and read and and reply to
Jake> that email.  Again subsequent replies are not filed with its parent.

Jake> Should gnus-registry handle these cases as I am expecting?  If so, what do I
Jake> need to do to make it work?

Note that if you have more than 2500 registry entries, the older ones
will get expired when you save, and the registry won't remember where
they lived.  Additionally, every time you visit a group with N visible
messages, the registry will make N entries by design.  Turn off
gnus-registry-max-entries while testing and see if it makes a
difference.  Each entry is not big, it's just a list, so you'll pay a
little more memory, that's all.

I think all these cases should be handled, and they work for me.  I need
the debug info.

Ted

Re: Problems with gnus-registry

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   TZ> this looks OK.  Can you send me your registry contents for a
   TZ> particular message that didn't get split properly, and the message
   TZ> itself?  Also the stuff in *messages* (make sure you increase
   TZ> gnus-verbose before you run it) printed by the registry would be
   TZ> helpful.

I apologize for following up my own email.

I just ran a controlled test.  I manually moved the incorrectly filed email
into its correct group.  That resulted in the following entry made to
.gnus.registry.eld:

("<0B72769B5EA61E45BFDAB9B849923FE703359839@hamilton.ppllc.com>" ((mtime 18343 4695 975321)) "INBOX/Principia/PM-295-296")))

I replied to that email and waited for a reply to come back.  When it did, my
Message-Log shows:

nnimap: Fetching (part of) article 850672 from INBOX...
nnimap: Fetching (part of) article 850672 from nil...done
gnus-registry-split-fancy-with-parent traced nil to group nil
IMAP split moved hamilton:INBOX:850672 to INBOX/Principia

That is the email that should have been traced to
"INBOX/Principia/PM-295-296".

What other information can I provide you?

...Jake

-- 
Jake Colman
Director of Software Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Mon, 04 Feb 2008 08:38:31 -0500 Jake Colman <colman@ppllc.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> this looks OK.  Can you send me your registry contents for a
TZ> particular message that didn't get split properly, and the message
TZ> itself?  Also the stuff in *messages* (make sure you increase
TZ> gnus-verbose before you run it) printed by the registry would be
TZ> helpful.

Jake> I apologize for following up my own email.

Jake> I just ran a controlled test.  I manually moved the incorrectly filed email
Jake> into its correct group.  That resulted in the following entry made to
Jake> .gnus.registry.eld:

Jake> ("<0B72769B5EA61E45BFDAB9B849923FE703359839@hamilton.ppllc.com>" ((mtime 18343 4695 975321)) "INBOX/Principia/PM-295-296")))

Jake> I replied to that email and waited for a reply to come back.  When it did, my
Jake> Message-Log shows:

Jake> nnimap: Fetching (part of) article 850672 from INBOX...
Jake> nnimap: Fetching (part of) article 850672 from nil...done
Jake> gnus-registry-split-fancy-with-parent traced nil to group nil
Jake> IMAP split moved hamilton:INBOX:850672 to INBOX/Principia

Jake> That is the email that should have been traced to
Jake> "INBOX/Principia/PM-295-296".

The parent message is known, then.  By the way, you can always run

M-: (gnus-registry-split-fancy-with-parent)

in a buffer showing the original article (I forget, you may have to hit
`t' to show all the headers).  The result of that command is what the
registry will say at splitting time, so you don't have to wait for a
mail reply to arrive to test things :) Right now it's probably returning
nil (which passes control to the next splitting rule); please test if
that's true.

Next, try running

M-: (message-fetch-field "references")
M-: (message-fetch-field "in-reply-to")

in a buffer showing the article you mention.  That will show the
references we know in this article.  It should return something.  Those
references are what we use to track the parent of the article (in
addition to the subject and the sender, if you turn those on in the
registry settings; see gnus-registry-track-extra).  Without references
we can't do anything, so if they are missing we need to fix that.

Next, if there are references, do

M-: (gnus-extract-references (message-fetch-field "references"))

which should give you the list of referenced message IDs.  Is one of
them in the registry?

It would be great if you could attach the message itself.  Send it to me
privately if you prefer.

Ted

Re: Problems with gnus-registry

[Ted Zlatanov]

On Mon, 04 Feb 2008 21:41:51 -0500 Jake Colman <colman@ppllc.com> wrote: 

Jake> I believe that my correspondent's email is processed via a really old version
Jake> of MS Exchange Server.  I'm wondering if the old Exchange Servers did not act
Jake> as a proper MUA.  We'll be upgrading our Exchange Server in the next few
Jake> weeks so we'll see if that helps.

I don't know how Exchange does references, sorry...  You see the
problem: only sender/subject tracking can help you if there are no
references.  I think Exchange has some sort of thread ID mechanism, but
I didn't see it in the message you forwarded.  cc-ing ding in case
someone has ideas.

Ted

Re: Problems with gnus-registry

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   TZ> On Mon, 04 Feb 2008 21:41:51 -0500 Jake Colman <colman@ppllc.com> wrote: 
   Jake> I believe that my correspondent's email is processed via a really
   Jake> old version of MS Exchange Server.  I'm wondering if the old
   Jake> Exchange Servers did not act as a proper MUA.  We'll be upgrading
   Jake> our Exchange Server in the next few weeks so we'll see if that
   Jake> helps.

   TZ> I don't know how Exchange does references, sorry...  You see the
   TZ> problem: only sender/subject tracking can help you if there are no
   TZ> references.  I think Exchange has some sort of thread ID mechanism,
   TZ> but I didn't see it in the message you forwarded.  cc-ing ding in case
   TZ> someone has ideas.

Ted,

If I send an email, does the registry track anything about the sent email so
that it knows where to possibly file a reply?  Or is it only anything
sent/received after that initial reply is processed (that is, that something
was done to it to cause the registry to see it) that will be auto-filed by
the registry?

Here is what I am getting to:

1) I sent an email from a specific group but nothing was entered into the
   registry at all.  If so, how will replies end up in the same group?

2) If I send email from the main gnus page (outside of any specific group) or
   from a group that does not specify a gcc-self, the email has an automatic
   gcc-self of an Archive group.  In this way, I automatically have a copy of
   all emails that I send.  Does this automatic copy to an Archive group mean
   that my replies might end up there as well?  If so, that's an issue.  I
   rarely, if ever, go to the Archive group unless I need to see when a
   message was sent.

Related to my original problem, if I track the extras of subject and sender,
are those guaranteed to work regardless of whether Exchange screws up the
standard headers?

...Jake

-- 
Jake Colman
Director of Software Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Tue, 05 Feb 2008 16:51:44 -0500 Jake Colman <colman@ppllc.com> wrote: 

Jake> If I send an email, does the registry track anything about the sent email so
Jake> that it knows where to possibly file a reply?  Or is it only anything
Jake> sent/received after that initial reply is processed (that is, that something
Jake> was done to it to cause the registry to see it) that will be auto-filed by
Jake> the registry?

When you send an e-mail, there is no article and thus no group name for
the registry to notice.

If the article has a gcc, that will trigger the registry when the copy
is spooled.  The outgoing article in itself doesn't do anything, though.

I could add a mail send hook to do this, but it could be incorrect.  For
instance, if you send to ding while in group A but you get a copy of the
mail back into group B, You want followups in B but they'll go to A.
Maybe if the outbound mail record was a low-priority hint it could work,
but I don't like it much and I think it's confusing to users.

Jake> Here is what I am getting to:

Jake> 1) I sent an email from a specific group but nothing was entered into the
Jake>    registry at all.  If so, how will replies end up in the same group?

(I assume first you saved the registry file if you used grep on it)

It depends on whether there was a gcc.  Without gcc, nothing would be
registered.

Jake> 2) If I send email from the main gnus page (outside of any specific group) or
Jake>    from a group that does not specify a gcc-self, the email has an automatic
Jake>    gcc-self of an Archive group.  In this way, I automatically have a copy of
Jake>    all emails that I send.  Does this automatic copy to an Archive group mean
Jake>    that my replies might end up there as well?  If so, that's an issue.  I
Jake>    rarely, if ever, go to the Archive group unless I need to see when a
Jake>    message was sent.

You can set your archive group to be ignored, see
gnus-registry-ignored-groups.

Jake> Related to my original problem, if I track the extras of subject and sender,
Jake> are those guaranteed to work regardless of whether Exchange screws up the
Jake> standard headers?

They are not guaranteed:

- if you trim your registry, records will be lost

- subject and sender tracking is experimental and works OK for me, but
  you may discover bugs

- multiple matches for a subject or a sender will disqualify the match,
  to avoid ambiguous results

If you have ideas on improving any part of the registry, let me know
what you think.

Ted

Re: Problems with gnus-registry

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   TZ> On Tue, 05 Feb 2008 16:51:44 -0500 Jake Colman <colman@ppllc.com> wrote: 

   Jake> If I send an email, does the registry track anything about the sent
   Jake> email so that it knows where to possibly file a reply?  Or is it
   Jake> only anything sent/received after that initial reply is processed
   Jake> (that is, that something was done to it to cause the registry to see
   Jake> it) that will be auto-filed by the registry?

   TZ> When you send an e-mail, there is no article and thus no group name
   TZ> for the registry to notice.

   TZ> If the article has a gcc, that will trigger the registry when the copy
   TZ> is spooled.  The outgoing article in itself doesn't do anything,
   TZ> though.

   TZ> I could add a mail send hook to do this, but it could be incorrect.
   TZ> For instance, if you send to ding while in group A but you get a copy
   TZ> of the mail back into group B, You want followups in B but they'll go
   TZ> to A.  Maybe if the outbound mail record was a low-priority hint it
   TZ> could work, but I don't like it much and I think it's confusing to
   TZ> users.

So with a gcc, when the backend causes the sent article to spool to the
gcc-ed folder, the registry will register that article.  This will work for
sent emails as well since it is the gcc that will make the registry see it.
Is this all correct?

   Jake> Here is what I am getting to:

   Jake> 1) I sent an email from a specific group but nothing was entered into the
   Jake> registry at all.  If so, how will replies end up in the same group?

   TZ> (I assume first you saved the registry file if you used grep on it)

   TZ> It depends on whether there was a gcc.  Without gcc, nothing would be
   TZ> registered.

   Jake> 2) If I send email from the main gnus page (outside of any specific
   Jake>    group) or from a group that does not specify a gcc-self, the
   Jake>    email has an automatic gcc-self of an Archive group.  In this
   Jake>    way, I automatically have a copy of all emails that I send.  Does
   Jake>    this automatic copy to an Archive group mean that my replies
   Jake>    might end up there as well?  If so, that's an issue.  I rarely,
   Jake>    if ever, go to the Archive group unless I need to see when a
   Jake>    message was sent.

   TZ> You can set your archive group to be ignored, see
   TZ> gnus-registry-ignored-groups.

My archive group is defined through the following code:

  (setq gnus-message-archive-group
        '((concat "nnimap+hamilton:Archive/" 
                  (format-time-string "%Y-%m" (current-time)))))

How can I set gnus-registry-ignored-groups so that I don't have my replies
registered to the archive (in the absence of any overriding gcc of course).

   Jake> Related to my original problem, if I track the extras of subject and sender,
   Jake> are those guaranteed to work regardless of whether Exchange screws up the
   Jake> standard headers?

   TZ> They are not guaranteed:

   TZ> - if you trim your registry, records will be lost

I am not trimming anymore.

   TZ> - subject and sender tracking is experimental and works OK for me, but
   TZ>   you may discover bugs

   TZ> - multiple matches for a subject or a sender will disqualify the match,
   TZ>   to avoid ambiguous results

How can I test if that is what's going on?

It is pretty clear that some of my emails are coming in without the standard
headers that the registry is looking for.  But one would think that sender
and subject would always be found!  I'd love to help shake this out for you
that I can get more of my emails handled by the registry.

   TZ> If you have ideas on improving any part of the registry, let me know
   TZ> what you think.

Certainly the level of support does not need any improvement!

...Jake

-- 
Jake Colman
Director of Software Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Tue, 05 Feb 2008 17:24:18 -0500 Jake Colman <colman@ppllc.com> wrote: 

Jake> So with a gcc, when the backend causes the sent article to spool to the
Jake> gcc-ed folder, the registry will register that article.  This will work for
Jake> sent emails as well since it is the gcc that will make the registry see it.
Jake> Is this all correct?

Yes, that's how it's supposed to work.

TZ> You can set your archive group to be ignored, see
TZ> gnus-registry-ignored-groups.

Jake> My archive group is defined through the following code:

Jake>   (setq gnus-message-archive-group
Jake>         '((concat "nnimap+hamilton:Archive/" 
Jake>                   (format-time-string "%Y-%m" (current-time)))))

Jake> How can I set gnus-registry-ignored-groups so that I don't have my replies
Jake> registered to the archive (in the absence of any overriding gcc of
Jake> course).

Sorry, I gave you the wrong variable name.  This is the right one with
the default value:

(setq gnus-registry-unfollowed-groups 
 '("delayed$" "drafts$" "queue$" "INBOX$"))

It means "any group ending with one of these words" ($ at the end means
"the string must end here", so "INBOX.hello" won't match the last one
for example).

So, depending on whether you want to keep the default, you can add
"Archive/" to the list (so anything with that word in it not be followed
for splitting).

If you have other groups with "Archive/" in the name, the regular
expression can be more precise to allow only the one above.  I would
keep things simple if possible.

TZ> - subject and sender tracking is experimental and works OK for me, but
TZ> you may discover bugs

TZ> - multiple matches for a subject or a sender will disqualify the match,
TZ> to avoid ambiguous results

Jake> How can I test if that is what's going on?

Jake> It is pretty clear that some of my emails are coming in without the standard
Jake> headers that the registry is looking for.  But one would think that sender
Jake> and subject would always be found!  I'd love to help shake this out for you
Jake> that I can get more of my emails handled by the registry.

The sender and the subject are in the registry as extra data, e.g.

("<47A802B8.6080400@gna.org>" ((mtime 18344 55485 405961) (sender . "\"Suraj N. Kurapati\" <snk@gna.org>") (subject . "[wmii] [ANN] Rumai 2.0.0")) "nnimap+blockstar.com:INBOX.ion")

This entry says:

message ID is first entry

extra data:
 mtime is creation/update date
 sender and subject are as shown

group name is any string after the message ID

So to see if a subject matches multiple groups, you need to:

1) enter a group with a message you're interested in, to make sure it's
in the registry.  Note the message ID, subject, and sender.

2) (after saving to make sure the file contains the latest data)

"grep subject ~/.gnus.registry.eld | grep YOUR_SUBJECT_HERE"

Same as above for the sender.  If multiple entries match, none will be
returned.  If no entries match, we'll need to do more debugging on why
that's happened; grep the registry file for the message ID as well, and
I'll need to see that message.

I will write functions to do the above directly through the registry to
make debugging easier, but for the time being stick with grep.

Ted

Re: Problems with gnus-registry

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   TZ> On Tue, 05 Feb 2008 17:24:18 -0500 Jake Colman <colman@ppllc.com>
   TZ> wrote:

   Jake> So with a gcc, when the backend causes the sent article to spool to
   Jake> the gcc-ed folder, the registry will register that article.  This
   Jake> will work for sent emails as well since it is the gcc that will make
   Jake> the registry see it.  Is this all correct?

   TZ> Yes, that's how it's supposed to work.

So after an email with a gcc is sent, should I immediately see an entry in
.gnus.registry.eld (assuming I did a 'save' and flushed it to disk)?  And is
the list ordered with the most recent entries simply appended to the end?  Or
is there some other kind of ordering?

   TZ> Sorry, I gave you the wrong variable name.  This is the right one with
   TZ> the default value:

   TZ> (setq gnus-registry-unfollowed-groups 
   TZ>  '("delayed$" "drafts$" "queue$" "INBOX$"))

Actually, the default value contains only "delayed", "drafts", and "queue".
Not that INBOX is missing and that none have the trailing "$".  Maybe I'm not
using the latest version of the registry?  Could that be the source of some
of my issues?

   TZ> I will write functions to do the above directly through the registry
   TZ> to make debugging easier, but for the time being stick with grep.

A function that indicates which match was actually used (since it can match
on quite a few things) would be helpful too.

-- 
Jake Colman
Director of Software Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Wed, 06 Feb 2008 11:54:53 -0500 Jake Colman <colman@ppllc.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> On Tue, 05 Feb 2008 17:24:18 -0500 Jake Colman <colman@ppllc.com>
TZ> wrote:

Jake> So with a gcc, when the backend causes the sent article to spool to
Jake> the gcc-ed folder, the registry will register that article.  This
Jake> will work for sent emails as well since it is the gcc that will make
Jake> the registry see it.  Is this all correct?

TZ> Yes, that's how it's supposed to work.

Jake> So after an email with a gcc is sent, should I immediately see an entry in
Jake> .gnus.registry.eld (assuming I did a 'save' and flushed it to disk)?  And is
Jake> the list ordered with the most recent entries simply appended to the end?  Or
Jake> is there some other kind of ordering?

I think the list is ordered by date, but the "special" entries (those
with extra data like article marks) are processed separately and not in
order.  I wouldn't rely on the ordering in any case.

TZ> Sorry, I gave you the wrong variable name.  This is the right one with
TZ> the default value:

TZ> (setq gnus-registry-unfollowed-groups 
TZ> '("delayed$" "drafts$" "queue$" "INBOX$"))

Jake> Actually, the default value contains only "delayed", "drafts", and "queue".
Jake> Not that INBOX is missing and that none have the trailing "$".  Maybe I'm not
Jake> using the latest version of the registry?  Could that be the source of some
Jake> of my issues?

Gnus v5.10.8 is not up to date with all my changes lately.  I'd suggest
using a CVS checkout if possible; you can try updating just
gnus-registry.el but it may break badly as the registry gets integrated
more into Gnus as a whole.

TZ> I will write functions to do the above directly through the registry
TZ> to make debugging easier, but for the time being stick with grep.

Jake> A function that indicates which match was actually used (since it can match
Jake> on quite a few things) would be helpful too.

Yes, that comes out in the *messages* buffer but it's definitely not as
nice as it should be.  I'll put it on my TODO list.

Ted

Re: Problems with gnus-registry

[Reiner Steib]

On Wed, Feb 06 2008, Ted Zlatanov wrote:

[ gnus-registry-ignored-groups ]
> Sorry, I gave you the wrong variable name.  This is the right one with
> the default value:
>
> (setq gnus-registry-unfollowed-groups 
>  '("delayed$" "drafts$" "queue$" "INBOX$"))

IMHO, something with "-ignore-"/"-ignored-" would be a better name for
consistency with many other Gnus variable including
`nnmail-split-fancy-with-parent-ignore-groups' and
`nnmail-cache-ignore-groups'.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Problems with gnus-registry

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   TZ> Gnus v5.10.8 is not up to date with all my changes lately.  I'd
   TZ> suggest using a CVS checkout if possible; you can try updating just
   TZ> gnus-registry.el but it may break badly as the registry gets
   TZ> integrated more into Gnus as a whole.

I will look into this.  Right now it looks like my unix box cannot connect
to cvs.gnus.org.  :-(

Are the nightly builds reasonably safe to check out?  It does not look like
this is any official build that is newer than v5.10.8, is there?


I am looking into using match by subject since that it is one header that
should always be there.  Ufortunately, the message included below (I am
showing all available headers) did not get registered and filed correctly
even though I think it should have matched by subject.  (BTW I am pretty
sure that the embedded space in the subject is not the issue).

==========================================================

  Received: by hamilton.ppllc.com 
  	  id <01C868E5.2C6CB3A0@hamilton.ppllc.com>; Wed, 6 Feb 2008 12:24:48 -0500
  Message-ID: <0B72769B5EA61E45BFDAB9B849923FE703359AC5@hamilton.ppllc.com>
  From: Andrew Vickers <vickers@ppllc.com>
  To: Jacob Colman <colman@ppllc.com>
  Cc: Chirag Patel <patel@ppllc.com>
  Subject: RE: PM-331 Asset Position Updates into VM FS - ready for tech rev  iew
  Date: Wed, 6 Feb 2008 12:24:39 -0500 
  MIME-Version: 1.0
  Content-Type: text/plain

==========================================================

Looking into the registry, the message is registered correctly.  But if I
execute (message-fetch-field "subject") on this message, I get the following:

"RE: PM-331 Asset Position Updates into VM FS - ready for tech rev  iew, Re: PM-331 Asset Position Updates into VM FS - ready for tech"

This looks like it duplicates the text in the subject header which, of
course, will not match that which is registered.

Is this a bug in the v5.10.8 version of the registry?

...Jake

-- 
Jake Colman
Director of Software Development
Principia Partners LLC
101 West Elm Street
Suite 620
Conshohocken, PA  19428
+1 (610) 755-9786
www.principiapartners.com

Re: Problems with gnus-registry

[Ted Zlatanov]

On Wed, 06 Feb 2008 21:05:26 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Wed, Feb 06 2008, Ted Zlatanov wrote:
RS> [ gnus-registry-ignored-groups ]
>> Sorry, I gave you the wrong variable name.  This is the right one with
>> the default value:
>> 
>> (setq gnus-registry-unfollowed-groups 
>> '("delayed$" "drafts$" "queue$" "INBOX$"))

RS> IMHO, something with "-ignore-"/"-ignored-" would be a better name for
RS> consistency with many other Gnus variable including
RS> `nnmail-split-fancy-with-parent-ignore-groups' and
RS> `nnmail-cache-ignore-groups'.

There's two variables:

(in gnus.el)

;; The Gnus registry's ignored groups
(gnus-define-group-parameter
 registry-ignore
 :type list
 :function-document
 "Whether this group should be ignored by the registry."
 :variable gnus-registry-ignored-groups
 :variable-default nil
 :variable-document
 "*Groups in which the registry should be turned off."
 :variable-group gnus-registry
 :variable-type '(repeat
		  (list
		   (regexp :tag "Group Name Regular Expression")
		   (boolean :tag "Ignored")))

 :parameter-type '(boolean :tag "Group Ignored by the Registry")
 :parameter-document
 "Whether the Gnus Registry should ignore this group.")

(in gnus-registry.el)

(defcustom gnus-registry-unfollowed-groups '("delayed$" "drafts$" "queue$" "INBOX$")
  "List of groups that gnus-registry-split-fancy-with-parent won't return.
The group names are matched, they don't have to be fully
qualified.  This parameter tells the Registry 'never split a
message into a group that matches one of these, regardless of
references.'"
  :group 'gnus-registry
  :type '(repeat regexp))

They are used differently.  Ignored groups are not saved in the
registry, but they are followed for splitting (this is incorrect, I
think).  Unfollowed groups are not followed by
gnus-registry-split-fancy-with-parent function but they are saved.

I think the two should be united in one variable, specifically
gnus-registry-ignored-groups since it's more powerful and should do the
same thing.  I'm not sure why I didn't do it so far.  Is it OK to make
that change?  I'll deprecate gnus-registry-unfollowed-groups, emit a
message when it's set, but still respect it as equivalent to
gnus-registry-ignored-groups.

Ted

Re: Problems with gnus-registry

[Reiner Steib]

On Thu, Feb 07 2008, Ted Zlatanov wrote:

> On Wed, 06 Feb 2008 21:05:26 +0100 Reiner Steib wrote: 
> RS> IMHO, something with "-ignore-"/"-ignored-" would be a better name for
> RS> consistency with many other Gnus variable including
> RS> `nnmail-split-fancy-with-parent-ignore-groups' and
> RS> `nnmail-cache-ignore-groups'.
>
> There's two variables:
[...]
> ;; The Gnus registry's ignored groups
> (gnus-define-group-parameter
>  registry-ignore
>  :type list
>  :function-document
>  "Whether this group should be ignored by the registry."
>  :variable gnus-registry-ignored-groups
[...]
>  "*Groups in which the registry should be turned off."
[...]
> (defcustom gnus-registry-unfollowed-groups '("delayed$" "drafts$" "queue$" "INBOX$")
>   "List of groups that gnus-registry-split-fancy-with-parent won't return.
[...]
> They are used differently.  Ignored groups are not saved in the
> registry, but they are followed for splitting (this is incorrect, I
> think).  Unfollowed groups are not followed by
> gnus-registry-split-fancy-with-parent function but they are saved.
>
> I think the two should be united in one variable, specifically
> gnus-registry-ignored-groups since it's more powerful and should do the
> same thing.

As we want to store marks (or "labels") in the registry, wouldn't it
make sense to have two different variables?  (Though both should/could
contain "ignore" in the name, like the related nnmail-* variables.)  A
user might want to ignore some groups for parent splitting, but want
to set marks in these groups.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Problems with gnus-registry

[David]

Reiner Steib <reinersteib+gmane@imap.cc> writes:
> As we want to store marks (or "labels") in the registry, wouldn't it
> make sense to have two different variables?  (Though both should/could
> contain "ignore" in the name, like the related nnmail-* variables.)  A
> user might want to ignore some groups for parent splitting, but want
> to set marks in these groups.

I second that, please keep both variables. But I think it should be made
clearer that the variable currently named
gnus-registry-unfollowed-groups is related to splitting only, so maybe
call it gnus-registry-split-with-parent-ignored-groups or something like
that.

By the way, I'm currently playing around with the registry labels and
was wondering: what would currently be the best way to get all message
ids with a certain label? This way, I could easily create nnmairix
groups containing only these messages.

-David

Re: Problems with gnus-registry

[Ted Zlatanov]

On Thu, 07 Feb 2008 18:29:59 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

>> They are used differently.  Ignored groups are not saved in the
>> registry, but they are followed for splitting (this is incorrect, I
>> think).  Unfollowed groups are not followed by
>> gnus-registry-split-fancy-with-parent function but they are saved.
>> 
>> I think the two should be united in one variable, specifically
>> gnus-registry-ignored-groups since it's more powerful and should do the
>> same thing.

RS> As we want to store marks (or "labels") in the registry, wouldn't it
RS> make sense to have two different variables?  (Though both
RS> should/could contain "ignore" in the name, like the related nnmail-*
RS> variables.)  A user might want to ignore some groups for parent
RS> splitting, but want to set marks in these groups.

How about naming the single variable gnus-registry-ignored-groups (with
the corresponding topic/group parameter registry-ignore), with values
like this

"nnrss.*" t         ; meaning "ignore"
"nntp.*"  'nofollow ; meaning "do not follow"
"nnml.*"  nil       ; treat normally

It's probably simpler for the users than two separate variables,
especially since "do not follow" a group is a milder form of "ignore
this group."

I think registry labels will not care about this, they don't go by the
group at all, they only use the message ID.  The entry could have no
groups and the labels should still work fine.

Ted

Re: Problems with gnus-registry

[Ted Zlatanov]

On Fri, 08 Feb 2008 12:58:17 +0100 David <de_bb@arcor.de> wrote: 

D> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>> As we want to store marks (or "labels") in the registry, wouldn't it
>> make sense to have two different variables?  (Though both should/could
>> contain "ignore" in the name, like the related nnmail-* variables.)  A
>> user might want to ignore some groups for parent splitting, but want
>> to set marks in these groups.

D> I second that, please keep both variables. But I think it should be made
D> clearer that the variable currently named
D> gnus-registry-unfollowed-groups is related to splitting only, so maybe
D> call it gnus-registry-split-with-parent-ignored-groups or something like
D> that.

I'd like to avoid the two variables, see my proposal separately.  If you
and Reiner feel strongly they should be separate, though, I'll go along.

D> By the way, I'm currently playing around with the registry labels and
D> was wondering: what would currently be the best way to get all message
D> ids with a certain label? This way, I could easily create nnmairix
D> groups containing only these messages.

Currently there's no reverse mapping.  We have it for subject and
sender, and it should be fairly easy to generalize it for multi-value
entries like the labels.  See gnus-registry-fetch-extra-entry for the
cache insertion part; gnus-registry-split-fancy-with-parent uses the
cache.  I think the code could be much better, BTW.

Ted

Re: Problems with gnus-registry

[Ted Zlatanov]

On Thu, 14 Feb 2008 11:51:29 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Fri, 08 Feb 2008 12:58:17 +0100 David <de_bb@arcor.de> wrote: 

D> By the way, I'm currently playing around with the registry labels and
D> was wondering: what would currently be the best way to get all message
D> ids with a certain label? This way, I could easily create nnmairix
D> groups containing only these messages.

TZ> Currently there's no reverse mapping.  We have it for subject and
TZ> sender, and it should be fairly easy to generalize it for multi-value
TZ> entries like the labels.  See gnus-registry-fetch-extra-entry for the
TZ> cache insertion part; gnus-registry-split-fancy-with-parent uses the
TZ> cache.  I think the code could be much better, BTW.

David, do you still need this?  Have you tried maphash against the whole
registry to see if it's fast enough, or do we still need labels?

Ted

Re: Problems with gnus-registry

[Ted Zlatanov]

On Thu, 14 Feb 2008 11:45:04 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Thu, 07 Feb 2008 18:29:59 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
>>> They are used differently.  Ignored groups are not saved in the
>>> registry, but they are followed for splitting (this is incorrect, I
>>> think).  Unfollowed groups are not followed by
>>> gnus-registry-split-fancy-with-parent function but they are saved.
>>> 
>>> I think the two should be united in one variable, specifically
>>> gnus-registry-ignored-groups since it's more powerful and should do the
>>> same thing.

RS> As we want to store marks (or "labels") in the registry, wouldn't it
RS> make sense to have two different variables?  (Though both
RS> should/could contain "ignore" in the name, like the related nnmail-*
RS> variables.)  A user might want to ignore some groups for parent
RS> splitting, but want to set marks in these groups.

TZ> How about naming the single variable gnus-registry-ignored-groups (with
TZ> the corresponding topic/group parameter registry-ignore), with values
TZ> like this

TZ> "nnrss.*" t         ; meaning "ignore"
TZ> "nntp.*"  'nofollow ; meaning "do not follow"
TZ> "nnml.*"  nil       ; treat normally

TZ> It's probably simpler for the users than two separate variables,
TZ> especially since "do not follow" a group is a milder form of "ignore
TZ> this group."

TZ> I think registry labels will not care about this, they don't go by the
TZ> group at all, they only use the message ID.  The entry could have no
TZ> groups and the labels should still work fine.

Hm, no response on this yet.  Is there a better way?  Any opinions for
or against?

Ted

Re: Problems with gnus-registry

[David]

Ted Zlatanov <tzz@lifelogs.com> writes:
> D> By the way, I'm currently playing around with the registry labels and
> D> was wondering: what would currently be the best way to get all message
> D> ids with a certain label? This way, I could easily create nnmairix
> D> groups containing only these messages.
>
> TZ> Currently there's no reverse mapping.  We have it for subject and
> TZ> sender, and it should be fairly easy to generalize it for multi-value
> TZ> entries like the labels.  See gnus-registry-fetch-extra-entry for the
> TZ> cache insertion part; gnus-registry-split-fancy-with-parent uses the
> TZ> cache.  I think the code could be much better, BTW.
>
> David, do you still need this?  Have you tried maphash against the whole
> registry to see if it's fast enough, or do we still need labels?

Yes, I've tried it with maphash. It takes about 0.1sec and my registry
has ~17k entries. I'd say that's fast enough. :-)

-David