Re: gnus-registry flags API

[Ted Zlatanov <tzz@lifelogs.com>]

On Fri, 04 Jan 2008 18:43:21 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Thu, Jan 03 2008, Ted Zlatanov wrote:
>> On Thu, 03 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
RS> We need documentation in the manual for this as well as for
RS> `gnus-registry.el' in general.  The only hits for "registry" in the
RS> manual are in the context of spam*.el.
>> 
>> Yes.  I've been avoiding that work, pretending I was waiting for the
>> registry to mature :)
>> 
>> I've been using it for a while now, and the only real problem was with
>> multilingual characters corrupting the registry, and I think I've found
>> all the places where that could happen.  So it's probably mature enough
>> to popularize properly.  I'll do the docs once the items below have been
>> finished.

RS> I'd prefer if one thing would be finished before starting a new
RS> one. ;-)

OK, but since these changes require gnus-registry internal and external
API changes (e.g. storing the server, see below), I think it makes sense
to wait with the docs.

If anyone wants to write some docs sooner, though, I'll gladly review
them and incorporate changes.  Similarly, if anyone has comments, feel
free.

RS> BTW: I'd like to switch from `nnmail-split-fancy-with-parent' to
RS> `gnus-registry-split-fancy-with-parent'.  
>> 
>> Great, I think a lot of people will like it better.  Once this and
>> tramp-imap.el work, I'll try to get to cross-server mail splitting,
>> which has always seemed to me a really nice TODO item and I've needed it
>> many times.  I mention cross-server splitting because it relates to
>> gnus-registry splitting, since the registry will notice articles across
>> servers.

RS> Could you explain, why you don't store the server?  Or is this going
RS> to change?

I think the server is stored in the group name.  Do you mean why not
store it separately from the group name?

RS> Is there a conversion function from `nnmail-message-id-cache-file'
RS> to `gnus-registry-cache-file'?
>> 
>> Not currently, but the Gnus registry is built every time you visit a
>> group and see articles, so I doubt this is a big deal.  

RS> Yes, but mail splitting also should work for message/groups which I
RS> haven't seen for a while.

Sure, but keep in mind the registry in its current form gets big
quickly.  It should probably be compressed, if not trimmed.  If it's
trimmed then you only have the latest anyhow (in my experience 50,000
entries cover 99% of my needs); if we decide to compress it that's extra
work but probably good for the user's disk space.

>> We could write a simple conversion function, does anyone need it?

RS> Me. ;-) Is the "((mtime 18302 25797 153059))" sexp optional?  Then it
RS> would be trivial to write a conversion function.  It would be
RS> sufficient to convert once, i.e. fetch the entries from
RS> `nnmail-message-id-cache-file' and add them to
RS> `gnus-registry-hashtb'/`gnus-registry-alist'.

It's necessary for trimming the registry, because we need to sort it by
mtime to remove the oldest ones.  It's just the current time, so it
should be easy to automate adding it in the conversion function.

RS> BTW, some time ago, I added this comment:

RS> ,----
RS> | ;; FIXME: Get rid of duplicated code, cf. `gnus-save-newsrc-file' in
RS> | ;; `gnus-start.el'.  --rsteib
RS> | (defun gnus-registry-cache-save ()
RS> `----

I'd like that too.  I haven't fixed it myself, due to lack of time
mostly.

I think I had a good reason originally to copy that code.  I think it
did too much for what I needed, and refactoring was not an option
because it's so critical to Gnus users.  WDYT?  Is this really a
necessity for a release?

RS> I did `M-: (gnus-registry-mark-article nil) RET' without initializing
RS> the registry.  AFAICS, this is a no-op -- at least no
RS> ~/.gnus.registry.eld was created.  It should probably signal an error.
>> 
>> On load, we do: [...]
>> So afterwards, all the operations will work on that hash table
>> automatically.  You need to save the registry (which is automatically
>> done on load with (add-hook 'gnus-save-newsrc-hook 'gnus-registry-save))
>> to get the file.   [...]

RS> I know that I should do `gnus-registry-initialize' from the commentary
RS> in `gnus-registry.el'.  But I think operations like
RS> `gnus-registry-mark-article' should warn the user about it.

Well, we get to two questions:

1) do we push the registry on everyone?  If yes, then we should do 

(when (file-exists-p gnus-registry-cache-file)
   (gnus-registry-initialize))

in gnus.el.  If not...

2) when we notify the user, we should offer to turn
gnus-registry-enabled on, and in gnus.el we'll have

(when gnus-registry-enabled 
   (gnus-registry-initialize))

Either approach is OK with me.  It really depends on how much everyone
likes the registry and wants it on by default.  IMO (2) is the better
approach but complicates the code since we have to do the enabled check
everywhere we offer user functionality.

RS> When called interactively, it should prompt for flags with completion.
RS> At least the Mozilla labels ("Important" "Work" "Personal" "To do"
RS> "Later" <http://www.mozilla.org/mailnews/specs/labels/>) should be
RS> offered by default.  The user may add additional flags to this list
RS> and also set arbitrary flags.
>> 
>> How do you propose I do this, with a customizable list in the registry
>> namespace or a Gnus-wide customization setting?
>> 
>> Either way it shouldn't be too hard, though I haven't done it myself.

RS> It doesn't matter much.  But as long as it is specific to
RS> `gnus-registry.el', it might be better to use it's namespace.

OK, I'll use those labels (TODO#1 for me).

>> Isn't the Important flag confusing?  It may be seen as conflicting with
>> the Gnus tick mark.  I'm OK with this list otherwise.

RS> It just needs to be documented properly, I think.

>>>> gnus-registry-article-marks ARTICLE: get list of marks for the article
>>>> 
>>>> - if ARTICLE is nil, use current article
>> 
RS> Same remark as for `rs-gnus-summary-line-label-alist'.
>> 
>> Not sure what you mean?

RS> [ Copy&paste ... ]
RS> When used interactively, the current article should be the default
RS> too.  Now it insists on "Please enter a number.".  Maybe `article'
RS> should be optional?

Ah OK.  rs-gnus-summary-line-label-alist threw me off in your statement.

>>>> Let me know if you find bugs or inconsistencies.  I am especially
>>>> curious how these should integrate with the other Gnus summary
>>>> functions, and if they should take prefix arguments, etc.
>> 
RS> It should accept Gnus' standard process/prefix convention.
>> 
>> Is there one prototype function I can copy for this?

RS> See e.g. the use of `gnus-summary-work-articles' in
RS> `gnus-draft-send-message'.  Though instead of (while (setq article
RS> (pop articles)) ...) you should probably use `dolist' (which was not
RS> available in all Emacsen previously).

OK (TODO#2).

RS> Some other thoughts/remarks...

RS> - As the user probably doesn't want the flags to expire,
RS>   `gnus-registry-trim' should not trim those MIDs.

That's extra data, and gnus-registry-trim should not kill articles with
extra data.  I thought I had that functionality but must have forgotten
to add it.  No problem.  (TODO#3).

RS> - How can I avoid that MIDs from nntp groups are stored?
RS>   Through `gnus-registry-ignored-groups'?  It's not clear if it
RS>   matches the full (prefixed) group name.

It will match an unanchored regular expression, so "^nntp" should do
it.  We could add a gnus-registry-ignored-methods, which by default is
("nntp" "nnrss" "nndraft" "nnil") or something similar, and then match
on "^method:".  Let me know which one you like better, I have no
preference.

Ted