gnus-registry flags API

gnus-registry flags API

[Ted Zlatanov]

I added an API to the gnus-registry for reading and writing flags.  It's
very simple (id is always the message ID):

;; get the list of flags (note it's a list)
(gnus-registry-fetch-extra-flags id)
;; set 'important and 'todo flags (note the parameters are not a list)
(gnus-registry-store-extra-flags id 'important 'todo)
;; remove just the 'important flag
(gnus-registry-delete-extra-flags id 'important)
;; is 'important set?
(gnus-registry-has-extra-flag id 'important)
;; remove all the flags
(gnus-registry-store-extra-flags id nil)
;; convenience function for the above
(gnus-registry-delete-all-extra-flags id)

Because of the design of the gnus-registry, these flags will persist as
a message gets moved between groups.  The flag data is *not* stored in
the newsrc file, which I think is an advantage over server-side flags
for certain situations.  Code could be written to synchronize
server-side flags with gnus-registry flags, though I think it's more
valuable as a parallel facility that backends don't use, only intended
for frontend functionality.

Caching the flags (for quick lookup of all the 'important messages, for
example) will need some extra work, as the current caching assumes that
the extras value associated with a particular message and extras key is
not a list.

The frontend functionality I would like:

- set a flag for an article (this should be a simple function a user can
call from a group or message buffer, which grabs the message ID and uses
the gnus-registry API afterwards)

- indicate the article flags somehow (fringe or status line maybe?)

- add summary line formatting support for arbitrary flags

Any suggestions or contributions (especially the three items above) are
welcome.  Please review and criticize the code if you can, I haven't
done Emacs Lisp in a bit :)

Ted

Re: gnus-registry flags API

[Bastien]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Any suggestions or contributions (especially the three items above)
> are welcome. Please review and criticize the code if you can, I
> haven't done Emacs Lisp in a bit :)

I cannot afford to work on this right night, but I'm really looking
forward to it!

I spend 50% of my Emacs-time using Org-mode, sometimes storing email as
TODO items, and it would be great to be able to decide the state of the
TODO item depending on such a flag.

Regards,

-- 
Bastien

Re: gnus-registry flags API

[Leo]

On 2007-09-25 17:26 +0100, Ted Zlatanov wrote:
> I added an API to the gnus-registry for reading and writing flags.  It's
> very simple (id is always the message ID):
>
> ;; get the list of flags (note it's a list)
> (gnus-registry-fetch-extra-flags id)
> ;; set 'important and 'todo flags (note the parameters are not a list)
> (gnus-registry-store-extra-flags id 'important 'todo)
> ;; remove just the 'important flag
> (gnus-registry-delete-extra-flags id 'important)
> ;; is 'important set?
> (gnus-registry-has-extra-flag id 'important)
> ;; remove all the flags
> (gnus-registry-store-extra-flags id nil)
> ;; convenience function for the above
> (gnus-registry-delete-all-extra-flags id)
[...]

This looks like one of the most exciting features over the past year.

-- 
.:  Leo  :.  [ sdl.web AT gmail.com ]  .:  [ GPG Key: 9283AA3F ]  :.

       Use the most powerful email client -- http://gnus.org/

Re: gnus-registry flags API

[Ted Zlatanov]

On Tue, 25 Sep 2007 11:26:08 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> The frontend functionality I would like:

TZ> - set a flag for an article (this should be a simple function a user can
TZ> call from a group or message buffer, which grabs the message ID and uses
TZ> the gnus-registry API afterwards)

TZ> - indicate the article flags somehow (fringe or status line maybe?)

TZ> - add summary line formatting support for arbitrary flags

TZ> Any suggestions or contributions (especially the three items above) are
TZ> welcome.  Please review and criticize the code if you can, I haven't
TZ> done Emacs Lisp in a bit :)

I added the essential functions (first on the list) in CVS, now we need
a status line / summary line integration for these.  Note that the MARK
here is really any Lisp symbol, so 'hello-there-flag and
'goodbye-here-but-I-like-verbosity-flag are OK.  Go wild.

gnus-registry-mark-article ARTICLE &optional MARK REMOVE

- if ARTICLE is nil, use current article

- if MARK is nil, use gnus-registry-flag-default (you don't want that,
  it's boring)

- if REMOVE is t, remove MARK from the flags

gnus-registry-article-marks ARTICLE: get list of marks for the article

- if ARTICLE is nil, use current article

Let me know if you find bugs or inconsistencies.  I am especially
curious how these should integrate with the other Gnus summary
functions, and if they should take prefix arguments, etc.  For me that's
not a big deal.

Ted

Re: gnus-registry flags API

[Reiner Steib]

On Wed, Dec 19 2007, Ted Zlatanov wrote:

> On Tue, 25 Sep 2007 11:26:08 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 
>
> TZ> The frontend functionality I would like:
>
> TZ> - set a flag for an article (this should be a simple function a user can
> TZ> call from a group or message buffer, which grabs the message ID and uses
> TZ> the gnus-registry API afterwards)
>
> TZ> - indicate the article flags somehow (fringe or status line maybe?)
>
> TZ> - add summary line formatting support for arbitrary flags
>
> TZ> Any suggestions or contributions (especially the three items above) are
> TZ> welcome.  Please review and criticize the code if you can, I haven't
> TZ> done Emacs Lisp in a bit :)
>
> I added the essential functions (first on the list) in CVS, now we need
> a status line / summary line integration for these.  

We need documentation in the manual for this as well as for
`gnus-registry.el' in general.  The only hits for "registry" in the
manual are in the context of spam*.el.

BTW: I'd like to switch from `nnmail-split-fancy-with-parent' to
`gnus-registry-split-fancy-with-parent'.  Is there a conversion
function from `nnmail-message-id-cache-file' to
`gnus-registry-cache-file'?

> Note that the MARK here is really any Lisp symbol, so
> 'hello-there-flag and 'goodbye-here-but-I-like-verbosity-flag are
> OK.  Go wild.
>
> gnus-registry-mark-article ARTICLE &optional MARK REMOVE
>
> - if ARTICLE is nil, use current article

When used interactively, the current article should be the default
too.  Now it insists on "Please enter a number.".  Maybe `article'
should be optional?

I did `M-: (gnus-registry-mark-article nil) RET' without initializing
the registry.  AFAICS, this is a no-op -- at least no
~/.gnus.registry.eld was created.  It should probably signal an error.

> - if MARK is nil, use gnus-registry-flag-default (you don't want that,
>   it's boring)

When called interactively, it should prompt for flags with completion.
At least the Mozilla labels ("Important" "Work" "Personal" "To do"
"Later" <http://www.mozilla.org/mailnews/specs/labels/>) should be
offered by default.  The user may add additional flags to this list
and also set arbitrary flags.

> - if REMOVE is t, remove MARK from the flags
>
> gnus-registry-article-marks ARTICLE: get list of marks for the article
>
> - if ARTICLE is nil, use current article

Same remark as for `rs-gnus-summary-line-label-alist'.

> Let me know if you find bugs or inconsistencies.  I am especially
> curious how these should integrate with the other Gnus summary
> functions, and if they should take prefix arguments, etc.

It should accept Gnus' standard process/prefix convention.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: gnus-registry flags API

[Ted Zlatanov]

On Thu, 03 Jan 2008 18:10:51 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Wed, Dec 19 2007, Ted Zlatanov wrote:
>> On Tue, 25 Sep 2007 11:26:08 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 
>> 
TZ> The frontend functionality I would like:
>> 
TZ> - set a flag for an article (this should be a simple function a user can
TZ> call from a group or message buffer, which grabs the message ID and uses
TZ> the gnus-registry API afterwards)
>> 
TZ> - indicate the article flags somehow (fringe or status line maybe?)
>> 
TZ> - add summary line formatting support for arbitrary flags
>> 
TZ> Any suggestions or contributions (especially the three items above) are
TZ> welcome.  Please review and criticize the code if you can, I haven't
TZ> done Emacs Lisp in a bit :)
>> 
>> I added the essential functions (first on the list) in CVS, now we need
>> a status line / summary line integration for these.  

RS> We need documentation in the manual for this as well as for
RS> `gnus-registry.el' in general.  The only hits for "registry" in the
RS> manual are in the context of spam*.el.

Yes.  I've been avoiding that work, pretending I was waiting for the
registry to mature :)

I've been using it for a while now, and the only real problem was with
multilingual characters corrupting the registry, and I think I've found
all the places where that could happen.  So it's probably mature enough
to popularize properly.  I'll do the docs once the items below have been
finished.

RS> BTW: I'd like to switch from `nnmail-split-fancy-with-parent' to
RS> `gnus-registry-split-fancy-with-parent'.  

Great, I think a lot of people will like it better.  Once this and
tramp-imap.el work, I'll try to get to cross-server mail splitting,
which has always seemed to me a really nice TODO item and I've needed it
many times.  I mention cross-server splitting because it relates to
gnus-registry splitting, since the registry will notice articles across
servers.

RS> Is there a conversion function from `nnmail-message-id-cache-file'
RS> to `gnus-registry-cache-file'?

Not currently, but the Gnus registry is built every time you visit a
group and see articles, so I doubt this is a big deal.  We could write a
simple conversion function, does anyone need it?

>> Note that the MARK here is really any Lisp symbol, so
>> 'hello-there-flag and 'goodbye-here-but-I-like-verbosity-flag are
>> OK.  Go wild.
>> 
>> gnus-registry-mark-article ARTICLE &optional MARK REMOVE
>> 
>> - if ARTICLE is nil, use current article

RS> When used interactively, the current article should be the default
RS> too.  Now it insists on "Please enter a number.".  Maybe `article'
RS> should be optional?

Yes, I think that's my error, and it should not be doing that
interactive query.  I'll add the necessary code for this plus the
process convention plus flag completion (see below).

RS> I did `M-: (gnus-registry-mark-article nil) RET' without initializing
RS> the registry.  AFAICS, this is a no-op -- at least no
RS> ~/.gnus.registry.eld was created.  It should probably signal an error.

On load, we do:

(defvar gnus-registry-hashtb (make-hash-table 			    
			      :size 256
			      :test 'equal)
  "*The article registry by Message ID.")

So afterwards, all the operations will work on that hash table
automatically.  You need to save the registry (which is automatically
done on load with (add-hook 'gnus-save-newsrc-hook 'gnus-registry-save))
to get the file.  You also need to read that file on startup; the
default setup is:

(when gnus-registry-install
  (gnus-registry-install-hooks)
  (gnus-registry-read))

>> - if MARK is nil, use gnus-registry-flag-default (you don't want that,
>> it's boring)

RS> When called interactively, it should prompt for flags with completion.
RS> At least the Mozilla labels ("Important" "Work" "Personal" "To do"
RS> "Later" <http://www.mozilla.org/mailnews/specs/labels/>) should be
RS> offered by default.  The user may add additional flags to this list
RS> and also set arbitrary flags.

How do you propose I do this, with a customizable list in the registry
namespace or a Gnus-wide customization setting?

Either way it shouldn't be too hard, though I haven't done it myself.

Isn't the Important flag confusing?  It may be seen as conflicting with
the Gnus tick mark.  I'm OK with this list otherwise.

>> - if REMOVE is t, remove MARK from the flags
>> 
>> gnus-registry-article-marks ARTICLE: get list of marks for the article
>> 
>> - if ARTICLE is nil, use current article

RS> Same remark as for `rs-gnus-summary-line-label-alist'.

Not sure what you mean?

>> Let me know if you find bugs or inconsistencies.  I am especially
>> curious how these should integrate with the other Gnus summary
>> functions, and if they should take prefix arguments, etc.

RS> It should accept Gnus' standard process/prefix convention.

Is there one prototype function I can copy for this?

Thanks
Ted

Re: gnus-registry flags API

[Reiner Steib]

On Thu, Jan 03 2008, Ted Zlatanov wrote:

> On Thu, 03 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
> RS> We need documentation in the manual for this as well as for
> RS> `gnus-registry.el' in general.  The only hits for "registry" in the
> RS> manual are in the context of spam*.el.
>
> Yes.  I've been avoiding that work, pretending I was waiting for the
> registry to mature :)
>
> I've been using it for a while now, and the only real problem was with
> multilingual characters corrupting the registry, and I think I've found
> all the places where that could happen.  So it's probably mature enough
> to popularize properly.  I'll do the docs once the items below have been
> finished.

I'd prefer if one thing would be finished before starting a new
one. ;-)

> RS> BTW: I'd like to switch from `nnmail-split-fancy-with-parent' to
> RS> `gnus-registry-split-fancy-with-parent'.  
>
> Great, I think a lot of people will like it better.  Once this and
> tramp-imap.el work, I'll try to get to cross-server mail splitting,
> which has always seemed to me a really nice TODO item and I've needed it
> many times.  I mention cross-server splitting because it relates to
> gnus-registry splitting, since the registry will notice articles across
> servers.

Could you explain, why you don't store the server?  Or is this going
to change?

> RS> Is there a conversion function from `nnmail-message-id-cache-file'
> RS> to `gnus-registry-cache-file'?
>
> Not currently, but the Gnus registry is built every time you visit a
> group and see articles, so I doubt this is a big deal.  

Yes, but mail splitting also should work for message/groups which I
haven't seen for a while.

> We could write a simple conversion function, does anyone need it?

Me. ;-) Is the "((mtime 18302 25797 153059))" sexp optional?  Then it
would be trivial to write a conversion function.  It would be
sufficient to convert once, i.e. fetch the entries from
`nnmail-message-id-cache-file' and add them to
`gnus-registry-hashtb'/`gnus-registry-alist'.

BTW, some time ago, I added this comment:

,----
| ;; FIXME: Get rid of duplicated code, cf. `gnus-save-newsrc-file' in
| ;; `gnus-start.el'.  --rsteib
| (defun gnus-registry-cache-save ()
`----

> RS> I did `M-: (gnus-registry-mark-article nil) RET' without initializing
> RS> the registry.  AFAICS, this is a no-op -- at least no
> RS> ~/.gnus.registry.eld was created.  It should probably signal an error.
>
> On load, we do: [...]
> So afterwards, all the operations will work on that hash table
> automatically.  You need to save the registry (which is automatically
> done on load with (add-hook 'gnus-save-newsrc-hook 'gnus-registry-save))
> to get the file.   [...]

I know that I should do `gnus-registry-initialize' from the commentary
in `gnus-registry.el'.  But I think operations like
`gnus-registry-mark-article' should warn the user about it.

I now did `gnus-registry-initialize' and after displaying some
articles their MIDs end in ~/.gnus.registry.eld.  But `M-:
(gnus-registry-mark-article nil) RET' doesn't seem to have any effect:
The MID is written, but I don't see anything like
`gnus-registry-flag-default' there.  Additionally, the command doesn't
work if there's no article buffer (e.g. enter the group using `1
RET').  You should use (gnus-summary-article-number) instead of
`gnus-current-article'.

>>> - if MARK is nil, use gnus-registry-flag-default (you don't want that,
>>> it's boring)
>
> RS> When called interactively, it should prompt for flags with completion.
> RS> At least the Mozilla labels ("Important" "Work" "Personal" "To do"
> RS> "Later" <http://www.mozilla.org/mailnews/specs/labels/>) should be
> RS> offered by default.  The user may add additional flags to this list
> RS> and also set arbitrary flags.
>
> How do you propose I do this, with a customizable list in the registry
> namespace or a Gnus-wide customization setting?
>
> Either way it shouldn't be too hard, though I haven't done it myself.

It doesn't matter much.  But as long as it is specific to
`gnus-registry.el', it might be better to use it's namespace.

> Isn't the Important flag confusing?  It may be seen as conflicting with
> the Gnus tick mark.  I'm OK with this list otherwise.

It just needs to be documented properly, I think.

>>> gnus-registry-article-marks ARTICLE: get list of marks for the article
>>> 
>>> - if ARTICLE is nil, use current article
>
> RS> Same remark as for `rs-gnus-summary-line-label-alist'.
>
> Not sure what you mean?

[ Copy&paste ... ]
When used interactively, the current article should be the default
too.  Now it insists on "Please enter a number.".  Maybe `article'
should be optional?

>>> Let me know if you find bugs or inconsistencies.  I am especially
>>> curious how these should integrate with the other Gnus summary
>>> functions, and if they should take prefix arguments, etc.
>
> RS> It should accept Gnus' standard process/prefix convention.
>
> Is there one prototype function I can copy for this?

See e.g. the use of `gnus-summary-work-articles' in
`gnus-draft-send-message'.  Though instead of (while (setq article
(pop articles)) ...) you should probably use `dolist' (which was not
available in all Emacsen previously).

Some other thoughts/remarks...

- As the user probably doesn't want the flags to expire,
  `gnus-registry-trim' should not trim those MIDs.

- How can I avoid that MIDs from nntp groups are stored?
  Through `gnus-registry-ignored-groups'?  It's not clear if it
  matches the full (prefixed) group name.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: gnus-registry flags API

[Ted Zlatanov]

On Fri, 04 Jan 2008 18:43:21 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Thu, Jan 03 2008, Ted Zlatanov wrote:
>> On Thu, 03 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
RS> We need documentation in the manual for this as well as for
RS> `gnus-registry.el' in general.  The only hits for "registry" in the
RS> manual are in the context of spam*.el.
>> 
>> Yes.  I've been avoiding that work, pretending I was waiting for the
>> registry to mature :)
>> 
>> I've been using it for a while now, and the only real problem was with
>> multilingual characters corrupting the registry, and I think I've found
>> all the places where that could happen.  So it's probably mature enough
>> to popularize properly.  I'll do the docs once the items below have been
>> finished.

RS> I'd prefer if one thing would be finished before starting a new
RS> one. ;-)

OK, but since these changes require gnus-registry internal and external
API changes (e.g. storing the server, see below), I think it makes sense
to wait with the docs.

If anyone wants to write some docs sooner, though, I'll gladly review
them and incorporate changes.  Similarly, if anyone has comments, feel
free.

RS> BTW: I'd like to switch from `nnmail-split-fancy-with-parent' to
RS> `gnus-registry-split-fancy-with-parent'.  
>> 
>> Great, I think a lot of people will like it better.  Once this and
>> tramp-imap.el work, I'll try to get to cross-server mail splitting,
>> which has always seemed to me a really nice TODO item and I've needed it
>> many times.  I mention cross-server splitting because it relates to
>> gnus-registry splitting, since the registry will notice articles across
>> servers.

RS> Could you explain, why you don't store the server?  Or is this going
RS> to change?

I think the server is stored in the group name.  Do you mean why not
store it separately from the group name?

RS> Is there a conversion function from `nnmail-message-id-cache-file'
RS> to `gnus-registry-cache-file'?
>> 
>> Not currently, but the Gnus registry is built every time you visit a
>> group and see articles, so I doubt this is a big deal.  

RS> Yes, but mail splitting also should work for message/groups which I
RS> haven't seen for a while.

Sure, but keep in mind the registry in its current form gets big
quickly.  It should probably be compressed, if not trimmed.  If it's
trimmed then you only have the latest anyhow (in my experience 50,000
entries cover 99% of my needs); if we decide to compress it that's extra
work but probably good for the user's disk space.

>> We could write a simple conversion function, does anyone need it?

RS> Me. ;-) Is the "((mtime 18302 25797 153059))" sexp optional?  Then it
RS> would be trivial to write a conversion function.  It would be
RS> sufficient to convert once, i.e. fetch the entries from
RS> `nnmail-message-id-cache-file' and add them to
RS> `gnus-registry-hashtb'/`gnus-registry-alist'.

It's necessary for trimming the registry, because we need to sort it by
mtime to remove the oldest ones.  It's just the current time, so it
should be easy to automate adding it in the conversion function.

RS> BTW, some time ago, I added this comment:

RS> ,----
RS> | ;; FIXME: Get rid of duplicated code, cf. `gnus-save-newsrc-file' in
RS> | ;; `gnus-start.el'.  --rsteib
RS> | (defun gnus-registry-cache-save ()
RS> `----

I'd like that too.  I haven't fixed it myself, due to lack of time
mostly.

I think I had a good reason originally to copy that code.  I think it
did too much for what I needed, and refactoring was not an option
because it's so critical to Gnus users.  WDYT?  Is this really a
necessity for a release?

RS> I did `M-: (gnus-registry-mark-article nil) RET' without initializing
RS> the registry.  AFAICS, this is a no-op -- at least no
RS> ~/.gnus.registry.eld was created.  It should probably signal an error.
>> 
>> On load, we do: [...]
>> So afterwards, all the operations will work on that hash table
>> automatically.  You need to save the registry (which is automatically
>> done on load with (add-hook 'gnus-save-newsrc-hook 'gnus-registry-save))
>> to get the file.   [...]

RS> I know that I should do `gnus-registry-initialize' from the commentary
RS> in `gnus-registry.el'.  But I think operations like
RS> `gnus-registry-mark-article' should warn the user about it.

Well, we get to two questions:

1) do we push the registry on everyone?  If yes, then we should do 

(when (file-exists-p gnus-registry-cache-file)
   (gnus-registry-initialize))

in gnus.el.  If not...

2) when we notify the user, we should offer to turn
gnus-registry-enabled on, and in gnus.el we'll have

(when gnus-registry-enabled 
   (gnus-registry-initialize))

Either approach is OK with me.  It really depends on how much everyone
likes the registry and wants it on by default.  IMO (2) is the better
approach but complicates the code since we have to do the enabled check
everywhere we offer user functionality.

RS> When called interactively, it should prompt for flags with completion.
RS> At least the Mozilla labels ("Important" "Work" "Personal" "To do"
RS> "Later" <http://www.mozilla.org/mailnews/specs/labels/>) should be
RS> offered by default.  The user may add additional flags to this list
RS> and also set arbitrary flags.
>> 
>> How do you propose I do this, with a customizable list in the registry
>> namespace or a Gnus-wide customization setting?
>> 
>> Either way it shouldn't be too hard, though I haven't done it myself.

RS> It doesn't matter much.  But as long as it is specific to
RS> `gnus-registry.el', it might be better to use it's namespace.

OK, I'll use those labels (TODO#1 for me).

>> Isn't the Important flag confusing?  It may be seen as conflicting with
>> the Gnus tick mark.  I'm OK with this list otherwise.

RS> It just needs to be documented properly, I think.

>>>> gnus-registry-article-marks ARTICLE: get list of marks for the article
>>>> 
>>>> - if ARTICLE is nil, use current article
>> 
RS> Same remark as for `rs-gnus-summary-line-label-alist'.
>> 
>> Not sure what you mean?

RS> [ Copy&paste ... ]
RS> When used interactively, the current article should be the default
RS> too.  Now it insists on "Please enter a number.".  Maybe `article'
RS> should be optional?

Ah OK.  rs-gnus-summary-line-label-alist threw me off in your statement.

>>>> Let me know if you find bugs or inconsistencies.  I am especially
>>>> curious how these should integrate with the other Gnus summary
>>>> functions, and if they should take prefix arguments, etc.
>> 
RS> It should accept Gnus' standard process/prefix convention.
>> 
>> Is there one prototype function I can copy for this?

RS> See e.g. the use of `gnus-summary-work-articles' in
RS> `gnus-draft-send-message'.  Though instead of (while (setq article
RS> (pop articles)) ...) you should probably use `dolist' (which was not
RS> available in all Emacsen previously).

OK (TODO#2).

RS> Some other thoughts/remarks...

RS> - As the user probably doesn't want the flags to expire,
RS>   `gnus-registry-trim' should not trim those MIDs.

That's extra data, and gnus-registry-trim should not kill articles with
extra data.  I thought I had that functionality but must have forgotten
to add it.  No problem.  (TODO#3).

RS> - How can I avoid that MIDs from nntp groups are stored?
RS>   Through `gnus-registry-ignored-groups'?  It's not clear if it
RS>   matches the full (prefixed) group name.

It will match an unanchored regular expression, so "^nntp" should do
it.  We could add a gnus-registry-ignored-methods, which by default is
("nntp" "nnrss" "nndraft" "nnil") or something similar, and then match
on "^method:".  Let me know which one you like better, I have no
preference.

Ted

Re: gnus-registry flags API

[Ted Zlatanov]

On Tue, 15 Jan 2008 15:56:36 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> Sure, but keep in mind the registry in its current form gets big
TZ> quickly.  It should probably be compressed, if not trimmed.  If it's
TZ> trimmed then you only have the latest anyhow (in my experience 50,000
TZ> entries cover 99% of my needs); if we decide to compress it that's extra
TZ> work but probably good for the user's disk space.
...
TZ> Well, we get to two questions:

TZ> 1) do we push the registry on everyone?  If yes, then we should do 

TZ> (when (file-exists-p gnus-registry-cache-file)
TZ>    (gnus-registry-initialize))

TZ> in gnus.el.  If not...

TZ> 2) when we notify the user, we should offer to turn
TZ> gnus-registry-enabled on, and in gnus.el we'll have

TZ> (when gnus-registry-enabled 
TZ>    (gnus-registry-initialize))

TZ> Either approach is OK with me.  It really depends on how much everyone
TZ> likes the registry and wants it on by default.  IMO (2) is the better
TZ> approach but complicates the code since we have to do the enabled check
TZ> everywhere we offer user functionality.
...
TZ> It will match an unanchored regular expression, so "^nntp" should do
TZ> it.  We could add a gnus-registry-ignored-methods, which by default is
TZ> ("nntp" "nnrss" "nndraft" "nnil") or something similar, and then match
TZ> on "^method:".  Let me know which one you like better, I have no
TZ> preference.

Any opinions on the three items above?

Labels are done, but I've called them 'marks' consistently.  Before I
had flags, labels, and marks.  It was not nice.  The label is always
just a symbol.

The mark functionality in gnus-registry.el uses process/prefix
correctly, I think.

gnus-registry-trim will always keep articles with marks and any other
extra symbols you put in gnus-registry-extra-entries-precious.

Please test and review if you have a chance.

Thanks
Ted

Re: gnus-registry flags API

[Ted Zlatanov]

On Wed, 16 Jan 2008 15:52:29 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Tue, 15 Jan 2008 15:56:36 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 
TZ> Sure, but keep in mind the registry in its current form gets big
TZ> quickly.  It should probably be compressed, if not trimmed.  If it's
TZ> trimmed then you only have the latest anyhow (in my experience 50,000
TZ> entries cover 99% of my needs); if we decide to compress it that's extra
TZ> work but probably good for the user's disk space.
TZ> ...
TZ> Well, we get to two questions:

TZ> 1) do we push the registry on everyone?  If yes, then we should do 

TZ> (when (file-exists-p gnus-registry-cache-file)
TZ> (gnus-registry-initialize))

TZ> in gnus.el.  If not...

TZ> 2) when we notify the user, we should offer to turn
TZ> gnus-registry-enabled on, and in gnus.el we'll have

TZ> (when gnus-registry-enabled 
TZ> (gnus-registry-initialize))

TZ> Either approach is OK with me.  It really depends on how much everyone
TZ> likes the registry and wants it on by default.  IMO (2) is the better
TZ> approach but complicates the code since we have to do the enabled check
TZ> everywhere we offer user functionality.
TZ> ...
TZ> It will match an unanchored regular expression, so "^nntp" should do
TZ> it.  We could add a gnus-registry-ignored-methods, which by default is
TZ> ("nntp" "nnrss" "nndraft" "nnil") or something similar, and then match
TZ> on "^method:".  Let me know which one you like better, I have no
TZ> preference.

TZ> Any opinions on the three items above?

TZ> Labels are done, but I've called them 'marks' consistently.  Before I
TZ> had flags, labels, and marks.  It was not nice.  The label is always
TZ> just a symbol.

TZ> The mark functionality in gnus-registry.el uses process/prefix
TZ> correctly, I think.

TZ> gnus-registry-trim will always keep articles with marks and any other
TZ> extra symbols you put in gnus-registry-extra-entries-precious.

TZ> Please test and review if you have a chance.

I haven't heard back about these questions, so:

1)

Has anyone started using registry marks?  They work for me but let me
know if you have issues or questions.  Also a formatting function that
shows them in the modeline would be nice, but since there can be many
flags and little space I'm not sure what's the best way to display them.

Also, what keys should I bind to set custom flags in the group and
article buffers?

2)

If no one has objections, I'll set up the interactive query for registry
first-time use in gnus.el.  There will be three values for
gnus-registry-enabled:

nil: unknown, ask the user next time
t: enable
'no-thanks: user doesn't want it

Every user will have to answer it by default.  Is that a bad idea?  What
else can I do?

Ted

Re: gnus-registry flags API

[Reiner Steib]

On Wed, Feb 06 2008, Ted Zlatanov wrote:

> On Wed, 16 Jan 2008 15:52:29 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 
> TZ> 1) do we push the registry on everyone?  

I think it should be optional.

[...]
> TZ> gnus-registry-trim will always keep articles with marks and any other
> TZ> extra symbols you put in gnus-registry-extra-entries-precious.

Good.

> TZ> Please test and review if you have a chance.
>
> I haven't heard back about these questions, so:
>
> 1)
>
> Has anyone started using registry marks?  

I didn't, sorry.

> Also a formatting function that shows them in the modeline would be
> nice, but since there can be many flags and little space I'm not
> sure what's the best way to display them.

I'd expected them to show up in the summary buffer just like ticked,
answered, etc.  We could display a string or a small image (I never
tried):

  '(("Important" "I"    "summary_important")
    ("Work"      "W"    "summary_work")
    ("Personal"  "P"    "summary_personal")
    ("To do"     "T"    "summary_todo")
    ("Later"     "L"    "summary_later"))
  "Alist of regular expressions and summary line indicators."

("summary_*": images corresponding to the labels.)

Can we have multiple registry marks for a single article?  If not, we
may either display all strings/images or add a plus sign.

> Also, what keys should I bind to set custom flags in the group

s/group/summary/ ?

> and article buffers?

Maybe use `M m' or `M M' as prefix command?  (e.g. `M M t' for "To
do", ...)

> 2)
>
> If no one has objections, I'll set up the interactive query for registry
> first-time use in gnus.el.  There will be three values for
> gnus-registry-enabled:
>
> nil: unknown, ask the user next time
> t: enable
> 'no-thanks: user doesn't want it
>
> Every user will have to answer it by default.  Is that a bad idea?  

Yes.

> What else can I do?

I'd prefer an interface like `message-use-idna':

  :type '(choice (const :tag "Ask" ask)
		 (const :tag "Never" nil)
		 (const :tag "Always" t))

nil and t are clear, I think.  When `ask', prompt the user to enable
the registry when trying to set a mark (cf. the safe-local-variables
interface).  When the user allows, store gnus-registry-enabled = t in
via customize.

Is that feasible?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: gnus-registry flags API

[Ted Zlatanov]

On Sat, 16 Feb 2008 21:26:36 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Wed, Feb 06 2008, Ted Zlatanov wrote:
>> Also a formatting function that shows them in the modeline would be
>> nice, but since there can be many flags and little space I'm not
>> sure what's the best way to display them.

RS> I'd expected them to show up in the summary buffer just like ticked,
RS> answered, etc.  We could display a string or a small image (I never
RS> tried):

RS>   '(("Important" "I"    "summary_important")
RS>     ("Work"      "W"    "summary_work")
RS>     ("Personal"  "P"    "summary_personal")
RS>     ("To do"     "T"    "summary_todo")
RS>     ("Later"     "L"    "summary_later"))
RS>   "Alist of regular expressions and summary line indicators."

RS> ("summary_*": images corresponding to the labels.)

RS> Can we have multiple registry marks for a single article?  If not, we
RS> may either display all strings/images or add a plus sign.

Yes, multiples are allowed.  That's what makes it tricky to display
them, so I haven't done it yet :)

Maybe reserve 4 characters, and if more than 4 labels are set, show the
first 3 and then '+'.  In the modeline or in the article buffer with a
post-processing function we can show them all.  I don't know much about
this area of Gnus, especially image display, sorry.

RS> Maybe use `M m' or `M M' as prefix command?  (e.g. `M M t' for "To
RS> do", ...)

Sounds OK to me.  The keys can come from the alist of summary line
indicators above, so they'll make sense in conjunction to the user.

>> If no one has objections, I'll set up the interactive query for registry
>> first-time use in gnus.el.  There will be three values for
>> gnus-registry-enabled:

RS> I'd prefer an interface like `message-use-idna':

RS>   :type '(choice (const :tag "Ask" ask)
RS> 		 (const :tag "Never" nil)
RS> 		 (const :tag "Always" t))

RS> nil and t are clear, I think.  When `ask', prompt the user to enable
RS> the registry when trying to set a mark (cf. the safe-local-variables
RS> interface).  When the user allows, store gnus-registry-enabled = t in
RS> via customize.

Works for me.  I'll do it when I get a chance.

Ted

Re: gnus-registry flags API

[Reiner Steib]

On Thu, Feb 28 2008, Ted Zlatanov wrote:

> On Sat, 16 Feb 2008 21:26:36 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
> RS> Can we have multiple registry marks for a single article?  If not, we
> RS> may either display all strings/images or add a plus sign.
>
> Yes, multiples are allowed.  That's what makes it tricky to display
> them, so I haven't done it yet :)
>
> Maybe reserve 4 characters, 

Or just use the number from the format specifier, if feasible.

> and if more than 4 labels are set, show the first 3 and then '+'.
> In the modeline or in the article buffer with a post-processing
> function we can show them all.  I don't know much about this area of
> Gnus, especially image display, sorry.

Me neither.  But I'd guess we can do it like in
`gnus-mode-line-buffer-identification' (`add-text-properties') or
`gnus-picon-transform-address' (`propertize').

E.g. try this in *scratch* (while running Gnus):

(add-text-properties 4 5
		     (list 'display gnus-mode-line-image-cache
			   'help-echo "To do")
		     (current-buffer))

> RS> Maybe use `M m' or `M M' as prefix command?  (e.g. `M M t' for "To
> RS> do", ...)
>
> Sounds OK to me.  The keys can come from the alist of summary line
> indicators above, so they'll make sense in conjunction to the user.

Good idea.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: gnus-registry flags API

[Ted Zlatanov]

On Thu, 28 Feb 2008 21:04:04 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> Maybe use `M m' or `M M' as prefix command?  (e.g. `M M t' for "To
RS> do", ...)
>> 
>> Sounds OK to me.  The keys can come from the alist of summary line
>> indicators above, so they'll make sense in conjunction to the user.

I worked on the gnus-registry-marks format for a while, and what's there
now is, I think, what we need.

This is the static list of keys, from
gnus-registry-install-shortcuts-and-menus:

  (gnus-define-keys (gnus-registry-mark-map "M" gnus-summary-mark-map)
    "i" gnus-registry-set-article-Important-mark
    "I" gnus-registry-remove-article-Important-mark
    "w" gnus-registry-set-article-Work-mark
    "W" gnus-registry-remove-article-Work-mark
    "l" gnus-registry-set-article-Later-mark
    "L" gnus-registry-remove-article-Later-mark
    "p" gnus-registry-set-article-Personal-mark
    "P" gnus-registry-remove-article-Personal-mark
    "t" gnus-registry-set-article-To-Do-mark
    "T" gnus-registry-remove-article-To-Do-mark))

(so `M M i' will set the Important mark for example)

The functions are dynamically generated from the gnus-registry-marks
list, but I couldn't figure out how to generate the gnus-define-keys
call (as I mention in the comments).  Also it seems that each call will
reset previously defined shortcuts.  If anyone knows, please tell me
using function-name and shortcut.

Where should the menu shortcuts live?

I'll add the formatting functions next.

Ted

Re: gnus-registry flags API

[Ted Zlatanov]

On Thu, 28 Feb 2008 21:04:04 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Thu, Feb 28 2008, Ted Zlatanov wrote:
>> On Sat, 16 Feb 2008 21:26:36 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
RS> Can we have multiple registry marks for a single article?  If not, we
RS> may either display all strings/images or add a plus sign.
>> 
>> Yes, multiples are allowed.  That's what makes it tricky to display
>> them, so I haven't done it yet :)
>> 
>> Maybe reserve 4 characters, 

RS> Or just use the number from the format specifier, if feasible.

>> and if more than 4 labels are set, show the first 3 and then '+'.
>> In the modeline or in the article buffer with a post-processing
>> function we can show them all.  I don't know much about this area of
>> Gnus, especially image display, sorry.

RS> Me neither.  But I'd guess we can do it like in
RS> `gnus-mode-line-buffer-identification' (`add-text-properties') or
RS> `gnus-picon-transform-address' (`propertize').

RS> E.g. try this in *scratch* (while running Gnus):

RS> (add-text-properties 4 5
RS> 		     (list 'display gnus-mode-line-image-cache
RS> 			   'help-echo "To do")
RS> 		     (current-buffer))

I couldn't figure out the text properties, or how to update the summary
line as soon as a flag is set.  For now I've comitted a function
gnus-registry-user-format-function-M that shows the flags (unsorted, in
the order they were set).  But the user has to exit the group and
re-enter it to see the new flags.  I'll take a look again but if someone
knows how to do it, please let me know to save me all the work.

Also the string-building construct in that function

(defun gnus-registry-user-format-function-M (headers)
  (let* ((id (mail-header-message-id headers))
	 (marks (when id (gnus-registry-fetch-extra-marks id)))
	 (out ""))
    (dolist (mark marks)
      (let ((c (plist-get
		(cdr-safe
		 (assoc mark gnus-registry-marks)) :char)))
	(setq out (format "%s%s"
			  out
			  (if c
			      (char-to-string c)
			    "")))))
      out))

is ugly.  How would I do the same with mapcar?  I couldn't find a nice
way to eliminate nil entries (when the flag has no :char property) from
the resulting list I would pass to concat; concat would get a list like
(?l ?i nil) and fail.

Ted

Re: gnus-registry flags API

[Ted Zlatanov]

On Tue, 04 Mar 2008 16:43:46 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> I couldn't figure out the text properties, or how to update the summary
TZ> line as soon as a flag is set.  For now I've comitted a function
TZ> gnus-registry-user-format-function-M that shows the flags (unsorted, in
TZ> the order they were set).  But the user has to exit the group and
TZ> re-enter it to see the new flags.  I'll take a look again but if someone
TZ> knows how to do it, please let me know to save me all the work.

I still haven't done text properties, but the rest is working now.  See
gnus-registry-user-format-function-M and gnus-registry-install-shortcuts
for the details.

TZ> Also the string-building construct in that function
...
TZ> 	(setq out (format "%s%s"
TZ> 			  out
TZ> 			  (if c
TZ> 			      (char-to-string c)
TZ> 			    "")))))
...
TZ> is ugly.  How would I do the same with mapcar?  I couldn't find a nice
TZ> way to eliminate nil entries (when the flag has no :char property) from
TZ> the resulting list I would pass to concat; concat would get a list like
TZ> (?l ?i nil) and fail.

I figured it out, you just do (concat list1 nil list2) and it does the
right thing.  I didn't read the docs carefully.

TZ> The functions are dynamically generated from the gnus-registry-marks
TZ> list, but I couldn't figure out how to generate the gnus-define-keys
TZ> call (as I mention in the comments).  Also it seems that each call will
TZ> reset previously defined shortcuts.  If anyone knows, please tell me
TZ> using function-name and shortcut.

I figured this out, and gnus-registry-install-shortcuts will DTRT for
keyboard shortcuts.  The menus are hard-coded in gnus-sum.el and I
didn't think it was useful to generate them dynamically.

TZ> Where should the menu shortcuts live?

I put them under Gnus->Registry Marks to be near the other mark
operations.

Remaining:

- call (gnus-registry-install-p) in Gnus startup

- make gnus-registry-install-p call Customize if appropriate, I'm not
  sure

- text properties and images for each flag

It's finally coming together, so please test the functionality and let
me know what you think.

Thanks
Ted

Re: gnus-registry flags API

[Ted Zlatanov]

On Wed, 05 Mar 2008 13:00:54 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Tue, 04 Mar 2008 16:43:46 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 
TZ> I couldn't figure out the text properties, or how to update the summary
TZ> line as soon as a flag is set.  For now I've comitted a function
TZ> gnus-registry-user-format-function-M that shows the flags (unsorted, in
TZ> the order they were set).  But the user has to exit the group and
TZ> re-enter it to see the new flags.  I'll take a look again but if someone
TZ> knows how to do it, please let me know to save me all the work.

TZ> I still haven't done text properties, but the rest is working now.  See
TZ> gnus-registry-user-format-function-M and gnus-registry-install-shortcuts
TZ> for the details.

I played with the text properties, but they just don't look very nice.
The image needs to be aligned exactly with the character size or the
line is skewed, and I didn't want to spend the time getting that perfect
alignment.  Here's an example user formatting function which inserts an
image (as Reiner showed me) at the insertion point, in case someone
wants to play with it further.

(defun gnus-registry-user-format-function-propertized-M (headers)
  (let* ((id (mail-header-message-id headers))
	 (marks (when id (gnus-registry-fetch-extra-marks id))))
    (with-temp-buffer
      (dolist (mark marks)
	(let* ((info (cdr-safe (assoc mark gnus-registry-marks)))
	       (c (plist-get info :char))
	       (i (plist-get info :image)))
	  (when c
	    (insert c)
	    (when i
	      (add-text-properties (1- (point)) (point)
				   (list 'display gnus-mode-line-image-cache
					 'help-echo "To do")
				   (current-buffer))))))
      (buffer-string))))

Users can just use a Unicode character until someone provides a better
user formatting function.  Here's my Greek letter setup (omega for
"work" is a stretch, but I didn't want to overthink this :)

(setq gnus-registry-marks  
      '((Important
     :char ?ι)
    (Work
     :char ?ω)
    (Personal
     :char ?π)
    (To-Do
     :char ?τ)
    (Later
     :char ?λ)))

Note :image is not specified as it is in the default specification of
the variable.  It's not used, and none of the current functionality
needs it, so it can be safely omitted.

Ted