`gnus-unseen-mark' everywhere

`gnus-unseen-mark' everywhere

[Robert Epprecht]

After a cvs update this morning *all* articles in all mail groups
('nnml') had the `gnus-unseen-mark'.

In news groups (that's another Gnus incarnations for me) there are
no gnus-unseen-mark's, but on some articles I think there should be
one...

Robert Epprecht

Re: `gnus-unseen-mark' everywhere

[Lars Magne Ingebrigtsen]

Robert Epprecht <epprecht@sunweb.ch> writes:

> After a cvs update this morning *all* articles in all mail groups
> ('nnml') had the `gnus-unseen-mark'.
>
> In news groups (that's another Gnus incarnations for me) there are
> no gnus-unseen-mark's, but on some articles I think there should be
> one...

What does the `seen' list in the groups look like?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: `gnus-unseen-mark' everywhere

[Robert Epprecht]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Robert Epprecht <epprecht@sunweb.ch> writes:
>
>> After a cvs update this morning *all* articles in all mail groups
>> ('nnml') had the `gnus-unseen-mark'.
>>
>> In news groups (that's another Gnus incarnations for me) there are
>> no gnus-unseen-mark's, but on some articles I think there should be
>> one...

I couldn't reproduce this part (news), so I'm only talking about *mail*.

> What does the `seen' list in the groups look like?

from ~/.newsrc.eld
("Aljoscha-IN"
 3 ((1 . 117))
 ((seen 54 (74 . 75) 78 (86 . 88) 102 109 113 (118 . 119))
  (forward 78 93)
  (reply 54 76 (78 . 79) 82 85 (87 . 90) 95 (97 . 100) 103 (105 . 111)
   (113 . 116))
  (tick 54 (74 . 75) 78 (86 . 88) 102 109 113)))

from ~/Mail/Aljoscha-IN/.marks
((tick 54 (74 . 75) 78 (86 . 88) 102 109 113)
 (read (1 . 117))
 (reply 54 76 (78 . 79) 82 85 (87 . 90) 95 (97 . 100) 103 (105 . 111)
  (113 . 116))
 (forward 78 93))

Why do some sublists show up in both files and some don't?
Maybe this is the problem?

Robert Epprecht

Re: `gnus-unseen-mark' everywhere

[Lars Magne Ingebrigtsen]

Robert Epprecht <epprecht@sunweb.ch> writes:

>>> After a cvs update this morning *all* articles in all mail groups
>>> ('nnml') had the `gnus-unseen-mark'.

[...]

>  ((seen 54 (74 . 75) 78 (86 . 88) 102 109 113 (118 . 119))

[...]

> from ~/Mail/Aljoscha-IN/.marks
> ((tick 54 (74 . 75) 78 (86 . 88) 102 109 113)
>  (read (1 . 117))
>  (reply 54 76 (78 . 79) 82 85 (87 . 90) 95 (97 . 100) 103 (105 . 111)
>   (113 . 116))
>  (forward 78 93))

So it would seem as if the `seen' marks don't land in .marks, and this
makes all your mail appear as unseen.  I'm not really familiar with
the .marks code, so could somebody else have a go at this?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>>  ((seen 54 (74 . 75) 78 (86 . 88) 102 109 113 (118 . 119))
>
> [...]
>
>> from ~/Mail/Aljoscha-IN/.marks
>> ((tick 54 (74 . 75) 78 (86 . 88) 102 109 113)
>>  (read (1 . 117))
>>  (reply 54 76 (78 . 79) 82 85 (87 . 90) 95 (97 . 100) 103 (105 . 111)
>>   (113 . 116))
>>  (forward 78 93))
>
> So it would seem as if the `seen' marks don't land in .marks, and this
> makes all your mail appear as unseen.  I'm not really familiar with
> the .marks code, so could somebody else have a go at this?

This is intended as a feature -- Gnus (i.e. .newsrc.eld) remembers
`seen' marks.  Backends remembers `recent' marks.  If the backend
would remember `seen', it would make the two flags have the same
semantics. (I'd admit that the semantic difference between the two is
a little obscure, but I find the distinction useful.)  See below.

But the seen mark should go away once you have seen the message once.
(Assuming you don't use several Gnus installations or delete
.newsrc.eld, or something.)  Doesn't this happen?

;; Propagate flags to server, with the following exceptions:
;; `seen' is private to each gnus installation
;; `cache' is a internal gnus flag for each gnus installation
;; `download' is a agent flag private to each gnus installation
;; `unsend' are for nndraft groups only
;; `score' is not a proper mark
(defconst gnus-article-unpropagated-mark-lists
  '(seen cache download unsend score)
  "Marks that shouldn't be propagated to backends.
Typical marks are those that make no sense in a standalone backend,
such as a mark that says whether an article is stored in the cache
(which doesn't make sense in a standalone backend).")

Re: `gnus-unseen-mark' everywhere

[Robert Epprecht]

Simon Josefsson <jas@extundo.com> writes:

> But the seen mark should go away once you have seen the message once.
> (Assuming you don't use several Gnus installations or delete
> .newsrc.eld, or something.)  Doesn't this happen?

No.  If I restart Gnus all the articles in my 'nnml' groups have the 
`gnus-unseen-mark'. If I leave a group and re-enter, they are gone.
But if I re-start Gnus, they are all back here again.

Robert Epprecht

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> This is intended as a feature -- Gnus (i.e. .newsrc.eld) remembers
> `seen' marks.  Backends remembers `recent' marks.  If the backend
> would remember `seen', it would make the two flags have the same
> semantics.

I still disagree on that.  It seems to me that the semantic difference
is only a matter of where changes are initiated for each mark.  This
ought to be orthogonal to how the marks are stored.


paul

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> This is intended as a feature -- Gnus (i.e. .newsrc.eld) remembers
>> `seen' marks.  Backends remembers `recent' marks.  If the backend
>> would remember `seen', it would make the two flags have the same
>> semantics.
>
> I still disagree on that.  It seems to me that the semantic difference
> is only a matter of where changes are initiated for each mark.  This
> ought to be orthogonal to how the marks are stored.

Hm, yes, but I think the issue is mixed if you consider multi-user
scenarios.  Consider a nnml folder which two persons access.  The
recent mark, maintained by nnml.el, is the same for both persons (now,
actually, nnml doesn't set the recent mark, but for sake of argument).
The seen mark however cannot be stored in the backend because they
would become the same for both persons.  The seen mark should be a
kind of "local" seenedness, and "recent" a remote/backend seenedness.

I guess the problem is that the marks are not divided between one set
of global marks that apply to all users of the backend and one local
set of user-specific marks.  Renaming "seen" to "seen-<username>"
would solve that, but perhaps keeping "seen" in .newsrc.eld and
"recent" in the backend is good enough.

Do you see the problem I think I see?

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> Consider a nnml folder which two persons access.  The recent mark,
> maintained by nnml.el, is the same for both persons (now, actually,
> nnml doesn't set the recent mark, but for sake of argument).  The
> seen mark however cannot be stored in the backend because they would
> become the same for both persons.

You're making assumptions about how sharing can be done that happen to
be false for nnmaildir.  In a shared nnmaildir group, typically the
marks are all not shared and only the messages are, but it's also
possible to share some kinds of marks and not others.  If other
backends cannot do this, then I think it should be those backends that
decide to not store (nor update Gnus's copy of) the seen marks, rather
than Gnus that assumes that backends are incapable of handling it
properly.

> Do you see the problem I think I see?

Yes, I just think the workaround is in the wrong place.


paul

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> Consider a nnml folder which two persons access.  The recent mark,
>> maintained by nnml.el, is the same for both persons (now, actually,
>> nnml doesn't set the recent mark, but for sake of argument).  The
>> seen mark however cannot be stored in the backend because they would
>> become the same for both persons.
>
> You're making assumptions about how sharing can be done that happen to
> be false for nnmaildir.  In a shared nnmaildir group, typically the
> marks are all not shared and only the messages are, but it's also
> possible to share some kinds of marks and not others.

This seems to be the same as for nnimap.

> If other backends cannot do this, then I think it should be those
> backends that decide to not store (nor update Gnus's copy of) the
> seen marks, rather than Gnus that assumes that backends are
> incapable of handling it properly.

Ah.  Hm, I think I understand what you are saying.  If the backend
doesn't support the distinction between per-user and global marks, it
should rely on .newsrc.eld and not touch the mark. Nnml/nnfolder would
not store seen in .marks, and would not reset the info handed to it
via nnchoke-request-update-info.  I think I agree now. :-) It wouldn't
change how anything would work.  Yes.

The gnus-article-unpropagatable-p etc stuff should be moved into
nnml/nnfolder (perhaps nnmail) then, because we wouldn't want `seen'
marks to be stored in nnml .marks.  Hm, but this means the backends
has knowledge about the semantics of marks, which is wrong as well.

So, either Gnus assumes that backends should only handle global marks
(current solution), or the assumption is that backends need to be
aware that, e.g., seen marks should not be stored in the backend if it
can only provide global marks.  It would be nice if the backend
wouldn't have to make that kind of distinction (because the backend
shouldn't have to "understand" all flags Gnus uses).

Ok, I now convinced myself that none of the solutions is good.  Time
to sleep..

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> The gnus-article-unpropagatable-p etc stuff should be moved into
> nnml/nnfolder (perhaps nnmail) then, because we wouldn't want `seen'
> marks to be stored in nnml .marks.

Well, at least some things other than "seen" should probably be still
dealt with as they are now.  E.g., "cache" really shouldn't be stored
in any backend, because the information "cache" represents has nothing
to do with what is stored in the backend.  We could add a new
mechanism for handling "seen" (and others like it, if there are any).

> Hm, but this means the backends has knowledge about the semantics of
> marks, which is wrong as well.

Gnus could define a list of marks that are supposed to be per-user in
shared groups; Gnus knows the semantics of the marks and thus knows
which marks should be included.  (And users might have different ideas
about what ought to be per-user, so this could even be defcustom'ed.)
Backends could then DTRT with whichever marks are included in that
variable, without knowing *why* they're in that variable.  Would that
be any better?


paul

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> The gnus-article-unpropagatable-p etc stuff should be moved into
>> nnml/nnfolder (perhaps nnmail) then, because we wouldn't want `seen'
>> marks to be stored in nnml .marks.
>
> Well, at least some things other than "seen" should probably be still
> dealt with as they are now.  E.g., "cache" really shouldn't be stored
> in any backend, because the information "cache" represents has nothing
> to do with what is stored in the backend.  We could add a new
> mechanism for handling "seen" (and others like it, if there are any).

Hm.  One could argue (as I probably did in our last discussion) that
the "seen" mark does not have to do with anything in the backend
either.

>> Hm, but this means the backends has knowledge about the semantics of
>> marks, which is wrong as well.
>
> Gnus could define a list of marks that are supposed to be per-user in
> shared groups; Gnus knows the semantics of the marks and thus knows
> which marks should be included.  (And users might have different ideas
> about what ought to be per-user, so this could even be defcustom'ed.)
> Backends could then DTRT with whichever marks are included in that
> variable, without knowing *why* they're in that variable.  Would that
> be any better?

I guess so, but it sounds like work.

Right now the dichotomy is between marks that belong in backends and
marks that doesn't belong there, and among the few marks that doesn't
belong in backends (seen cache download unsend score) only "seen" is
questionable.  Having a, erhm, trichotomy with marks that do not
belong in backends and per-user backend marks and global backend marks
would perhaps be more flexible but rather complex.

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Well, at least some things other than "seen" should probably be still
>> dealt with as they are now.  E.g., "cache" really shouldn't be stored
>> in any backend, because the information "cache" represents has nothing
>> to do with what is stored in the backend.
>
> Hm.  One could argue (as I probably did in our last discussion) that
> the "seen" mark does not have to do with anything in the backend
> either.

It doesn't have to do with the backend itself, but it does have to do
with what is stored via the backend - i.e., the articles.  OTOH,
"cache" has to do with information always stored outside the backend.
There's nothing outside the backend that "seen" depends on, or must be
synchronized with, etc.

>> Gnus could define a list of marks that are supposed to be per-user in
>> shared groups; [...]  Backends could then DTRT with whichever marks
>> are included in that variable
...
> Right now the dichotomy is between marks that belong in backends and
> marks that doesn't belong there, and among the few marks that doesn't
> belong in backends (seen cache download unsend score) only "seen" is
> questionable.

Or else all the others are; I think last time, the idea appeared that
the information represented by those marks should perhaps not be
represented by marks at all, but instead by something else.  Did
anything ever come of the proposed cache/agent unification?  I think
such a project could also handle the de-mark-ification of at least
"cache" and "download", right?


paul

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> prj@po.cwru.edu (Paul Jarc) writes:
>>> Well, at least some things other than "seen" should probably be still
>>> dealt with as they are now.  E.g., "cache" really shouldn't be stored
>>> in any backend, because the information "cache" represents has nothing
>>> to do with what is stored in the backend.
>>
>> Hm.  One could argue (as I probably did in our last discussion) that
>> the "seen" mark does not have to do with anything in the backend
>> either.
>
> It doesn't have to do with the backend itself, but it does have to do
> with what is stored via the backend - i.e., the articles.  OTOH,
> "cache" has to do with information always stored outside the backend.
> There's nothing outside the backend that "seen" depends on, or must be
> synchronized with, etc.

You could say that ".newsrc.eld" is outside of the backend and the
seen mark depends on it and is specific to .newsrc.eld.  Or even
.gnus.  Point is, seen is specific to one instance of Gnus and has to
be synchronized with that instance.  This is one view at least.  I'm
not sure we'll gut much further on this point.

>>> Gnus could define a list of marks that are supposed to be per-user in
>>> shared groups; [...]  Backends could then DTRT with whichever marks
>>> are included in that variable
> ...
>> Right now the dichotomy is between marks that belong in backends and
>> marks that doesn't belong there, and among the few marks that doesn't
>> belong in backends (seen cache download unsend score) only "seen" is
>> questionable.
>
> Or else all the others are; I think last time, the idea appeared that
> the information represented by those marks should perhaps not be
> represented by marks at all, but instead by something else.

Yup.  The marks could ideally be maintained by the Gnus Agent and Gnus
Cache themselves.  Seen could be maintained by Gnus.  Everything else
could be placed in the backends.  But this sounds like even more work.

> Did anything ever come of the proposed cache/agent unification?  

Water over head.  It would take some work to do it cleanly, and will
probably break everything in the process.  I wish I had more time to
work on it and then support it..

> I think such a project could also handle the de-mark-ification of at
> least "cache" and "download", right?

That could be another step in that project.  It is probably best to
unify the agent and the cache without touching marks first though.

Maybe we should start porting Gnus to Guile and clean up the design in
the process. But this sounds like really more work. :-)

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> ["seen"] doesn't have to do with the backend itself, but it does
>> have to do with what is stored via the backend - i.e., the
>> articles.  OTOH, "cache" has to do with information always stored
>> outside the backend.  There's nothing outside the backend that
>> "seen" depends on, or must be synchronized with, etc.
>
> You could say that ".newsrc.eld" is outside of the backend and the
> seen mark depends on it and is specific to .newsrc.eld.

Depending on the user's configuration and wishes, it's possible that
the backend could have a more up-to-date copy of the "seen" marks than
.newsrc.eld does[*], so it makes sense to store "seen" in the backend
and allow the backend to update Gnus's "seen" marks.  This cannot be
said for "cache"; the backend does not store the cache itself, and so
its copy of those marks (were they to be stored in the backend, which
of course they aren't) will never be more up-to-date than Gnus's copy.
So there's no reason for the backend to update Gnus's copy, and thus
no reason for the backend - any backend, regardless of its
capabilities - to store "cache".

[*] You would probably say that the most up-to-date copy of "seen" is,
by definition, always in .newsrc.eld.  But note two facts:
- "seen" does not carry very much meaning for Gnus (less meaning than
  is carried by, e.g., "expire"); it is displayed in the *Summary*
  buffer, and it is automatically set for new articles.  That's all.
- Gnus does not make it easy for the user to define arbitrary new mark
  types.
As a consequence of these two facts, a user might want to adjust the
meaning of "seen" in such a way that the backend could have the most
up-to-date copy.  I think it would be nice if it weren't too hard to
do this.  But it would be even better to allow arbitrary new
user-defined mark types.  So I guess focusing on "seen" is the wrong
way to go anyway.

>> I think such a project could also handle the de-mark-ification of at
>> least "cache" and "download", right?
>
> That could be another step in that project.  It is probably best to
> unify the agent and the cache without touching marks first though.
>
> Maybe we should start porting Gnus to Guile and clean up the design in
> the process. But this sounds like really more work. :-)

Well, "cleaning up the design" sounds like it could easily include the
cache/agent unification as well. :)


paul

Re: `gnus-unseen-mark' everywhere

[Simon Josefsson]

prj@po.cwru.edu (Paul Jarc) writes:

> Simon Josefsson <jas@extundo.com> wrote:
>> prj@po.cwru.edu (Paul Jarc) writes:
>>> ["seen"] doesn't have to do with the backend itself, but it does
>>> have to do with what is stored via the backend - i.e., the
>>> articles.  OTOH, "cache" has to do with information always stored
>>> outside the backend.  There's nothing outside the backend that
>>> "seen" depends on, or must be synchronized with, etc.
>>
>> You could say that ".newsrc.eld" is outside of the backend and the
>> seen mark depends on it and is specific to .newsrc.eld.
>
> Depending on the user's configuration and wishes, it's possible that
> the backend could have a more up-to-date copy of the "seen" marks than
> .newsrc.eld does[*], so it makes sense to store "seen" in the backend
> and allow the backend to update Gnus's "seen" marks.

Then it would be a "recent" mark, wouldn't it?  The difference between
recent and seen is that backends decide which articles is recent or
not.  If a article is seen or not is determined by Gnus.  Now, seen
marks might be stored in the backend but if the backend would start to
modify the value, it would have the same semantics as recent, no?

Well, OK, a backend could use different algorithms for deciding which
articles are seen and which are recent, but then we would have to
invent another mark that reflects Gnus's opinion and not the
backend's.  I'm not sure that having two backend-controlled marks that
indicate readedness is needed.  (Even having both recent and seen is
somewhat excessive.)

> [*] You would probably say that the most up-to-date copy of "seen" is,
> by definition, always in .newsrc.eld.

Yes, I think so.

> But note two facts:
> - "seen" does not carry very much meaning for Gnus (less meaning than
>   is carried by, e.g., "expire"); it is displayed in the *Summary*
>   buffer, and it is automatically set for new articles.  That's all.

Right, seen is used as an indicator to the user to display what Gnus
thinks of the article.  Recent is used as an indicator to the user to
display what the backend thinks of the article.  If seen were to be
controlled by the backend as well, I would like another mark that told
me what Gnus thought.

> - Gnus does not make it easy for the user to define arbitrary new mark
>   types.

I think it is pretty easy.  Just write a command, jas-add-flonk-mark,
that adds a mark to the current article (using either the backend
interface or modifying the group info or modifying a summary local
variable, depending on what you want).  Then write a summary buffer %U
function to display the mark.

> As a consequence of these two facts, a user might want to adjust the
> meaning of "seen" in such a way that the backend could have the most
> up-to-date copy.

Isn't controlling the recent mark in the backend enough?

> I think it would be nice if it weren't too hard to do this.  But it
> would be even better to allow arbitrary new user-defined mark types.

Just write a function that queries the user for a name of the mark and
set it.  I guess Group Info wasn't made to support this usage, but the
only problem would be slowing things down.  If another data structure
for group info was used, it would be fast.

> So I guess focusing on "seen" is the wrong way to go anyway.

Yup.

>>> I think such a project could also handle the de-mark-ification of at
>>> least "cache" and "download", right?
>>
>> That could be another step in that project.  It is probably best to
>> unify the agent and the cache without touching marks first though.
>>
>> Maybe we should start porting Gnus to Guile and clean up the design in
>> the process. But this sounds like really more work. :-)
>
> Well, "cleaning up the design" sounds like it could easily include the
> cache/agent unification as well. :)

So much to do, so little time...

Re: `gnus-unseen-mark' everywhere

[Paul Jarc]

Simon Josefsson <jas@extundo.com> wrote:
> If a article is seen or not is determined by Gnus.

Yes, but I'm thinking of a user who doesn't care whether Gnus thinks
it has seen the article before, and might find the "seen" mark a
convenient way of representing some other information instead.

> Now, seen marks might be stored in the backend but if the backend
> would start to modify the value, it would have the same semantics as
> recent, no?

I think that would depend on what sort of action would automatically
set or unset the mark, if any.

> Well, OK, a backend could use different algorithms for deciding which
> articles are seen and which are recent, but then we would have to
> invent another mark that reflects Gnus's opinion and not the
> backend's.

No, users who like "seen" the way it is would simply not abuse it to
make it mean something different.  If backends were to store "seen"
marks, but never change them automatically, then the semantics would
be unaffected.

> I'm not sure that having two backend-controlled marks that indicate
> readedness is needed.

If the mark is being abused, then it may well have nothing to do with
readness.

> Right, seen is used as an indicator to the user to display what Gnus
> thinks of the article.  Recent is used as an indicator to the user to
> display what the backend thinks of the article.

If the user is in control of the marks, then the marks mean whatever
the user wants them to mean.  (But that includes the possibility of
the meanings you describe, if that's what the user wants.)

>> - Gnus does not make it easy for the user to define arbitrary new mark
>>   types.
>
> I think it is pretty easy.  Just write a command, jas-add-flonk-mark,
> that adds a mark to the current article (using either the backend
> interface or modifying the group info or modifying a summary local
> variable, depending on what you want).  Then write a summary buffer %U
> function to display the mark.

Ah, I didn't know that.  But are we sure that no part of Gnus will
break when finding a mark it has never heard of?  And does the manual
say we can rely on that?

>> a user might want to adjust the meaning of "seen" in such a way
>> that the backend could have the most up-to-date copy.
>
> Isn't controlling the recent mark in the backend enough?

User-defined marks are enough.  "Recent" is partly under the control
of the server behind the backend (when there is one), so it might not
be easily subverted.

> Just write a function that queries the user for a name of the mark and
> set it.  I guess Group Info wasn't made to support this usage, but the
> only problem would be slowing things down.  If another data structure
> for group info was used, it would be fast.

What's wrong with the group info data structure?  Why would it be any
slower for user-defined marks than for standard marks?


paul

Re: `gnus-unseen-mark' everywhere

[Maciej Matysiak]

On the 2nd of January 2002 at 08:24, Robert Epprecht <epprecht@sunweb.ch> wrote:

> If I restart Gnus all the articles in my 'nnml' groups have the 
> `gnus-unseen-mark'. If I leave a group and re-enter, they are gone.
> But if I re-start Gnus, they are all back here again.

i've just updated from before-christmas-oort-cvs to current cvs and got
the same problem :( all my nnml groups are full of unseen marks.

 m.m.
-- 
 use gnus not guns!