Gnus development mailing list
 help / color / mirror / Atom feed
* nnml suddenly not activating groups on startup
@ 2020-08-26  2:22 Russ Allbery
  2020-08-26  3:59 ` Russ Allbery
  2020-08-26  7:59 ` Adam Sjøgren
  0 siblings, 2 replies; 11+ messages in thread
From: Russ Allbery @ 2020-08-26  2:22 UTC (permalink / raw)
  To: ding

I'm having a very strange issue and am not sure where to look in debugging
it.

I have Gnus configured with nntp as the primary select method and nnml as
the secondary select method, and have a ton of nnml groups.  Due to an
unrelated problem, I've recently had to C-g during a "g" command to check
for new mail and news a few times.  Other than that, no versions or other
configuration have changed.

Suddenly today when I start Gnus most (but not all!) of my nnml groups do
not activate and show in the group buffer with * for the article count.
Pressing enter on a group to read it activates it and then everything is
fine, including the marks.  C-c M-g to activate all groups fixes the
problem.  But when I exit and restart Gnus, or use R, the problem recurs.

Nothing has changed in my startup files and I can't find anything in
newsrc.eld that would be relevant.  Clearly Gnus is storing some state
somewhere, but I have no idea where and can't find it.

Where should I be looking?

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26  2:22 nnml suddenly not activating groups on startup Russ Allbery
@ 2020-08-26  3:59 ` Russ Allbery
  2020-08-26  7:59 ` Adam Sjøgren
  1 sibling, 0 replies; 11+ messages in thread
From: Russ Allbery @ 2020-08-26  3:59 UTC (permalink / raw)
  To: ding

Russ Allbery <eagle@eyrie.org> writes:

> Suddenly today when I start Gnus most (but not all!) of my nnml groups
> do not activate and show in the group buffer with * for the article
> count.  Pressing enter on a group to read it activates it and then
> everything is fine, including the marks.  C-c M-g to activate all groups
> fixes the problem.  But when I exit and restart Gnus, or use R, the
> problem recurs.

This is with Gnus v5.13 as shipped with Emacs 26.3.

To hopefully narrow this down, if after pressing R, I run:

M-: (gnus-get-unread-articles)
l

then all the groups are activated and everything is fine.  So running
gnus-get-unread-articles (such as via gnus-group-get-new-news) does
activate all of the groups.  I think that rules out any problems with
group activation levels (which regardless haven't changed).

gnus-setup-news should be calling gnus-get-unread-articles, however, so
I'm a bit mystified by this.  I think there's either some state which is
not properly intialized when gnus-setup-news calls
gnus-get-unread-articles but that is then working properly when I call it
later, or somehow dont-connect is set and is being passed in.  If that's
the case, I'm not sure why -- I'm certainly not passing it in!

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26  2:22 nnml suddenly not activating groups on startup Russ Allbery
  2020-08-26  3:59 ` Russ Allbery
@ 2020-08-26  7:59 ` Adam Sjøgren
  2020-08-26 17:19   ` Russ Allbery
  1 sibling, 1 reply; 11+ messages in thread
From: Adam Sjøgren @ 2020-08-26  7:59 UTC (permalink / raw)
  To: ding

Russ writes:

> Nothing has changed in my startup files and I can't find anything in
> newsrc.eld that would be relevant.  Clearly Gnus is storing some state
> somewhere, but I have no idea where and can't find it.
>
> Where should I be looking?

Could it be that an .overview or .marks file, that nnml stores in each
folder, got corrupted?

  $ ls -l Mail/normal/.overview Mail/normal/.marks 
  -rw-r--r-- 1 asjo asjo    6032 Feb 15  2012 Mail/normal/.marks
  -rw------- 1 asjo asjo 5606188 Aug 25 21:49 Mail/normal/.overview
  $ 

It's a long shot, but was what sprung to mind as another place that Gnus
stores state for nnml.

I think there is a function to update/rebuild them, but I don't remember
the last time I had to use it.


  Best regards,

    Adam

-- 
 "Vi är inte riktigt kloka när man tänker efter."           Adam Sjøgren
                                                       asjo@koldfront.dk



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26  7:59 ` Adam Sjøgren
@ 2020-08-26 17:19   ` Russ Allbery
  2020-08-26 18:00     ` Adam Sjøgren
  0 siblings, 1 reply; 11+ messages in thread
From: Russ Allbery @ 2020-08-26 17:19 UTC (permalink / raw)
  To: ding

Adam Sjøgren <asjo@koldfront.dk> writes:

> Could it be that an .overview or .marks file, that nnml stores in each
> folder, got corrupted?

>   $ ls -l Mail/normal/.overview Mail/normal/.marks 
>   -rw-r--r-- 1 asjo asjo    6032 Feb 15  2012 Mail/normal/.marks
>   -rw------- 1 asjo asjo 5606188 Aug 25 21:49 Mail/normal/.overview
>   $ 

> It's a long shot, but was what sprung to mind as another place that Gnus
> stores state for nnml.

It looks like the .marks files aren't used any more (mine were all last
updated in 2014).  I regenerated overview with no change.

I did find a proximate cause, but I don't understand it.  I noticed that a
few nnml groups were activated by default but most weren't, so I inspected
the group data with G E.  The nnml groups that were activated have
(nnml "") as the last element of their data.  The ones that aren't
activated have "nnml:"  as the last element of their data.  Presumably I
can fix this by editing all of the group metadata accordingly, but I think
new groups are created using the "nnml:" syntax.

I vaguely recall seeing soemthing on StackOverflow that said this was an
intentional change from some time back, but it seems to suddenly be
causing me problems?  The two things that happened right before this
problem started was that I aborted several "g" new-news checks, and I
added an NNTP group with a new secondary server.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 17:19   ` Russ Allbery
@ 2020-08-26 18:00     ` Adam Sjøgren
  2020-08-26 21:00       ` Russ Allbery
  0 siblings, 1 reply; 11+ messages in thread
From: Adam Sjøgren @ 2020-08-26 18:00 UTC (permalink / raw)
  To: ding

Russ writes:

> I noticed that a few nnml groups were activated by default but most
> weren't, so I inspected the group data with G E. The nnml groups that
> were activated have (nnml "") as the last element of their data. The
> ones that aren't activated have "nnml:" as the last element of their
> data. Presumably I can fix this by editing all of the group metadata
> accordingly, but I think new groups are created using the "nnml:"
> syntax.

All¹ of my mail groups have "nnml:" as the last element in
gnus-newsrc-alist.

I have a feeling that Gnus sometimes gets confused about servers that
are the same but mentioned by different definitions. Maybe going all
"nnml:" or all (nnml "") will unconfuse things?

Does your server buffer show multiple nnml-servers?


  Best regards,

    Adam


¹ Not my archive groups, though, they seem to all carry a copy of the full
server definition:
   (nnml "archive" (nnml-directory "~/Mail/archive")
         (nnml-active-file "~/Mail/archive/active")
         (nnml-get-new-mail nil)
         (nnml-inhibit-expiry t))
as the last element. That seems a little much.

-- 
 "Your father traded the tools for M&Ms again"              Adam Sjøgren
                                                       asjo@koldfront.dk



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 18:00     ` Adam Sjøgren
@ 2020-08-26 21:00       ` Russ Allbery
  2020-08-26 22:09         ` Eric Abrahamsen
  2020-08-27  9:43         ` Adam Sjøgren
  0 siblings, 2 replies; 11+ messages in thread
From: Russ Allbery @ 2020-08-26 21:00 UTC (permalink / raw)
  To: ding

Adam Sjøgren <asjo@koldfront.dk> writes:

> All¹ of my mail groups have "nnml:" as the last element in
> gnus-newsrc-alist.

> I have a feeling that Gnus sometimes gets confused about servers that
> are the same but mentioned by different definitions. Maybe going all
> "nnml:" or all (nnml "") will unconfuse things?

I tried changing everything to "nnml:" and that caused all of the groups
to not be activated and then, in the server buffer, the nnml server was
left closed after I started Gnus.

I then changed them all to (nnml "") and that fixes the problem (but I
suspect new groups won't have that setting).

So (nnml "") seems to be doing something different than "nnml:" and is
working correctly, and this appears to have something to do with when the
server is opened.  Maybe it's a sequencing issue during startup?

For the record, my select-method configuration is:

 '(gnus-select-method
   '(nntp "news.eyrie.org"
          (nntp-open-connection-function network-only)
          (nntp-connection-timeout 3)))
 '(gnus-secondadry-select-methods '((nnml "")))

> Does your server buffer show multiple nnml-servers?

It does not, but now that you mention this, I feel like at one point I did
see this and "fixed it" by deleting one of the servers.  I should have
remembered that, and that's probably related, although I'm fairly sure my
problems started before that and this was one of my attempts to fix it.

> ¹ Not my archive groups, though, they seem to all carry a copy of the full
> server definition:
>    (nnml "archive" (nnml-directory "~/Mail/archive")
>          (nnml-active-file "~/Mail/archive/active")
>          (nnml-get-new-mail nil)
>          (nnml-inhibit-expiry t))
> as the last element. That seems a little much.

Mine had this in gnus-server-alist and just a reference to "archive" in
the group information, but while I've been working on this problem (and
after it started), I finally got rid of the second nnml server for
archiving and just merged all that stuff into my main nnml server because
I've had no end of problems with having two nnml servers for years.  It
seems to confuse Gnus and I have regularly had Gnus write things into the
archive server instead of the main server in weird edge cases.  The last
straw was that while I was restarting Gnus to try to get to the bottom of
this problem, one time Gnus split all of my incoming mail into the archive
server instead of the main nnml server.

The multiple nnml server support seems kind of buggy, in other words, and
has been for years, although it previously had never irritated me enough
to change it.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 21:00       ` Russ Allbery
@ 2020-08-26 22:09         ` Eric Abrahamsen
  2020-08-26 22:25           ` Russ Allbery
  2020-08-27  9:43         ` Adam Sjøgren
  1 sibling, 1 reply; 11+ messages in thread
From: Eric Abrahamsen @ 2020-08-26 22:09 UTC (permalink / raw)
  To: ding

Russ Allbery <eagle@eyrie.org> writes:

> Adam Sjøgren <asjo@koldfront.dk> writes:
>
>> All¹ of my mail groups have "nnml:" as the last element in
>> gnus-newsrc-alist.
>
>> I have a feeling that Gnus sometimes gets confused about servers that
>> are the same but mentioned by different definitions. Maybe going all
>> "nnml:" or all (nnml "") will unconfuse things?
>
> I tried changing everything to "nnml:" and that caused all of the groups
> to not be activated and then, in the server buffer, the nnml server was
> left closed after I started Gnus.
>
> I then changed them all to (nnml "") and that fixes the problem (but I
> suspect new groups won't have that setting).
>
> So (nnml "") seems to be doing something different than "nnml:" and is
> working correctly, and this appears to have something to do with when the
> server is opened.  Maybe it's a sequencing issue during startup?
>
> For the record, my select-method configuration is:
>
>  '(gnus-select-method
>    '(nntp "news.eyrie.org"
>           (nntp-open-connection-function network-only)
>           (nntp-connection-timeout 3)))
>  '(gnus-secondadry-select-methods '((nnml "")))

Just checking in to confirm that the above is a typo, and not actual
copy-and-paste from your config?



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 22:09         ` Eric Abrahamsen
@ 2020-08-26 22:25           ` Russ Allbery
  2020-08-26 23:01             ` Eric Abrahamsen
  0 siblings, 1 reply; 11+ messages in thread
From: Russ Allbery @ 2020-08-26 22:25 UTC (permalink / raw)
  To: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:
> Russ Allbery <eagle@eyrie.org> writes:

>> For the record, my select-method configuration is:

>>  '(gnus-select-method
>>    '(nntp "news.eyrie.org"
>>           (nntp-open-connection-function network-only)
>>           (nntp-connection-timeout 3)))
>>  '(gnus-secondadry-select-methods '((nnml "")))

> Just checking in to confirm that the above is a typo, and not actual
> copy-and-paste from your config?

Sigh.  No, it was a cut and paste, and this is why one always cuts and
pastes one's configuration.

Thank you.

So I broke it when adding a secondary server by misspelling the variable
name, and Gnus was smart enough to figure out that it should activate the
server anyway if I pushed it hard enough, which kept me from realizing
that the configuration was just wrong.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 22:25           ` Russ Allbery
@ 2020-08-26 23:01             ` Eric Abrahamsen
  2020-08-26 23:23               ` Russ Allbery
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Abrahamsen @ 2020-08-26 23:01 UTC (permalink / raw)
  To: ding

Russ Allbery <eagle@eyrie.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>> Russ Allbery <eagle@eyrie.org> writes:
>
>>> For the record, my select-method configuration is:
>
>>>  '(gnus-select-method
>>>    '(nntp "news.eyrie.org"
>>>           (nntp-open-connection-function network-only)
>>>           (nntp-connection-timeout 3)))
>>>  '(gnus-secondadry-select-methods '((nnml "")))
>
>> Just checking in to confirm that the above is a typo, and not actual
>> copy-and-paste from your config?
>
> Sigh.  No, it was a cut and paste, and this is why one always cuts and
> pastes one's configuration.

Very true!

> Thank you.
>
> So I broke it when adding a secondary server by misspelling the variable
> name, and Gnus was smart enough to figure out that it should activate the
> server anyway if I pushed it hard enough, which kept me from realizing
> that the configuration was just wrong.

Gnus will also happily make you a server that you haven't configured, if
it sees groups that specify that server.

Glad it wasn't more complicated than this. I was keeping quiet and
hoping Adam would solve it :)



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 23:01             ` Eric Abrahamsen
@ 2020-08-26 23:23               ` Russ Allbery
  0 siblings, 0 replies; 11+ messages in thread
From: Russ Allbery @ 2020-08-26 23:23 UTC (permalink / raw)
  To: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Glad it wasn't more complicated than this. I was keeping quiet and
> hoping Adam would solve it :)

Me too!  And thank you very much to Adam for all of the help!

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: nnml suddenly not activating groups on startup
  2020-08-26 21:00       ` Russ Allbery
  2020-08-26 22:09         ` Eric Abrahamsen
@ 2020-08-27  9:43         ` Adam Sjøgren
  1 sibling, 0 replies; 11+ messages in thread
From: Adam Sjøgren @ 2020-08-27  9:43 UTC (permalink / raw)
  To: ding

Russ writes:

> So (nnml "") seems to be doing something different than "nnml:" and is
> working correctly, and this appears to have something to do with when the
> server is opened.  Maybe it's a sequencing issue during startup?

I was chasing something similar at one point I think (related to
nnir/notmuch and searching), when I changed my .gnus to use the full
definition for each server where ever it is used, I stopped having
problems.

> For the record, my select-method configuration is:
>
>  '(gnus-select-method
>    '(nntp "news.eyrie.org"
>           (nntp-open-connection-function network-only)
>           (nntp-connection-timeout 3)))
>  '(gnus-secondadry-select-methods '((nnml "")))
                 ^
              I guess this typo is only in the email?

I have:

  (setq asjo-local-mail-method '(nnml ""
                                      (nnir-search-engine notmuch)))
  (setq gnus-secondary-select-methods
        `(
          ,asjo-local-mail-method
            [....]

  (setq gnus-check-new-newsgroups `(,asjo-local-mail-method))

So I always give the complete and identical definition whenever I use
it.

>> Does your server buffer show multiple nnml-servers?
>
> It does not, but now that you mention this, I feel like at one point I did
> see this and "fixed it" by deleting one of the servers.  I should have
> remembered that, and that's probably related, although I'm fairly sure my
> problems started before that and this was one of my attempts to fix it.

That could account for the two different "types" of groups, at least, I guess.

>> ¹ Not my archive groups, though, they seem to all carry a copy of the full
>> server definition:
>>    (nnml "archive" (nnml-directory "~/Mail/archive")
>>          (nnml-active-file "~/Mail/archive/active")
>>          (nnml-get-new-mail nil)
>>          (nnml-inhibit-expiry t))
>> as the last element. That seems a little much.

> Mine had this in gnus-server-alist and just a reference to "archive" in
> the group information, but while I've been working on this problem (and
> after it started), I finally got rid of the second nnml server for
> archiving and just merged all that stuff into my main nnml server because
> I've had no end of problems with having two nnml servers for years.

I gave my archive server the same treatment as I described above, so I
have:

  (setq asjo-archive-method '(nnml "archive"
                                   (nnml-directory "~/Mail/archive")
                                   (nnml-active-file "~/Mail/archive/active")
                                   (nnml-get-new-mail nil)
                                   (nnir-search-engine notmuch)
                                   (nnml-inhibit-expiry t)))
  (setq gnus-message-archive-method asjo-archive-method)

  (setq gnus-refer-article-method
        ;; First try to load the article from your local newsserver or nnml
        '(current
          ;; Then try the archive (?):
          (list asjo-archive-method)
             ....

to make sure that every time I mention the server, the definition is
exactly identical.

I think I chased that problem down to some elaborate function checking
if two definitions were the same, somewhere deep inside Gnus, and I
concluded that it was too hairy for me to try to fix.

> The last straw was that while I was restarting Gnus to try to get to
> the bottom of this problem, one time Gnus split all of my incoming
> mail into the archive server instead of the main nnml server.

Uggghh :-/


  Best regards,

    Adam

-- 
 "Dom 999 saker                                             Adam Sjøgren
  jag aldrig skulle göra                               asjo@koldfront.dk
  Dom vägs upp av 999 värre
  saker jag redan har gjort"



^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2020-08-27 15:03 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-26  2:22 nnml suddenly not activating groups on startup Russ Allbery
2020-08-26  3:59 ` Russ Allbery
2020-08-26  7:59 ` Adam Sjøgren
2020-08-26 17:19   ` Russ Allbery
2020-08-26 18:00     ` Adam Sjøgren
2020-08-26 21:00       ` Russ Allbery
2020-08-26 22:09         ` Eric Abrahamsen
2020-08-26 22:25           ` Russ Allbery
2020-08-26 23:01             ` Eric Abrahamsen
2020-08-26 23:23               ` Russ Allbery
2020-08-27  9:43         ` Adam Sjøgren

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).