nnimap+agent has stopped caching when reading

nnimap+agent has stopped caching when reading

[Steinar Bang]

In my ~/.gnus.el file I have:

 ;; Store article in agent when reading it while online
 (add-hook 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

This used to make the agent save a copy of all articles I've read.

But sometime between last night and today, this stopped working: read
articles aren't saved in the agent.

Re: nnimap+agent has stopped caching when reading

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> In my ~/.gnus.el file I have:
>
>  ;; Store article in agent when reading it while online
>  (add-hook 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)
>
> This used to make the agent save a copy of all articles I've read.

Have you tried checking whether that function is actually called, and if
so, what it (isn't) doing?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Steinar Bang <sb@dod.no> writes:
>> In my ~/.gnus.el file I have:
>> 
>> ;; Store article in agent when reading it while online
>> (add-hook 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

>> This used to make the agent save a copy of all articles I've read.

> Have you tried checking whether that function is actually called, and if
> so, what it (isn't) doing?

Not yet.  I've been looking at checkins on nnimap.el and gnus-agent.el,
and the gnus-completing-read changes on gnus-agent.el fit the time
frame, at least.

I will edebug the function in a little while.

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:
>> Steinar Bang <sb@dod.no> writes:
>>> In my ~/.gnus.el file I have:
>>> 
>>> ;; Store article in agent when reading it while online
>>> (add-hook 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

>>> This used to make the agent save a copy of all articles I've read.

>> Have you tried checking whether that function is actually called, and if
>> so, what it (isn't) doing?

> I will edebug the function in a little while.

gnus-agent-fetch-selected-article is called and calls
gnus-agent-fetch-articles with the nnimap group name, and a list
containing the article number as the single element.

That method does what seems to be sensible, but at the end of the read
no article has been saved.  The .agentview file is updated, though.
It's read at the start, and written back at the end.

In between there, something is loaded and put into a buffer named
*Temp*, or something like that.  Whether that buffer is saved or not, I
couldn't figure out.

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> gnus-agent-fetch-selected-article is called and calls
> gnus-agent-fetch-articles with the nnimap group name, and a list
> containing the article number as the single element.

> That method does what seems to be sensible, but at the end of the read
> no article has been saved.  The .agentview file is updated, though.
> It's read at the start, and written back at the end.

> In between there, something is loaded and put into a buffer named
> *Temp*, or something like that.  Whether that buffer is saved or not, I
> couldn't figure out.

The code uses the function nntp-server-buffer a lot.  I've no idea what
that means...?  Is the article saved in nntp-server-buffer and from
there into the agent?  Or is the use of the function nntp-server-buffer
from nnimap completely... weird...?

I tried doing `C-x v g' in gnus-agent.el.  But the command doesn't work
for me (git claims that it doesn't understand the argument -rmaster to
git blame), so I can't see when those lines were last touched.

Re: nnimap+agent has stopped caching when reading

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> The code uses the function nntp-server-buffer a lot.

That's a variable.  :-)

> I've no idea what that means...?  Is the article saved in
> nntp-server-buffer and from there into the agent?  Or is the use of
> the function nntp-server-buffer from nnimap completely... weird...?

Yes, the articles are transferred from the backends to Gnus itself via
that buffer.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

>> I've no idea what that means...?  Is the article saved in
>> nntp-server-buffer and from there into the agent?  Or is the use of
>> the function nntp-server-buffer from nnimap completely... weird...?

> Yes, the articles are transferred from the backends to Gnus itself via
> that buffer.

Hm... anyway, the gnus-agent-fetch-articles function returns with a nil,
which I think means it wasn't able to load the articles it was asked to
load, because 
          (cdr fetched-articles))))))
at the end there is nil.

Too much bytecode to make sense of.  I've restarted and loaded
gnus-agent.el and then done
 (debug-on-entry 'gnus-agent-fetch-selected-article)

But now I'm too sleepy to make sense of what I'm debugging...

Ah well!  Good night.  I'm calling it a day.

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> Hm... anyway, the gnus-agent-fetch-articles function returns with a nil,
> which I think means it wasn't able to load the articles it was asked to
> load, because 
>           (cdr fetched-articles))))))
> at the end there is nil.

In gnus-agent-fetch-articles it comes out of 
          ;; Skip headers preceeding this article
          (while (> article
                    (setq header-number
                          (let* ((header (car headers)))
                            (if header
                                (mail-header-number header)
                              (1+ article)))))
            (setq headers (cdr headers)))

with header-number set to article + 1 (ie. article is 7899 and
header-number is 7900).

Is this what is expected to happen?

In any case in:
          (when (< gnus-agent-max-fetch-size
                   (setq current-set-size
                   ...
current-set-size ends up as:
 0 (#o0, #x0, ?\C-@)

Does that mean that it has decided to fetch 0 articles?

What's the relationship between the article number gnus uses and what
identifies the article on the IMAP server?

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> In any case in:
>           (when (< gnus-agent-max-fetch-size
>                    (setq current-set-size
>                    ...
> current-set-size ends up as:
>  0 (#o0, #x0, ?\C-@)

> Does that mean that it has decided to fetch 0 articles?

No, that wasn't it either.  On the next debug run I saw the same thing,
except this time I could follow him copying the article from buffer to
buffer (from " *nntpd*" to the temp buffer and back and then narrow to
region and write-region of the article).  And when I looked into the
agent directory there was an article with that number.

One difference is that I peeked into the *nntpd* buffer on occasion to
see what's in there (it contained the contents of the 7899 article).

One observation I made: the article downloaded into the agent was
stripped of its attachments.  I have 
  (setq nnimap-fetch-partial-articles "text/")
in my .gnus.el.

When I did `A C' the article buffer contained the article with its
attachments, but the agent still contained the article without its
attachments. 

I really would have liked the agent-representation of the article to be
replaced with the full one.

Re: nnimap+agent has stopped caching when reading

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> I really would have liked the agent-representation of the article to be
> replaced with the full one.

That's not implemented.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Steinar Bang <sb@dod.no> writes:
>> I really would have liked the agent-representation of the article to be
>> replaced with the full one.

> That's not implemented.

I know.

Re: nnimap+agent has stopped caching when reading

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> Hm... anyway, the gnus-agent-fetch-articles function returns with a nil,
> which I think means it wasn't able to load the articles it was asked to
> load, because 
>           (cdr fetched-articles))))))
> at the end there is nil.

Okay... it seems to be like this, consistently:
 - The first time I step through gnus-agent-fetch-articles for an
   article that isn't already in the agent cache, the *nntpd* buffer is
   empty when it hits the 
                (with-current-buffer nntp-server-buffer
   and the part containing the write-region is skipped, and the function
   returns with a nil
 - The second time I step through gnus-agent-fetch-articles for the same
   article the *nntpd* contains the article at this point, and the
   article is written (the write-region bit is evaluated), and the
   function returns a list containing the article number

I haven't seen anything yet that could cause this strangeness.