articles which won't cache

articles which won't cache

[Roderick Schertler]

Has anybody else noticed any weirdness with an active-type
(tick/dormant) cache in 0.43?  (This might also have existed with older
versions, I just recently switched to sgnus.)  Today I had an article
which wouldn't cache properly.  I would tick the article then exit the
group and exit Gnus and would be left with the article in the cache
active file but not in the cache directory nor in the .overview file.
As soon as I ticked the article it would show up in the cache directory
but it would be removed when I exited the group.

I'm afraid this behavior went away after about a half dozen tries while
I was trying to to track it down (this might or might not have
correlated with exiting emacs rather than just exiting and restarting
Gnus) so I'm afraid I don't have any hard information to offer, I mainly
wanted to send out a feeler to see if anybody else had seen something
similar.

-- 
Roderick Schertler
roderick@gate.net

Re: articles which won't cache

[Lars Magne Ingebrigtsen]

Roderick Schertler <roderick@gate.net> writes:

> I'm afraid this behavior went away after about a half dozen tries while
> I was trying to to track it down (this might or might not have
> correlated with exiting emacs rather than just exiting and restarting
> Gnus) so I'm afraid I don't have any hard information to offer, I mainly
> wanted to send out a feeler to see if anybody else had seen something
> similar.

I have had similar reports, but I've been unsuccessful in trying to
duplicate the bug.  It would be most helpful if you could find a way
to reliably trigger the bug.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: articles which won't cache

[Roderick Schertler]

September Gnus v0.45; nntp 4.0; nnspool 2.0; nnmh 1.0; nndraft 1.0; nnfolder 1.0
GNU Emacs 19.30.1 (m88k-dg-dgux5.4R3.10, X toolkit) of Fri Dec  1 1995 on david

I haven't grokked what's going on, but I'm at least able to reproduce a
non-passive caching problem.  With any luck this bug is also the elusive
one I saw the other day.  What I'm seeing is this:

If I read an article and then tick it it is immediately saved in the
cache.  If I then exit the group the article is removed from the cache
(oops), yet it remains ticked.  (If I re-enter the group at that point
the article shows as ticked but not cached, and if I save .newsrc.eld
the article is in the ticked listed but not the cached list.)  I think
this behavior is due to the article being entered in
gnus-cache-removable-articles and then left there even after I've ticked
it.

If I re-enter the group after the above and display the article (which
is still ticked but not cached) it is again saved in the cache, if I
then exit the group it is again removed from the cache.  I believe the
problem here is that gnus-cache-removable-articles is not always cleared
out after I exit a group.

If you don't see this behavior I'll try to pare down my environment
until I can spot what the confounding factor is.

PS:  The new pp thing in gnus-debug is still off, note the double-quoted
variable names and "\n"s below.

------------------ Environment follows ------------------

(setq "gnus-default-nntp-server" "news" "\n")
(setq "gnus-local-domain" "ibcinc.com" "\n")
(setq "gnus-local-organization" "International Bonded Couriers/Miami" "\n")
(setq "gnus-use-generic-from" t "\n")
(setq "gnus-use-generic-path" "not-for-mail" "\n")
(setq "gnus-select-method"
      '(nnspool "junior.ibcinc.com")
      "\n")
(setq "gnus-refer-article-method"
      '(nntp "news")
      "\n")
(setq "gnus-dribble-directory" "~/tmp" "\n")
(setq "gnus-article-save-directory" "~/news/" "\n")
(setq "gnus-kill-files-directory" "~/news/" "\n")
(setq "gnus-default-article-saver"
      'gnus-summary-save-in-folder "\n")
(setq "gnus-folder-save-name"
      'rjs/gnus-folder-save-name "\n")
(setq "gnus-use-adaptive-scoring" t "\n")
(setq "gnus-use-cache" t "\n")
(setq "gnus-fetch-old-headers"
      'some "\n")
(setq "gnus-summary-make-false-root"
      'empty "\n")
(setq "gnus-summary-gather-subject-limit"
      'fuzzy "\n")
(setq "gnus-build-sparse-threads"
      'more "\n")
(setq "gnus-summary-same-subject" "-||-" "\n")
(setq "gnus-decode-encoded-word-method" #[nil "@\a" [nil] 1] "\n")
(setq "gnus-thread-ignore-subject" t "\n")
(setq "gnus-ignored-headers" "^\\(A\\(pproved\\|rticle-I.D.\\)\\|Content-\\(Length\\|T\\(ransfer-Encoding\\|ype\\)\\)\\|D\\(ate\\|istribution\\)\\|In-Reply-To\\|Li\\(nes\\|st-Name\\)\\|M\\(essage-ID\\|ime-Version\\)\\|NNTP-Posting-\\(Host\\|User\\)\\|Or\\(ganization\\|iginator\\)\\|P\\(ath\\|recedence\\)\\|Re\\(ceived\\|ferences\\)\\|Sender\\|X\\(-\\(Attribution\\|Disclaimer\\|Face\\|Geek-3\\|Loop\\|M\\(ail\\(er\\|ing-List\\)\\|oderator-.*\\|ts\\)\\|N\\(NTP-Posting-Host\\|SA-Fodder\\|ewsreader\\)\\|Quote\\(\\|-I-Like\\)\\|Ribbon\\|Submissions-To\\|UR[IL]\\|Zippy\\)\\|ref\\)\\):" "\n")
(setq "gnus-visible-headers" nil "\n")
(setq "gnus-sorted-header-list"
      '("^From:" "^Subject:" "^X-Sent:" "^Newsgroups:" "^Followup-To:")
      "\n")
(setq "gnus-boring-article-headers"
      '(empty followup-to newsgroups reply-to)
      "\n")
(setq "gnus-saved-headers" "^From:\\|^Newsgroups:\\|^Subject:\\|^Date:\\|^Followup-To:\\|^Reply-To:\\|^Organization:\\|^Summary:\\|^Keywords:\\|^To:\\|^Cc:\\|^Posted-To:\\|^Mail-Copies-To:\\|^Apparently-To:\\|^Gnus-Warning:\\|^Resent-" "\n")
(setq "gnus-auto-select-first" nil "\n")
(setq "gnus-subscribe-newsgroup-method"
      'gnus-subscribe-hierarchically "\n")
(setq "gnus-group-sort-function"
      '(gnus-group-sort-by-alphabet gnus-group-sort-by-level)
      "\n")
(setq "gnus-group-line-format" "%M%p%P %2ud %2ut %S %5y: %L %(%-40uc%) %18ue\n" "\n")
(setq "gnus-summary-line-format" "%U%R %5i %(%[%3L: %-18,18n%]%)%I %s\n" "\n")
(setq "gnus-article-mode-line-format" "Gnus A: %S" "\n")
(setq "gnus-mode-non-string-length" 33 "\n")
(setq "gnus-thread-sort-functions"
      '(gnus-thread-sort-by-number gnus-thread-sort-by-total-score)
      "\n")
(setq "gnus-summary-prepare-exit-hook"
      '(gnus-summary-expire-articles)
      "\n")
(setq "gnus-startup-hook"
      '(#[nil "@AB\"\b@AC\"\bD\x15@FB\"\a" [remove-hook gnus-summary-prepare-hook hilit-rehighlight-buffer-quietly hilit-install-line-hooks
	      (gnus-summary-mark-read-and-unread-as-read)
	      gnus-mark-article-hook gnus-article-prepare-hook] 3])
      "\n")
(setq "gnus-summary-prepare-hook"
      '(rjs/gnus-score-select-score-file)
      "\n")
(setq "gnus-article-prepare-hook"
      '(gnus-article-date-lapsed)
      "\n")
(setq "gnus-select-group-hook"
      '((lambda nil
	  (gnus-group-set-parameter group
				    'gnus-group-date-last-entered
				    (current-time))))
      "\n")
(setq "gnus-author-copy" "|rcvstore +post" "\n")
(setq "gnus-message-archive-group" nil "\n")
(setq "gnus-user-from-line" "Roderick Schertler <roderick@ibcinc.com>" "\n")
(setq "gnus-required-headers"
      '(From Date Newsgroups Subject Message-ID Lines)
      "\n")
(setq "gnus-auto-mail-to-author" t "\n")
(setq "gnus-mail-courtesy-message" "[This is a courtesy copy of a reply which was also posted to Usenet.]\n\n" "\n")
(setq "gnus-mail-method"
      'mh "\n")
(setq "gnus-score-expiry-days" 28 "\n")
(setq "gnus-score-after-write-file-function"
      'rjs/gnus-score-after-write-file "\n")
(setq "nnmail-spool-file" nil "\n")

-- 
Roderick Schertler
roderick@gate.net

Re: articles which won't cache

[John Griffith]

Lars Magne Ingebrigtsen writes:
>Roderick Schertler <roderick@gate.net> writes:
>
>> I'm afraid this behavior went away after about a half dozen tries while
>> I was trying to to track it down (this might or might not have
>> correlated with exiting emacs rather than just exiting and restarting
>> Gnus) so I'm afraid I don't have any hard information to offer, I mainly
>> wanted to send out a feeler to see if anybody else had seen something
>> similar.
>
>I have had similar reports, but I've been unsuccessful in trying to
>duplicate the bug.  It would be most helpful if you could find a way
>to reliably trigger the bug.

I seem to have no problem duplicating this bug.  It think as well that
I have tracked it down and have a possible fix, though I haven't tried
to fix the code.

Sorry that this is a bit long.  The possible fix is at the end.

Here's my setup using September 0.46 and XEmacs 19.13.

.emacs
------------------------------------------------------------
(setq load-path (cons (expand-file-name "~/emacs/sgnus-0.46/lisp") load-path))
------------------------------------------------------------

.gnus
------------------------------------------------------------
(setq gnus-nntp-server	"newsserv.zdv.uni-tuebingen.de" )
(setq gnus-use-cache t)
(setq gnus-cache-directory "~/mhbox/News/cache")
------------------------------------------------------------

Initially the directory ~/mhbox/News/cache/news.announce.newusers
does not exist.

Group buffer:
------------------------------------------------------------
       6: news.announce.newusers
   77380: news.newusers.questions
    1071: news.answers
------------------------------------------------------------

" " on n.a.n to enter group and select the first article.

Summary buffer:
------------------------------------------------------------
R  [1061: Chris Lewis         ] How to become a Usenet site
   [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
   [  82: Mark Moraes         ] Welcome to Usenet!
   [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------

"?" to mark the first article as dormant.

Summary buffer:
------------------------------------------------------------
?* [1061: Chris Lewis         ] How to become a Usenet site
   [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
   [  82: Mark Moraes         ] Welcome to Usenet!
   [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------

The directory ~/mhbox/News/cache/news.announce.newusers is created and
the article is now in the cache.

"c y" to catch up and leave the group.

*** Here's the first bug ***

The article is no longer in the cache.  (In fact the directory is also
removed, but this seems to only happen if it's empty.)

Group buffer:
------------------------------------------------------------
       0: news.announce.newusers
   77380: news.newusers.questions
    1071: news.answers
------------------------------------------------------------

"C-u =" on n.a.n to reread group.

Summary buffer:
------------------------------------------------------------
?  [1061: Chris Lewis         ] How to become a Usenet site
O  [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
O  [  82: Mark Moraes         ] Welcome to Usenet!
O  [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------

Article still not in cache.  Note there is no "*" mark.

" " to read first article.

Summary buffer:
------------------------------------------------------------
?  [1061: Chris Lewis         ] How to become a Usenet site
O  [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
O  [  82: Mark Moraes         ] Welcome to Usenet!
O  [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------

Now the article is back in the cache and the "*" mark has returned.

"c y" again to leave and the article remains in the cache.

Weird, huh?

============================================================
Now some related bugs (perhaps):

Enter n.a.n again with "C-u ="

Summary buffer:
------------------------------------------------------------
?* [1061: Chris Lewis         ] How to become a Usenet site
O  [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
O  [  82: Mark Moraes         ] Welcome to Usenet!
O  [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------

[Note that caching and dormant stuff is fine now.]

I decide that I don't want the first article marked as dormant any
more, so I do "M c" on the first article.

Summary buffer:
------------------------------------------------------------
 * [1061: Chris Lewis         ] How to become a Usenet site
O  [ 829: David Alex Lamb     ] FAQ: How to find people's E-mail addresses
O  [  82: Mark Moraes         ] Welcome to Usenet!
O  [ 492: Aliza R. Panitz     ] How to find the right place to post (FAQ)
------------------------------------------------------------
"M c" removes the "?" mark but not the "*", nor does it remove the
article from the cache. (even after "c y" to leave the group).

*** Is this the correct behavior? ***

Perhaps it is the correct behavior since because GNUS doesn't seem to
distinguish between articles that I cached because they were dormant
and those that I explicitly cache.  (Maybe it should make this
distinction?)

Anyway, suppose I decide that I want to uncache the article too.
I type "M-*" on the first article.

*** Second bug ****

Beep. Error in minibuffer: "Wrong type argument: stringp, nil"

The article *is* removed from the cache, but the "*" is not removed
from the summary buffer.  The cache directory is not removed however
since there is now a .overview file in it.  (perhaps a buglet?)

This second bug smells a bit like an XEmacs specific one.

============================================================

I can duplicate these bugs over and over again.

Here's my attempt at debugging the first bug and a possible solution.

A trace:

gnus-summary-catchup-and-exit
  calls: gnus-summary-exit
    calls: gnus-cache-possibly-remove-articles
       calls: gnus-cache-possibly-remove-articles-1
           calls: gnus-cache-possibly-remove-article
              calls: gnus-cache-member-of-class

This last call looks like:

    `(gnus-cache-member-of-class (read) nil (1090) nil)'

which returns `(read)' and consequently causes
gnus-cache-possibly-remove-article to delete the file.

Here's the definition of gnus-cache-member-of-class:

(defun gnus-cache-member-of-class (class ticked dormant unread)
  (or (and ticked (memq 'ticked class))
      (and dormant (memq 'dormant class))
      (and unread (memq 'unread class))
      (and (not unread) (memq 'read class))))

The problem seems to be that all read articles are removed from the
cache even if they are marked as dormant and gnus-use-cache is t.

I guess one solution is that gnus-cache-possibly-remove-article should
check if the article is dormant and gnus-use-cache is t, if so then do
not delete the file.  But not (yet) having intimate knowledge of the
code I'm not sure if this test should go here or somewhere earlier.

Have a nice day.