Bug regarding reading sent-mail folder

Bug regarding reading sent-mail folder

[Jarl Friis]

Hello.

I recently upgraded from gnus 5.8.8 to 5.10.6 on XEmacs 21.4.10

Now I can't read my sent mails.

I believe I found a bug regarding regarding sent-mail. My relevant
"~/.gnus" part is (pretty standard/default):=20

(setq gnus-message-archive-method
      '(nnfolder "archive"
                 (nnfolder-get-new-mail nil)
                 (nnfolder-inhibit-expiry t)
                 (nnfolder-directory "~/Mail-softace/sent-mail/")
                 (nnfolder-active-file "~/Mail-softace/sent-mail/active")
                 )
      )
(setq gnus-message-archive-group "all")

When I hit the group "nnfolder+archive:all" in "*Group*" buffer. I get:

Signaling: (error "Couldn't request group nnfolder+archive:all: No such group: all")
  signal(error ("Couldn't request group nnfolder+archive:all: No such group: all"))
  cerror("Couldn't request group %s: %s" "nnfolder+archive:all" "No such group: all")
  apply(cerror "Couldn't request group %s: %s" ("nnfolder+archive:all" "No such group: all"))
  error("Couldn't request group %s: %s" "nnfolder+archive:all" "No such group: all")
  gnus-select-newsgroup("nnfolder+archive:all" nil nil)
  gnus-summary-read-group-1("nnfolder+archive:all" nil t nil nil nil)
  gnus-summary-read-group("nnfolder+archive:all" nil t nil nil nil nil)
  gnus-group-read-group(nil t)
  gnus-group-select-group(nil)
  gnus-topic-select-group(nil)
  call-interactively(gnus-topic-select-group)


I further realised that when gnus has started up, there is a
difference of the value of `nnfolder-group-alist' variable.
In gnus 5.8.8 this value is `(("all" (1 . 1340)))'.
In gnus 5.10.6 this value is nil.

I further figured out that the following lines where removed from
5.8.8 to 5.10.6 in nnfolder.el (approx. line 600):
----------------------------------------------
	;; The group doesn't exist, so we create a new entry for it.
	(push (list group (cons 1 0)) nnfolder-group-alist)
	(nnfolder-save-active nnfolder-group-alist nnfolder-active-file))
----------------------------------------------

Is this a bug?=20
Can someone help?
Can I provide you with more information.

Jarl

-- 
Jarl Friis
Softace
Omøgade 8, 2.sal
2100 København Ø.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl@softace.dk

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

Hi,

I read your message last week and tried the same archiving method.
However, I could neither reproduce any problem nor look at the
nnfolder-group-alist variable as nil.

>>>>> In <m3llmk7auy.fsf@zeus.intra.softace.dk>
>>>>>	Jarl Friis <jarl@softace.dk> wrote:

> Hello.

> I recently upgraded from gnus 5.8.8 to 5.10.6 on XEmacs 21.4.10

There is a remarkable difference between 5.8.8 and 5.10.6.
Recent Gnus uses .mrk files for nnfolder groups, for instance:

% cd ~/Mail-softace/sent-mail
% ls -aF
./ ../ active  all  all.mrk  all.nov  all~

About this, you will find a note in the GNUS-NEWS file as follows:

--8<---------------cut here---------------start------------->8---
** Upgrading from previous (stable) version if you have used Oort.

If you have tried Oort (the unstable Gnus branch leading to this
release) but went back to a stable version, be careful when upgrading
to this version.  In particular, you will probably want to remove all
.marks (nnml) and .mrk (nnfolder) files, so that flags are read from
your ~/.newsrc.eld instead of from the .marks/.mrk file where this
release store flags.  See a later entry for more information about
marks.  Note that downgrading isn't safe in general.
--8<---------------cut here---------------end--------------->8---

If .mrk files are broken, you can ignore them by setting the
nnfolder-marks-is-evil variable as t.  This may be unrelated
though.

> Now I can't read my sent mails.

> I believe I found a bug regarding regarding sent-mail. My relevant
> "~/.gnus" part is (pretty standard/default):=20

> (setq gnus-message-archive-method
>       '(nnfolder "archive"
>                  (nnfolder-get-new-mail nil)
>                  (nnfolder-inhibit-expiry t)
>                  (nnfolder-directory "~/Mail-softace/sent-mail/")
>                  (nnfolder-active-file "~/Mail-softace/sent-mail/active")
>                  )
>       )
> (setq gnus-message-archive-group "all")

> When I hit the group "nnfolder+archive:all" in "*Group*" buffer. I get:

> Signaling: (error "Couldn't request group nnfolder+archive:all: No such group: all")

[...]

> I further realised that when gnus has started up, there is a
> difference of the value of `nnfolder-group-alist' variable.
> In gnus 5.8.8 this value is `(("all" (1 . 1340)))'.
> In gnus 5.10.6 this value is nil.

> I further figured out that the following lines where removed from
> 5.8.8 to 5.10.6 in nnfolder.el (approx. line 600):
> ----------------------------------------------
> 	;; The group doesn't exist, so we create a new entry for it.
> 	(push (list group (cons 1 0)) nnfolder-group-alist)
> 	(nnfolder-save-active nnfolder-group-alist nnfolder-active-file))
> ----------------------------------------------

Hm, it has been deleted early in the Oort Gnus (2001-07-09).

> Is this a bug?=20
> Can someone help?
> Can I provide you with more information.

> Jarl
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: Bug regarding reading sent-mail folder

[Jarl Friis]

Hello Katsumi.

Thanks for the response.

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Hi,
>
> I read your message last week and tried the same archiving method.
> However, I could neither reproduce any problem nor look at the
> nnfolder-group-alist variable as nil.
>
>>>>>> In <m3llmk7auy.fsf@zeus.intra.softace.dk>
>>>>>>	Jarl Friis <jarl@softace.dk> wrote:
>
>> Hello.
>
>> I recently upgraded from gnus 5.8.8 to 5.10.6 on XEmacs 21.4.10
>
> There is a remarkable difference between 5.8.8 and 5.10.6.
> Recent Gnus uses .mrk files for nnfolder groups, for instance:
>
> % cd ~/Mail-softace/sent-mail
> % ls -aF
> ./ ../ active  all  all.mrk  all.nov  all~

Hmm... that seems not to be the content of my directory, i have:
#all#  ./  ../  active  all  all.nov  all~

It seems that I am missing the all.mrk file, right?

In other folder I have a ".marks" file.

>
> About this, you will find a note in the GNUS-NEWS file as follows:
>
> --8<---------------cut here---------------start------------->8---
[...]
> --8<---------------cut here---------------end--------------->8---
>
> If .mrk files are broken, you can ignore them by setting the
> nnfolder-marks-is-evil variable as t.  This may be unrelated
> though.

What if it is completely missing?

Even without setting this var to anything in .gnus, gnus (when
starting up) changes the value from nil to t.

-- 
Jarl Friis
Softace
Omøgade 8, 2.sal
2100 København Ø.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl@softace.dk

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

>>>>> In <m3smgoth21.fsf@zeus.intra.softace.dk>
>>>>>	Jarl Friis <jarl@softace.dk> wrote:

>> % cd ~/Mail-softace/sent-mail
>> % ls -aF
>> ./ ../ active  all  all.mrk  all.nov  all~

> Hmm... that seems not to be the content of my directory, i have:
> #all#  ./  ../  active  all  all.nov  all~

> It seems that I am missing the all.mrk file, right?

Normally, the all.mrk file will be created automatically.
I guess the reason why Gnus doesn't create it is because
nnfolder-group-alist is nil.

> In other folder I have a ".marks" file.

[...]

>> If .mrk files are broken, you can ignore them by setting the
>> nnfolder-marks-is-evil variable as t.  This may be unrelated
>> though.

> What if it is completely missing?

> Even without setting this var to anything in .gnus, gnus (when
> starting up) changes the value from nil to t.

That's quite strange.  I couldn't find out what sets it as t.

Well, the saved data for the "nnfolder+archive:all" group may be
broken in the ~/.newsrc.eld file.  Could you let me know it?  To
do that, copy the following Lisp form into the *scratch* buffer
and evaluate it (type C-j at the end of the line).  Gnus needs
to be running.

(assoc "nnfolder+archive:all" gnus-newsrc-alist)
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: Bug regarding reading sent-mail folder

[Jarl Friis]

Dear Katsumi.

Thanks for your response. It is long time we communicated. I have been
very busy, yet I haven't solved my problem. Here is some more
information for you. Thanks for helping me.

Katsumi Yamaoka <yamaoka@jpl.org> writes:

>>>>>> In <m3smgoth21.fsf@zeus.intra.softace.dk>
>>>>>>	Jarl Friis <jarl@softace.dk> wrote:
>
>>> % cd ~/Mail-softace/sent-mail
>>> % ls -aF
>>> ./ ../ active  all  all.mrk  all.nov  all~
>
>> Hmm... that seems not to be the content of my directory, i have:
>> #all#  ./  ../  active  all  all.nov  all~
>
>> It seems that I am missing the all.mrk file, right?
>
> Normally, the all.mrk file will be created automatically.
> I guess the reason why Gnus doesn't create it is because
> nnfolder-group-alist is nil.

Allright... But in my case it seems like Gnus-5.10.6 have not done that yet,
probably because it claims that the group "all" does not exist in the
first place, so it has not yet initialised that group (and created the
.mrk/.marks file)

>
>> In other folder I have a ".marks" file.

These are all the nnml backend folders (that can be open/initialised)
hence created by Gnus.

>
> [...]
>
>>> If .mrk files are broken, you can ignore them by setting the
>>> nnfolder-marks-is-evil variable as t.  This may be unrelated
>>> though.
>
>> Even without setting this var to anything in .gnus, gnus (when
>> starting up) changes the value from nil to t.
>
> That's quite strange.  I couldn't find out what sets it as t.

Sorry, I must have mistyped here (or things have changed). It changes
the value from t to nil. Even when setting it manually and putting
`(setq nnfolder-marks-is-evil t)' in ~/.gnus

> Well, the saved data for the "nnfolder+archive:all" group may be
> broken in the ~/.newsrc.eld file.  Could you let me know it? 
> To do that, copy the following Lisp form into the *scratch* buffer
> and evaluate it (type C-j at the end of the line).  Gnus needs to be
> running.
>
> (assoc "nnfolder+archive:all" gnus-newsrc-alist)
=> ("nnfolder+archive:all" 6 nil nil (nnfolder "archive" (nnfolder-inhibit-expiry t) (nnfolder-directory "~/Mail-softace/sent-mail/") (nnfolder-active-file "~/Mail-softace/sent-mail/active")))

Any help is appreciated.

Jarl

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

The old articles can be found in:

http://news.gmane.org/group/gmane.emacs.gnus.general/thread=56470

>>>>> In <m34qae85hy.fsf@zeus.intra.softace.dk> Jarl Friis wrote:

> Thanks for your response. It is long time we communicated. I have been
> very busy, yet I haven't solved my problem. Here is some more
> information for you. Thanks for helping me.

I retried it and succeeded in archiving and reading messages
again.  When I think of it now (I'm not quite sure of all,
though), it doesn't seem to be related to the marks nor to the
change you pointed out as follows:

>>>>> In <m3llmk7auy.fsf@zeus.intra.softace.dk> Jarl Friis wrote:

> I further figured out that the following lines where removed from
> 5.8.8 to 5.10.6 in nnfolder.el (approx. line 600):
> ----------------------------------------------
> 	;; The group doesn't exist, so we create a new entry for it.
> 	(push (list group (cons 1 0)) nnfolder-group-alist)
> 	(nnfolder-save-active nnfolder-group-alist nnfolder-active-file))
> ----------------------------------------------

In addition, there seems to be no problem in the newsrc entry
for your archive group, except that it suggests there's no
article.

>> (assoc "nnfolder+archive:all" gnus-newsrc-alist)

> => ("nnfolder+archive:all" 6 nil nil (nnfolder "archive" (nnfolder-inhibit-expiry t) (nnfolder-directory "~/Mail-softace/sent-mail/") (nnfolder-active-file "~/Mail-softace/sent-mail/active")))

I guess there might be another "archive" method in the
`gnus-server-alist' variable and it might be used by some groups
other than the archive group.  For instance, in the .newsrc.eld
file, if `gnus-server-alist' contains

("archive" nnml "")

and a certain nnml group uses it in `gnus-newsrc-alist' as

("nnml:some" 1 (...) (...) "archive" (...))

rather then

("nnml:some" 1 (...) (...) "nnml:" (...))

, there's no problem to use the group, but the error you've
reported first will occur with the real archive group.  I've
ever experienced a similar accident actually, though I don't
know why it occurred.  The way to fix it is only to edit the
.newsrc.eld file manually.

If it is possible, please show me personally your .newsrc.eld
file.  I may be able to write a Lisp program to fix it.

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

>>>>> In <b4m64uqm7nw.fsf@jpl.org> Katsumi Yamaoka wrote:

> If it is possible, please show me personally your .newsrc.eld
> file.  I may be able to write a Lisp program to fix it.

Thank you for sending it.  However, I didn't find any oddity in
your .newsrc.eld file.  `gnus-newsrc-alist' contains some nntp
groups connecting to some servers, nnml groups using the
nnml:softace server and one nnfolder group in question.
`gnus-server-alist' has a single element which defines the
"archive" server.  They are summarized as follows:

(setq gnus-newsrc-alist '(...
      ("nnfolder+archive:all" 6 nil nil
       (nnfolder "archive"
		 (nnfolder-inhibit-expiry t)
		 (nnfolder-directory "~/Mail-softace/sent-mail/")
		 (nnfolder-active-file "~/Mail-softace/sent-mail/active")))
      ...))

(setq gnus-server-alist
      '(("archive" nnfolder "archive"
	 (nnfolder-get-new-mail nil)
	 (nnfolder-inhibit-expiry t)
	 (nnfolder-directory "~/Mail-softace/sent-mail/")
	 (nnfolder-active-file "~/Mail-softace/sent-mail/active"))))

The `all' group entry specifies the server explicitly by itself,
and there's neither other `all' group nor other `archive' server.
The problem is `nil nil' in the `all' group entry which suggests
there's no article.  Though I couldn't reproduce the problem,
could anyone please try it again?  The message Jarl posted first
is below:


[-- Attachment #2: Type: message/rfc822, Size: 2713 bytes --]

From: Jarl Friis <jarl@softace.dk>
Subject: Bug regarding reading sent-mail folder
Date: Mon, 01 Mar 2004 18:09:25 +0100
Message-ID: <m3llmk7auy.fsf@zeus.intra.softace.dk>

Hello.

I recently upgraded from gnus 5.8.8 to 5.10.6 on XEmacs 21.4.10

Now I can't read my sent mails.

I believe I found a bug regarding regarding sent-mail. My relevant
"~/.gnus" part is (pretty standard/default):=20

(setq gnus-message-archive-method
      '(nnfolder "archive"
                 (nnfolder-get-new-mail nil)
                 (nnfolder-inhibit-expiry t)
                 (nnfolder-directory "~/Mail-softace/sent-mail/")
                 (nnfolder-active-file "~/Mail-softace/sent-mail/active")
                 )
      )
(setq gnus-message-archive-group "all")

When I hit the group "nnfolder+archive:all" in "*Group*" buffer. I get:

Signaling: (error "Couldn't request group nnfolder+archive:all: No such group: all")
  signal(error ("Couldn't request group nnfolder+archive:all: No such group: all"))
  cerror("Couldn't request group %s: %s" "nnfolder+archive:all" "No such group: all")
  apply(cerror "Couldn't request group %s: %s" ("nnfolder+archive:all" "No such group: all"))
  error("Couldn't request group %s: %s" "nnfolder+archive:all" "No such group: all")
  gnus-select-newsgroup("nnfolder+archive:all" nil nil)
  gnus-summary-read-group-1("nnfolder+archive:all" nil t nil nil nil)
  gnus-summary-read-group("nnfolder+archive:all" nil t nil nil nil nil)
  gnus-group-read-group(nil t)
  gnus-group-select-group(nil)
  gnus-topic-select-group(nil)
  call-interactively(gnus-topic-select-group)


I further realised that when gnus has started up, there is a
difference of the value of `nnfolder-group-alist' variable.
In gnus 5.8.8 this value is `(("all" (1 . 1340)))'.
In gnus 5.10.6 this value is nil.

I further figured out that the following lines where removed from
5.8.8 to 5.10.6 in nnfolder.el (approx. line 600):
----------------------------------------------
	;; The group doesn't exist, so we create a new entry for it.
	(push (list group (cons 1 0)) nnfolder-group-alist)
	(nnfolder-save-active nnfolder-group-alist nnfolder-active-file))
----------------------------------------------

Is this a bug?=20
Can someone help?
Can I provide you with more information.

Jarl

-- 
Jarl Friis
Softace
Omøgade 8, 2.sal
2100 København Ø.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl@softace.dk

Re: Bug regarding reading sent-mail folder

[Jarl Friis]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

>>>>>> In <b4m64uqm7nw.fsf@jpl.org> Katsumi Yamaoka wrote:
>
>> If it is possible, please show me personally your .newsrc.eld
>> file.  I may be able to write a Lisp program to fix it.
>
> Thank you for sending it.  However, I didn't find any oddity in
> your .newsrc.eld file.  `gnus-newsrc-alist' contains some nntp
> groups connecting to some servers, nnml groups using the
> nnml:softace server and one nnfolder group in question.
> `gnus-server-alist' has a single element which defines the
> "archive" server.  They are summarized as follows:
>
> (setq gnus-newsrc-alist '(...
>       ("nnfolder+archive:all" 6 nil nil
>        (nnfolder "archive"
> 		 (nnfolder-inhibit-expiry t)
> 		 (nnfolder-directory "~/Mail-softace/sent-mail/")
> 		 (nnfolder-active-file "~/Mail-softace/sent-mail/active")))
>       ...))
>
> (setq gnus-server-alist
>       '(("archive" nnfolder "archive"
> 	 (nnfolder-get-new-mail nil)
> 	 (nnfolder-inhibit-expiry t)
> 	 (nnfolder-directory "~/Mail-softace/sent-mail/")
> 	 (nnfolder-active-file "~/Mail-softace/sent-mail/active"))))
>
> The `all' group entry specifies the server explicitly by itself,
> and there's neither other `all' group nor other `archive' server.
> The problem is `nil nil' in the `all' group entry which suggests
> there's no article. 

What values should there be instead of `nil nil'? Coule I manually
edit them? Whould it help?

Jarl

-- 
Jarl Friis
Softace ApS
Omøgade 8, 2.sal
2100 København Ø.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl@softace.dk

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

>>>>> In <m3aci9grq9.fsf@zeus.intra.softace.dk> Jarl Friis wrote:

> What values should there be instead of `nil nil'?

The first one registers the range of article numbers you've
read, the second one registers an alist of marks and the ranges
of article numbers.  You can see what they should be at the
entries of groups you can read correctly.

>> (setq gnus-newsrc-alist '(...
>>       ("nnfolder+archive:all" 6 nil nil
>>        (nnfolder "archive"
>> 		 (nnfolder-inhibit-expiry t)
>> 		 (nnfolder-directory "~/Mail-softace/sent-mail/")
>> 		 (nnfolder-active-file "~/Mail-softace/sent-mail/active")))
>>       ...))

The value `nil nil' means you've never read articles and you've
never marked articles in that group, but it is syntactically
correct.  If you enter that group, you should be able to read
all articles as unread.  The real problem is that Gnus doesn't
open the group in question correctly.

> Coule I manually edit them? Whould it help?

The "nnfolder+archive:all" group entry in the gnus-newsrc-alist
variable saved in your .newsrc.eld file should be maintained by
Gnus.  I don't think it gets better even if you edit it manually.

I'm sorry that I cannot reproduce the same bad behavior.  If it
occurs on me, I'll investigate all the Gnus functions related to
this step by step using edebug for example.

(I personally will not use the mail directory which is not the
default or the methods which aren't popular since I don't
believe they have been fully tested, though.)

Re: Bug regarding reading sent-mail folder

[Jarl Friis]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

>>>>>> In <m3aci9grq9.fsf@zeus.intra.softace.dk> Jarl Friis wrote:
>
>> What values should there be instead of `nil nil'?
>
> The first one registers the range of article numbers you've
> read, the second one registers an alist of marks and the ranges
> of article numbers.  You can see what they should be at the
> entries of groups you can read correctly.

Thanks.

> The value `nil nil' means you've never read articles and you've
> never marked articles in that group, but it is syntactically
> correct.  If you enter that group, you should be able to read
> all articles as unread.  The real problem is that Gnus doesn't
> open the group in question correctly.

Right, that is exactly the Gnus bug!

> I'm sorry that I cannot reproduce the same bad behavior.  If it
> occurs on me, I'll investigate all the Gnus functions related to
> this step by step using edebug for example.

Have you tried reproducing using the same backend (method)?

> (I personally will not use the mail directory which is not the
> default or the methods which aren't popular since I don't
> believe they have been fully tested, though.)

So what you say is that I might solve the problem by converting my
sent-mail to another backend (method of storing my sent-mail), right?

How do I do that?

Jarl

-- 
Jarl Friis
Softace ApS
Omøgade 8, 2.sal
2100 København Ø.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl@softace.dk

Re: Bug regarding reading sent-mail folder

[Katsumi Yamaoka]

>>>>> In <m31x3kgkbn.fsf@zeus.intra.softace.dk> Jarl Friis wrote:

>> I'm sorry that I cannot reproduce the same bad behavior.  If it
>> occurs on me, I'll investigate all the Gnus functions related to
>> this step by step using edebug for example.

> Have you tried reproducing using the same backend (method)?

Yes, I tried it more than three times using exactly the same
method (and directory) as yours.  I also tried some various ways
in order to make the problem clear, but I've never succeeded in
it yet.  I hope other developers see the following thread and
try it too.

http://news.gmane.org/group/gmane.emacs.gnus.general/thread=56470

>> (I personally will not use the mail directory which is not the
>> default or the methods which aren't popular since I don't
>> believe they have been fully tested, though.)

> So what you say is that I might solve the problem by converting my
> sent-mail to another backend (method of storing my sent-mail), right?

Probably.  I use the local nntp server for `gnus-select-method'
and the settings similar to the following:

(setq gnus-secondary-select-methods '(nnml "")
      gnus-message-archive-method '(nnml "")
      gnus-message-archive-group '((if (message-news-p)
				       "sent-news"
				     "sent-mail")))