strange behaviour when `B c'ing an empty article

strange behaviour when `B c'ing an empty article

[Dan Christensen]

In my nnfolder:Inbox group, I have two articles, which I have
attached below.  If I put the point on the first one, and do
`B c nnfolder:test5', I get prompted for whether I want to
create the group and I say yes.  In the resulting group,
the article incorrectly keeps its article number from the
Inbox group.  The active file thinks that the article is
number 1.

When I do the same thing with the second article attached below,
everything is fine.  Could it have to do with the fact that the
first article is empty?

Note:  Further testing shows that this has nothing to do with the fact
that the groups test5 and test6 are created in the process.  The same
error happens whenever the first article is copied to an nnfolder group.

In the following, the spaces before the "From " lines are added by a
hook I have and are not really in the files.  The blank lines *are*
in the files.

The two articles:

-----------
 From jdc@chow.mat.jhu.edu Wed Oct 06 16:28:32 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxfr-0006FN-00; Wed, 6 Oct 1999 16:28:31 -0400
To: jdc@scratchy
Subject: test of orphan scoring
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:28:31 -0400
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <874sg48c9c.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Lines: 1
Xref: scratchy Inbox:10385
X-Gnus-Article-Number: 10385   Wed Oct  6 16:28:37 1999

 From jdc@chow.mat.jhu.edu Wed Oct 06 16:29:27 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxgl-0006Fd-00; Wed, 6 Oct 1999 16:29:27 -0400
To: jdc@scratchy
Subject: Re: test of orphan scoring
References: <874sg48c9c.fsf@chow.mat.jhu.edu>
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:29:26 -0400
In-Reply-To: Dan Christensen's message of "06 Oct 1999 16:28:31 -0400"
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <87puys6xnd.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Lines: 3
Xref: scratchy Inbox:10386
X-Gnus-Article-Number: 10386   Wed Oct  6 16:29:29 1999

this is the followup to jdc


----------------

The nnfolder file for the group test5, which is where the first
article was moved to:

----------------


 From jdc@chow.mat.jhu.edu Wed Oct 06 16:28:32 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxfr-0006FN-00; Wed, 6 Oct 1999 16:28:31 -0400
To: jdc@scratchy
Subject: test of orphan scoring
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:28:31 -0400
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <874sg48c9c.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Lines: 1
Xref: scratchy Inbox:10385
X-Gnus-Article-Number: 10385   Wed Oct  6 16:28:37 1999
-------------------

The nnfolder file for the group test6, which is where the second
article was moved to:

-------------------


 From jdc@chow.mat.jhu.edu Wed Oct 06 16:29:27 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxgl-0006Fd-00; Wed, 6 Oct 1999 16:29:27 -0400
To: jdc@scratchy
Subject: Re: test of orphan scoring
References: <874sg48c9c.fsf@chow.mat.jhu.edu>
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:29:26 -0400
In-Reply-To: Dan Christensen's message of "06 Oct 1999 16:28:31 -0400"
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <87puys6xnd.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Xref: scratchy test6:1
Lines: 3
X-Gnus-Article-Number: 1   Wed Oct  6 16:36:52 1999

this is the followup to jdc


---------------------

And the relevant lines of the active file:

test5 1 1 y
test6 1 1 y

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

Re: strange behaviour when `B c'ing an empty article

[Shenghuo ZHU]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

>>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:

[...]

Dan> When I do the same thing with the second article attached below,
Dan> everything is fine.  Could it have to do with the fact that the
Dan> first article is empty?

[...]

I guess I just fixed it.

The patch follows and has been committed.

-- 
Shenghuo ZHU

1999-10-06  Shenghuo ZHU  <zsh@cs.rochester.edu>

	* nnfolder.el (nnfolder-request-move-article): For empty article,
	search till (point-max).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 97-106.diff --]
[-- Type: text/x-patch, Size: 654 bytes --]

Index: nnfolder.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnfolder.el,v
retrieving revision 5.18
diff -u -r5.18 nnfolder.el
--- nnfolder.el	1999/08/27 13:18:17	5.18
+++ nnfolder.el	1999/10/06 23:21:37
@@ -368,7 +368,8 @@
 	 (goto-char (point-min))
 	 (while (re-search-forward
 		 (concat "^" nnfolder-article-marker)
-		 (save-excursion (search-forward "\n\n" nil t) (point)) t)
+		 (save-excursion (and (search-forward "\n\n" nil t) (point))) 
+		 t)
 	   (delete-region (progn (beginning-of-line) (point))
 			  (progn (forward-line 1) (point))))
 	 (setq result (eval accept-form))

Re: strange behaviour when `B c'ing an empty article

[Dan Christensen]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> >>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:
> 
> [...]
> 
> Dan> When I do the same thing with the second article attached below,
> Dan> everything is fine.  Could it have to do with the fact that the
> Dan> first article is empty?
> 
> [...]
> 
> I guess I just fixed it.
> 
> The patch follows and has been committed.

I don't see any change in the behaviour.

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

Re: strange behaviour when `B c'ing an empty article

[Shenghuo ZHU]

[-- Attachment #1: Type: text/plain, Size: 562 bytes --]

>>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:

Dan> I don't see any change in the behaviour.

The change is that 'B m' works. 

Now, another patch make more things work. (Committed)

(Now, it's time to remove my a bunch of test nnfolder.)

-- 
Shenghuo ZHU

1999-10-06  Shenghuo ZHU  <zsh@cs.rochester.edu>

	* nnfolder.el (nnfolder-request-move-article): For empty article,
	search till (point-max).
	(nnfolder-retrieve-headers): Ditto.
	(nnfolder-request-accept-article): Ditto.
	(nnfolder-save-mail): Ditto.
	(nnfolder-insert-newsgroup-line): Ditto.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 97-107.diff --]
[-- Type: text/x-patch, Size: 2204 bytes --]

Index: nnfolder.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnfolder.el,v
retrieving revision 5.19
diff -u -r5.19 nnfolder.el
--- nnfolder.el	1999/10/06 23:23:20	5.19
+++ nnfolder.el	1999/10/07 03:14:29
@@ -119,8 +119,9 @@
 	    (set-buffer nnfolder-current-buffer)
 	    (when (nnfolder-goto-article article)
 	      (setq start (point))
-	      (search-forward "\n\n" nil t)
-	      (setq stop (1- (point)))
+	      (setq stop (if (search-forward "\n\n" nil t)
+			     (1- (point))
+			   (point-max)))
 	      (set-buffer nntp-server-buffer)
 	      (insert (format "221 %d Article retrieved.\n" article))
 	      (insert-buffer-substring nnfolder-current-buffer start stop)
@@ -401,8 +402,9 @@
        (save-excursion
 	 (set-buffer buf)
 	 (goto-char (point-min))
-	 (search-forward "\n\n" nil t)
-	 (forward-line -1)
+	 (if (search-forward "\n\n" nil t)
+	     (forward-line -1)
+	   (goto-char (point-max)))
 	 (while (re-search-backward (concat "^" nnfolder-article-marker) nil t)
 	   (delete-region (point) (progn (forward-line 1) (point))))
 	 (when nnmail-cache-accepted-message-ids
@@ -648,8 +650,9 @@
     (while (setq group-art (pop group-art-list))
       ;; Kill any previous newsgroup markers.
       (goto-char (point-min))
-      (search-forward "\n\n" nil t)
-      (forward-line -1)
+      (if (search-forward "\n\n" nil t)
+	  (forward-line -1)
+	(goto-char (point-max)))
       (while (search-backward (concat "\n" nnfolder-article-marker) nil t)
 	(delete-region (1+ (point)) (progn (forward-line 2) (point))))
 
@@ -678,10 +681,11 @@
 (defun nnfolder-insert-newsgroup-line (group-art)
   (save-excursion
     (goto-char (point-min))
-    (when (search-forward "\n\n" nil t)
-      (forward-char -1)
-      (insert (format (concat nnfolder-article-marker "%d   %s\n")
-		      (cdr group-art) (current-time-string))))))
+    (if (search-forward "\n\n" nil t)
+	(forward-char -1)
+      (goto-char (point-max)))
+    (insert (format (concat nnfolder-article-marker "%d   %s\n")
+		    (cdr group-art) (current-time-string)))))
 
 (defun nnfolder-active-number (group)
   ;; Find the next article number in GROUP.

Re: strange behaviour when `B c'ing an empty article

[Dan Christensen]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> Now, another patch make more things work. (Committed)

Getting closer.  :-)  I copied the articles to groups test8 and
test9 (with `B c') and have attached the files below.

Here are various things that seem odd:

o The number of lines in the non-empty article got changed from 3
  to 1 when the move is done.  What is the correct way to count
  lines in an article?

o The number of lines in the empty article is calculated as 1
  (both before and after the move).  Is this correct?

o The Xref lines are different for the two articles.  One is
  self-referential, and the other points back to my Inbox.
  I think the latter is correct, right?

o On the short article, if display the full headers by
  hitting `t', everything looks ok except that I get two
  headers combined on one line:

X-Gnus-Article-Number: 1   Thu Oct  7 00:55:51 1999From: Dan Christensen <jdc@chow.mat.jhu.edu>

Mbox format sure is awkward...

Here is test8:

---------------


 From jdc@chow.mat.jhu.edu Wed Oct 06 16:28:32 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxfr-0006FN-00; Wed, 6 Oct 1999 16:28:31 -0400
To: jdc@scratchy
Subject: test of orphan scoring
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:28:31 -0400
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <874sg48c9c.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Lines: 1
Xref: scratchy Inbox:10385
X-Gnus-Article-Number: 1   Thu Oct  7 00:55:51 1999
------------------

and test9:

------------------


 From jdc@chow.mat.jhu.edu Wed Oct 06 16:29:27 1999
Received: from jdc by scratchy with local (Exim 2.05 #1 (Debian))
	id 11Yxgl-0006Fd-00; Wed, 6 Oct 1999 16:29:27 -0400
To: jdc@scratchy
Subject: Re: test of orphan scoring
References: <874sg48c9c.fsf@chow.mat.jhu.edu>
From: Dan Christensen <jdc@chow.mat.jhu.edu>
Date: 06 Oct 1999 16:29:26 -0400
In-Reply-To: Dan Christensen's message of "06 Oct 1999 16:28:31 -0400"
X-Gnus-Mail-Source: file:/var/mail/jdc
Message-ID: <87puys6xnd.fsf@chow.mat.jhu.edu>
User-Agent: Gnus/5.07009701 (Pterodactyl Gnus v0.97.1) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Xref: scratchy test9:1
Lines: 1
X-Gnus-Article-Number: 1   Thu Oct  7 00:58:24 1999

this is the followup to jdc
------------------

Thanks a lot for looking into this...

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

Re: strange behaviour when `B c'ing an empty article

[Shenghuo ZHU]

>>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:

Dan> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> Now, another patch make more things work. (Committed)

Dan> Getting closer.  :-)  I copied the articles to groups test8 and
Dan> test9 (with `B c') and have attached the files below.

Dan> Here are various things that seem odd:

Dan> o The number of lines in the non-empty article got changed from 3
Dan>   to 1 when the move is done.  What is the correct way to count
Dan>   lines in an article?

    (search-forward "\n\n" nil t)
    (setq lines (count-lines (point) (point-max)))

It seems only 1 line in test9.

Dan> o The number of lines in the empty article is calculated as 1
Dan>   (both before and after the move).  Is this correct?

Maybe 0 or 1. Maybe a hide blank line is counted.

Dan> o The Xref lines are different for the two articles.  One is
Dan>   self-referential, and the other points back to my Inbox.
Dan>   I think the latter is correct, right?

The former is correct. Xref is for cross-post. Each item refers to the
copy in one group. If only one in the Xref, it must be
self-referential. I guess Gnus does not treat 'B c' as cross-post.

Dan> o On the short article, if display the full headers by
Dan>   hitting `t', everything looks ok except that I get two
Dan>   headers combined on one line:

Dan> X-Gnus-Article-Number: 1   Thu Oct  7 00:55:51 1999From: Dan Christensen <jdc@chow.mat.jhu.edu>

This is awkward. Anyway, now I insert one blank line for others can
not find "\n\n". I guess it helps. I'll commit it later.

[...]

-- 
Shenghuo ZHU

Re: strange behaviour when `B c'ing an empty article

[Dan Christensen]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> Dan> o The Xref lines are different for the two articles.  One is
> Dan>   self-referential, and the other points back to my Inbox.
> Dan>   I think the latter is correct, right?
> 
> The former is correct. Xref is for cross-post. Each item refers to the
> copy in one group. If only one in the Xref, it must be
> self-referential. I guess Gnus does not treat 'B c' as cross-post.

So the test8 article still triggers a bug?

/home/jdc/Gnus/Mail/test8:Xref: scratchy Inbox:10385
/home/jdc/Gnus/Mail/test9:Xref: scratchy test9:1

(The test8 one is the empty one, and both were copied with `B c'.  To
be honest, it seems to me that the test8 article's Xref is better than
the test9 article's Xref, but I'm not really sure what's supposed to
happen.)

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

Re: strange behaviour when `B c'ing an empty article

[Shenghuo ZHU]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

>>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:


[...]

Dan> So the test8 article still triggers a bug?

Dan> /home/jdc/Gnus/Mail/test8:Xref: scratchy Inbox:10385
Dan> /home/jdc/Gnus/Mail/test9:Xref: scratchy test9:1

Dan> (The test8 one is the empty one, and both were copied with `B c'.
Dan> To be honest, it seems to me that the test8 article's Xref is
Dan> better than the test9 article's Xref, but I'm not really sure
Dan> what's supposed to happen.)

[...]

It is a bug. The same about the line number.

Fixed and committed.

-- 
Shenghuo ZHU

1999-10-07 17:09:34  Shenghuo ZHU  <zsh@cs.rochester.edu>

	* nnmail.el (nnmail-insert-xref): Dealing with empty article.
	(nnmail-insert-lines): Ditto.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 97-109.diff --]
[-- Type: text/x-patch, Size: 2383 bytes --]

Index: nnmail.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnmail.el,v
retrieving revision 5.39
diff -u -r5.39 nnmail.el
--- nnmail.el	1999/09/27 15:19:04	5.39
+++ nnmail.el	1999/10/07 21:12:27
@@ -999,35 +999,39 @@
   (let (lines chars)
     (save-excursion
       (goto-char (point-min))
-      (when (search-forward "\n\n" nil t)
-	(setq chars (- (point-max) (point)))
-	(setq lines (count-lines (point) (point-max)))
-	(forward-char -1)
-	(save-excursion
-	  (when (re-search-backward "^Lines: " nil t)
-	    (delete-region (point) (progn (forward-line 1) (point)))))
-	(beginning-of-line)
-	(insert (format "Lines: %d\n" (max lines 0)))
-	chars))))
+      (unless (search-forward "\n\n" nil t) 
+	(goto-char (point-max))
+	(insert "\n"))
+      (setq chars (- (point-max) (point)))
+      (setq lines (count-lines (point) (point-max)))
+      (forward-char -1)
+      (save-excursion
+	(when (re-search-backward "^Lines: " nil t)
+	  (delete-region (point) (progn (forward-line 1) (point)))))
+      (beginning-of-line)
+      (insert (format "Lines: %d\n" (max lines 0)))
+      chars)))
 
 (defun nnmail-insert-xref (group-alist)
   "Insert an Xref line based on the (group . article) alist."
   (save-excursion
     (goto-char (point-min))
-    (when (search-forward "\n\n" nil t)
-      (forward-char -1)
-      (when (re-search-backward "^Xref: " nil t)
-	(delete-region (match-beginning 0)
-		       (progn (forward-line 1) (point))))
-      (insert (format "Xref: %s" (system-name)))
-      (while group-alist
-	(insert (format " %s:%d"
-			(mm-encode-coding-string
-			 (caar group-alist)
-			 nnmail-pathname-coding-system)
-			(cdar group-alist)))
-	(setq group-alist (cdr group-alist)))
-      (insert "\n"))))
+    (unless (search-forward "\n\n" nil t)
+      (goto-char (point-max))
+      (insert "\n"))
+    (forward-char -1)
+    (when (re-search-backward "^Xref: " nil t)
+      (delete-region (match-beginning 0)
+		     (progn (forward-line 1) (point))))
+    (insert (format "Xref: %s" (system-name)))
+    (while group-alist
+      (insert (format " %s:%d"
+		      (mm-encode-coding-string
+		       (caar group-alist)
+		       nnmail-pathname-coding-system)
+		      (cdar group-alist)))
+      (setq group-alist (cdr group-alist)))
+    (insert "\n")))
 
 ;;; Message washing functions
 

Re: strange behaviour when `B c'ing an empty article

[Dan Christensen]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> >>>>> "Dan" == Dan Christensen <jdc@jhu.edu> writes:
> Dan> So the test8 article still triggers a bug?
> 
> Dan> /home/jdc/Gnus/Mail/test8:Xref: scratchy Inbox:10385
> Dan> /home/jdc/Gnus/Mail/test9:Xref: scratchy test9:1
> 
> Dan> (The test8 one is the empty one, and both were copied with `B c'.
> Dan> To be honest, it seems to me that the test8 article's Xref is
> Dan> better than the test9 article's Xref, but I'm not really sure
> Dan> what's supposed to happen.)
> 
> It is a bug. The same about the line number.
> 
> Fixed and committed.

Looks excellent!  Thanks very much.  I also learned about the `B B'
command which keeps the old Xref entry as well (the behaviour I
thought was "better" in the parenthetical remark above).  So
all the problems with empty articles that I've noticed seem
to be fixed.

Is anyone else having the problems with orphan scoring that I
mentioned in a separate post?

Dan

-- 
Dan Christensen
jdc@math.jhu.edu