Behaviour of flow-fill before signature

Behaviour of flow-fill before signature

[Michal Jankowski]

Hello,

I have noticed that some "format=flowed;" messages are displayed
incorrectly (?) - the signature is treated as part of the message and
"flowed" together with it.

It seems to happen whenever the last non-blank line before signature
ends with a space, like this (quotes added for clarity):

"Last line "
""
"-- "
"Sig"

gets displayed as
"Last line -- "
"Sig"

I'm using Gnus v5.10.2.

For real example, see the usenet message
<djlo5r$hom$1@atlantis.news.tpi.pl>

Any ideas? One could argue that messages shouldn't end with "flowed"
line and therefore the above message is malformed (?). But I'm seeing
more and more messages formatted like this and would like to see Gnus
displaying them better.

Thanks

  Michal

Re: Behaviour of flow-fill before signature

[Simon Josefsson]

Michal Jankowski <Michal.Jankowski@fuw.edu.pl> writes:

> Hello,
>
> I have noticed that some "format=flowed;" messages are displayed
> incorrectly (?) - the signature is treated as part of the message and
> "flowed" together with it.
>
> It seems to happen whenever the last non-blank line before signature
> ends with a space, like this (quotes added for clarity):
>
> "Last line "
> ""
> "-- "
> "Sig"
>
> gets displayed as
> "Last line -- "
> "Sig"
>
> I'm using Gnus v5.10.2.
>
> For real example, see the usenet message
> <djlo5r$hom$1@atlantis.news.tpi.pl>
>
> Any ideas? One could argue that messages shouldn't end with "flowed"
> line and therefore the above message is malformed (?). But I'm seeing
> more and more messages formatted like this and would like to see Gnus
> displaying them better.

The RFC is clear on this, see quote below, the 
above would be incorrect.  However, this should be 
fixed in CVS, both for Gnus 5.10 and No Gnus, so an 
upgrade should solve the problem.

   An interpreting agent SHOULD allow for three exceptions to the rule
   that paragraphs end with a fixed line.  These exceptions are
   improperly constructed messages: a flowed line SHOULD be considered
   to end the paragraph if it is followed by a line of a different quote
   depth (see 4.5) or by a signature separator (see 4.3); the end of the
   body also ends the paragraph.

I have made this message an example of this problem, so that people
can test whether code works or not.  I had to fix a bug in No Gnus to
make unquoted lines flow too.

Last line 

-- 
Sig

Re: Behaviour of flow-fill before signature

[Reiner Steib]

On Wed, Oct 26 2005, Simon Josefsson wrote:

> However, this should be fixed in CVS, both for Gnus 5.10 and No Gnus, so an
> upgrade should solve the problem.

Beside the point-at-[be]ol stuff (required for compatibility with old
Emacsen), there is this diff:

--- v5-10/lisp/flow-fill.el     2005-08-26 15:41:39.000000000 +0200
+++ No/lisp/flow-fill.el        2005-10-26 11:27:06.000000000 +0200
@@ -113,6 +102,11 @@
   (save-excursion
     (set-buffer (or (current-buffer) buffer))
     (goto-char (point-min))
+    ;; Remove space stuffing.
+    (while (re-search-forward "^ " nil t)
+      (delete-char -1)
+      (forward-line 1))
+    (goto-char (point-min))
     (while (re-search-forward " $" nil t)
       (when (save-excursion
              (beginning-of-line)

Should it be added to Gnus 5.10 too?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Behaviour of flow-fill before signature

[Simon Josefsson]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Wed, Oct 26 2005, Simon Josefsson wrote:
>
>> However, this should be fixed in CVS, both for Gnus 5.10 and No Gnus, so an
>> upgrade should solve the problem.
>
> Beside the point-at-[be]ol stuff (required for compatibility with old
> Emacsen), there is this diff:
>
> --- v5-10/lisp/flow-fill.el     2005-08-26 15:41:39.000000000 +0200
> +++ No/lisp/flow-fill.el        2005-10-26 11:27:06.000000000 +0200
> @@ -113,6 +102,11 @@
>    (save-excursion
>      (set-buffer (or (current-buffer) buffer))
>      (goto-char (point-min))
> +    ;; Remove space stuffing.
> +    (while (re-search-forward "^ " nil t)
> +      (delete-char -1)
> +      (forward-line 1))
> +    (goto-char (point-min))
>      (while (re-search-forward " $" nil t)
>        (when (save-excursion
>               (beginning-of-line)
>
> Should it be added to Gnus 5.10 too?

I think that patch is incorrect -- according to RFC 3767, space
stuffing should be done after checking for quoting.  I don't see a
simple way to fix this though, right now the 5.10 logic is a bit
backwards, in that it searches for flowed lines first and then for
quoting on those lines.  So it may miss space stuffing on fixed lines.

I don't have time to work out the code properly right now, but if
someone would like to read RFC 3767 and construct examples for all
corner cases, that would really help.  It is difficult to modify the
algorithm without accidentally cause regressions.  Making a test
harness from the examples is the next step (there is one example and
very simple test harness in flow-fill.el now).  After that, tinkering
with the algorithm is much simpler.

So I don't care strongly whether 5.10 has this patch or not, as the
code appear slightly broken regardless.