"> > >" space removal removes too many spaces?

"> > >" space removal removes too many spaces?

[Kai Großjohann]

Look at this:

% ftp
ftp> open host
ftp> quit
% date

If you hit `F' or `R' on this message you will see that some lines are
quoted like "> % ftp" whereas other lines are quoted like ">ftp> quit".
Note how a space is missing in the `ftp>' lines.

Is this a bug?

kai
-- 
~/.signature: No such file or directory

Re: "> > >" space removal removes too many spaces?

[Norbert Koch]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Look at this:
> 
> % ftp
>ftp> open host
>ftp> quit
> % date
> 
> If you hit `F' or `R' on this message you will see that some lines are
> quoted like "> % ftp" whereas other lines are quoted like ">ftp> quit".
> Note how a space is missing in the `ftp>' lines.
> 
> Is this a bug?

I guess, Gnus assumes 'ftp>' is a quote string, eg. like the ones
SuperCite uses.  Thus, it treats these lines differently.  IMO this is
not a bug.

norbert.

Re: "> > >" space removal removes too many spaces?

[Kai Großjohann]

Norbert Koch <nk@LF.net> writes:

> I guess, Gnus assumes 'ftp>' is a quote string, eg. like the ones
> SuperCite uses.  Thus, it treats these lines differently.

Obviously.

> IMO this is not a bug.

Hm.  But maybe I would prefer Gnus to simplify `> > >' but not `> x>'.
What do you think?

kai
-- 
~/.signature: No such file or directory

Re: "> > >" space removal removes too many spaces?

[Norbert Koch]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

Hi!

>> IMO this is not a bug.
> 
> Hm.  But maybe I would prefer Gnus to simplify `> > >' but not `> x>'.
> What do you think?

Yes, this might be a good thing to do.

norbert.

Re: "> > >" space removal removes too many spaces?

[Simon Josefsson]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Look at this:
> 
> % ftp
> ftp> open host
> ftp> quit
> % date
> 
> If you hit `F' or `R' on this message you will see that some lines are
> quoted like "> % ftp" whereas other lines are quoted like ">ftp> quit".
> Note how a space is missing in the `ftp>' lines.
> 
> Is this a bug?

I think so; `message-yank-cited-prefix' (">") should probably only be
used if the original article used the "correct" quotation style.
Right now `message-cite-prefix-regexp' is used to match quoted lines,
but that regexp matches lots of weird styles, and there might exist
uglier missmatches than this.

Also, consider the following style as well:

,----
| Kai> Hm.  The agent.  Hm.  I have (gnus-agentize) in my .gnus and the
| Kai> server is under the agent, but I'm always plugged.
| 
| If plugged, it shouldn't matter then as it shouldn't be talking to the
| agent (AFAIK).
`----

When quoting that, I think

,----
| > Kai> Hm.  The agent.  Hm.  I have (gnus-agentize) in my .gnus and the
| > Kai> server is under the agent, but I'm always plugged.
| > 
| > If plugged, it shouldn't matter then as it shouldn't be talking to the
| > agent (AFAIK).
`----

looks nicer than

,----
| >Kai> Hm.  The agent.  Hm.  I have (gnus-agentize) in my .gnus and the
| >Kai> server is under the agent, but I'm always plugged.
| > 
| > If plugged, it shouldn't matter then as it shouldn't be talking to the
| > agent (AFAIK).
`----

Perhaps.

So, maybe the following?  Maybe it shouldn't match ">From"?  And maybe
it shouldn't be hardcoded.  Or maybe, both that string and
`message-yank-{cited-,}prefix' really _should_ be hardcoded (users
should not really change them).  Perhaps it's enough that these
variables aren't documented.

--- message.el.~6.108.~	Sat Jul 28 19:01:54 2001
+++ message.el	Sun Aug  5 14:43:53 2001
@@ -2101,7 +2101,7 @@
       (save-excursion
 	(goto-char start)
 	(while (< (point) (mark t))
-	  (if (looking-at message-cite-prefix-regexp)
+	  (if (looking-at ">")
 	      (insert message-yank-cited-prefix)
 	    (insert message-yank-prefix))
 	  (forward-line 1))))

Re: "> > >" space removal removes too many spaces?

[Simon Josefsson]

Simon Josefsson <jas@extundo.com> writes:

> Also, consider the following style as well:
> 
> ,----
>| Kai> Hm.  The agent.  Hm.  I have (gnus-agentize) in my .gnus and the
>| Kai> server is under the agent, but I'm always plugged.
>| 
>| If plugged, it shouldn't matter then as it shouldn't be talking to the
>| agent (AFAIK).
> `----

And consider quoting boxquote's as well. :-)

Without the patch I get the above.  With the patch I get

> Also, consider the following style as well:
> 
>  ,----
> | Kai> Hm.  The agent.  Hm.  I have (gnus-agentize) in my .gnus and the
> | Kai> server is under the agent, but I'm always plugged.
> | 
> | If plugged, it shouldn't matter then as it shouldn't be talking to the
> | agent (AFAIK).
>  `----

which is better.

Sorry for following up on myself, but I just noticed this.

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

Simon Josefsson <jas@extundo.com> writes:

> --- message.el.~6.108.~	Sat Jul 28 19:01:54 2001
>+++ message.el	Sun Aug  5 14:43:53 2001
> @@ -2101,7 +2101,7 @@
>        (save-excursion
>  	(goto-char start)
>  	(while (< (point) (mark t))
> -	  (if (looking-at message-cite-prefix-regexp)
>+	  (if (looking-at ">")
>  	      (insert message-yank-cited-prefix)
>  	    (insert message-yank-prefix))
>  	  (forward-line 1))))

Hmm, diffs aren't quoted properly either.  Oh my...

Well, what I wanted to say is the following.  Looking just at one line
is certainly not enough.  Consider at the following example:

> According to our analysis, we have a total of
>>50 cases in which our software fails.  This
> is, of course, not acceptable.

'>' may appear at the beginning of a line accidently, and we certainly
want to supply the space character in this case.

Re: "> > >" space removal removes too many spaces?

[Simon Josefsson]

Florian Weimer <fw@deneb.enyo.de> writes:

> Well, what I wanted to say is the following.  Looking just at one line
> is certainly not enough.  Consider at the following example:
> 
>> According to our analysis, we have a total of
>>>50 cases in which our software fails.  This
>> is, of course, not acceptable.
> 
> '>' may appear at the beginning of a line accidently, and we certainly
> want to supply the space character in this case.

First, wouldn't it be very complicated to solve your case?  It's much
easier to assume that ">" in the first position indicate a quoted
line.  I don't see how you programatically can easily separate your
case from e.g.

> According to your analysis
>>42 is a nice number
> you indicate that 42 is a nice number.

but suggestions are welcome. :)

Secondly, RFC 2646 compliant MUAs should never generate this case in
the first place.  Even if you don't generate RFC 2646 (Gnus doesn't)
you could QP encode the character to prevent the ambiguity.  In Gnus
you could put text properties on quote ">" characters to separate them
from any ">" characters the user enters (of course a user should be
able to insert quote ">" characters as well) and when sending the
mail, QP encode the quote ">" characters.  But this becomes quite
complicated for little gain, I think.

Re: "> > >" space removal removes too many spaces?

[Nuutti Kotivuori]

Simon Josefsson wrote:
> Secondly, RFC 2646 compliant MUAs should never generate this case in
> the first place.  Even if you don't generate RFC 2646 (Gnus doesn't)
> you could QP encode the character to prevent the ambiguity.  In Gnus
> you could put text properties on quote ">" characters to separate
> them from any ">" characters the user enters (of course a user
> should be able to insert quote ">" characters as well) and when
> sending the mail, QP encode the quote ">" characters.  But this
> becomes quite complicated for little gain, I think.

I'm partly working on this (RFC2646, generating it) already. But
expect no progress for a little while atleast.

-- Naked

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

Nuutti Kotivuori <nuutti.kotivuori@smarttrust.com> writes:

> I'm partly working on this (RFC2646, generating it) already.

Interesting.  I thought this would be impossible, since Emacs is
line-oriented, not paragraph-oriented.

Re: "> > >" space removal removes too many spaces?

[Nuutti Kotivuori]

Florian Weimer wrote:
> Nuutti Kotivuori <nuutti.kotivuori@smarttrust.com> writes:
> 
>> I'm partly working on this (RFC2646, generating it) already.
> 
> Interesting.  I thought this would be impossible, since Emacs is
> line-oriented, not paragraph-oriented.

Well there's a minor mode 'use-hard-newlines' - and I have other ideas
as well. Also replying on RFC2646 mails should preserve the
format=flowed so that even if we don't generate it ourselves, other
paragraphs would be flowed. But this requires quite a lot of work as
well.

-- Naked

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

Simon Josefsson <jas@extundo.com> writes:

>> '>' may appear at the beginning of a line accidently, and we certainly
>> want to supply the space character in this case.
> 
> First, wouldn't it be very complicated to solve your case?

No, I don't think so.  I think it helps to consider a leading '>' to
be a quote character only if the preceding line is empty (or consists
entirely of whitespace), or if it starts in turn with a '>'.

I've just implemented this logic (at least I think so).  The diff is a
bit more complicated then it needs to be, but if you don't like this
implementation, I'm going to add some backtracking code which should
do somewhat better.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/diff, Size: 1552 bytes --]

Index: message.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/message.el,v
retrieving revision 6.108
diff -u -r6.108 message.el
--- message.el	2001/07/28 16:43:18	6.108
+++ message.el	2001/08/05 14:26:26
@@ -2100,11 +2100,31 @@
 	(indent-rigidly start (mark t) message-indentation-spaces)
       (save-excursion
 	(goto-char start)
-	(while (< (point) (mark t))
-	  (if (looking-at message-cite-prefix-regexp)
-	      (insert message-yank-cited-prefix)
-	    (insert message-yank-prefix))
-	  (forward-line 1))))
+	(let (last-line)
+	  ;; `last-line' describes the contents of the last line
+	  ;; encountered in the loop below. nil means "empty line",
+	  ;; spaces "line consisting entirely of whitespace",
+	  ;; right-angle "line starts with >", quoted "quote character
+	  ;; at the beginning of the line", text "the remaining cases".
+	  (while (< (point) (mark t))
+	    (cond
+	     ((eolp) 
+	      (insert message-yank-prefix)
+	      (setq last-line nil))
+	     ((looking-at ">")
+	      (if (memq last-line '(nil spaces right-angle quoted))
+		  (progn
+		    (insert message-yank-cited-prefix)
+		    (setq last-line 'quoted))
+		(insert message-yank-prefix)
+		(setq last-line 'right-angle)))
+	     ((looking-at "\\s-+$")
+	      (insert message-yank-prefix)
+	      (setq last-line 'spaces))
+	     (t
+	      (insert message-yank-prefix)
+	      (setq last-line 'text)))
+	    (forward-line 1)))))
     (goto-char start)))
 
 (defun message-yank-original (&optional arg)

[-- Attachment #3: Type: text/plain, Size: 756 bytes --]


> I don't see how you programatically can easily separate your case
> from e.g.
> 
>> According to your analysis
>>>42 is a nice number
>> you indicate that 42 is a nice number.
> 
> but suggestions are welcome. :)

Not leaving blank lines before and after quotes is not my preferred
style of writing messages. ;-)

> Secondly, RFC 2646 compliant MUAs should never generate this case in
> the first place.  Even if you don't generate RFC 2646 (Gnus doesn't)
> you could QP encode the character to prevent the ambiguity.

Does RFC 2646 really try to solve the problem this way?  In this
case, the RFC is horribly broken.  Attaching semantic to QP encoding
vs. not-encoding is clearly an extremely bad idea, and it certainly
contradicts the spirit of MIME.

Re: "> > >" space removal removes too many spaces?

[Kai Großjohann]

Oh, boy, what have I done...  I shall try Florian's patch.  Hm.  Do I
also need Simon's patch?  I get confused.  But I will see what
happens.

Florian, do you want me to commit your patch?

kai
-- 
~/.signature: No such file or directory

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Oh, boy, what have I done...  I shall try Florian's patch.  Hm.  Do I
> also need Simon's patch?  I get confused.  But I will see what
> happens.

My patch doesn't depend on Simon's.

> Florian, do you want me to commit your patch?

No, thanks, I'm going to do that myself as soon as someone says it's
the right thing(TM).

Re: "> > >" space removal removes too many spaces?

[Simon Josefsson]

Florian Weimer <fw@deneb.enyo.de> writes:

> Simon Josefsson <jas@extundo.com> writes:
> 
>>> '>' may appear at the beginning of a line accidently, and we certainly
>>> want to supply the space character in this case.
>> 
>> First, wouldn't it be very complicated to solve your case?
> 
> No, I don't think so.  I think it helps to consider a leading '>' to
> be a quote character only if the preceding line is empty (or consists
> entirely of whitespace), or if it starts in turn with a '>'.

Ah.  It's not a complete solution then, but probably covers enough
cases to be worth it.

> I've just implemented this logic (at least I think so).

Looks nice.

>> I don't see how you programatically can easily separate your case
>> from e.g.
>> 
>>> According to your analysis
>>>>42 is a nice number
>>> you indicate that 42 is a nice number.
>> 
>> but suggestions are welcome. :)
> 
> Not leaving blank lines before and after quotes is not my preferred
> style of writing messages. ;-)

Mine neither, but I see it from time to time.

>> Secondly, RFC 2646 compliant MUAs should never generate this case in
>> the first place.  Even if you don't generate RFC 2646 (Gnus doesn't)
>> you could QP encode the character to prevent the ambiguity.
> 
> Does RFC 2646 really try to solve the problem this way?  In this
> case, the RFC is horribly broken.  Attaching semantic to QP encoding
> vs. not-encoding is clearly an extremely bad idea, and it certainly
> contradicts the spirit of MIME.

No, I think that approach is suggested in a couple of places, RFC 2015
being the only one I can find now (they use it to protect "From").
RFC 2646 suggests an alternate approach, space-stuffing the lines.  It
might be cleaner, but it's (slightly) less visually pleasing to
non-RFC 2646 clients.  The idea above looks good on all clients that
support QP.

The semantic of QP encoding vs unencoded is only in the generating
client, the receiver doesn't care.

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

Simon Josefsson <jas@extundo.com> writes:

>> I've just implemented this logic (at least I think so).
> 
> Looks nice.

Fine.  Kai, do you think it's okay to commit it?

>> Does RFC 2646 really try to solve the problem this way?  In this
>> case, the RFC is horribly broken.  Attaching semantic to QP encoding
>> vs. not-encoding is clearly an extremely bad idea, and it certainly
>> contradicts the spirit of MIME.
> 
> No, I think that approach is suggested in a couple of places, RFC 2015
> being the only one I can find now (they use it to protect "From").

This is a detail at the transport layer.  RFC 2015 (and, in
particular, its successor) doesn't fit nicely into the MIME
environment anyway.

> RFC 2646 suggests an alternate approach, space-stuffing the lines.  It
> might be cleaner, but it's (slightly) less visually pleasing to
> non-RFC 2646 clients.  The idea above looks good on all clients that
> support QP.

But your approach is not compatible with MIME, and RFC 2646 is.  I
think the following is required by MIME (implicitly or explicitly, I
don't know): The transfer encoding shall not alter the semantics of a
part, and no information shall be lost during decoding.  This goes
along with the possiblity of one-pass processing, and it greatly
simplifies the implementation of MIME agents.

RFCs 2015 and 2646 adhere to this, your proposal does not, I'm afraid.

Re: "> > >" space removal removes too many spaces?

[Kai Großjohann]

Florian Weimer <fw@deneb.enyo.de> writes:

> Simon Josefsson <jas@extundo.com> writes:
> 
>>> I've just implemented this logic (at least I think so).
>> 
>> Looks nice.
> 
> Fine.  Kai, do you think it's okay to commit it?

Yep :-)

kai
-- 
~/.signature: No such file or directory

Re: "> > >" space removal removes too many spaces?

[Kai Großjohann]

Florian Weimer <fw@deneb.enyo.de> writes:

> Not leaving blank lines before and after quotes is not my preferred
> style of writing messages. ;-)

I wish my boss would change his opinion.  I think for him, empty lines
are a valuable resource not to be wasted thoughtlessly.

kai
-- 
~/.signature: No such file or directory

Re: "> > >" space removal removes too many spaces?

[Paul Jarc]

Simon Josefsson <jas@extundo.com> writes:
> --- message.el.~6.108.~	Sat Jul 28 19:01:54 2001
> +++ message.el	Sun Aug  5 14:43:53 2001
> @@ -2101,7 +2101,7 @@
>        (save-excursion
>  	(goto-char start)
>  	(while (< (point) (mark t))
> -	  (if (looking-at message-cite-prefix-regexp)
> +	  (if (looking-at ">")
>  	      (insert message-yank-cited-prefix)
>  	    (insert message-yank-prefix))
>  	  (forward-line 1))))

How about:
(if (and (string= "> " message-yank-prefix)
         (looking-at ">+ "))
    (insert ">")
  (insert message-yank-prefix))

People who use anything other than "> " for message-yank-prefix are
doing something different anyway, so this doesn't try to deal with
that.


paul

Re: "> > >" space removal removes too many spaces?

[Florian Weimer]

prj@po.cwru.edu (Paul Jarc) writes:

> People who use anything other than "> " for message-yank-prefix are
> doing something different anyway, so this doesn't try to deal with
> that.

It doesn't matter IMHO.  If people don't use '>', omitting the space
doesn't look very nice.