Re: More liberal MIME decoding (unencoded question marks in encoded words)

[Reiner Steib <reinersteib+gmane@imap.cc>]

On Tue, Nov 27 2007, Katsumi Yamaoka wrote:

>>>>>> Reiner Steib wrote:
>> On Mon, Nov 26 2007, Katsumi Yamaoka wrote:
>>> Would we be able to make complete test cases?

I don't think providing *complete* test cases is possible.  But
regression tests would be very nice to have.

How about defining a variable containing list of (decoded . encoded)
pairs, decode/encode the strings and put the results in a buffer (or
file) and compare them?

[...]
>> +\\|Q\\?\\(?:\\?+[ -<>@-~]\\)?\\(?:[ ->@-~]+\\?+[ -<>@-~]\\)*[ ->@-~]*\\?*\
>> +\\)\\?="))
>
> This pattern is similar to:
>
> "Q\\?\\(\\?+[^\n=?]\\)?\\([^\n?]+\\?+[^\n=?]\\)*[^\n?]*\\?*"
>      <--------1-------><----------2,3----------><--4--><-5->
>
> 1. After "Q?", allow "?"s that follow a character other than "=".
> 2. Allow "=" after "Q?"; it isn't regarded as the terminator.
> 3. In the middle of an encoded word, allow "?"s that follow a
>    character other than "=".
> 4. Allow any characters other than "?" in the middle of an
>    encoded word.
> 5. At the end, allow "?"s.

Could you please add such explanations as comments in `rfc2047.el'?

>> And we probably should have an option to toggle strict/loose
>> decoding.
>
> I've introduced the `rfc2047-allow-irregular-q-encoded-words'
> option.  I wish that it is tested widely, so I've set the default
> value to t.  But it might have to be nil when it is imported into
> the stable branch.  

I see that we already have quite a few of these variables: At least
`rfc2047-allow-irregular-q-encoded-words',
`rfc2047-allow-incomplete-encoded-text' and finally
`gnus-article-loose-mime'.

How about deriving the defaults from a single variable
`rfc2047-allow-loose-mime' (or `rfc2047-loose-mime-decoding'):

(defcustom rfc2047-allow-loose-mime t
  "Allow loose MIME decoding.  ...")
(devar rfc2047-allow-irregular-q-encoded-words rfc2047-allow-loose-mime
  ...)
(devar rfc2047-allow-incomplete-encoded-text rfc2047-allow-loose-mime
  ...)
(defcustom gnus-article-loose-mime rfc2047-allow-loose-mime
  "...
See also `rfc2047-allow-loose-mime'.")

Or even, if we don't need such a fine tuning, use
`rfc2047-allow-loose-mime' directly.

> Now there are two regexps; one is `rfc2047-encoded-word-regexp' for
> strict decoding, the other is `rfc2047-encoded-word-regexp-loose'.

You you explain the purpose of using `eval-and-compile'?

> I agree with you.  I've made `rfc2047-encodable-p' use
> `rfc2047-encoded-word-regexp' instead of "=?".

The defconst needs to be before `rfc2047-encodable-p':

In rfc2047-encodable-p:
rfc2047.el:302:37:Warning: reference to free variable
    `rfc2047-encoded-word-regexp'

> It will be hard to be found out even if this change causes another
> trouble, though.

Yes.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/