Re: More liberal MIME decoding (unencoded question marks in encoded words)

[Katsumi Yamaoka <yamaoka@jpl.org>]

>>>>> Reiner Steib wrote:

>>> On Mon, Nov 26 2007, Katsumi Yamaoka wrote:
>>>> Would we be able to make complete test cases?

> I don't think providing *complete* test cases is possible.  But
> regression tests would be very nice to have.

> How about defining a variable containing list of (decoded . encoded)
> pairs, decode/encode the strings and put the results in a buffer (or
> file) and compare them?

That's a good idea.  Since rfc2047.el does some special treatments,
it will be a good means not only for the test but also for
explaining them.  However, though I've faced various instances
causing troubles while improving rfc2047.el, to my regret I lost
all of them.  I'll scan my articles archived and functions when
I have time in the future.

>>> +\\|Q\\?\\(?:\\?+[ -<>@-~]\\)?\\(?:[ ->@-~]+\\?+[ -<>@-~]\\)*[ ->@-~]*\\?*\
>>> +\\)\\?="))

[...]

> Could you please add such explanations as comments in `rfc2047.el'?

Done.

[...]

> I see that we already have quite a few of these variables: At least
> `rfc2047-allow-irregular-q-encoded-words',
> `rfc2047-allow-incomplete-encoded-text' and finally
> `gnus-article-loose-mime'.

> How about deriving the defaults from a single variable
> `rfc2047-allow-loose-mime' (or `rfc2047-loose-mime-decoding'):

> (defcustom rfc2047-allow-loose-mime t
>   "Allow loose MIME decoding.  ...")
> (devar rfc2047-allow-irregular-q-encoded-words rfc2047-allow-loose-mime
>   ...)
> (devar rfc2047-allow-incomplete-encoded-text rfc2047-allow-loose-mime
>   ...)
> (defcustom gnus-article-loose-mime rfc2047-allow-loose-mime
>   "...
> See also `rfc2047-allow-loose-mime'.")

> Or even, if we don't need such a fine tuning, use
> `rfc2047-allow-loose-mime' directly.

I agree with integrating those options into the only
`rfc2047-allow-loose-mime'.  However, for `gnus-article-loose-mime',
I don't think making it refer to the rfc2047- variable is proper
because rfc2047 is not necessarily a representative for the MIME
features.  Needless to say, making rfc2047.el refer to the value
of `gnus-article-loose-mime' is bad, too.  Also
(gnus|mm|gmm)-utils.el seem not to be a good place to define MIME-
related things.  Hmm, I have no good idea.

>> Now there are two regexps; one is `rfc2047-encoded-word-regexp' for
>> strict decoding, the other is `rfc2047-encoded-word-regexp-loose'.

> You you explain the purpose of using `eval-and-compile'?

Because those values are the constants and the values derived
from them are hard coded in `rfc2047-decode-region'.  Without
`eval-and-compile' Emacs complains as:

Error: Symbol's value as variable is void:
 rfc2047-encoded-word-regexp-loose

There might be room for considering whether they should be
constants or not, though.

> The defconst needs to be before `rfc2047-encodable-p':

> In rfc2047-encodable-p:
> rfc2047.el:302:37:Warning: reference to free variable
>     `rfc2047-encoded-word-regexp'

Fixed.  I didn't notice it until I tried compiling rfc2047.el
individually.  Thanks.