Re: gnus-search & imap: always "CHARSET UTF-8" when literal+ is supported

[Eric Abrahamsen <eric@ericabrahamsen.net>]

David Edmondson <dme@dme.org> writes:

> On Friday, 2021-10-22 at 10:47:42 -07, Eric Abrahamsen wrote:
>
>> David Edmondson <dme@dme.org> writes:
>>
>>> Using current emacs git head, talking to outlook.office365.com over
>>> IMAP.
>>>
>>> Attempts to use gnus-search always fail with the server reporting:
>>>
>>> (("NO" ("BADCHARSET" "(US-ASCII)") "The" "specified" "charset" "is" "not" "supported."))
>>>
>>> Looking at gnus-search.el, `gnus-search-imap-search-command' always
>>> sends CHARSET UTF-8 if the server supports literal+ (which this one
>>> does). Sending US-ASCII (or no charset at all) causes the server to
>>> return the required results in simple test cases.
>>>
>>> Is there a way to determine whether a server supports UTF-8 in searches,
>>> and adjust the command sent accordingly? If not, could the use of UTF-8
>>> be controlled with (yet another!) variable?
>>
>> A bit of internet research seems to indicate that Exchange can't handle
>> UTF-8 encoded search strings, and also there's no way to test that in
>> advance apart from simply seeing if it errors. Awesome!
>
> That's also the impression I gained.
>
>> I think what this means is that it's impossible to search for non-ascii
>> text on an Exchange server (can that be true?!). If that's true, then
>> the imap search command should be using the presence of a multibyte
>> string as the test for whether to use CHARSET UTF-8 or not. You're not
>> going to be able to search for a multibyte string, anyway.
>>
>> Would you try eval'ling the below, and tell me if it works okay when
>> searching for a string with no non-ascii characters in it?
>
> This works in a few simple tests, yes. Thanks!

Great, I'll put this change in.

>> Also, when you do get the error message above, how does that present to
>> the user? Did you have to go digging to find it?
>
> I had to dig. The observed behaviour is that no messages match the
> search.

I'm hoping to make some changes to Gnus error reporting that should make
this less difficult. The user should definitely see the difference
between "no results" and an actual error.

Thanks for the report.