Gnus development mailing list
 help / color / mirror / Atom feed
* problem with non-ascii group-names
@ 2021-06-03 10:29 Peter Münster
  2021-06-03 16:00 ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-06-03 10:29 UTC (permalink / raw)
  To: ding

Hi,

Very often, when saving a new message to an nnml-group, that has
accented characters (for example "stéphane"), and then entering that
group, the summary buffer does not show this new message.

Only after several (gnus-group-get-new-news) and switching back and
forth to other groups, after a while the new message shows up.

How could this be solved (and/or debugged) please?

TIA for any hints,
-- 
           Peter



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 10:29 problem with non-ascii group-names Peter Münster
@ 2021-06-03 16:00 ` Eric Abrahamsen
  2021-06-03 17:21   ` Peter Münster
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 16:00 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Peter Münster <pm@a16n.net> writes:

> Hi,
>
> Very often, when saving a new message to an nnml-group, that has
> accented characters (for example "stéphane"), and then entering that
> group, the summary buffer does not show this new message.
>
> Only after several (gnus-group-get-new-news) and switching back and
> forth to other groups, after a while the new message shows up.
>
> How could this be solved (and/or debugged) please?
>
> TIA for any hints,

Can you tell us your Gnus version? Things have changed fairly
dramatically in this area over the past couple of years. Your agent and
cache settings will be relevant, too.

Thanks,
Eric


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 16:00 ` Eric Abrahamsen
@ 2021-06-03 17:21   ` Peter Münster
  2021-06-03 18:31     ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-06-03 17:21 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> Can you tell us your Gnus version?

It's in the headers:
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)


> Your agent and cache settings will be relevant, too.

Concerning agent and cache, this is the only thing, that I've
customized:

--8<---------------cut here---------------start------------->8---
nnmail-message-id-cache-file is a variable defined in ‘nnmail.el’.
Its value is "~peter/.emacs.d/nnmail-cache"
Original value was "~/.nnmail-cache"
The file name of the nnmail Message-ID cache.
--8<---------------cut here---------------end--------------->8---

-- 
           Peter



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 17:21   ` Peter Münster
@ 2021-06-03 18:31     ` Eric Abrahamsen
  2021-06-03 18:56       ` Peter Münster
  2021-09-14 20:12       ` Peter Münster
  0 siblings, 2 replies; 15+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 18:31 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Peter Münster <pm@a16n.net> writes:

> On Thu, Jun 03 2021, Eric Abrahamsen wrote:
>
>> Can you tell us your Gnus version?
>
> It's in the headers:
> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)

I don't think there will be a whole lot you can do -- in mid-2019 I made
some fairly large changes to Gnus to use decoded group names everywhere,
and in the process fixed several bugs related to non-ascii group name. I
don't think any of that is available in Emacs 27. It's possible if we
found the exact bug that the fix could be added to 27, but I don't even
think there will be any more releases of 27 anyway (?, not sure). Your
best bet may be to wait until Emacs 28 is released.

In the mean time, can you look at the active file for this server and
see how the group name is encoded there?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 18:31     ` Eric Abrahamsen
@ 2021-06-03 18:56       ` Peter Münster
  2021-06-03 19:55         ` Eric Abrahamsen
  2021-09-14 20:12       ` Peter Münster
  1 sibling, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-06-03 18:56 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> I don't think any of that is available in Emacs 27.

Ok, then I'll wait for version 28. Do you know, when it's going to be
released?


> In the mean time, can you look at the active file for this server and
> see how the group name is encoded there?

It's UTF-8. And now I notice, that there are several lines for the same
group, if its name has accented characters. For example:

--8<---------------cut here---------------start------------->8---
stéphane 277 1 y
stéphane 280 1 y
stéphane 281 1 y
stéphane 282 1 y
stéphane 284 1 y
stéphane 285 1 y
--8<---------------cut here---------------end--------------->8---

-- 
           Peter



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 18:56       ` Peter Münster
@ 2021-06-03 19:55         ` Eric Abrahamsen
  2021-06-03 20:36           ` Peter Münster
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 19:55 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Peter Münster <pm@a16n.net> writes:

> On Thu, Jun 03 2021, Eric Abrahamsen wrote:
>
>> I don't think any of that is available in Emacs 27.
>
> Ok, then I'll wait for version 28. Do you know, when it's going to be
> released?

I sure don't! Much less do I know when the various platforms will update
their packages. I sort of get the sense that it might be released within
the next year?

>> In the mean time, can you look at the active file for this server and
>> see how the group name is encoded there?
>
> It's UTF-8. And now I notice, that there are several lines for the same
> group, if its name has accented characters. For example:
>
> stéphane 277 1 y
> stéphane 280 1 y
> stéphane 281 1 y
> stéphane 282 1 y
> stéphane 284 1 y
> stéphane 285 1 y

Well that's interesting! I wonder how that happened. That's definitely a
clue to what's going on, though again I'm not optimistic about getting a
fix to you. If you just delete all the extra lines (leaving the one with
the highest active number) does it fix the behavior you're seeing?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 19:55         ` Eric Abrahamsen
@ 2021-06-03 20:36           ` Peter Münster
  2021-06-03 21:02             ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-06-03 20:36 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> If you just delete all the extra lines (leaving the one with the
> highest active number) does it fix the behavior you're seeing?

I've tried with one example: yes, it seems to fix it.

-- 
           Peter



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 20:36           ` Peter Münster
@ 2021-06-03 21:02             ` Eric Abrahamsen
  0 siblings, 0 replies; 15+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 21:02 UTC (permalink / raw)
  To: ding

Peter Münster <pm@a16n.net> writes:

> On Thu, Jun 03 2021, Eric Abrahamsen wrote:
>
>> If you just delete all the extra lines (leaving the one with the
>> highest active number) does it fix the behavior you're seeing?
>
> I've tried with one example: yes, it seems to fix it.

Which doesn't mean that the problem won't come back again, but at least
narrows the scope of the bug. Thanks for confirming that.

I still think it's unlikely that I'll be able to get a fix to you, but
I'll look into this over the next few weeks, and who knows. In the
meantime, thanks for the bug report :)



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-06-03 18:31     ` Eric Abrahamsen
  2021-06-03 18:56       ` Peter Münster
@ 2021-09-14 20:12       ` Peter Münster
  2021-09-14 20:20         ` Peter Münster
  1 sibling, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-09-14 20:12 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> in mid-2019 I made some fairly large changes to Gnus to use decoded
> group names everywhere, and in the process fixed several bugs related
> to non-ascii group name.

Hello Eric,

Now I use Emacs from git-master. Still the same problem.

Here some details for the group "céline":

- There is this line in ~/Mail/active: "céline 153 1 y"
- Then I reply to a message in this group with "Gcc: céline"
- This appears in the *Messages*:
  Sending...
  Sending via mail...
  Wrote /home/peter/Mail/c\303\251line/154  (2 non-printable characters)
  Sending...done
- When entering the group "céline", the new message is hidden.
- There are now 2 lines in the active file:
  céline 154 1 y
  céline 153 1 y

I guess, that this should be reproducible with any nnml group with
accented characters in the group name.

TIA for any help,
-- 
           Peter

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-14 20:12       ` Peter Münster
@ 2021-09-14 20:20         ` Peter Münster
  2021-09-20 18:09           ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-09-14 20:20 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

And here a related problem:

When using "S D r" (gnus-summary-resend-message), a local copy of the
message is written to the group.

-- 
           Peter

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-14 20:20         ` Peter Münster
@ 2021-09-20 18:09           ` Eric Abrahamsen
  2021-09-20 18:14             ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Abrahamsen @ 2021-09-20 18:09 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Peter Münster <pm@a16n.net> writes:

> And here a related problem:
>
> When using "S D r" (gnus-summary-resend-message), a local copy of the
> message is written to the group.

I'm guessing that there's something particular to archive group writing,
since this doesn't happen with regular nnml groups with non-ascii names.
Are you using "Céline" as your archive group, or your Gcc header for
this particular set of messages? And you're in this group when you hit
"S D r" as above, right?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-20 18:09           ` Eric Abrahamsen
@ 2021-09-20 18:14             ` Eric Abrahamsen
  2021-09-20 18:53               ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Abrahamsen @ 2021-09-20 18:14 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Peter Münster <pm@a16n.net> writes:
>
>> And here a related problem:
>>
>> When using "S D r" (gnus-summary-resend-message), a local copy of the
>> message is written to the group.
>
> I'm guessing that there's something particular to archive group writing,
> since this doesn't happen with regular nnml groups with non-ascii names.
> Are you using "Céline" as your archive group, or your Gcc header for
> this particular set of messages? And you're in this group when you hit
> "S D r" as above, right?

Okay, I was able to reproduce this problem when archiving messages to a
group with a non-ascii name. Give me a few days and I ought have this
fixed.

Thanks,
Eric


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-20 18:14             ` Eric Abrahamsen
@ 2021-09-20 18:53               ` Eric Abrahamsen
  2021-09-20 21:13                 ` Peter Münster
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Abrahamsen @ 2021-09-20 18:53 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Peter Münster <pm@a16n.net> writes:
>>
>>> And here a related problem:
>>>
>>> When using "S D r" (gnus-summary-resend-message), a local copy of the
>>> message is written to the group.
>>
>> I'm guessing that there's something particular to archive group writing,
>> since this doesn't happen with regular nnml groups with non-ascii names.
>> Are you using "Céline" as your archive group, or your Gcc header for
>> this particular set of messages? And you're in this group when you hit
>> "S D r" as above, right?
>
> Okay, I was able to reproduce this problem when archiving messages to a
> group with a non-ascii name. Give me a few days and I ought have this
> fixed.

That was much easier than I thought, I'd simply missed a group-name
encoding call when I was yanking all this stuff out a year or so ago.

If you'd like, please test the attached diff to make sure it works,
though I'm fairly sure this is it.

Eric


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dont-encode-gcc-group-name.diff --]
[-- Type: text/x-patch, Size: 649 bytes --]

diff --git a/lisp/gnus/gnus-msg.el b/lisp/gnus/gnus-msg.el
index 66501ef7d5..0e1ba70d1e 100644
--- a/lisp/gnus/gnus-msg.el
+++ b/lisp/gnus/gnus-msg.el
@@ -1584,10 +1584,7 @@ gnus-inews-do-gcc
 			(message-tokenize-header gcc " ,\n\t")))
 	  ;; Copy the article over to some group(s).
 	  (while (setq group (pop groups))
-	    (setq method (gnus-inews-group-method group)
-		  group (encode-coding-string
-			 group
-			 (gnus-group-name-charset method group)))
+	    (setq method (gnus-inews-group-method group))
 	    (unless (gnus-check-server method)
 	      (error "Can't open server %s" (if (stringp method) method
 					      (car method))))

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-20 18:53               ` Eric Abrahamsen
@ 2021-09-20 21:13                 ` Peter Münster
  2021-09-20 22:34                   ` Eric Abrahamsen
  0 siblings, 1 reply; 15+ messages in thread
From: Peter Münster @ 2021-09-20 21:13 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

On Mon, Sep 20 2021, Eric Abrahamsen wrote:

> -	    (setq method (gnus-inews-group-method group)
> -		  group (encode-coding-string
> -			 group
> -			 (gnus-group-name-charset method group)))
> +	    (setq method (gnus-inews-group-method group))

Hello Eric,

Thank you! This fixes apparently the first issue. But not the second
with the resending. Here the *Messages*:

Resending message to XXX...
Sending via mail...
Wrote /home/peter/Mail/c\303\251line/123
Resending message to XXX...done

Kind regards,
-- 
           Peter

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: problem with non-ascii group-names
  2021-09-20 21:13                 ` Peter Münster
@ 2021-09-20 22:34                   ` Eric Abrahamsen
  0 siblings, 0 replies; 15+ messages in thread
From: Eric Abrahamsen @ 2021-09-20 22:34 UTC (permalink / raw)
  To: Peter Münster; +Cc: ding

Peter Münster <pm@a16n.net> writes:

> On Mon, Sep 20 2021, Eric Abrahamsen wrote:
>
>> -	    (setq method (gnus-inews-group-method group)
>> -		  group (encode-coding-string
>> -			 group
>> -			 (gnus-group-name-charset method group)))
>> +	    (setq method (gnus-inews-group-method group))
>
> Hello Eric,
>
> Thank you! This fixes apparently the first issue. But not the second
> with the resending. Here the *Messages*:
>
> Resending message to XXX...
> Sending via mail...
> Wrote /home/peter/Mail/c\303\251line/123

Yes! There were two more instances of group name encoding in that file.
I've removed both of them, will test a bit, and then push this fix later
today.

Thanks for reporting,
Eric


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-09-20 22:34 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-03 10:29 problem with non-ascii group-names Peter Münster
2021-06-03 16:00 ` Eric Abrahamsen
2021-06-03 17:21   ` Peter Münster
2021-06-03 18:31     ` Eric Abrahamsen
2021-06-03 18:56       ` Peter Münster
2021-06-03 19:55         ` Eric Abrahamsen
2021-06-03 20:36           ` Peter Münster
2021-06-03 21:02             ` Eric Abrahamsen
2021-09-14 20:12       ` Peter Münster
2021-09-14 20:20         ` Peter Münster
2021-09-20 18:09           ` Eric Abrahamsen
2021-09-20 18:14             ` Eric Abrahamsen
2021-09-20 18:53               ` Eric Abrahamsen
2021-09-20 21:13                 ` Peter Münster
2021-09-20 22:34                   ` Eric Abrahamsen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).