nnml groups showing up with '*' at startup

nnml groups showing up with '*' at startup

[Adam Sjøgren]

After a change recently, every time I start Gnus, all my
nnml+archive:-groups show up with '*' in front of them, along with 3 of
my (many more) nnml:-groups.

Usually Gnus knows that all the articles in those groups are read, and
the groups aren't shown at startup.

Bisecting finds this commit to be where this started:

  0814c236bf5b827fd046819bbd6ad591986c9caf is the first bad commit
  commit 0814c236bf5b827fd046819bbd6ad591986c9caf
  Author: Michael Sperber <mike@xemacs.org>
  Date:   Mon Oct 19 14:26:00 2015 +0000

      Bind `rmail-dont-reply-to-names'

      * message.el (message-get-reply-headers): In addition to
      `mail-dont-reply-to-names', bind `rmail-dont-reply-to-names', which is
      used in XEmacs.

    - http://git.gnus.org/cgit/gnus.git/commit/?id=0814c236bf5b827fd046819bbd6ad591986c9caf

I'm not quite sure what the changes in nnml.el in that commit achieves,
or how they are related to rmail-dont-reply-to-names?


  Best regards,

    Adam

-- 
 "God must've been punting angels left and right."            Adam Sjøgren
                                                         asjo@koldfront.dk

Re: nnml groups showing up with '*' at startup

[Peter Münster]

On Sun, Nov 22 2015, Adam Sjøgren wrote:

> Bisecting finds this commit to be where this started:
>
>   0814c236bf5b827fd046819bbd6ad591986c9caf is the first bad commit
>   commit 0814c236bf5b827fd046819bbd6ad591986c9caf
>   Author: Michael Sperber <mike@xemacs.org>
>   Date:   Mon Oct 19 14:26:00 2015 +0000

Hi Adam,

Does your message get through to Michael?

This is, what I got:

"<mike@xemacs.org>: delivery to host xemacs.org[failed] timed out "

(see subject: Re: tab-completion with gnus-group-jump-to-group)

-- 
           Peter

Re: nnml groups showing up with '*' at startup

[Adam Sjøgren]

Peter writes:

> Does your message get through to Michael?
>
> This is, what I got:
>
> "<mike@xemacs.org>: delivery to host xemacs.org[failed] timed out "

I guess not:

  (host mail.xemacs.org[207.172.156.133] said: 452 4.4.5 Insufficient
  disk space; try again later (in reply to MAIL FROM command))


  Best regards,

    Adam

-- 
 "Lef ma nine imma Jeep"                                      Adam Sjøgren
                                                         asjo@koldfront.dk

Re: nnml groups showing up with '*' at startup

[Michael Sperber]

asjo@koldfront.dk (Adam Sjøgren) writes:

> After a change recently, every time I start Gnus, all my
> nnml+archive:-groups show up with '*' in front of them, along with 3 of
> my (many more) nnml:-groups.
>
> Usually Gnus knows that all the articles in those groups are read, and
> the groups aren't shown at startup.
>
> Bisecting finds this commit to be where this started:
>
>   0814c236bf5b827fd046819bbd6ad591986c9caf is the first bad commit
>   commit 0814c236bf5b827fd046819bbd6ad591986c9caf
>   Author: Michael Sperber <mike@xemacs.org>
>   Date:   Mon Oct 19 14:26:00 2015 +0000
>
>       Bind `rmail-dont-reply-to-names'
>
>       * message.el (message-get-reply-headers): In addition to
>       `mail-dont-reply-to-names', bind `rmail-dont-reply-to-names', which is
>       used in XEmacs.
>
>     - http://git.gnus.org/cgit/gnus.git/commit/?id=0814c236bf5b827fd046819bbd6ad591986c9caf
>
> I'm not quite sure what the changes in nnml.el in that commit achieves,
> or how they are related to rmail-dont-reply-to-names?

Two separate changes got conflated on committing.  (Probably my bad,
sorry about that.)

The other change - the one probably responsible for this problem -
unbreaks the functionality of the per-group mail source.

If you're not using this feature, could you pinpoint the hunk (out of 2)
that's causing it?  (Easily done by first removing the
`nnml-retrieve-groups` defun.)  That'd help me find out where it's
coming from.

-- 
Regards,
Mike

Re: nnml groups showing up with '*' at startup

[Adam Sjøgren]

Michael writes:

> The other change - the one probably responsible for this problem -
> unbreaks the functionality of the per-group mail source.

I don't know what that means. Likely that I am not using it :-)

> If you're not using this feature, could you pinpoint the hunk (out of 2)
> that's causing it?  (Easily done by first removing the
> `nnml-retrieve-groups` defun.)  That'd help me find out where it's
> coming from.

Yup, this makes the symptom disappear:

---
 lisp/nnml.el | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/lisp/nnml.el b/lisp/nnml.el
index 33eae1c..a2947c0 100644
--- a/lisp/nnml.el
+++ b/lisp/nnml.el
@@ -268,23 +268,6 @@ non-nil.")
 			   (max (1+ (- (cdr active) (car active))) 0)
 			   (car active) (cdr active) group)))))))
 
-(deffoo nnml-retrieve-groups (groups &optional server)
-  (when nnml-get-new-mail
-    (if (nnmail-get-new-mail-per-group)
-	(dolist (group groups)
-	  (nnml-request-scan group server))
-      (nnml-request-scan nil server)))
-  (with-current-buffer nntp-server-buffer
-    (erase-buffer)
-    (dolist (group groups)
-      (let* ((entry (assoc group nnml-group-alist))
-	     (active (nth 1 entry)))
-	(if (consp active)
-	    (insert (format "211 %d %d %d %s\n"
-			    (max (1+ (- (cdr active) (car active))) 0)
-			    (car active) (cdr active) group))))))
-  'group)
-
 (deffoo nnml-request-scan (&optional group server)
   (setq nnml-article-file-alist nil)
   (nnml-possibly-change-directory group server)
-- 
2.6.2

-- 
 "It's not denial. I'm just very selective about the          Adam Sjøgren
  reality I accept."                                     asjo@koldfront.dk

Re: nnml groups showing up with '*' at startup

[Katsumi Yamaoka]

On Sun, 22 Nov 2015 13:22:36 +0100, Adam Sjøgren wrote:
> After a change recently, every time I start Gnus, all my
> nnml+archive:-groups show up with '*' in front of them, along with 3 of
> my (many more) nnml:-groups.

I found that my "nnml:テスト" group doesn't show up.  That is
the group I used to use for testing non-ASCII group names.
Only the `L' command is able to show it up in the group buffer,
but it indicates `*' for the number of articles even if its
group level is 1.

On Mon, 23 Nov 2015 19:02:15 +0100, Adam Sjøgren wrote:
> --- a/lisp/nnml.el
> +++ b/lisp/nnml.el
> @@ -268,23 +268,6 @@ non-nil.")
>  			   (max (1+ (- (cdr active) (car active))) 0)
>  			   (car active) (cdr active) group)))))))

> -(deffoo nnml-retrieve-groups (groups &optional server)
> -  (when nnml-get-new-mail
> -    (if (nnmail-get-new-mail-per-group)
> -	(dolist (group groups)
> -	  (nnml-request-scan group server))
> -      (nnml-request-scan nil server)))
> -  (with-current-buffer nntp-server-buffer
> -    (erase-buffer)
> -    (dolist (group groups)
> -      (let* ((entry (assoc group nnml-group-alist))
> -	     (active (nth 1 entry)))
> -	(if (consp active)
> -	    (insert (format "211 %d %d %d %s\n"
> -			    (max (1+ (- (cdr active) (car active))) 0)
> -			    (car active) (cdr active) group))))))
> -  'group)

Removing this made Gnus work as before, too.  So far I don't
know why/how it does wrong, but I think we should remove it if
it is nonessential and nonurgent.  The function has already been
in the emacs-25 branch (for releasing Emacs 25.1).

WDYT Michael?

Re: nnml groups showing up with '*' at startup

[Katsumi Yamaoka]

On Tue, 24 Nov 2015 11:40:59 +0900, Katsumi Yamaoka wrote:
>> --- a/lisp/nnml.el
>> +++ b/lisp/nnml.el
>> -(deffoo nnml-retrieve-groups (groups &optional server)
>> -  (when nnml-get-new-mail
>> -    (if (nnmail-get-new-mail-per-group)
[...]

> Removing this made Gnus work as before, too.  So far I don't
> know why/how it does wrong,

As for at least a non-ASCII group name like "nnml:テスト" I found
the cause of the problem.

・Gnus uses an encoded group name of the unibyte version to access
  that group.  By default utf-8 is used for encoding, i.e.:
  "nnml:テスト" -> "nnml:\343\203\206\343\202\271\343\203\210"
・`nnml-retrieve-groups' inserts an encoded group name to the
  `nntp-server-buffer', in which the multibyteness is turned on.
・`gnus-groups-to-gnus-format' reads it using `read'.  Because
  the multibyteness is turned on, the result will be a symbol of
  the multibyte version, and the active info will be created in
  `gnus-active-hashtb' with a multibyte group name.  That cannot
  be accessed by Gnus.

A quick hack is attached below, although I don't want to install
it because it will probably damage Gnus' speed.  Moreover, it
won't be a fix to Peter's and Adam's problems; both don't seem to
mention groups of non-ASCII names.

> but I think we should remove it if
> it is nonessential and nonurgent.  The function has already been
> in the emacs-25 branch (for releasing Emacs 25.1).

> WDYT Michael?

--- gnus-start.el~	2015-10-01 01:52:23.531908800 +0000
+++ gnus-start.el	2015-11-24 10:38:38.449503300 +0000
@@ -2256,8 +2256,10 @@
 		    (skip-chars-forward " \t")
 		    (insert prefix)
 		    (goto-char opoint)
-		    (set (let ((obarray hashtb)) (read cur))
-			 (cons min max)))
+		    (gnus-sethash
+		     (string-as-unibyte (symbol-name (read cur)))
+		     (cons min max)
+		     hashtb))
 		(error (and group (symbolp group) (set group nil))))
 	      (forward-line 1)))
 	(let (min max group)

Re: nnml groups showing up with '*' at startup

[Peter Münster]

On Tue, Nov 24 2015, Katsumi Yamaoka wrote:

> A quick hack is attached below, although I don't want to install
> it because it will probably damage Gnus' speed.  Moreover, it
> won't be a fix to Peter's and Adam's problems; both don't seem to
> mention groups of non-ASCII names.

My problem is a very minor one: after a while, the tab-completion works,
it seems, that there is a list of group-names used by
gnus-group-completing-read that is not complete at start-up time, but
later groups are added to this list. After some uptime, it works...

But I agree: sometimes, there are issues with non-ASCII names (but I
don't remember what exactly...)

-- 
           Peter

Re: nnml groups showing up with '*' at startup

[Katsumi Yamaoka]

On Tue, 24 Nov 2015 21:48:39 +0100, Peter Münster wrote:
> On Tue, Nov 24 2015, Katsumi Yamaoka wrote:

>> A quick hack is attached below, although I don't want to install
>> it because it will probably damage Gnus' speed.  Moreover, it
>> won't be a fix to Peter's and Adam's problems; both don't seem to
>> mention groups of non-ASCII names.

> My problem is a very minor one: after a while, the tab-completion works,
> it seems, that there is a list of group-names used by
> gnus-group-completing-read that is not complete at start-up time, but
> later groups are added to this list. After some uptime, it works...

Probably the group names list will be updated after a while by
some means other than `nnml-retrieve-groups'.  It's no more than
my vaguely guess, though.

OTOH, I verified that the `mail-source' group parameter[1] works
(as Michael intended, probably) and the existence of
`nnml-retrieve-groups' is unrelated to it to work.  So, I'll
delete `nnml-retrieve-groups' later[2].

[1] The `(group)' source is required in `mail-sources'.
    cf. (info "(gnus)Mail Source Specifiers")
[2] I want to add the urls of these threads to the commit log
    for the future reference.  However, the Gmane web interface
    seems down now.

Re: nnml groups showing up with '*' at startup

[Adam Sjøgren]

Katsumi writes:

> [2] I want to add the urls of these threads to the commit log
>     for the future reference.  However, the Gmane web interface
>     seems down now.

You can get these from the Archived-At:-headers of the articles over
NNTP, e.g.:

  Archived-At: <http://permalink.gmane.org/gmane.emacs.gnus.general/86339>

(I usually change "permalink" to "article", because I like that look
better.)


  Best regards,

    Adam

-- 
 "It's like somebody with obsessive-compulsive                Adam Sjøgren
  disorder constantly rearranging the names to fit       asjo@koldfront.dk
  into some sort of perfect ontology they have in
  their heads."

Re: nnml groups showing up with '*' at startup

[Katsumi Yamaoka]

On Wed, 25 Nov 2015 09:02:13 +0100, Adam Sjøgren wrote:
> Katsumi writes:

>> [2] I want to add the urls of these threads to the commit log
>>     for the future reference.  However, the Gmane web interface
>>     seems down now.

> You can get these from the Archived-At:-headers of the articles over
> NNTP, e.g.:

>   Archived-At: <http://permalink.gmane.org/gmane.emacs.gnus.general/86339>

> (I usually change "permalink" to "article", because I like that look
> better.)

Thanks.  What I'm about to add to the commit log are
<http://thread.gmane.org/gmane.emacs.gnus.general/86308> and
<http://thread.gmane.org/gmane.emacs.gnus.general/86321>.
Maybe these are correct, but I want to make assurance double sure
because a git log is very hard (to me) to rewrite. ;)

Btw I have:
,----[ C-h f w3m-gmane-url-at-point RET ]
| w3m-gmane-url-at-point is a compiled Lisp function in ‘w3m.el’.
|
| (w3m-gmane-url-at-point)
|
| Return a url that indicates the thread page in Gmane.
| This function works only when the cursor stays in the References
| header or the Message-ID header, otherwise returns nil.
| [...]
`----

Re: nnml groups showing up with '*' at startup

[Adam Sjøgren]

Katsumi writes:

> <http://thread.gmane.org/gmane.emacs.gnus.general/86308> and
> <http://thread.gmane.org/gmane.emacs.gnus.general/86321>.
> Maybe these are correct, but I want to make assurance double sure
> because a git log is very hard (to me) to rewrite. ;)

Ah, yes, that makes sense. I guess you'll have to wait until Lars gets
around to fixing the web-interface, again...

> Btw I have:
> ,----[ C-h f w3m-gmane-url-at-point RET ]

Cool!


  Best regards,

    Adam

-- 
 "Even if you don't have all the things you want, be          Adam Sjøgren
  grateful for the things you don't have that you        asjo@koldfront.dk
  don't want."

Re: nnml groups showing up with '*' at startup

[Michael Sperber]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Removing this made Gnus work as before, too.  So far I don't
> know why/how it does wrong, but I think we should remove it if
> it is nonessential and nonurgent.  The function has already been
> in the emacs-25 branch (for releasing Emacs 25.1).

Yes, you should.  I couldn't make it work without that definition when I
originally made the change (> 1 year ago), but it seems superfluous
now.  Sorry for causing problems!

-- 
Regards,
Mike

groups with non-ASCII names (was: nnml groups showing up with '*' at startup)

[Peter Münster]

On Tue, Nov 24 2015, Peter Münster wrote:

> But I agree: sometimes, there are issues with non-ASCII names (but I
> don't remember what exactly...)

Now I do: tab-completion does not work in message-mode on the "Gcc:"
line, when the group name has non-ASCII characters.

What could be done, to get tab-completion for such groups please?

TIA for any hints,
-- 
           Peter

Re: groups with non-ASCII names

[Katsumi Yamaoka]

On Wed, 02 Dec 2015 23:02:51 +0100, Peter Münster wrote:
> Now I do: tab-completion does not work in message-mode on the "Gcc:"
> line, when the group name has non-ASCII characters.

> What could be done, to get tab-completion for such groups please?

Only the non-ASCII group doesn't appear in the candidates list
for the completion, whereas that group appears in the Group buffer,
doesn't it?  If so, unfortunately I have no idea how it happens.
Does this form return a non-nil value?

(intern-soft
 (encode-coding-string "nnml:テスト" 'utf-8)
 gnus-active-hashtb)

;; Replace "nnml:テスト" with your group name in question.

If this returns nil but the next one returns non-nil,

(intern-soft
 (string-as-multibyte
  (encode-coding-string "nnml:テスト" 'utf-8))
 gnus-active-hashtb)

there would still be something wrong in the Gnus code (or your
customization).  Even so, `message-tab' should work.

FYI:
By default, `message-tab' runs `message-expand-group'[1] that
extracts group names for the candidates from `gnus-active-hashtb'.
If all goes well, a non-ASCII group name in it is encoded by the
utf-8 coding system[2] in the unibyte mode.

[1] The completion function may be customized by
    `message-completion-alist'.
[2] `gnus-group-name-charset-group-alist' specifies it.

Re: groups with non-ASCII names

[Peter Münster]

On Thu, Dec 03 2015, Katsumi Yamaoka wrote:

> Only the non-ASCII group doesn't appear in the candidates list
> for the completion, whereas that group appears in the Group buffer,
> doesn't it?  If so, unfortunately I have no idea how it happens.

Strange: it works now. Yesterday, I got completion with
gnus-group-jump-to-group but not in message-mode in the Gcc: line with
the same group. Today it works.

I guess, that it has nothing to do with utf-8 characters but it's
related to the "tab-completion with gnus-group-jump-to-group" issue.
Sorry for the noise...

-- 
           Peter