Gnus development mailing list
 help / color / mirror / Atom feed
* emacs 28 search not working with nnml
@ 2021-06-23 12:54 Thomas Alexander Gerds
  2021-06-23 13:51 ` Eric S Fraga
  0 siblings, 1 reply; 8+ messages in thread
From: Thomas Alexander Gerds @ 2021-06-23 12:54 UTC (permalink / raw)
  To: ding

hello

a recent upgrade to emacs 28.0.50 and my homegrown search functions
stopped working. Eric Abrahamsen kindly helped me to translate my
obsolete nnir-settings (thank you Eric!). but there is another problem
deeper in the system which I describe here.

I am using nnml and have all my email downloaded to my laptop using
notmuch for searching. now I am confident that notmuch is finding some
articles when I enter via GG or envoke this very simple function:

(defun test-search ()
  (interactive)
  (gnus-group-read-ephemeral-search-group
   nil
   `((search-query-spec (query . "from:Alessandra"))
     (search-group-spec ("nnml:" . ("nnml:research"))))))

some hours debugging showed that nnselect-retrieve-headers calls the
macro nnselect-get-artlist which returns a too long group name, i.e.,
"nnml:home.tag.mail.research" where I think that it should simply return
"research". at least this helps a little bit because

a) now nnmail-group-pathname does not produce the buggy result
/home/tag/mail/nnml/home/tag/mail

b) the following hack

,----
| (setq nnselect-retrieve-headers-override-function 'my-retrieve-headers)
| (defun my-retrieve-headers (sequence &optional group server fetch-old)
|   (setq group (replace-in-string group "nnml.home.tag.mail." ""))
|     (when (nnml-possibly-change-directory group server)
|     ...
`----

shows me a summary buffer with the search results. only the body of
articles are all empty. I guess this happens because
(replace-in-string group "nnml.home.tag.mail." "") should be applied not only when
retrieving headers but also when retrieving articles.

now I am hoping that this is a bug which can be fixed somewhere earlier
on in this complex chain of commands? but, any help is very much
appreciated.

cheers, Thomas



-- 
7LL-6 Stop thinking too much, it’s alright not to know all the answers.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-23 12:54 emacs 28 search not working with nnml Thomas Alexander Gerds
@ 2021-06-23 13:51 ` Eric S Fraga
  2021-06-23 15:01   ` Eric Abrahamsen
  0 siblings, 1 reply; 8+ messages in thread
From: Eric S Fraga @ 2021-06-23 13:51 UTC (permalink / raw)
  To: ding

On Wednesday, 23 Jun 2021 at 14:54, Thomas Alexander Gerds wrote:
> I am using nnml and have all my email downloaded to my laptop using

I have a similar configuration.  My settings look like this:

#+begin_src emacs-lisp
  (setq gnus-secondary-select-methods
        '((nnml "work"
                (gnus-search-engine gnus-search-notmuch
                                    (remove-prefix "/HOME/Mail")
                                    (config-file "/HOME/.notmuch-config")
                                    )
                (get-new-mail t))))

  (add-to-list 'gnus-search-default-engines '(nnml . notmuch))
  (setq gnus-search-notmuch-remove-prefix "/HOME/Mail")
  (with-eval-after-load "gnus-search"
    (cl-defmethod gnus-search-indexed-parse-output :around
      ((e gnus-search-notmuch) s q groups)
      (let ((gs (mapcar (lambda (g) (replace-regexp-in-string "\\." "/" g))
                        groups)))
        (cl-call-next-method e s q gs))))
#+end_src

where HOME in various places above should be replaced by your home
directory.  I have gnus splitting email into various sub-directories of
Mail/.

The last bit was suggested by Jose A. Ortega Ruiz on the gnus newsgroup
[1] sometime in late 2020 and proved to be necessary (at that time) to
get the new search engine to work with nnml.  It basically changes the
file paths from . to / separators.



Footnotes:
[1]  gnus:nntp+news.gwene.org:gmane.emacs.gnus.general#87pn4hc97h.fsf@gnus.jao.io

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4.6 on Debian bullseye/sid



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-23 13:51 ` Eric S Fraga
@ 2021-06-23 15:01   ` Eric Abrahamsen
  2021-06-24  5:45     ` Thomas Alexander Gerds
  2021-06-24 10:01     ` Eric S Fraga
  0 siblings, 2 replies; 8+ messages in thread
From: Eric Abrahamsen @ 2021-06-23 15:01 UTC (permalink / raw)
  To: ding

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Wednesday, 23 Jun 2021 at 14:54, Thomas Alexander Gerds wrote:
>> I am using nnml and have all my email downloaded to my laptop using
>
> I have a similar configuration.  My settings look like this:
>
> #+begin_src emacs-lisp
>   (setq gnus-secondary-select-methods
>         '((nnml "work"
>                 (gnus-search-engine gnus-search-notmuch
>                                     (remove-prefix "/HOME/Mail")
>                                     (config-file "/HOME/.notmuch-config")
>                                     )
>                 (get-new-mail t))))
>
>   (add-to-list 'gnus-search-default-engines '(nnml . notmuch))
>   (setq gnus-search-notmuch-remove-prefix "/HOME/Mail")
>   (with-eval-after-load "gnus-search"
>     (cl-defmethod gnus-search-indexed-parse-output :around
>       ((e gnus-search-notmuch) s q groups)
>       (let ((gs (mapcar (lambda (g) (replace-regexp-in-string "\\." "/" g))
>                         groups)))
>         (cl-call-next-method e s q gs))))
> #+end_src
>
> where HOME in various places above should be replaced by your home
> directory.  I have gnus splitting email into various sub-directories of
> Mail/.
>
> The last bit was suggested by Jose A. Ortega Ruiz on the gnus newsgroup
> [1] sometime in late 2020 and proved to be necessary (at that time) to
> get the new search engine to work with nnml.  It basically changes the
> file paths from . to / separators.

FWIW, this hack isn't necessary any more, if you're running a recent
Emacs from master (looks like since early February of this year).

Thomas, the bug you're seeing is something I'm aware of, I just haven't
come up with a great solution for it yet (actually I'd forgotten about
it). I'll get that fixed soon, and report back here.

Eric



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-23 15:01   ` Eric Abrahamsen
@ 2021-06-24  5:45     ` Thomas Alexander Gerds
  2021-06-24 15:21       ` Eric Abrahamsen
  2021-06-24 10:01     ` Eric S Fraga
  1 sibling, 1 reply; 8+ messages in thread
From: Thomas Alexander Gerds @ 2021-06-24  5:45 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding


Eric(1): thank you so much, this helped and solved my problem!
Eric(2): you are right: the hack is not needed and my search functions work
         smoothly without it. not sure what remaining bug there might
         be, but you may want to consider to change the default value
         here for people like me who use "mail" instead of "Mail":

(defcustom gnus-search-notmuch-remove-prefix gnus-directory
  "The prefix to remove from each file name returned by notmuch
in order to get a group name (albeit with / instead of .).  This is a
regular expression.

This variable can also be set per-server."
  :type 'regexp
  :version "28.1")

cheers Thomas

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>
>> On Wednesday, 23 Jun 2021 at 14:54, Thomas Alexander Gerds wrote:
>>> I am using nnml and have all my email downloaded to my laptop using
>>
>> I have a similar configuration.  My settings look like this:
>>
>> #+begin_src emacs-lisp
>>   (setq gnus-secondary-select-methods
>>         '((nnml "work"
>>                 (gnus-search-engine gnus-search-notmuch
>>                                     (remove-prefix "/HOME/Mail")
>>                                     (config-file "/HOME/.notmuch-config")
>>                                     )
>>                 (get-new-mail t))))
>>
>>   (add-to-list 'gnus-search-default-engines '(nnml . notmuch))
>>   (setq gnus-search-notmuch-remove-prefix "/HOME/Mail")
>>   (with-eval-after-load "gnus-search"
>>     (cl-defmethod gnus-search-indexed-parse-output :around
>>       ((e gnus-search-notmuch) s q groups)
>>       (let ((gs (mapcar (lambda (g) (replace-regexp-in-string "\\." "/" g))
>>                         groups)))
>>         (cl-call-next-method e s q gs))))
>> #+end_src
>>
>> where HOME in various places above should be replaced by your home
>> directory.  I have gnus splitting email into various sub-directories of
>> Mail/.
>>
>> The last bit was suggested by Jose A. Ortega Ruiz on the gnus newsgroup
>> [1] sometime in late 2020 and proved to be necessary (at that time) to
>> get the new search engine to work with nnml.  It basically changes the
>> file paths from . to / separators.
>
> FWIW, this hack isn't necessary any more, if you're running a recent
> Emacs from master (looks like since early February of this year).
>
> Thomas, the bug you're seeing is something I'm aware of, I just haven't
> come up with a great solution for it yet (actually I'd forgotten about
> it). I'll get that fixed soon, and report back here.
>
> Eric
>
>
>
>

-- 
Thomas A. Gerds -- Department of Biostatistics Copenhagen
University of Copenhagen, Oester Farimagsgade 5, 1014 Copenhagen,
Denmark


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-23 15:01   ` Eric Abrahamsen
  2021-06-24  5:45     ` Thomas Alexander Gerds
@ 2021-06-24 10:01     ` Eric S Fraga
  1 sibling, 0 replies; 8+ messages in thread
From: Eric S Fraga @ 2021-06-24 10:01 UTC (permalink / raw)
  To: ding

On Wednesday, 23 Jun 2021 at 08:01, Eric Abrahamsen wrote:
> FWIW, this hack isn't necessary any more, if you're running a recent
> Emacs from master (looks like since early February of this year).

Thank you.  I will remove this hack.

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4.6 on Debian bullseye/sid



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-24  5:45     ` Thomas Alexander Gerds
@ 2021-06-24 15:21       ` Eric Abrahamsen
  2021-06-24 19:15         ` Thomas Alexander Gerds
  0 siblings, 1 reply; 8+ messages in thread
From: Eric Abrahamsen @ 2021-06-24 15:21 UTC (permalink / raw)
  To: ding

Thomas Alexander Gerds <grb615@ku.dk> writes:

> Eric(1): thank you so much, this helped and solved my problem!
> Eric(2): you are right: the hack is not needed and my search functions work
>          smoothly without it. not sure what remaining bug there might
>          be,

The remaining bug is subtler: if you search a group whose name is a
substring of a different group, the code will also attempt to return
results from that other group, which will fail.

> but you may want to consider to change the default value here for
> people like me who use "mail" instead of "Mail":

Wait, which default value are you referring to here? `gnus-directory'?

> (defcustom gnus-search-notmuch-remove-prefix gnus-directory
>   "The prefix to remove from each file name returned by notmuch
> in order to get a group name (albeit with / instead of .).  This is a
> regular expression.
>
> This variable can also be set per-server."
>   :type 'regexp
>   :version "28.1")
>
> cheers Thomas
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>>
>>> On Wednesday, 23 Jun 2021 at 14:54, Thomas Alexander Gerds wrote:
>>>> I am using nnml and have all my email downloaded to my laptop using
>>>
>>> I have a similar configuration.  My settings look like this:
>>>
>>> #+begin_src emacs-lisp
>>>   (setq gnus-secondary-select-methods
>>>         '((nnml "work"
>>>                 (gnus-search-engine gnus-search-notmuch
>>>                                     (remove-prefix "/HOME/Mail")
>>>                                     (config-file "/HOME/.notmuch-config")
>>>                                     )
>>>                 (get-new-mail t))))
>>>
>>>   (add-to-list 'gnus-search-default-engines '(nnml . notmuch))
>>>   (setq gnus-search-notmuch-remove-prefix "/HOME/Mail")
>>>   (with-eval-after-load "gnus-search"
>>>     (cl-defmethod gnus-search-indexed-parse-output :around
>>>       ((e gnus-search-notmuch) s q groups)
>>>       (let ((gs (mapcar (lambda (g) (replace-regexp-in-string "\\." "/" g))
>>>                         groups)))
>>>         (cl-call-next-method e s q gs))))
>>> #+end_src
>>>
>>> where HOME in various places above should be replaced by your home
>>> directory.  I have gnus splitting email into various sub-directories of
>>> Mail/.
>>>
>>> The last bit was suggested by Jose A. Ortega Ruiz on the gnus newsgroup
>>> [1] sometime in late 2020 and proved to be necessary (at that time) to
>>> get the new search engine to work with nnml.  It basically changes the
>>> file paths from . to / separators.
>>
>> FWIW, this hack isn't necessary any more, if you're running a recent
>> Emacs from master (looks like since early February of this year).
>>
>> Thomas, the bug you're seeing is something I'm aware of, I just haven't
>> come up with a great solution for it yet (actually I'd forgotten about
>> it). I'll get that fixed soon, and report back here.
>>
>> Eric
>>
>>
>>
>>



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-24 15:21       ` Eric Abrahamsen
@ 2021-06-24 19:15         ` Thomas Alexander Gerds
  2021-06-26  3:48           ` Eric Abrahamsen
  0 siblings, 1 reply; 8+ messages in thread
From: Thomas Alexander Gerds @ 2021-06-24 19:15 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Thomas Alexander Gerds <grb615@ku.dk> writes:
>
>> Eric(1): thank you so much, this helped and solved my problem!
>> Eric(2): you are right: the hack is not needed and my search functions work
>>          smoothly without it. not sure what remaining bug there might
>>          be,
>
> The remaining bug is subtler: if you search a group whose name is a
> substring of a different group, the code will also attempt to return
> results from that other group, which will fail.

aah. haha, and I thought it was a neat feature that in addition to
nnml:research the search was also applied to nnml:research.old

>
>> but you may want to consider to change the default value here for
>> people like me who use "mail" instead of "Mail":
>
> Wait, which default value are you referring to here? `gnus-directory'?

no the default value of gnus-search-notmuch-remove-prefix. I have
forgotten if there also is a variable that has a default value equal to
"~/Mail". it seems that gnus-directory defaults to "~/News".


>> (defcustom gnus-search-notmuch-remove-prefix gnus-directory
>>   "The prefix to remove from each file name returned by notmuch
>> in order to get a group name (albeit with / instead of .).  This is a
>> regular expression.
>>
>> This variable can also be set per-server."
>>   :type 'regexp
>>   :version "28.1")
>>
>> cheers Thomas
>>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>>>
>>>> On Wednesday, 23 Jun 2021 at 14:54, Thomas Alexander Gerds wrote:
>>>>> I am using nnml and have all my email downloaded to my laptop using
>>>>
>>>> I have a similar configuration.  My settings look like this:
>>>>
>>>> #+begin_src emacs-lisp
>>>>   (setq gnus-secondary-select-methods
>>>>         '((nnml "work"
>>>>                 (gnus-search-engine gnus-search-notmuch
>>>>                                     (remove-prefix "/HOME/Mail")
>>>>                                     (config-file "/HOME/.notmuch-config")
>>>>                                     )
>>>>                 (get-new-mail t))))
>>>>
>>>>   (add-to-list 'gnus-search-default-engines '(nnml . notmuch))
>>>>   (setq gnus-search-notmuch-remove-prefix "/HOME/Mail")
>>>>   (with-eval-after-load "gnus-search"
>>>>     (cl-defmethod gnus-search-indexed-parse-output :around
>>>>       ((e gnus-search-notmuch) s q groups)
>>>>       (let ((gs (mapcar (lambda (g) (replace-regexp-in-string "\\." "/" g))
>>>>                         groups)))
>>>>         (cl-call-next-method e s q gs))))
>>>> #+end_src
>>>>
>>>> where HOME in various places above should be replaced by your home
>>>> directory.  I have gnus splitting email into various sub-directories of
>>>> Mail/.
>>>>
>>>> The last bit was suggested by Jose A. Ortega Ruiz on the gnus newsgroup
>>>> [1] sometime in late 2020 and proved to be necessary (at that time) to
>>>> get the new search engine to work with nnml.  It basically changes the
>>>> file paths from . to / separators.
>>>
>>> FWIW, this hack isn't necessary any more, if you're running a recent
>>> Emacs from master (looks like since early February of this year).
>>>
>>> Thomas, the bug you're seeing is something I'm aware of, I just haven't
>>> come up with a great solution for it yet (actually I'd forgotten about
>>> it). I'll get that fixed soon, and report back here.
>>>
>>> Eric
>>>
>>>
>>>
>>>
>
>
>
>

-- 
Thomas A. Gerds -- Department of Biostatistics Copenhagen
University of Copenhagen, Oester Farimagsgade 5, 1014 Copenhagen,
Denmark


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: emacs 28 search not working with nnml
  2021-06-24 19:15         ` Thomas Alexander Gerds
@ 2021-06-26  3:48           ` Eric Abrahamsen
  0 siblings, 0 replies; 8+ messages in thread
From: Eric Abrahamsen @ 2021-06-26  3:48 UTC (permalink / raw)
  To: ding

Thomas Alexander Gerds <grb615@ku.dk> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Thomas Alexander Gerds <grb615@ku.dk> writes:
>>
>>> Eric(1): thank you so much, this helped and solved my problem!
>>> Eric(2): you are right: the hack is not needed and my search functions work
>>>          smoothly without it. not sure what remaining bug there might
>>>          be,
>>
>> The remaining bug is subtler: if you search a group whose name is a
>> substring of a different group, the code will also attempt to return
>> results from that other group, which will fail.
>
> aah. haha, and I thought it was a neat feature that in addition to
> nnml:research the search was also applied to nnml:research.old

No, sorry, that's a bug :) But you can certainly use a mark-by-regexp
command to mark all your "*research*" groups and then search them at
once! Only fourteen more keystrokes.

>>
>>> but you may want to consider to change the default value here for
>>> people like me who use "mail" instead of "Mail":
>>
>> Wait, which default value are you referring to here? `gnus-directory'?
>
> no the default value of gnus-search-notmuch-remove-prefix. I have
> forgotten if there also is a variable that has a default value equal to
> "~/Mail". it seems that gnus-directory defaults to "~/News".

Oh, I see. That's just the way that's always been, it's not something I
set. It's hard to imagine any default value that would actually be of
use -- I imagine everyone's got something slightly different.



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-06-26  3:48 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-23 12:54 emacs 28 search not working with nnml Thomas Alexander Gerds
2021-06-23 13:51 ` Eric S Fraga
2021-06-23 15:01   ` Eric Abrahamsen
2021-06-24  5:45     ` Thomas Alexander Gerds
2021-06-24 15:21       ` Eric Abrahamsen
2021-06-24 19:15         ` Thomas Alexander Gerds
2021-06-26  3:48           ` Eric Abrahamsen
2021-06-24 10:01     ` Eric S Fraga

Gnus development mailing list

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.vuxu.org/ding

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 ding ding/ http://inbox.vuxu.org/ding \
		ding@inbox.vuxu.org
	public-inbox-index ding

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.emacs.gnus.general


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git